Tagged statistics
10 write-ups.
Reading a Reinforcement Learning Taxonomy Against My Own Tools
Nine of the list's fifteen sections need a scalar reward my generator has not got. The one branch that does apply separates good builds at AUC 0.784, and the name of the website a request was scraped from scores 0.676 on the same rows.
A Measured Headroom That Nothing Could Reach
A per-position oracle over my code completer's mixing weight is worth +0.0705 top-1, with a confidence interval excluding zero. Three feature families captured 0.0%, -3.1% and -0.9% of it, and the whole signal available to a per-class router turned out to be the same size as the bias from fitting one.
A Learner That Scored Below Its Own Random Control
Growing hidden units scored 15.5 of 75 where random units of the same shape scored 17.0 and a hand-written keyword rule scored 47. In a separate protocol, handing the learner the rule's own answer as one more feature was worth a reliable +7.8 rows and still did not reach the rule that supplied it.
Words Fitted to a Rule's Own Mistakes Do Not Travel
Seven words selected from a keyword rule's own errors gained 8 rows of 87, above the maximum of 200 random draws from the same pool. Four words fitted the other way round gained 4 of 47 and moved the held-out corpus none, in the direction where a third of random draws move it by a row.
Code Completion From Your Own Repository's Statistics
One number decides whether it helps: the recital rate, 13.5% to 72.9% across nine corpora. Above 60% recital the n-gram index beats a ten-line frequency table; at 13.5% it beats nothing, the harness prints the null, and one published row was wrong and is corrected in the README.
Measuring How Cost Scales by Counting Instead of Timing
Insertion sort on seed 17 at n=64 performs reads 3812 and writes 1848, the same integers in Python and JavaScript, because the generator, the iteration rule and the subscript rule are part of the contract. The same parity suite caught its own dependency formatting one number two ways above a million.
A Curve Fitter That Refuses to Answer
A tool that always produced a constant would be useless and would still pass every test that checks it produces one. 0.2.0 fixed a defect the suite had pinned: with error bars from scatter alone, a quantity measured exactly was reported as one that could not be determined.
When a Zero-Parameter Cache Overtakes a Transformer
A count table over the current document has no parameters and no training. Somewhere between 60 and 250 tokens of document it passes a 1.43M-parameter transformer, and by 1000 tokens it wins top-1 by 0.064, a 43% relative margin. Adding the transformer on top then buys 0.002.
The Last Non-Neural Candidate, and It Did Not Clear the Bar
The bar was a slope: keep converting extra data into accuracy after exact-context statistics saturate. A full hierarchical Pitman-Yor model with Gibbs sweeps and inferred discounts moved the intercept and left the slope alone, halving with every doubling exactly as cruder count models did.
Ranking Language Models by How Well They Spot Liars
I built a benchmark to rank language models on spotting liars, then ran 522 games of Mafia through it. The same model scored -0.400 and +0.078 in one run, 52 games apart. Day 1 accusations came in at -0.005 against chance on n=2,103, and the whole thing reproduces in 11 seconds with no API keys.