Why NEAT Stopped Forming Species When the Starting Genome Grew

NEAT, short for NeuroEvolution of Augmenting Topologies, evolves a population of neural networks, adding connections and nodes as it goes. A newly added node usually makes a network worse until its weights are tuned, so NEAT sorts the population into species. A network with a new node then competes with its relatives rather than with the whole population, which gives the new structure time to pay off. I had it forming species on XOR and assumed the same code would do the same on a larger problem. On a Monopoly player with 180 starting connections it formed exactly one species for ten generations straight. Nothing about the run looked wrong. It ran, mutated and bred as usual, and the species count simply stayed at 1.

How NEAT decides two networks are different

A genome is the list of connections that describes one network, and each connection in it is a gene. Every gene carries an innovation number recording when that connection first appeared anywhere in the population, which lets two genomes be lined up gene by gene. Every genome starts as a copy of the same minimal network, with each input and a bias wired straight to each output. For XOR that is 2 inputs plus a bias to 1 output, 3 genes. For my Monopoly player it is 44 inputs plus a bias to 4 outputs, 180 genes, and every member of the population shares all of them. The original paper measures the difference between two genomes as a compatibility distance:

δ = c1·E/N + c2·D/N + c3·W̄

E counts excess genes and D counts disjoint genes. If one genome has innovations 1 to 5 and the other has 1 to 3 and 7, then 4 and 5 are disjoint (gaps in the middle) and 7 is excess (past the end of the other genome). W̄ is the average weight difference over the genes both share, and N is the gene count of the larger genome. Each generation, a network joins the first species whose representative, a random member of that species from the previous generation, is closer than a threshold; if none is, it founds a new species. Because it is compared with a representative rather than with every other member, a species can hold pairs that are further apart than the threshold. I used the paper’s XOR coefficients of 1, 1 and 0.4 and a threshold of 3. The paper also allows N to be 1 when both genomes have fewer than 20 genes, and my implementation does that.

The division by N is the part that matters here. It makes two large genomes that differ by a few genes count as similar. That makes sense when every gene in a genome was added by evolution, as it nearly is on XOR. It makes much less sense when most of the genes were in the starting genome and every member of the population has them.

The measurement

I ran the project’s NEAT for ten generations on two problems, twice each, from one script with a single fixed random seed. Then I sampled 800 random pairs from each final population and took the median distance. The last column lists the species count after each of the ten generations.

The XOR population had 150 networks and was scored on the four XOR cases. The Monopoly population had 100 networks, each with 44 inputs and 4 outputs. Each was scored on eight games against three scripted players that buy everything they land on.

After ten generations the average Monopoly genome had 181.6 genes, only 1.6 more than it started with. Two genomes typically differed by a handful of genes out of about 180. Even five differing genes, divided by 180, add under 0.03 to the distance, so the median of 0.368 is almost entirely the weight term. For the weight term alone to reach 3, the shared weights would have to differ by 7.5 on average, and they never did. The 90th percentile distance was 0.403, so almost no pair came anywhere near 3.

The change

The question was whether distance should account for the starting genome’s size, and the change I made was small. N now counts the genes added since the start rather than all of them:

let added = Math.max(this.connections.length, other.connections.length) - baseGenes;
let n = (added < 20) ? 1 : added;

baseGenes is the starting genome’s gene count. The NEAT object records it when it is created and passes it to every distance call during speciation, and with a default of 0 the function behaves exactly as before. The Monopoly genomes had added about three genes on average, far short of 20, so N fell to 1 and each added gene counted fully. The median distance went from 0.368 to 6.379, and the population held six species from the sixth generation on.

On XOR the change does nothing, because the genomes have fewer than 20 genes and N is 1 either way. A test in the project checks that on 100 random pairs of small genomes. The two XOR rows still differ, 7.203 against 8.266 and 13 species against 14, because the seed was set once for the whole script, so the second run drew different random numbers from the first. That difference is what ordinary run-to-run variation looks like. The Monopoly gap is 17 times the distance and five extra species, which variation of that size does not explain.

A unit test shows the same effect on a smaller network without a full run. It builds a 60-network population from a 54-gene starting genome, grows half of it, and speciates: 31 species with the change and 1 without.

What this does and does not show

• It shows that the species mechanism was switched off, not that turning it back on helps. I have not compared Monopoly play from a population with 6 species against one with 1. The paper argues that speciation protects new structure long enough to tune it. Whether that matters here is a separate experiment.
• The species count can run away the other way. With N at 1, every added gene adds a full point to the distance, so a population that keeps adding structure keeps splitting. XOR reached 14 species out of 150 networks by generation 10. Spread across dozens of species, 150 networks leave each one too small to evolve much. Some NEAT implementations adjust the threshold as they go to hold the species count near a target; mine does not yet.
• One seed per configuration. The Monopoly result is large enough that I do not expect another seed to reverse it, but I have not run one.

The code is a learning project and is not public. I wrote the NEAT implementation; Claude Code wrote the tests I built it against and the script that produced the table above. The fix is two lines. What I would check first in any other NEAT port is whether its distance formula assumes a small starting genome. This failure produces a run that looks like it is working, with the one counter nobody watches stuck on 1.