stockfish increasing returns with low nodes

[Uri Blass]

I thought that generally there is a low of diminishing returns but when I look at the data in the following link I am surprised to find increasing returns for stockfish at low number of nodes



results with 200 nodes against 100 nodes is 4237-3322 with 441 draws that is only 39.91 elo difference
results with 6250 nodes against 3125 nodes is 285.97 elo advantage 6373-959 and 668 draws
results with 12500 nodes against 6250 nodes is 286.45 elo advantage 3098-388 and 514 draws and only with more nodes we start to see diminishing returns.

Note that I expected maybe not to see diminishing returns with small number of nodes but I did not expect to see a significant increasing returns.

results at 102,400,000 nodes against 51,200,000 nodes is 11-3 and 86 draws that gives an estimate of 27.85 elo after 100 games but I think that you need more games for a better estimate.

[Laskos]

I thought that generally there is a low of diminishing returns but when I look at the data in the following link I am surprised to find increasing returns for stockfish at low number of nodes



results with 200 nodes against 100 nodes is 4237-3322 with 441 draws that is only 39.91 elo difference
results with 6250 nodes against 3125 nodes is 285.97 elo advantage 6373-959 and 668 draws
results with 12500 nodes against 6250 nodes is 286.45 elo advantage 3098-388 and 514 draws and only with more nodes we start to see diminishing returns.

Note that I expected maybe not to see diminishing returns with small number of nodes but I did not expect to see a significant increasing returns.

results at 102,400,000 nodes against 51,200,000 nodes is 11-3 and 86 draws that gives an estimate of 27.85 elo after 100 games but I think that you need more games for a better estimate.

The precision of node counter in SF is not that high, on my hardware about 1000 or so nodes. So, 200 against 100 nodes is more like 1200 vs 1100 or something like that, you understand, so the result is not to be taken seriously. There are engines with very precise counter, but I forgot which exactly.

[Laskos]

In fact that was an interesting issue, very low nodes behavior. It is a pretty much uncharted territory. I saw that Critter 1.6 obeys precisely the "go nodes" command. I am not sure what nodes it considers as nodes, q-search and such, but it's probably not that different from what other top engines do. The first reliable doubling I got was in 200:100 nodes games, which is the lowest reliable node count games I had. Each data point is from 4000 games matches, the error 2SD is about 10 Elo points.

Code: Select all

Critter 1.6


200:100 nodes
Elo: 108 
draw rate: 1.8%

400:200 nodes
Elo: 111
draw rate: 1.7%

800:400 nodes
Elo: 188
draw rate: 2.0%

1600:800 nodes
Elo: 404
draw rate: 3.0%

3200:1600 nodes
Elo: 356
draw rate: 9.6%

6400:3200 nodes
Elo: 302
draw rate: 17.0%

12800:6400 nodes
Elo: 272
draw rate: 20.5%

25600:12800 nodes
Elo: 231
draw rate: 23.6%

51200:25600 nodes
Elo: 201
draw rate: 30.7%

Observe also the draw rate, at very low node counts, it is extremely low, very much lower than that of the random player. After these node counts in the table, we are in the region of the ultra-fast games, which is known. Then blitz games, longer time control games, and finally TCEC.

Here is the plot: red line represents the new data, blue line is its continuation according to our past knowledge (the diminishing returns behavior from ultra-fast to LTC is known).