TalkChess.com

LTC (This exact version. . . more simplified)
LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
Total: 5392 W: 729 L: 631 D: 4032 Elo +6.32

This is endgame patch and it's difficult to believe it's much/any gain there. And NCM test from same patch gives -5 ELO! Don't expect SF 12 before 2022 .

If we're talking the usual 50 Elo increase between releases, I'd say that well into 2022 by the looks of it:



The graph covers data from ncm spanning more than 6 months and shows an increase of less than 10 Elo points for that time period.

Jouni wrote: ↑Thu Mar 05, 2020 9:07 am LTC (This exact version. . . more simplified)
LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
Total: 5392 W: 729 L: 631 D: 4032 Elo +6.32

This is endgame patch and it's difficult to believe it's much/any gain there. And NCM test from same patch gives -5 ELO! Don't expect SF 12 before 2022 .

The patch apparently replaces two arrays of values with linear formulas. Hard to believe that this alters Elo significantly in either direction.

https://github.com/official-stockfish/S ... c26L57-R59

I'm not surprised.

A memory lookup is expensive. If you can calculate the same thing from a formula it will be much faster. If it is something that happens a lot, you could see a big benefit.

Bench numbers tell the tale.
5123316/4923286 = 4% faster for the entire program. That is amazing.

Since it is an endgame change, I guess that endgame speedup will be even more dramatic than the average.

It is clear to me that the NCM measurement is the one that is buggy. There really isn't a functional change and the new version is much faster.

Maybe in some endgames there are big tuning changes still waiting, and this functionally changes just something right? As Protospring is saying, the bench right now is not really affected by a lot of endgame changes, they are not triggered in the searched positions. But it is a functional change from the change in benchmark. But that is not a speedtest so you can't tell nodes per second from it? It will be interesting what Stefan Pohl's testing may say about performance against other programs. Can't test much myself right now, maybe someone is interested in testing some endgames?

"Bench numbers tell the tale. 5123316/4923286 = 4% faster for the entire program. That is amazing." Really Dann I don't believe this 4%. Bench is not nps!

Fewer nodes is equally good.

Did you actually interpret the node count of bench as meaningful for strength ?

Extremely minor changes that have about 0 elo impact can impact bench's nodes count by more than 10%.

While massive differences in bench node count are indicative of different amounts of pruning and re-searching, minor differences between close versions are pseudo-random.