LTC (This exact version. . . more simplified)
LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
Total: 5392 W: 729 L: 631 D: 4032 Elo +6.32
This is endgame patch and it's difficult to believe it's much/any gain there. And NCM test from same patch gives -5 ELO! Don't expect SF 12 before 2022 .
Stockfish and latest +6 ELO patch!
Moderators: hgm, Rebel, chrisw
-
- Posts: 1534
- Joined: Sun Oct 25, 2009 2:30 am
Re: Stockfish and latest +6 ELO patch!
If we're talking the usual 50 Elo increase between releases, I'd say that well into 2022 by the looks of it:
The graph covers data from ncm spanning more than 6 months and shows an increase of less than 10 Elo points for that time period.
The graph covers data from ncm spanning more than 6 months and shows an increase of less than 10 Elo points for that time period.
-
- Posts: 6442
- Joined: Tue Jan 09, 2007 12:31 am
- Location: PA USA
- Full name: Louis Zulli
Re: Stockfish and latest +6 ELO patch!
The patch apparently replaces two arrays of values with linear formulas. Hard to believe that this alters Elo significantly in either direction.Jouni wrote: ↑Thu Mar 05, 2020 9:07 am LTC (This exact version. . . more simplified)
LLR: 2.95 (-2.94,2.94) {-1.50,0.50}
Total: 5392 W: 729 L: 631 D: 4032 Elo +6.32
This is endgame patch and it's difficult to believe it's much/any gain there. And NCM test from same patch gives -5 ELO! Don't expect SF 12 before 2022 .
https://github.com/official-stockfish/S ... c26L57-R59
-
- Posts: 12538
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Stockfish and latest +6 ELO patch!
I'm not surprised.
A memory lookup is expensive. If you can calculate the same thing from a formula it will be much faster. If it is something that happens a lot, you could see a big benefit.
Bench numbers tell the tale.
5123316/4923286 = 4% faster for the entire program. That is amazing.
Since it is an endgame change, I guess that endgame speedup will be even more dramatic than the average.
It is clear to me that the NCM measurement is the one that is buggy. There really isn't a functional change and the new version is much faster.
A memory lookup is expensive. If you can calculate the same thing from a formula it will be much faster. If it is something that happens a lot, you could see a big benefit.
Bench numbers tell the tale.
5123316/4923286 = 4% faster for the entire program. That is amazing.
Since it is an endgame change, I guess that endgame speedup will be even more dramatic than the average.
It is clear to me that the NCM measurement is the one that is buggy. There really isn't a functional change and the new version is much faster.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 4564
- Joined: Sun Mar 12, 2006 2:40 am
- Full name:
Re: Stockfish and latest +6 ELO patch!
Maybe in some endgames there are big tuning changes still waiting, and this functionally changes just something right? As Protospring is saying, the bench right now is not really affected by a lot of endgame changes, they are not triggered in the searched positions. But it is a functional change from the change in benchmark. But that is not a speedtest so you can't tell nodes per second from it? It will be interesting what Stefan Pohl's testing may say about performance against other programs. Can't test much myself right now, maybe someone is interested in testing some endgames?
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
-
- Posts: 3283
- Joined: Wed Mar 08, 2006 8:15 pm
Re: Stockfish and latest +6 ELO patch!
"Bench numbers tell the tale. 5123316/4923286 = 4% faster for the entire program. That is amazing." Really Dann I don't believe this 4%. Bench is not nps!
Jouni
-
- Posts: 12538
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Stockfish and latest +6 ELO patch!
Fewer nodes is equally good.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 550
- Joined: Tue Nov 19, 2019 8:48 pm
- Full name: Alayan Feh
Re: Stockfish and latest +6 ELO patch!
Did you actually interpret the node count of bench as meaningful for strength ?
Extremely minor changes that have about 0 elo impact can impact bench's nodes count by more than 10%.
While massive differences in bench node count are indicative of different amounts of pruning and re-searching, minor differences between close versions are pseudo-random.
Extremely minor changes that have about 0 elo impact can impact bench's nodes count by more than 10%.
While massive differences in bench node count are indicative of different amounts of pruning and re-searching, minor differences between close versions are pseudo-random.