Stockfish developmental progress is tested at https://nextchessmove.com/dev-builds
Each change is tested against Stockfish 7 in a trial of 20,000 random games. The randomness leaves room for variation.
However, in the last 15 trials covering the last 15 changes, the Elo differential above Stockfish 7, has been under 365 elo points.
If we look at the 14 trials prior to the last 15 trials, 10 out of the 14 had an Elo differential above 365 points.
Something seems to have gone wrong after the versions of 2020.22.10. On that date, 2 trials were above 370, and since then none have been above 365. It could be due to randomness, or perhaps some bug was accidentally introduced.
Stockfish developmental progess on pause?
Moderators: hgm, Rebel, chrisw
-
- Posts: 1056
- Joined: Thu Mar 09, 2006 4:15 pm
- Location: Long Island, NY, USA
Stockfish developmental progess on pause?
Last edited by Norm Pollock on Sun Nov 15, 2020 5:46 pm, edited 3 times in total.
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: Stockfish developmental progess on pause?
Or maybe Stockfish has reached perfect play and no more progress is possible.Norm Pollock wrote: ↑Sun Nov 15, 2020 5:41 pm Stockfish developmental progress is tested at https://nextchessmove.com/dev-builds
Each change is tested against Stockfish 7 in a trial of 20,000 random games. The randomness leaves room for variation.
However, in the last 10 trials covering the last 10 changes, the Elo differential above Stockfish 7, has been under 365 elo points.
If we look at the 12 trials prior to the last 10 trials, 9 out of the 12 had an Elo differential above 365 points.
Something seems to have gone wrong after the versions of 2020.22.10. On that date, 2 trials were above 370, and since then none have been above 365. It could be due to randomness or perhaps, or perhaps some bug was accidentally introduced.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 4605
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Stockfish developmental progess on pause?
IMHO opinion SF7 is simply too weak meanwhile to get reliable results against a much stronger SF12 and newer dev versions.Norm Pollock wrote: ↑Sun Nov 15, 2020 5:41 pm Stockfish developmental progress is tested at https://nextchessmove.com/dev-builds
Each change is tested against Stockfish 7 in a trial of 20,000 random games. The randomness leaves room for variation.
However, in the last 15 trials covering the last 15 changes, the Elo differential above Stockfish 7, has been under 365 elo points.
If we look at the 14 trials prior to the last 15 trials, 10 out of the 14 had an Elo differential above 365 points.
Something seems to have gone wrong after the versions of 2020.22.10. On that date, 2 trials were above 370, and since then none have been above 365. It could be due to randomness, or perhaps some bug was accidentally introduced.
The error bars will be larger and larger, if the target is >350 always. (also don't forget it is still just a kind of selfplay test)
They should update to at least SF9 or SF10 now at NCM.