Stockfish 14 has been released

[Chessqueen]

Apparently it was tested vs. SF13 in 60.000 games and turned out as +30 elo.

The difference is hardly Noticeable, therefore, take Stockfis14 and rename it Stockfish Beta

[Rebel]

Apparently it was tested vs. SF13 in 60.000 games and turned out as +30 elo.

1. In self-play I assume. And that could be derived from the current result (1600 from the 1800 games) playing SF12, 42.4%, meaning 57.6% for SF14 = +53 elo.

2. I tested one of the earlier 40Mb nets (Stockfish 21-05-18) and it shows a similar pattern, not better.

Code: Select all

   # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)     W     D     L  D(%)  OppAvg
   1 Stockfish 13          :  3677.1   17.1  1555.5    2000    78     100  1157   797    46    40  3434.5
   2 Stockfish 21-05-18    :  3651.2   17.5   795.0    1000    80     100   610   370    20    37  3387.5
   3 Stockfish 12          :  3617.6   28.2  1130.5    1500    75      99   817   627    56    42  3399.7

I will run SF14 vs SF13 afterwards.

[Rebel]

Code: Select all

Gambit Rating List
Running      : Gauntlet Stockfish 14 
Time Control : Time control 40/120
Games        : 1800

Results from file gauntlet-sf14.pgn:

No. Name            Win Draw Loss Unf.  Score Games       %
-----------------------------------------------------------
  1 Stockfish 14   +1133 =639  -28   *0 1452.5  1800   80.7%
  2 Stockfish 12    +12 =141  -47   *0   82.5   200   41.2%
  3 Komodo-Dragon    +9 =123  -68   *0   70.5   200   35.2%
  4 SlowChess 2.6    +5  =78 -117   *0   44.0   200   22.0%
  5 Pedone 3.1       +2  =60 -138   *0   32.0   200   16.0%
  6 RubiChess 2.1    +0  =60 -140   *0   30.0   200   15.0%
  7 Igel 3.0.5       +0  =53 -147   *0   26.5   200   13.2%
  8 Ethereal 12.75   +0  =47 -153   *0   23.5   200   11.8%
  9 Nemorino 6.00    +0  =41 -159   *0   20.5   200   10.2%
 10 Booot 6.5        +0  =36 -164   *0   18.0   200    9.0%

Total Games:    1800
White Wins:      564 (31.3%)
Black Wins:      597 (33.2%)
Draws:           639 (35.5%)
Unfinished:        0 (0.0%)

Estimated elo gain for Stockfish_14
Elo pool : 3404
Stockfish 13 : 3677.0
Stockfish_14 : 3649.3
Difference : -27.7

Deeply unsatisfying.

[bmp1974]

A quick check for 100 game match: SF13 vs SF14
Hardware: i9-9900k @ 3.6ghz, turbo 5ghz
GUI: Cutechess
Thread: 1cpu
Book: book-ply8-unifen-Q-0.0-0.25.pgn
TC: 3min+1sec

Result:
Score of SF13 vs SF14: 1 - 11 - 88 [0.450]
White vs Black: 12 - 0 - 88 [0.560] 100
Elo difference: -34.9 +/- 22.8, LOS: 0.2 %, DrawRatio: 88.0 %
100 of 100 games finished.

[AndrewGrant]

Code: Select all

Gambit Rating List
Running      : Gauntlet Stockfish 14 
Time Control : Time control 40/120
Games        : 1800

Results from file gauntlet-sf14.pgn:

No. Name            Win Draw Loss Unf.  Score Games       %
-----------------------------------------------------------
  1 Stockfish 14   +1133 =639  -28   *0 1452.5  1800   80.7%
  2 Stockfish 12    +12 =141  -47   *0   82.5   200   41.2%
  3 Komodo-Dragon    +9 =123  -68   *0   70.5   200   35.2%
  4 SlowChess 2.6    +5  =78 -117   *0   44.0   200   22.0%
  5 Pedone 3.1       +2  =60 -138   *0   32.0   200   16.0%
  6 RubiChess 2.1    +0  =60 -140   *0   30.0   200   15.0%
  7 Igel 3.0.5       +0  =53 -147   *0   26.5   200   13.2%
  8 Ethereal 12.75   +0  =47 -153   *0   23.5   200   11.8%
  9 Nemorino 6.00    +0  =41 -159   *0   20.5   200   10.2%
 10 Booot 6.5        +0  =36 -164   *0   18.0   200    9.0%

Total Games:    1800
White Wins:      564 (31.3%)
Black Wins:      597 (33.2%)
Draws:           639 (35.5%)
Unfinished:        0 (0.0%)

Estimated elo gain for Stockfish_14
Elo pool : 3404
Stockfish 13 : 3677.0
Stockfish_14 : 3649.3
Difference : -27.7

Deeply unsatisfying.

Wonder if running the most recent non-Leela net would be of use.

[Sylwy]

Right or wrong ?

viewtopic.php?f=6&t=77506

[Rebel]

Code: Select all

Gambit Rating List
Running      : Gauntlet Stockfish 14 
Time Control : Time control 40/120
Games        : 1800

Results from file gauntlet-sf14.pgn:

No. Name            Win Draw Loss Unf.  Score Games       %
-----------------------------------------------------------
  1 Stockfish 14   +1133 =639  -28   *0 1452.5  1800   80.7%
  2 Stockfish 12    +12 =141  -47   *0   82.5   200   41.2%
  3 Komodo-Dragon    +9 =123  -68   *0   70.5   200   35.2%
  4 SlowChess 2.6    +5  =78 -117   *0   44.0   200   22.0%
  5 Pedone 3.1       +2  =60 -138   *0   32.0   200   16.0%
  6 RubiChess 2.1    +0  =60 -140   *0   30.0   200   15.0%
  7 Igel 3.0.5       +0  =53 -147   *0   26.5   200   13.2%
  8 Ethereal 12.75   +0  =47 -153   *0   23.5   200   11.8%
  9 Nemorino 6.00    +0  =41 -159   *0   20.5   200   10.2%
 10 Booot 6.5        +0  =36 -164   *0   18.0   200    9.0%

Total Games:    1800
White Wins:      564 (31.3%)
Black Wins:      597 (33.2%)
Draws:           639 (35.5%)
Unfinished:        0 (0.0%)

Estimated elo gain for Stockfish_14
Elo pool : 3404
Stockfish 13 : 3677.0
Stockfish_14 : 3649.3
Difference : -27.7

Deeply unsatisfying.

Wonder if running the most recent non-Leela net would be of use.

What I am suspecting as a possible theory, if you change playing style (I am pretty sure Leela does have its influence) it can have a good or bad effect when faced with gambits. Benjamin is the best example. Maybe other engines suffer or profit gambit positions.

[Sopel]

Regression test gone wrong?! Not expecting negative results for this version 14

Yes, 60000 games produced an incorrect result. Let's trust the 100 game matches instead.

[Rebel]

Code: Select all

Gambit Rating List
Running      : Match SF14 vs SF13
Time Control : Time control 40/120
Games        : 200

Results from file gauntlet-sf14-sf13.pgn:

No. Name          Win Draw Loss Unf.  Score Games       %
---------------------------------------------------------
  1 Stockfish 14  +28 =149  -23   *0  102.5   200   51.2%
  2 sf13          +23 =149  -28   *0   97.5   200   48.8%

Total Games:     200
White Wins:       21 (10.5%)
Black Wins:       30 (15.0%)
Draws:           149 (74.5%)
Unfinished:        0 (0.0%)

Estimated ratings for this elo 3677 pool

   # PLAYER          :  RATING  POINTS  PLAYED   (%)
   1 Stockfish 14    :  3681.4   102.5     200    51
   2 sf13            :  3672.6    97.5     200    49

A bit of relief, but not good for my theory of gambit openings

[FormazChar]

Code: Select all

Gambit Rating List
Running      : Gauntlet Stockfish 14 
Time Control : Time control 40/120
Games        : 1800

Results from file gauntlet-sf14.pgn:

No. Name            Win Draw Loss Unf.  Score Games       %
-----------------------------------------------------------
  1 Stockfish 14   +1133 =639  -28   *0 1452.5  1800   80.7%
  2 Stockfish 12    +12 =141  -47   *0   82.5   200   41.2%
  3 Komodo-Dragon    +9 =123  -68   *0   70.5   200   35.2%
  4 SlowChess 2.6    +5  =78 -117   *0   44.0   200   22.0%
  5 Pedone 3.1       +2  =60 -138   *0   32.0   200   16.0%
  6 RubiChess 2.1    +0  =60 -140   *0   30.0   200   15.0%
  7 Igel 3.0.5       +0  =53 -147   *0   26.5   200   13.2%
  8 Ethereal 12.75   +0  =47 -153   *0   23.5   200   11.8%
  9 Nemorino 6.00    +0  =41 -159   *0   20.5   200   10.2%
 10 Booot 6.5        +0  =36 -164   *0   18.0   200    9.0%

Total Games:    1800
White Wins:      564 (31.3%)
Black Wins:      597 (33.2%)
Draws:           639 (35.5%)
Unfinished:        0 (0.0%)

Estimated elo gain for Stockfish_14
Elo pool : 3404
Stockfish 13 : 3677.0
Stockfish_14 : 3649.3
Difference : -27.7

Deeply unsatisfying.

Wonder if running the most recent non-Leela net would be of use.

What I am suspecting as a possible theory, if you change playing style (I am pretty sure Leela does have its influence) it can have a good or bad effect when faced with gambits. Benjamin is the best example. Maybe other engines suffer or profit gambit positions.

Maybe the elo calculation is way of because pool is different. The result in every minimatch is better for sf14 compared to the sf13 gauntlet. More games and corrections to the ratings are needed here.

SlowChess 2.6 isn't even a common opponent.