13 hard test positions July 2016

[MikeGL]

Searching: r4rk1/pb3pp1/1p1qpR2/2ppN3/3P4/2P4P/PP1pQ1P1/3R2K1 w - - 0 1
infinite: 0 ponder: 0 time: 0 increment: 0 moves to go: 0

...
...

Nodes: 3118389811
Nodes/second: 663486
Best move: Ng4
Ponder move: Rf8xf6=R

Averaging at 3 billion nodes. What engine is this and what's your hardware?

[Dann Corbit]

Searching: r4rk1/pb3pp1/1p1qpR2/2ppN3/3P4/2P4P/PP1pQ1P1/3R2K1 w - - 0 1
infinite: 0 ponder: 0 time: 0 increment: 0 moves to go: 0

...
...

Nodes: 3118389811
Nodes/second: 663486
Best move: Ng4
Ponder move: Rf8xf6=R

Averaging at 3 billion nodes. What engine is this and what's your hardware?

The node counts are doubtful, since I have not bothered to maintain that (I add the logging back into SF, but I only maintain the parts that I need and I do not care about the node counts).

I don't know for sure which machine this was.

It had somewhere between 4 and 12 cores, probably running over 3 GHz. The 12 core machines have 32 GB Ram and 256 GB SSD along with a few TB of disk.

[peter]

[The node counts are doubtful, since I have not bothered to maintain that (I add the logging back into SF, but I only maintain the parts that I need and I do not care about the node counts).

Hi Dann!
So this is a MateFinder- version of you own again?
Still with scalable nullmove-verification?
Do you offer it for free download still?

[Dann Corbit]

[The node counts are doubtful, since I have not bothered to maintain that (I add the logging back into SF, but I only maintain the parts that I need and I do not care about the node counts).

Hi Dann!
So this is a MateFinder- version of you own again?
Still with scalable nullmove-verification?
Do you offer it for free download still?

Current SF with my tweaks:
https://www.dropbox.com/s/cp4a8iokmef65 ... r3.7z?dl=0

Matefinder (not the latest):
https://www.dropbox.com/s/795nepxvva1ie ... r2.7z?dl=0

There is a newer version of Matefinder that I have built with my changes, but I did not post it yet. I will post a link tomorrow.

[peter]

Thanks a lot

[Dann Corbit]

The latest matefinder build that I have:
https://www.dropbox.com/s/u1pwpo8papegk ... er.7z?dl=0

[peter]

[Eelco de Groot]

Version _035 had a bug unfortunately, so it will give false positive wins. Thanks to Mindbreaker's post on Fishcooking, unrelated to this but with a difficult fortress, I found out quickly otherwise it might have been ages. Elo did not seem to suffer. There was only a one line change and 034 was still okay so the location of the bug was also not so hard. Exactly why the codechange was wrong is not so easy. But it involved setting ss->skipEarlyPruning from within the node, mainly to skip nullmove pruning, that is still okay but it was never set to false again, that is asking for trouble. Well, I managed to find another way to skip nullmove that hopefully is a bit more correct and that still can find a win score here! Unfortunately the PVs are still not really to write home about, as far as I can see, even when the score goes up. Fir instance the move Bh4 is pushed farther away each time it seems. But just finding a win seems a big deal here (compared to Stockfish). I hear Sting has received an upgrade for solving this position as well, I have not tried it myself.

1B1k4/3p4/1Kp5/PpP2p2/1PqP1P2/8/1P6/8 w - -

Engine: Rainbow Serpent 20160626_039 MP (i7 6700, 4 thread, 512 MB)
The main switch compared to 034 now reads like this, it's still a one line change but needs another Boolean switch allowNull:

Code: Select all

// Can ttValue be used as a better position evaluation?
if (ttValue != VALUE_NONE)
    if (tte->bound() & (ttValue > eval ? BOUND_LOWER : BOUND_UPPER))
    {
	    eval = ttValue;
	    if (ttValue < ss->staticEval)
		     allowNull = false;                
    }

Line 692-699 in search.cpp This probably loses Elo, have not measured how much yet. Advantage is we now still can do IID. Futility pruning Parent node is skipped by the same Boolean.

by T. Romstad, M. Costalba, J. Kiiski, G. Linscott

29/17 0:01 0.00 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa1
5.Bb8 Qa2 (18.612.499) 11019

30/17 0:01 0.00 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa1
5.Bb8 Qa2 (18.786.684) 11012
.
.
.
48/17 4:43 0.00 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa1
5.Bb8 Qa2 (3.233.538.095) 11405

49/17 6:07 0.00 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa1
5.Bb8 Qa2 (4.158.762.888) 11311

50/17 6:44 +0.07++ 1.a6 (4.594.183.874) 11345

50/53 8:51 +0.15++ 1.a6 (6.042.951.346) 11366

50/53 12:18 +0.27++ 1.a6 (8.370.999.821) 11341

50/53 14:01 +0.43++ 1.a6 (9.535.685.805) 11332

50/53 16:08 +0.65++ 1.a6 (10.991.892.474) 11344

50/61 18:47 +0.94++ 1.a6 (12.809.329.678) 11356

50/61 20:50 +1.33++ 1.a6 (14.193.974.631) 11350

50/62 24:26 +1.84++ 1.a6 (16.637.199.402) 11343

50/62 28:44 +2.48++ 1.a6 (19.535.814.706) 11326

50/62 35:58 +3.31++ 1.a6 (24.433.096.089) 11320

50/62 43:19 +4.36++ 1.a6 (29.421.162.454) 11316

50/62 49:59 +5.69++ 1.a6 (33.898.097.276) 11301

50/62 59:59 +7.37++ 1.a6 (40.529.433.182) 11259

50/70 61:24 +5.94-- 1.a6 Kc8 (41.464.444.543) 11253

50/70 68:06 +7.73++ 1.a6 (45.873.182.698) 11224

50/82 87:40 +11.08++ 1.a6 (58.205.167.103) 11064

50/88 102:45 +8.26-- 1.a6 Kc8 (68.373.004.391) 11089

50/88 118:54 +11.78++ 1.a6 (79.074.096.437) 11084

50/94 145:04 +12.36 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa2
5.b3 Qa3 6.Bf6 Qa2 7.Bh4 Qa3 8.Bg3 Qa2
9.Bf2 Qa1 10.Be1 Qa2 11.Bd2 Kd8
12.Kb7 Qxd2 13.a8Q+ Ke7 14.d5 (96.419.065.894) 11077
.
.
.
51/94 284:43 +14.74 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa1
5.b3 Qa3 6.Bf6 Qa2 7.Bh4 Qa3 8.Bg5 Qa1
9.Bh6 Qa3 10.Bg7 Qa2 11.Bf6 Qa3
12.Bh4 Qa1 13.Be1 Qa2 14.Bd2 (189.188.306.220) 11074
.
.
.
52/94 297:37 +15.05 1.a6 Kc8 2.a7 Qa2 3.Bd6 Qa4 4.b3 Qa2
5.Be7 Qa3 6.Bf6 Qa2 7.Be5 Qa3 8.Bg7 Qa2
9.Bf6 Qa3 10.Bh4 Qa2 11.Bf2 Qa1
12.Be1 Qa2 13.Bd2 Qa1 14.Bc1 (197.731.670.117) 11072
.
.
.
53/94 323:58 +9.48 1.a6 Kc8 2.a7 Qa2 3.Be5 Qa1 4.Bd6 Qa4
5.Be7 Qa2 6.Bf8 Qa4 7.b3 Qa2 8.Bd6 Qa3
9.Be7 Qa2 10.Bg5 Qa3 11.Bf6 Qa2
12.Be7 Qa3 13.Bg5 Qa2 14.Bh4 (215.376.126.225) 11080
.
.
.
54/94 755:52 +16.24 1.a6 Kc8 2.a7 Qa2 3.Bd6 Qa1 4.b3 Qa3
5.Be7 Qa1 6.Bf6 Qa2 7.Be5 Qa3 8.Bd6 Qa2
9.Bf8 Qa1 10.Bg7 Qa2 11.Bh6 Qa3
12.Bg5 Qa2 13.Bh4 Qa3 14.Be1 (505.523.097.640) 11146
.
.
.
55/94 878:52 +18.03 1.a6 Kc8 2.a7 Qa2 3.Bd6 Qa1 4.Bc7 Qa2
5.Be5 Qa4 6.Bf6 Qa2 7.Bh4 Qa4 8.Bg5 Qa1
9.b3 Qa3 10.Bh4 Qa2 11.Bf6 Qa3
12.Be7 Qa2 13.Bh4 Qa3 14.Bg3 (588.675.740.328) 11163

56/94 880:07 +17.17-- 1.a6 Kc8 (589.527.271.774) 11163

56/94 895:55 +17.17-- 1.a6 Kc8 (600.112.986.851) 11163

best move: a5-a6 time: 915:05.062 min n/s: 11.163.792 nodes: 612.562.706.728

Still lots of search instability The last aspiration window staying at 17.17 was strange, never seen that before.

[peter]

by T. Romstad, M. Costalba, J. Kiiski, G. Linscott


50/17 6:44 +0.07++ 1.a6 (4.594.183.874) 11345

50/53 8:51 +0.15++ 1.a6 (6.042.951.346) 11366

50/53 12:18 +0.27++ 1.a6 (8.370.999.821) 11341

50/53 14:01 +0.43++ 1.a6 (9.535.685.805) 11332

50/53 16:08 +0.65++ 1.a6 (10.991.892.474) 11344

50/61 18:47 +0.94++ 1.a6 (12.809.329.678) 11356

50/61 20:50 +1.33++ 1.a6 (14.193.974.631) 11350

50/62 24:26 +1.84++ 1.a6 (16.637.199.402) 11343

50/62 28:44 +2.48++ 1.a6 (19.535.814.706) 11326

50/62 35:58 +3.31++ 1.a6 (24.433.096.089) 11320

50/62 43:19 +4.36++ 1.a6 (29.421.162.454) 11316

50/62 49:59 +5.69++ 1.a6 (33.898.097.276) 11301

50/62 59:59 +7.37++ 1.a6 (40.529.433.182) 11259

50/70 61:24 +5.94-- 1.a6 Kc8 (41.464.444.543) 11253

50/70 68:06 +7.73++ 1.a6 (45.873.182.698) 11224

50/82 87:40 +11.08++ 1.a6 (58.205.167.103) 11064

50/88 102:45 +8.26-- 1.a6 Kc8 (68.373.004.391) 11089

50/88 118:54 +11.78++ 1.a6 (79.074.096.437) 11084

50/94 145:04 +12.36 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa2
5.b3 Qa3 6.Bf6 Qa2 7.Bh4 Qa3 8.Bg3 Qa2
9.Bf2 Qa1 10.Be1 Qa2 11.Bd2 Kd8
12.Kb7 Qxd2 13.a8Q+ Ke7 14.d5 (96.419.065.894) 11077
.
.
.
51/94 284:43 +14.74 1.a6 Kc8 2.a7 Qa2 3.Bc7 Qa4 4.Be5 Qa1
5.b3 Qa3 6.Bf6 Qa2 7.Bh4 Qa3 8.Bg5 Qa1
9.Bh6 Qa3 10.Bg7 Qa2 11.Bf6 Qa3
12.Bh4 Qa1 13.Be1 Qa2 14.Bd2 (189.188.306.220) 11074
.
.
.
52/94 297:37 +15.05 1.a6 Kc8 2.a7 Qa2 3.Bd6 Qa4 4.b3 Qa2
5.Be7 Qa3 6.Bf6 Qa2 7.Be5 Qa3 8.Bg7 Qa2
9.Bf6 Qa3 10.Bh4 Qa2 11.Bf2 Qa1
12.Be1 Qa2 13.Bd2 Qa1 14.Bc1 (197.731.670.117) 11072
.
.
.
53/94 323:58 +9.48 1.a6 Kc8 2.a7 Qa2 3.Be5 Qa1 4.Bd6 Qa4
5.Be7 Qa2 6.Bf8 Qa4 7.b3 Qa2 8.Bd6 Qa3
9.Be7 Qa2 10.Bg5 Qa3 11.Bf6 Qa2
12.Be7 Qa3 13.Bg5 Qa2 14.Bh4 (215.376.126.225) 11080
.
.
.
54/94 755:52 +16.24 1.a6 Kc8 2.a7 Qa2 3.Bd6 Qa1 4.b3 Qa3
5.Be7 Qa1 6.Bf6 Qa2 7.Be5 Qa3 8.Bd6 Qa2
9.Bf8 Qa1 10.Bg7 Qa2 11.Bh6 Qa3
12.Bg5 Qa2 13.Bh4 Qa3 14.Be1 (505.523.097.640) 11146
.
.
.
55/94 878:52 +18.03 1.a6 Kc8 2.a7 Qa2 3.Bd6 Qa1 4.Bc7 Qa2
5.Be5 Qa4 6.Bf6 Qa2 7.Bh4 Qa4 8.Bg5 Qa1
9.b3 Qa3 10.Bh4 Qa2 11.Bf6 Qa3
12.Be7 Qa2 13.Bh4 Qa3 14.Bg3 (588.675.740.328) 11163

56/94 880:07 +17.17-- 1.a6 Kc8 (589.527.271.774) 11163

56/94 895:55 +17.17-- 1.a6 Kc8 (600.112.986.851) 11163

best move: a5-a6 time: 915:05.062 min n/s: 11.163.792 nodes: 612.562.706.728

Very well, Eelco, didn't see another engine solve this Blathy- study.
Maybe I stopped some trials too soon, but dev. SF gets depth 50 and much higher than that very quickly without any sign of "seeing" the triangle- maneuvers of the Bishop, MateFinder needs much more time than your version to reach depth 50, in which Rainbow Serpent starts finding kind of Bishop- moves into right direction, even if still seemingly undecided how to get straight forward.
Thought till now, there had to be some "lazy Bishop"- algo for that position especially, yet your engine shows it works without that too, doesn't it?

[Eelco de Groot]

Thanks for the compliment Peter! It is very rewarding if Rainbow Serpent does something that Stockfish absolutely can't do at the moment I have to be careful though because there is no way to guess how much the Elo price is going to be. There is a very small chance that it is very small, but it could cost easily 40 Elo. The previous version seemed to lose this much:

Stockfish 260616 64 - BMI2 Rainbow Serpent 20160626_036 (+4 =6 -0) 7 - 3 70% ca. +147 Elo according to Shredder (01)

This is only ten games of course but it seems bad. 039 I hope is losing slightly less bad to Stockfish, but once again I only have ten games so far for 039

Stockfish 260616 64 - BMI2 Rainbow Serpent 20160626_039 (+1 =7 -2) 4½ - 5½ 45% ca. -34 Elo according to Shredder (01)

If Serpent scores about 40% that is normal, not good because Stockfish is actually a slow compile from Abrok.eu, TDM-GCC can do better on my machine. Well 40% was for the best versions of Serpent that were actually 32 bit compiles, so TDM-GCC should do better than 40%. But I could have a large error margin in that 40%...

I am 90% sure the code for replacing a static eval with a search result that goes below the static eval is buggy as it is in Stockfish. Replacing static eval with a lower limit above the static eval is fine, but an upper limit that you get back from search and if that still is above alpha can be used for going into nullmove, Futility pruning etc, can actually be as low as the minimum limit for eval. Because it is an upper limit. I would call that a bug

It's been there for a long time though so I could be wrong here

If Komodo would follow Stockfish here, both can easily adopt my one line code in time for the TCEC final This assuming Stockfish will get into the final of course. Komodo can do so without cost if it has the bug and decides only to change Analysis mode. Stockfish does not have an analysis mode anymore.

It is possible however that the 'buggy code' from Stockfish is really a feature, that it brings Elo and maybe much. I don't know at this point. If my code is actually a discovery it is possible it is a rediscovery of a trick that was introduced in the private (closed code, but otherwise now free versions of Fruit) Fruits from Ryan Benitez. That would be nice too, but if not that one might still be out there...