Couple more ideas

[vincenegri]

14. O-O makes my point. Here's what SF "plans" after a real search:

Code: Select all

info depth 43 seldepth 55 multipv 1 score cp 24 nodes 15156824762 nps 21011107 hashfull 999 tbhits 0 time 721372 pv e2c2 c6a7 b5e2 b7a6 f3h2 a6e2 c2e2 c7c5 h2g4 a7c6 g4f6 e7f6 g5f6 c8c7 a1c1 c7d7 c1a1 d7c7 e1g1 c6e5 d4e5 c7e5 e2f3 e5f6 f3f6 h5f6 f1e1 f8e8 a1d1 f6h5 e3h6 h5g7 h6g5 g7h5 b3a1 h8g7 a1c2 f7f6 g5d2 a8d8 g1g2 e6e5 c2a3 g7f7 a3b5 h5g7 b5c7 e8e7

Yes, that's much more like it.. Bd2, Nh2->g4->f6… with enough depth the king safety terms guide SF into better manoeuvring.

It's not an easy problem to solve, though. Test at much longer time controls and you no longer have statistical rigour - the odds of coding a patch to address something specific you have seen and never knowing it had a bad effect overall increase.

If many more volunteers contributed CPU time to fish test (or a similar effort) the story might be different. Imagine if the framework had 20,000 cpus at its disposal. You could run a SPRT at TCEC controls in a day! Or at least you could if not for the fact the draw rate goes up a lot… more sensibly you could use 15 minutes per game instead of 15 seconds. Would require a huge recruitment drive...

[Lyudmil Tsvetkov]

Well here's your problem..

During local testing this position was obtained in one game by the engine with hotspot bonus:

[d]r1q2r1k/1bp1bp1p/1pn1p1p1/pB1pP1Pn/P2P3P/1NP1BN2/1P2QP2/R3K2R w KQ - 0 14

The position is just about perfect and should play itself, no? Put N on f6, open h-file, goodnight Vienna, as they say.

However hotspot-SF did not find this plan and while it did make efforts at K-side play, the game petered into a draw.

Too deep.

But still, chances for SF to find a win here are better than if the definition was murkier.

[Lyudmil Tsvetkov]

[d]rnbqk1nr/pppp1ppp/4p3/8/1b4P1/3P4/PPP1PP1P/RNBQKBNR w KQkq - 0 1

Guys, I just saw what start positions the SF 2-move book contains, thanks to Vince posting this above.

Is this really possible?
Maybe Vince is kidding after all and this is not into the SF book?

If it is, then I do not know if I will go back to watch how SF patches perform on the framework, as it is pretty meaningless.

You really can not hope tuning your eval and search well, starting from such positions, could you?

[zullil]

14. O-O makes my point. Here's what SF "plans" after a real search:

Code: Select all

info depth 43 seldepth 55 multipv 1 score cp 24 nodes 15156824762 nps 21011107 hashfull 999 tbhits 0 time 721372 pv e2c2 c6a7 b5e2 b7a6 f3h2 a6e2 c2e2 c7c5 h2g4 a7c6 g4f6 e7f6 g5f6 c8c7 a1c1 c7d7 c1a1 d7c7 e1g1 c6e5 d4e5 c7e5 e2f3 e5f6 f3f6 h5f6 f1e1 f8e8 a1d1 f6h5 e3h6 h5g7 h6g5 g7h5 b3a1 h8g7 a1c2 f7f6 g5d2 a8d8 g1g2 e6e5 c2a3 g7f7 a3b5 h5g7 b5c7 e8e7

It's not an easy problem to solve, though. Test at much longer time controls and you no longer have statistical rigour - the odds of coding a patch to address something specific you have seen and never knowing it had a bad effect overall increase.

I understand that without a large number of games one cannot decide if a patch is positive, negative or neutral. (And part of me is thinking: "so what, no lives will be lost if a bad change is made. Life is short. Take risks. Have fun!) Of course, anyone is free to fork Stockfish and do whatever he wants, free from the constraints imposed by the fishtest protocols.

I just find myself wondering more and more if the current testing constraints almost preclude improving aspects of Stockfish's play that involve positional play and "long-term planning". Maybe Stockfish has now developed to the point where a new testing protocol is needed, in order for the engine to reach its true potential?

[Lyudmil Tsvetkov]

I wonder how Joerg is going to introduce a kingside fianchettoe bonus, starting from this position, how Joerg?

The fianchettoe is already long gone.

Besides, this position is lost for white, no matter the relatively low scores, those are at just extremely low depth.

Real shame.

[Lyudmil Tsvetkov]

Well, here is a rule, we talked about a bit in the past, but let me devote it a whole subthread now.

It concerns closed positions, and, as far as I know, is one of the rules with almost no exceptions.

The rules goes like that: if all 4 files on one side of the board are closed with pairs of blocked pawns, and one player has an advanced pawn on the 5th rank on either e5,f5,d5 or c5, then this player should get an additional bonus of 50cps or so.

Of course, this bonus is for the fact that you are attacking, while the opponent does not have counterplay on the other side of the board.

[d]6k1/8/p3p3/Pp1pP3/1PpP4/2P5/8/6K1 w - - 0 1
all 4 files on the queen side, a,b,c,d are closed with pairs of blocked pawns, white has an advanced e5 pawn, therefore white gets additional 50cps bonus.

[d]6k1/8/8/p1p1pP2/PpPpP3/1P1P4/8/6K1 w - - 0 1
all 4 files on the queen side are closed, e file is also closed, white has f5 pawn, therefore white gets the additional 50cps bonus.

[d]6k1/8/3p4/p1p1pP2/PpP1P3/1P1P4/8/6K1 w - - 0 1
same here, white gets the bonus

[d]6k1/8/2p1p3/1p1pP3/pP1P4/P1P5/8/6K1 w - - 0 1
same here, white gets the bonus because of the e5 pawn

Same would hold true with all files on the king side closed, and one player having an advanced pawn on d5 or c5 on the queen side.

So, taking into account the above, and assuming the caveat that still some positions will not be fully defined because of being too complicated, the rule should run as follows in its simplest form:

Center closed according to the closed positions flag definition with 4 main closed types in the center with blocked and backward-fated pawns, and all remaining files on one side of the board also closed, one player having e5,f5,d5 or c5 pawn on the other side of the board, 50cps bonus for the player with the e5,f5,d5,c5 pawn.

Of course, if you like an even simpler implementation, skipping some cases, all 4 a,b,c,d files closed with pairs of blocked pawns, one player having e5 or f5 pawn, 50cps bonus for the e5,f5 pawn.
Or, all h,g,f,e files closed with pairs of blocked pawns, one player having d5 or c5 pawn, 50cps bonus for the d5,c5 pawn.

Any comments about this?

I have been winning hundreds of games against SF based on this rule, and so far I have not witnessed even single exception...

[vincenegri]

Maybe Vince is kidding after all and this is not into the SF book?

If it is, then I do not know if I will go back to watch how SF patches perform on the framework, as it is pretty meaningless.

You really can not hope tuning your eval and search well, starting from such positions, could you?

Yes, this is really a position from the 2-move book.

The patches and tuning that result from using the 2-move book are periodically checked in a regression test that uses a normal 8-move book with sensible openings. The 8-move test has, to date, always corroborated the improvement.

Unintuitive? Yes. But the results speak for themselves. SF is arguably the strongest engine in the world, and unarguably one of the top three. Computer chess is not always intuitive!

Marco is someone who listens to results above words. If someone ran two tuning runs on the framework for a parameter, one using 2-move book and one using 8-move book, got two different results, and then the 8-move tuned parameter beat the 2-moved tuned parameter, then you'd have something to talk about.

As you can see, that 'crazy' opening produced a closed-centre position that, if I hadn't posted the whole game, you would have supposed came from a normal opening. Chess has no memory - how you arrived at a position is immaterial to an engine (barring marginal hash table effects)

[Lyudmil Tsvetkov]

Of course, another caveat is that, even if engines have such positions, they might still not know what to do: presumably, they might want to open them as soon as possible...

[vincenegri]

I have been winning hundreds of games against SF based on this rule, and so far I have not witnessed even single exception...

1) At what time control?

2) Are you claiming you would beat SF in a controlled match? You do realise that you would be the toast of the scene and any number of sites would be delighted to host such an event.

[vincenegri]

Of course, another caveat is that, even if engines have such positions, they might still not know what to do: presumably, they might want to open them as soon as possible...

Given that opening the position would cause the bonus to be lost, they might no longer want to open them.