PST-only Evaluation for MinimalChess 0.4

[Uri Blass]

What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine.

Well, afaik that would be the original PeSTO which plays at 3100 ELO!

So the question becomes what is to be considered minimal and what is to be considered incomplete.

I don't think an engine playing above 3000 ELO is incomplete in any meaningful way. But my question wasn't really about the proper definition of the term "minimal" in relation to chess engines. That's a very subjective concept anyway. I could try to give my definition but that's not the point of this thread as far as I'm concerned.

Instead for me the question was what makes PeSTO so strong and how a set of PSTs with similar strength can be created. Everyone who uses it to replace their own tuned values (TSCP, Dumber) or in conjunction with an existing evaluation (ProDeo 3.1) sees a significant ELO gain. Isn't that strange?

I care because currently I've "borrowed" PeSTO's values and before I make the next release I want to replace it with an original set of values that ideally are equally strong. Or stronger!

I doubt if everyone who use Pesto to replace their own tuned values sees a significant elo gain.

I do not know about the values of Dumber but I believe that TSCP has not tuned values for piece square tables otherwise they could make knight and bishop equal more than rook and a pawn.

Note that I do not claim to have tuned piece square table in movei but I strongly believe that my piece square tables is better than TSCP's piece square table.

Here are my piece square table from movei magic number file that people can modify by editing the changes ini file.

The table is from white point of view start with a8,b8,c8,...h8,a7,b7,....
pawn at the 8 rank is basically a queen so it is the reason for the big values for pawns in these squares

Values for the 7th rank with only piece square table of course could be different and movei basically has also bonus for passed pawns(every pawn in the 7th rank is a passed pawn)

If I had to write an engine with only piece square table I think that I could have bigger bonus for advanced pawns.

The king table unlike the other tables has 128 numbers because it has numbers for the opening and number for the endgame.

pawn_square_table 1020 1030 1030 1030 1030 1030 1030 1020 130 130 130 130 130 130 130 130 115 115 115 115 115 115 115 115 100 100 100 105 105 100 100 100 90 92 95 100 100 95 92 90 90 91 92 85 85 92 91 90 91 90 90 70 70 90 90 91 0 0 0 0 0 0 0 0knight_square_table 210 300 340 340 340 340 300 210 300 340 350 350 350 350 340 300 330 350 355 355 355 355 350 330 330 350 355 360 360 355 350 330 330 350 355 360 360 355 350 330 340 350 355 355 355 355 350 340 320 340 350 350 350 350 340 320 220 320 340 340 340 340 320 220 bishop_square_table 350 350 350 350 350 350 350 350 300 360 360 360 360 360 360 300 360 360 365 365 365 365 360 360 350 360 365 370 370 365 360 350 350 360 365 370 370 365 360 350 350 360 360 360 360 360 360 350 350 360 360 360 360 360 360 350 340 340 340 350 350 340 340 340 rook_square_table 540 540 540 540 540 540 540 540 540 540 540 540 540 540 540 540 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 530 540 540 540 540 530 530 queen_square_table 1020 1030 1030 1030 1030 1030 1030 1020 1010 1020 1020 1020 1020 1020 1020 1010 1030 1020 1020 1020 1020 1020 1020 1030 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1020 1010 1020 1020 1020 1020 1020 1020 1010 1010 1020 1020 1020 1020 1020 1020 1010 king_table -60 -60 -60 -60 -60 -60 -60 -60 -50 -50 -50 -50 -50 -50 -50 -50 -45 -45 -45 -45 -45 -45 -45 -45 -40 -40 -40 -40 -40 -40 -40 -40 -35 -35 -35 -35 -35 -35 -35 -35 -30 -30 -30 -30 -30 -30 -30 -30 -10 -10 -20 -20 -20 -20 -10 -10 30 20 0 0 0 0 20 30 -24 -12 0 4 4 0 -12 -24 -12 8 20 24 24 20 8 -12 0 20 32 36 36 32 20 0 4 24 36 40 40 36 24 4 4 24 36 40 40 36 24 4 0 20 32 36 36 32 20 0 -12 8 20 24 24 20 8 -12 -24 -12 0 4 4 0 -12 -24

[mvanthoor]

Channeling my inner Spock - It is illogical that a chess engine with only static PSTs can achieve a 3100 ccrl elo when many of the positions that are evaluated would simply be wrong. Channeling my inner Kirk - Not so Mr. Spock, a tuned PST can get a good start for a chess engine and then a superior search can negate any deficiencies of the PSTs.
https://www.youtube.com/watch?v=5vrXKlO2Jbw

If you could analyse the game to the final end, you wouldn't even _need_ an evaluation: it would be enough to count material, and know about check and checkmate. As this is obviously not (yet) possible, the engine needs _some_ knowledge about positions. The required knowledge becomes less and less, as opening books and endgame databases get larger.

In checkers, this is already the case; opening books and endgame databases are so large, and engines can see so far ahead, that the best checkers engines can see straight into the endgame database from the opening book. The only thing they have to do on their own is survive some moves in the middle game, in such a way that they reach an endgame they don't lose. Therefore checkers is considered to be weakly solved. At some point, chess will be at that point as well. I've already had openings that simplify so much, that Stockfish starts hitting the endgame database when thinking about move 17...

[Ronald]

I guess Piece Square Tables are the minimum for creating a seriously strong chess engine. Piece values alone are not enough, because the engine will not have any clue where to put it's pieces.

It's nice to see that other people are using the PeSTO tables and that some are even gain an advantage with it

The PST themselves are just the result of some "plain" Texel tuning. If I remember correctly I used the "quiet-labeled.epd" created by the Zurichess developper for the tuning proces. With plain Texel Tuning I mean: For every PST value, increment it, calculate the error, if it's lower then next parameter, else decrement it and check the error etc. I believe I started with a higher increment first(for instance add 9), and if incr/decr didn't lower the error I tried a smaller increment etc.

The Texel tuning proces will normally find a local minimum of the error, so there are multiple tuning results possible depending on initial values/sequence of PST values. These different results will normally also result in different "strength". So for getting "optimal" results, you will probably have to play with initial values of the PST's and or sequence of elements.

A final remark on using the PeSTO PST's: The PST' values are relative and are related to the basic value of a pawn (around 90 centipawns). Many of the pruning logic in chess engines (futility pruning etc.) are also related to the value of a pawn, but of the current value of the pawn in the engine. If the PeSTO PST's are used it might be useful to tweak the pruning values to get better results.

[Mike Sherwin]

I guess Piece Square Tables are the minimum for creating a seriously strong chess engine. Piece values alone are not enough, because the engine will not have any clue where to put it's pieces.

It's nice to see that other people are using the PeSTO tables and that some are even gain an advantage with it

The PST themselves are just the result of some "plain" Texel tuning. If I remember correctly I used the "quiet-labeled.epd" created by the Zurichess developper for the tuning proces. With plain Texel Tuning I mean: For every PST value, increment it, calculate the error, if it's lower then next parameter, else decrement it and check the error etc. I believe I started with a higher increment first(for instance add 9), and if incr/decr didn't lower the error I tried a smaller increment etc.

The Texel tuning proces will normally find a local minimum of the error, so there are multiple tuning results possible depending on initial values/sequence of PST values. These different results will normally also result in different "strength". So for getting "optimal" results, you will probably have to play with initial values of the PST's and or sequence of elements.

A final remark on using the PeSTO PST's: The PST' values are relative and are related to the basic value of a pawn (around 90 centipawns). Many of the pruning logic in chess engines (futility pruning etc.) are also related to the value of a pawn, but of the current value of the pawn in the engine. If the PeSTO PST's are used it might be useful to tweak the pruning values to get better results.

Ronald,
Thanks for your work and for sharing it. It is quite amazing! I'm still not a believer in theory that static PSTs are superior to dynamic algorithmic created PST's, before every search. It is my current goal for my new engine to do tuning of dynamically created PSTs during search by using a portion of the allotted time to play a few thousand very fast games and using the results to modify the PSTs before the main search.

[Mike Sherwin]

Bricabrac currently has the PSTs from PeSTO. I once said that PESTO is tuned for the 90%. I wanted to do a short and simple test to demonstrate the points that I was trying to make. So I manually played a rare first move, f2f4. Then I started a self play session. The reply as black was d7d5. The next move as white was d2d4 showing a total lack of positional understanding for the opening played.

r n b q k b n r
p p p . p p p p
. . . . . . . .
. . . p . . . .
. . . . . P . .
. . . . . . . .
P P P P P . P P
R N B Q K B N R

Enter Command(w): go

Begin Depth 1
1 -17 0 20 g1f3

Begin Depth 2
2 -53 0 171 g1f3 g8f6

Begin Depth 3
3 -18 0 1418 g1f3 g8f6 d2d4

Begin Depth 4
4 -51 0 9410 g1f3 g8f6 d2d4 b8c6

Begin Depth 5
5 -18 0 36003 g1f3 g8f6 d2d4 b8c6 b1c3

Begin Depth 6
6 -46 1 70497 g1f3 g8f6 d2d4 b8c6 b1c3 c8e6

Begin Depth 7
7 -18 2 171471 g1f3 g8f6 d2d4 b8c6 b1c3 c8e6 c1e3

Begin Depth 8
8 -38 5 556234 d2d4 g8f6 g1f3 b8c6 b1c3 c8e6 c1e3 a8c8

Begin Depth 9
9 -23 18 2072024 g2g3 g8f6 f1g2 e7e6 g1f3 f8d6 e1g1 e8g8 d2d4

Begin Depth 10
10 -46 49 6022284 g1f3 g8f6 d2d4 g7g6 e2e3 f8g7 f1d3 e8g8 e1g1 b8c6

Begin Depth 11
11 -25 89 10852886 g1f3 b8c6 d2d4 c8f5 e2e3 c6b4 f1b5 c7c6 b5a4 g8f6 e1g1

Begin Depth 12
12 -41 224 27693884 g1f3 g8f6 d2d4 c8f5 b1c3 e7e6 f3h4 f5g4 c1e3 b8c6 h2h3 g4h5

Begin Depth 13
13 -25 563 70248992 d2d4 g8f6 g1f3 b8c6 b1c3 c8f5 e2e3 e7e6 f1d3 f5d3 d1d3 c6b4 d3d1

Begin Depth 14
14 -44 1589 203375627 d2d4 g8f6 g1f3 c8f5 e2e3 e7e6 f1e2 b8c6 e1g1 f8d6 b1c3 e8g8 c3b5 f5e4

Begin Depth 15

Move Played: D2D4 Score: -44
r n b q k b n r
p p p . p p p p
. . . . . . . .
. . . p . . . .
. . . P . P . .
. . . . . . . .
P P P . P . P P
R N B Q K B N R

Enter Command(b):

[abulmo2]

I doubt if everyone who use Pesto to replace their own tuned values sees a significant elo gain.

I do not know about the values of Dumber but I believe that TSCP has not tuned values for piece square tables otherwise they could make knight and bishop equal more than rook and a pawn.

Dumb/Dumber eval weights are tuned, but kept symmetrical (except for the king's one). Pesto's weights are fully assymetrical and better than Dumber's one. The Elo gain was 67 Elo (+/- 25).

Note that I do not claim to have tuned piece square table in movei but I strongly believe that my piece square tables is better than TSCP's piece square table.

Just by looking at your weights, I dare to claim they are much weaker than Pesto's one, I guess by by 100-200 Elo.

[Uri Blass]

I doubt if everyone who use Pesto to replace their own tuned values sees a significant elo gain.

I do not know about the values of Dumber but I believe that TSCP has not tuned values for piece square tables otherwise they could make knight and bishop equal more than rook and a pawn.

Dumb/Dumber eval weights are tuned, but kept symmetrical (except for the king's one). Pesto's weights are fully assymetrical and better than Dumber's one. The Elo gain was 67 Elo (+/- 25).

Note that I do not claim to have tuned piece square table in movei but I strongly believe that my piece square tables is better than TSCP's piece square table.

Just by looking at your weights, I dare to claim they are much weaker than Pesto's one, I guess by by 100-200 Elo.

Some comments:
1)one of the disadvantages of my piece square tables is the fact that except for the king I do not have special endgame and special opening tables.
2)I wonder how much elo you lose if you force the piece square table of Pesto to be symmetrical(except for the king's one).

It is possible to modify every assymetrical piece square table to symmetrical piece square table simply by making average(for example if white pawn at h7 is evaluated as x and white pawn at a7 is evaluated as y you evaluate both of them as (x+y)/2

3)I saw that a white pawn at h7 in the middle game is bad unlike a white pawn at a7 and I think that I understand the reason for it.
usually black castle short so white pawn at h7 is often blocked by the black king and defend the black king from an attack unlike white pawn at a7.

4)I wonder how much piece square mobility table can be stronger than only piece square table
The idea is that for every piece you also count the number of moves the piece can make and not only use the square.

It means for example that for a rook there is going to be 15 weights for every square(because the rook can move between 0 and 14 moves and you need to count the possible moves of it inside the evaluation).

[Ronald]

Thanks for your work and for sharing it. It is quite amazing! I'm still not a believer in theory that static PSTs are superior to dynamic algorithmic created PST's, before every search. It is my current goal for my new engine to do tuning of dynamically created PSTs during search by using a portion of the allotted time to play a few thousand very fast games and using the results to modify the PSTs before the main search.

Interesting idea. How are you going to change the PST values based on the results of the played games?

[Rebel]

Thanks for your work and for sharing it. It is quite amazing! I'm still not a believer in theory that static PSTs are superior to dynamic algorithmic created PST's, before every search. It is my current goal for my new engine to do tuning of dynamically created PSTs during search by using a portion of the allotted time to play a few thousand very fast games and using the results to modify the PSTs before the main search.

Interesting idea. How are you going to change the PST values based on the results of the played games?

A long time ago this idea gave me 10-15 elo - Before the search starts ProDeo modifies its Piece-Square-Tables with the information from the HISTORY table from the previous search.

http://rebel13.nl/rebel13/ideas.html#FIVE

[Mike Sherwin]

Thanks for your work and for sharing it. It is quite amazing! I'm still not a believer in theory that static PSTs are superior to dynamic algorithmic created PST's, before every search. It is my current goal for my new engine to do tuning of dynamically created PSTs during search by using a portion of the allotted time to play a few thousand very fast games and using the results to modify the PSTs before the main search.

Interesting idea. How are you going to change the PST values based on the results of the played games?

The feedback to the evaluation function that Rebel mentions was pioneered in RomiChess back in ~2007. And it is worth a little bit of elo. What the plan is now is to do Reinforcement Learning in real time. In RomiChess the RL was after game learning. The idea is to save all the moves of the fast games into the hashtable with their scores. Then with each fast game to modify an existing score. But in addition to that the plan is to use the scores in the hashtable to modify the piece square tables.