PST-only Evaluation for MinimalChess 0.4

[lithander]

Right after releasing version 0.3 of MinimalChess, which used non-tuned, non-tapered PSTs for it's evaluation I tried the approach of PeSTO, faithfully implementing the PST values shared by Ronald Friederich in this forum. My engine gained ~200 ELO from changing the evaluation in that way! The new eval wasn't much more complicated than the old one (the concept is the same, you just interpolate between two sets of PSTs depending on the game's phase) and thus it fits my project's vision ("minimal") pretty well.

I want to release the next version of MinimalChess soon and the only thing left to do is to replace the evaluation I borrowed from PeSTO with something original. I'm in the process of writing a tuner and I hope to create values from scratch that can rival or even surpass the PeSTO values.

I've seen a lot of threads where people added PeSTO to their engines (TSCP, Dumber, Rebel) making them significantly stronger. But has anybody managed to reproduce it's surprisingly good performance with their own PST-only eval? Or surpassed it? If so - how? If not - what do you think makes PeSTO special?

[Mike Sherwin]

Right after releasing version 0.3 of MinimalChess, which used non-tuned, non-tapered PSTs for it's evaluation I tried the approach of PeSTO, faithfully implementing the PST values shared by Ronald Friederich in this forum. My engine gained ~200 ELO from changing the evaluation in that way! The new eval wasn't much more complicated than the old one (the concept is the same, you just interpolate between two sets of PSTs depending on the game's phase) and thus it fits my project's vision ("minimal") pretty well.

I want to release the next version of MinimalChess soon and the only thing left to do is to replace the evaluation I borrowed from PeSTO with something original. I'm in the process of writing a tuner and I hope to create values from scratch that can rival or even surpass the PeSTO values.

I've seen a lot of threads where people added PeSTO to their engines (TSCP, Dumber, Rebel) making them significantly stronger. But has anybody managed to reproduce it's surprisingly good performance with their own PST-only eval? Or surpassed it? If so - how? If not - what do you think makes PeSTO special?

I think PeSTO is "special" for a weak engine. I do not know if PeSTO is special for a stronger engine. What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine. It has PST's, a minimal pawn evaluator and that is about it when it comes to evaluation. The only difference between Romi's PST and PeSTO is that Romi's PST is algorithmically created at the root and after the first couple of ply to account for the changes to the position.

So the question becomes what is to be considered minimal and what is to be considered incomplete. If all an engine has is static PST's tapered or not I would not call it minimal. I would call it incomplete. How can a chess engine be considered minimal that does not attempt to use any chess knowledge at all specific to the position on the board?

[lithander]

What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine.

Well, afaik that would be the original PeSTO which plays at 3100 ELO!

So the question becomes what is to be considered minimal and what is to be considered incomplete.

I don't think an engine playing above 3000 ELO is incomplete in any meaningful way. But my question wasn't really about the proper definition of the term "minimal" in relation to chess engines. That's a very subjective concept anyway. I could try to give my definition but that's not the point of this thread as far as I'm concerned.

Instead for me the question was what makes PeSTO so strong and how a set of PSTs with similar strength can be created. Everyone who uses it to replace their own tuned values (TSCP, Dumber) or in conjunction with an existing evaluation (ProDeo 3.1) sees a significant ELO gain. Isn't that strange?

I care because currently I've "borrowed" PeSTO's values and before I make the next release I want to replace it with an original set of values that ideally are equally strong. Or stronger!

[Mike Sherwin]

What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine.

Well, afaik that would be the original PeSTO which plays at 3100 ELO!

So the question becomes what is to be considered minimal and what is to be considered incomplete.

I don't think an engine playing above 3000 ELO is incomplete in any meaningful way. But my question wasn't really about the proper definition of the term "minimal" in relation to chess engines. That's a very subjective concept anyway. I could try to give my definition but that's not the point of this thread as far as I'm concerned.

Instead for me the question was what makes PeSTO so strong and how a set of PSTs with similar strength can be created. Everyone who uses it to replace their own tuned values (TSCP, Dumber) or in conjunction with an existing evaluation (ProDeo 3.1) sees a significant ELO gain. Isn't that strange?

I care because currently I've "borrowed" PeSTO's values and before I make the next release I want to replace it with an original set of values that ideally are equally strong. Or stronger!

Then the next question becomes, how are the PeSTO PSTs modified with additional evaluation. I got from your post that you planned on stopping at tuned PSTs and not have any further evaluation. So let me define my original question more precisely. What is the highest ccrl rating of a chess engine that only uses the PST of PeSTO and does not have any other evaluation except maybe an endleaf pawn structure evaluator? Let's keep this in the context of what you are trying to do!

[hgm]

Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.

[mvanthoor]

I think that it depends on the engine the Pesto values are tuned against. Often, an engine such as Stockfish is run for X thousand of positions, for 2 seconds per position, to obtain an evaluation. Then the tuner tries to change the PST and material value in such a way to get as close as possible to the Stockfish evaluation.

In short: tuned PST's contain a lot of positional knowledge provided by a different, much stronger engine.

(If I'm mistaken and tuners work differently nowadays, I'd be happy to be re-educated; this is a subject I haven't looked into yet. Regrettably I haven't had much time for chess programming the past few weeks.)

[Mike Sherwin]

Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.

Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo.

[mvanthoor]

Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.

Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo.

Why not? I it has all the sorting and pruning tricks of other high-end engines, I'm sure it will be possible to tune a lot of positional awareness into an evaluation and material values that are tuned and tapered.

[mar]

Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.

Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo.

well, 3100 elo but 4CPU - I get (most do in fact) +100 by 4CPU as well (but today it's sort of a privilege of getting your SMP engine tested unlike 5 years ago )
single core, that's sligtly below 3k in CCRL 40/15.

so most of the strength comes from Rofchade's search.
I tried to replace my texel-tuned HCE with PeSTO and immediately lost ~200 elo, which means (considering my engine is slightly below PeSTO in CCRL) that PeSTO's search is at least 200 elo better than mine.

[Mike Sherwin]

Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.

Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo.

Why not? I it has all the sorting and pruning tricks of other high-end engines, I'm sure it will be possible to tune a lot of positional awareness into an evaluation and material values that are tuned and tapered.

Channeling my inner Spock - It is illogical that a chess engine with only static PSTs can achieve a 3100 ccrl elo when many of the positions that are evaluated would simply be wrong. Channeling my inner Kirk - Not so Mr. Spock, a tuned PST can get a good start for a chess engine and then a superior search can negate any deficiencies of the PSTs.
https://www.youtube.com/watch?v=5vrXKlO2Jbw