PST-only Evaluation for MinimalChess 0.4

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
lithander
Posts: 353
Joined: Sun Dec 27, 2020 1:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

PST-only Evaluation for MinimalChess 0.4

Post by lithander » Thu Apr 15, 2021 10:50 am

Right after releasing version 0.3 of MinimalChess, which used non-tuned, non-tapered PSTs for it's evaluation I tried the approach of PeSTO, faithfully implementing the PST values shared by Ronald Friederich in this forum. My engine gained ~200 ELO from changing the evaluation in that way! The new eval wasn't much more complicated than the old one (the concept is the same, you just interpolate between two sets of PSTs depending on the game's phase) and thus it fits my project's vision ("minimal") pretty well.

I want to release the next version of MinimalChess soon and the only thing left to do is to replace the evaluation I borrowed from PeSTO with something original. I'm in the process of writing a tuner and I hope to create values from scratch that can rival or even surpass the PeSTO values.

I've seen a lot of threads where people added PeSTO to their engines (TSCP, Dumber, Rebel) making them significantly stronger. But has anybody managed to reproduce it's surprisingly good performance with their own PST-only eval? Or surpassed it? If so - how? If not - what do you think makes PeSTO special?
Minimal Chess. A simple engine written in C#! Details on Youtube & Github

Mike Sherwin
Posts: 369
Joined: Thu Aug 20, 2020 11:25 pm
Full name: Michael J Sherwin

Re: PST-only Evaluation for MinimalChess 0.4

Post by Mike Sherwin » Thu Apr 15, 2021 3:32 pm

lithander wrote:
Thu Apr 15, 2021 10:50 am
Right after releasing version 0.3 of MinimalChess, which used non-tuned, non-tapered PSTs for it's evaluation I tried the approach of PeSTO, faithfully implementing the PST values shared by Ronald Friederich in this forum. My engine gained ~200 ELO from changing the evaluation in that way! The new eval wasn't much more complicated than the old one (the concept is the same, you just interpolate between two sets of PSTs depending on the game's phase) and thus it fits my project's vision ("minimal") pretty well.

I want to release the next version of MinimalChess soon and the only thing left to do is to replace the evaluation I borrowed from PeSTO with something original. I'm in the process of writing a tuner and I hope to create values from scratch that can rival or even surpass the PeSTO values.

I've seen a lot of threads where people added PeSTO to their engines (TSCP, Dumber, Rebel) making them significantly stronger. But has anybody managed to reproduce it's surprisingly good performance with their own PST-only eval? Or surpassed it? If so - how? If not - what do you think makes PeSTO special?
I think PeSTO is "special" for a weak engine. I do not know if PeSTO is special for a stronger engine. What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine. It has PST's, a minimal pawn evaluator and that is about it when it comes to evaluation. The only difference between Romi's PST and PeSTO is that Romi's PST is algorithmically created at the root and after the first couple of ply to account for the changes to the position.

So the question becomes what is to be considered minimal and what is to be considered incomplete. If all an engine has is static PST's tapered or not I would not call it minimal. I would call it incomplete. How can a chess engine be considered minimal that does not attempt to use any chess knowledge at all specific to the position on the board?

User avatar
lithander
Posts: 353
Joined: Sun Dec 27, 2020 1:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

Re: PST-only Evaluation for MinimalChess 0.4

Post by lithander » Thu Apr 15, 2021 3:59 pm

Mike Sherwin wrote:
Thu Apr 15, 2021 3:32 pm
What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine.
Well, afaik that would be the original PeSTO which plays at 3100 ELO!
Mike Sherwin wrote:
Thu Apr 15, 2021 3:32 pm
So the question becomes what is to be considered minimal and what is to be considered incomplete.
I don't think an engine playing above 3000 ELO is incomplete in any meaningful way. But my question wasn't really about the proper definition of the term "minimal" in relation to chess engines. That's a very subjective concept anyway. I could try to give my definition but that's not the point of this thread as far as I'm concerned.

Instead for me the question was what makes PeSTO so strong and how a set of PSTs with similar strength can be created. Everyone who uses it to replace their own tuned values (TSCP, Dumber) or in conjunction with an existing evaluation (ProDeo 3.1) sees a significant ELO gain. Isn't that strange?

I care because currently I've "borrowed" PeSTO's values and before I make the next release I want to replace it with an original set of values that ideally are equally strong. Or stronger! :twisted:
Minimal Chess. A simple engine written in C#! Details on Youtube & Github

Mike Sherwin
Posts: 369
Joined: Thu Aug 20, 2020 11:25 pm
Full name: Michael J Sherwin

Re: PST-only Evaluation for MinimalChess 0.4

Post by Mike Sherwin » Thu Apr 15, 2021 4:45 pm

lithander wrote:
Thu Apr 15, 2021 3:59 pm
Mike Sherwin wrote:
Thu Apr 15, 2021 3:32 pm
What is the ccrl rating of the strongest engine that uses PeSTO? RomiChess is a "minimal" chess engine.
Well, afaik that would be the original PeSTO which plays at 3100 ELO!
Mike Sherwin wrote:
Thu Apr 15, 2021 3:32 pm
So the question becomes what is to be considered minimal and what is to be considered incomplete.
I don't think an engine playing above 3000 ELO is incomplete in any meaningful way. But my question wasn't really about the proper definition of the term "minimal" in relation to chess engines. That's a very subjective concept anyway. I could try to give my definition but that's not the point of this thread as far as I'm concerned.

Instead for me the question was what makes PeSTO so strong and how a set of PSTs with similar strength can be created. Everyone who uses it to replace their own tuned values (TSCP, Dumber) or in conjunction with an existing evaluation (ProDeo 3.1) sees a significant ELO gain. Isn't that strange?

I care because currently I've "borrowed" PeSTO's values and before I make the next release I want to replace it with an original set of values that ideally are equally strong. Or stronger! :twisted:
Then the next question becomes, how are the PeSTO PSTs modified with additional evaluation. I got from your post that you planned on stopping at tuned PSTs and not have any further evaluation. So let me define my original question more precisely. What is the highest ccrl rating of a chess engine that only uses the PST of PeSTO and does not have any other evaluation except maybe an endleaf pawn structure evaluator? Let's keep this in the context of what you are trying to do!

User avatar
hgm
Posts: 26570
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: PST-only Evaluation for MinimalChess 0.4

Post by hgm » Thu Apr 15, 2021 4:51 pm

Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.

User avatar
mvanthoor
Posts: 1284
Joined: Wed Jul 03, 2019 2:42 pm
Location: Netherlands
Full name: Marcel Vanthoor
Contact:

Re: PST-only Evaluation for MinimalChess 0.4

Post by mvanthoor » Thu Apr 15, 2021 5:22 pm

I think that it depends on the engine the Pesto values are tuned against. Often, an engine such as Stockfish is run for X thousand of positions, for 2 seconds per position, to obtain an evaluation. Then the tuner tries to change the PST and material value in such a way to get as close as possible to the Stockfish evaluation.

In short: tuned PST's contain a lot of positional knowledge provided by a different, much stronger engine.

(If I'm mistaken and tuners work differently nowadays, I'd be happy to be re-educated; this is a subject I haven't looked into yet. Regrettably I haven't had much time for chess programming the past few weeks.)
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL

Mike Sherwin
Posts: 369
Joined: Thu Aug 20, 2020 11:25 pm
Full name: Michael J Sherwin

Re: PST-only Evaluation for MinimalChess 0.4

Post by Mike Sherwin » Thu Apr 15, 2021 5:40 pm

hgm wrote:
Thu Apr 15, 2021 4:51 pm
Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.
Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo. :?

User avatar
mvanthoor
Posts: 1284
Joined: Wed Jul 03, 2019 2:42 pm
Location: Netherlands
Full name: Marcel Vanthoor
Contact:

Re: PST-only Evaluation for MinimalChess 0.4

Post by mvanthoor » Thu Apr 15, 2021 5:52 pm

Mike Sherwin wrote:
Thu Apr 15, 2021 5:40 pm
hgm wrote:
Thu Apr 15, 2021 4:51 pm
Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.
Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo. :?
Why not? I it has all the sorting and pruning tricks of other high-end engines, I'm sure it will be possible to tune a lot of positional awareness into an evaluation and material values that are tuned and tapered.
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL

mar
Posts: 2494
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: PST-only Evaluation for MinimalChess 0.4

Post by mar » Thu Apr 15, 2021 6:11 pm

Mike Sherwin wrote:
Thu Apr 15, 2021 5:40 pm
hgm wrote:
Thu Apr 15, 2021 4:51 pm
Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.
Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo. :?
well, 3100 elo but 4CPU - I get (most do in fact) +100 by 4CPU as well (but today it's sort of a privilege of getting your SMP engine tested unlike 5 years ago :roll:)
single core, that's sligtly below 3k in CCRL 40/15.

so most of the strength comes from Rofchade's search.
I tried to replace my texel-tuned HCE with PeSTO and immediately lost ~200 elo, which means (considering my engine is slightly below PeSTO in CCRL) that PeSTO's search is at least 200 elo better than mine.
Martin Sedlak

Mike Sherwin
Posts: 369
Joined: Thu Aug 20, 2020 11:25 pm
Full name: Michael J Sherwin

Re: PST-only Evaluation for MinimalChess 0.4

Post by Mike Sherwin » Thu Apr 15, 2021 6:15 pm

mvanthoor wrote:
Thu Apr 15, 2021 5:52 pm
Mike Sherwin wrote:
Thu Apr 15, 2021 5:40 pm
hgm wrote:
Thu Apr 15, 2021 4:51 pm
Well, that is PeSTO itself, of course. And it has no Pawn evaluator. Just PST, as the name suggests.
Then that is absolutely amazing! I would never have imagined that an engine with no more than static PST's could ever attain anywhere near a 3100 elo. :?
Why not? I it has all the sorting and pruning tricks of other high-end engines, I'm sure it will be possible to tune a lot of positional awareness into an evaluation and material values that are tuned and tapered.
Channeling my inner Spock - It is illogical that a chess engine with only static PSTs can achieve a 3100 ccrl elo when many of the positions that are evaluated would simply be wrong. Channeling my inner Kirk - Not so Mr. Spock, a tuned PST can get a good start for a chess engine and then a superior search can negate any deficiencies of the PSTs.
https://www.youtube.com/watch?v=5vrXKlO2Jbw

Post Reply