New engine: Peacekeeper

[Sazgr]

I have released v1.0, which has tuned piece square tables. They seem to be around the strength of the PeSTO tables, so I have gone with them. Other bit of search modifications, and the engine seems a bit stronger (but still weaker than those of other people around here). However finally the important components of Peacekeeper are original!

[Sazgr]

Release v1.01 to fix a crashing bug: https://github.com/Sazgr/peacekeeper/releases/tag/v1.01

Aside from fixing that bug though, I've been making no other progress at all which I suspect may have to do with extremely messy code. Almost no comments at all, pain in the neck to add on new features such as killer tables or pawn hash. I suppose I will have to do a major code restructuring before continuing progress

[emadsen]

Hang in there. It's all part of the learning process.

I do recommend incrementally refactoring code to keep it clean. You don't want to accumulate such a mess that you're discouraged from working on your engine. Make small changes, then run a gauntlet tournament to ensure you didn't introduce a performance regression. Accept the fact that part of chess programming is waiting for game results.

[lithander]

I do recommend incrementally refactoring code to keep it clean. You don't want to accumulate such a mess that you're discouraged from working on your engine. Make small changes, then run a gauntlet tournament to ensure you didn't introduce a performance regression. Accept the fact that part of chess programming is waiting for game results.

That's solid advice!

If you don't test every new feature in isolation and instead make a bunch of changes and test them together you'll risk that some features become part of your engine that are underperforming. Good luck finding these problematic parts later!

Even if you test thoroughly the question is: how much Elo should a new feature add so that you can be pretty sure it's not underperforming? I haven't found a good answer to that one but reading the forum helps.

Besides running selfplay matches or gauntlets (and enough of them for the results to be statistically significant) I found it useful to have a set of a few hundred positions with a known best move. You can search all positions in the testset to a specific depth and see how many correct moves you found and how long that took. When you are just refactoring or performance optimizing you can skip the gauntlet test as long as the duration of the test goes down while the number of solved positions and total number of searched nodes stay the same. Making your program just faster is never a reason for concern.

[Sazgr]

I do recommend incrementally refactoring code to keep it clean. You don't want to accumulate such a mess that you're discouraged from working on your engine. Make small changes, then run a gauntlet tournament to ensure you didn't introduce a performance regression. Accept the fact that part of chess programming is waiting for game results.

That's solid advice!

If you don't test every new feature in isolation and instead make a bunch of changes and test them together you'll risk that some features become part of your engine that are underperforming. Good luck finding these problematic parts later!

Even if you test thoroughly the question is: how much Elo should a new feature add so that you can be pretty sure it's not underperforming? I haven't found a good answer to that one but reading the forum helps.

Besides running selfplay matches or gauntlets (and enough of them for the results to be statistically significant) I found it useful to have a set of a few hundred positions with a known best move. You can search all positions in the testset to a specific depth and see how many correct moves you found and how long that took. When you are just refactoring or performance optimizing you can skip the gauntlet test as long as the duration of the test goes down while the number of solved positions and total number of searched nodes stay the same. Making your program just faster is never a reason for concern.

Hi, thank you two!
At the early stages of development I neglected to test some features, so now I am disabling features one by one to see elo gain separately. I will look into testsets such as the WAC. Hopefully I can find some features that are not working up to their full strength. LMR may be a culprit, as in tests, it gives a benefit of usually in the 10-20 elo range which seems suspiciously low.

On another note, right now I am using 5+0.05 time control with concurrency 4 on a laptop with 4 cores and 8 processors (I also do work simultaneously). Should I modify any of those settings for testing matches?

Thank you for all these suggestions

[JacquesRW]

Release v1.01 to fix a crashing bug: https://github.com/Sazgr/peacekeeper/releases/tag/v1.01

Aside from fixing that bug though, I've been making no other progress at all which I suspect may have to do with extremely messy code. Almost no comments at all, pain in the neck to add on new features such as killer tables or pawn hash. I suppose I will have to do a major code restructuring before continuing progress

Congratulations on the release!
At first glance there's a lot of the below in your pvs and quiescence code that makes it pretty difficult to read, imo.

Code: Select all

if (!(nodes & 8191) && timer.check(nodes)) {/* something */; return 0;}

But you only really need to do this right at the top of the pvs function (and not at all in quiescence), and the search will probably be faster from not having to check so many branches in each node.

[Robert Pope]

If he doesn't check in quiesce, doesn't he risk going many multiples of 8192 without checking, since many will occur in quiesce?

[JacquesRW]

If he doesn't check in quiesce, doesn't he risk going many multiples of 8192 without checking, since many will occur in quiesce?

Good point, he could also check at the top of quiescence as well then, though I've never had any issue not doing that (although I check every 2048 nodes) as these small blocks of nodes are completed so quickly.
Either way its way more readable than having code for checking over a dozen times at different locations.

[Sazgr]

Release v1.01 to fix a crashing bug: https://github.com/Sazgr/peacekeeper/releases/tag/v1.01

Aside from fixing that bug though, I've been making no other progress at all which I suspect may have to do with extremely messy code. Almost no comments at all, pain in the neck to add on new features such as killer tables or pawn hash. I suppose I will have to do a major code restructuring before continuing progress

Congratulations on the release!
At first glance there's a lot of the below in your pvs and quiescence code that makes it pretty difficult to read, imo.

Code: Select all

if (!(nodes & 8191) && timer.check(nodes)) {/* something */; return 0;}

But you only really need to do this right at the top of the pvs function (and not at all in quiescence), and the search will probably be faster from not having to check so many branches in each node.

But say the time allocated by the time management is all used up, and I returned 0 from a PVS call. On the PVS call it returned to, if there is only a check at the top, the outer PVS call will proceed semi-normally. So I put a check after each call of PVS as well. How else should I arrange the checks so that I don't need as much of them? Or will not checking after PVS calls work fine? I am not sure.

[JacquesRW]

But say the time allocated by the time management is all used up, and I returned 0 from a PVS call. On the PVS call it returned to, if there is only a check at the top, the outer PVS call will proceed semi-normally. So I put a check after each call of PVS as well. How else should I arrange the checks so that I don't need as much of them? Or will not checking after PVS calls work fine? I am not sure.

Mine goes like this

Code: Select all

fn search(...) {
	if (stop) {return 0;}
	if (!(nodes & 2047) && check_time() > allocated time) {
		stop = true;
		return 0;
	}
	...
}

So after a node finds that the time is up, it sets the stop flag, and then any new node that is entered immediately returns 0. This propagates up the search tree extremely quickly as at each ply 0 is returned on all remaining moves to be searched. Then of course because the stop flag is set you ignore the search results and return a best move.

One part where it is important though, is to make sure you don't write to the hash table (or other tables) with one of these values, so just adding

Code: Select all

if (!stop && /* other conditions */) { hash_table.push(...) }

Alternatively you can check the stop flag once more after the moves loop in a node to explicitly return 0.