Pedantic Developer's Log Stardate...

Discussion of chess software programming and technical issues.

Moderator: Ras

Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Pedantic Developer's Log Stardate...

Post by Guenther »

JoAnnP38 wrote: Mon Jan 09, 2023 2:55 pm
...

Question to the group -- does anyone know of any UCI engines rated between 1800 and 2200? I'm guessing this is the around the ELO strength of Pedantic, but I really need to see it win some games before I can be sure.

...


Rating classifications between C2 and B3 cover this. (scaled similarily to CCRL blitz ratings)
A more precise explanation can be found in the TOC+ sheet.


You can use sort/filter columns as you like even with just reading access.

Guenther (maintainer of this XB/UCI chronology)
https://rwbc-chess.de

[Trolls n'existent pas...]
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: Pedantic Developer's Log Stardate...

Post by JoAnnP38 »

Okay, I found an engine supposedly rated 2050 ELO and I actually won a game for the first time. However, in the rematch I am scratching my head. Why doesn't Pedantic see the move a2a1Q at 40 that leads to mate? I've debugged it a little and see that the increase in material is being updated. The PV pedantic suggests doesn't even push it into the horizon. Sigh... more work, more work.

okidoki
Posts: 17
Joined: Thu Jun 30, 2022 12:46 pm
Full name: Kabir Kumar

Re: Pedantic Developer's Log Stardate...

Post by okidoki »

JoAnnP38 wrote: Tue Jan 10, 2023 7:39 pm d see that the increase in material is being updated. The PV pedantic suggests doesn't even push it into the horizon. Sigh... more work, more work.
I haven't checked the source if Pedantic implements extra pruning. However you can check if plain negamax(full search without pruning) finds the mate . This quick check might save lot of hours IMO.
JVMerlino
Posts: 1398
Joined: Wed Mar 08, 2006 10:15 pm
Location: San Francisco, California

Re: Pedantic Developer's Log Stardate...

Post by JVMerlino »

JoAnnP38 wrote: Tue Jan 10, 2023 7:39 pm Okay, I found an engine supposedly rated 2050 ELO and I actually won a game for the first time. However, in the rematch I am scratching my head. Why doesn't Pedantic see the move a2a1Q at 40 that leads to mate? I've debugged it a little and see that the increase in material is being updated. The PV pedantic suggests doesn't even push it into the horizon. Sigh... more work, more work.
[fen]r2n2k1/3P3p/1R4p1/2B5/4p3/2P1P2P/p4rP1/2KR4 w - - 0 40[/fen]

It's a mate in 4 for Black regardless. So unless Pedantic has a bug with movegen or make/unmake of promotions, I'd take a wild guess that the problem is that you're not correctly adjusting a hash value for mate scores based on the remaining depth.
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: Pedantic Developer's Log Stardate...

Post by JoAnnP38 »

JVMerlino wrote: Fri Jan 13, 2023 5:13 pm It's a mate in 4 for Black regardless. So unless Pedantic has a bug with movegen or make/unmake of promotions, I'd take a wild guess that the problem is that you're not correctly adjusting a hash value for mate scores based on the remaining depth.
okidoki wrote: Fri Jan 13, 2023 4:40 pm I haven't checked the source if Pedantic implements extra pruning. However, you can check if plain negamax(full search without pruning) finds the mate. This quick check might save lot of hours IMO.
I think you guys were both right. I only solved the problem by going back to basics and creating the simplest Negamax search I could -- no quiesence, no transposition tables, no pruning and no null window searches. An as @okidoki points out I discovered my problems. Biggest one had to do with the transposition table. I had some logic reversed which of course broke the search. As I slowly added features back into the search I found other bugs, so it was a very fruitful endeavor. I wish I had started like that. I depended on my unit tests, but if you are developing code around ideas that you don't fully grok... Well, the tests are only as good as my knowledge.

I'm very close to putting a pin in the engine development so I can start writing the client that will adjudicate the genetic algorithm and hopefully result in a strong evaluation. I'm making progress!
JVMerlino
Posts: 1398
Joined: Wed Mar 08, 2006 10:15 pm
Location: San Francisco, California

Re: Pedantic Developer's Log Stardate...

Post by JVMerlino »

Excellent! Looking forward to your first release. :-)
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: Pedantic Developer's Log Stardate...

Post by JoAnnP38 »

I have finally finished tinkering around with Pedantic to the point where I can announce a Pedantic v0.1 "pre-release." There will not be an official release until I have completed the portions of the project required to optimize the evaluation. The entire project at this point is a bloated piece of @#%@ (okay, maybe not quite that bad.) It currently implements three flavors of search:
  1. MTD(f) - memory enhanced test-driver algorithm.
  2. Principal Variation Search
  3. MinimalChess 0.6 Emulation (a search inspired by MinimalChess)
Please don't confuse my rough emulation of MinimalChess search with the actual app. Pedantic is slower and really only uses the heuristic parts of MinimalChess's search design. It is still integrated with the Pedantic evaluation, quiesence search, and it is currently being called with aspiration windows (MinimalChess doesn't use them). I needed a baseline (that I wouldn't be tempted to change) to compare my development on other searches. At some point it will probably be removed from the Pedantic application.

This engine supports enough of the UCI protocol to run in Arena and has its own opening book and supports pondering. I have not tested it recently with pondering turned off so you may run into problems there. It has a few command-line switches that can be used to control its function. Here is the output of help that shows each option:

Code: Select all

>Pedantic /?
Description:
  The pedantic chess engine.

Usage:
  Pedantic [command] [options]

Options:
  --version       Show version information
  -?, -h, --help  Show help and usage information

Commands:
  uci    Start the pedantic application in UCI mode.
  perft  Run a standard Perft test.
  
 >Pedantic perft /?
Description:
  Run a standard Perft test.

Usage:
  Pedantic perft [options]

Options:
  --type <Average|Details|Normal>  Specifies the perft variant to execute. [default: Normal]
  --depth <depth>                  Specifies the maximum depth for the perft test. [default: 6]
  --fen <fen>                      Specifies the starting position if other than the default. []
  -?, -h, --help                   Show help and usage information
  
 >Pedantic uci /?
Description:
  Start the pedantic application in UCI mode.

Usage:
  Pedantic uci [options]

Options:
  --search <Minimal|Mtd|Pv>  Specifies the search type to use. [default: Pv]
  --input <input>            Specify a file read UCI commands from. []
  --error <error>            Output errors to specified file. []
  -?, -h, --help             Show help and usage information
Note: The performance of the perft tests is currently much lower than I have posted here on this forum because my perft tests reuse the same move generation and execution logic that normal game play uses. This includes all of the incremental updates required for evaluation.

Using the MTD(f) search it currently has a rating around 1700 (a bit lower than I was shooting for) and I believe that is currently its strongest search method. Its end game play is unfortunately pretty bad. Hopefully, I can pick up a 100-200 or so ELO after I have finished the next phase of my project that will optimize all of the weights used by the evaluation function. Once I have finished the optimization stage, I will officially release Pedantic and start on a C++ port that removes the bloat. My goal is to get to a competition worthy engine, but I still have a lot to learn so that is a long way off. So many tricks of the trade still evade my understanding. :oops: The evaluation currently evaluates the following positional features:
  • Game Phase
  • Material
  • Piece position
  • Mobility
  • Development
  • King Safety
  • Pawn Structure (blocked pawns, isolated pawns, backward pawns, doubled pawns, passed pawns, pawn majorities, and others...)
  • Knight/Bishop outposts
  • Other miscellaneous positional qualities
If anyone dares to run this "pre-release" or look at its code (see my signature for github url) and would like to offer friendly criticisms, please feel free. Just keep in mind that it is still far from what I would called polished or for that matter fully tested. (Can any software ever be called "fully" tested?) There are still a lot of quirks I have to live with (like knowing when to exit an iterative search when forced mate or draw is detected) but it doesn't currently inhibit me from testing the engine in Arena.
Last edited by JoAnnP38 on Wed Jan 25, 2023 6:33 am, edited 1 time in total.
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: Pedantic Developer's Log Stardate...

Post by JoAnnP38 »

lithander wrote: Tue Jan 10, 2023 11:43 am
JoAnnP38 wrote: Mon Jan 09, 2023 2:55 pm Question to the group -- does anyone know of any UCI engines rated between 1800 and 2200? I'm guessing this is the around the ELO strength of Pedantic, but I really need to see it win some games before I can be sure.
You could play against earlier version like 0.4 (1890 Elo) or version 0.5 (2270 Elo) of MinimalChess of course. And you may want to try Leorik 1.0 which is also in the desired Elo range (2150) but has the interesting property that it does not implement any unsafe pruning techniques, reductions, extensions etc. Not even null-move pruning. This causes a high branching factor and the search remains quite shallow even at higher time controls. But it solves all mate puzzle with the shortest path for example because it never overlooks a move. This could expose some weaknesses in your engine if you overdo it with the pruning and reductions!
Sorry, but both of those implementations are too strong for Pedantic at the moment. I am currently using Celestial 1.0 rated about 1700 to test against. My raw nps speed is around 2-3 times slower than Celestial which was developed in rust, so I have to adjust all my heuristics to minimize the number of nodes I actually evaluate. I thought that optimizing the evaluation would be the challenging part, but this entire process of tinkering with pruning, reductions, and extensions has had me pulling my hair out at times. Hopefully, Pedantic will be strong enough to test against your engines in the future.
User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

Re: Pedantic Developer's Log Stardate...

Post by lithander »

JoAnnP38 wrote: Wed Jan 25, 2023 6:29 am I thought that optimizing the evaluation would be the challenging part, but this entire process of tinkering with pruning, reductions, and extensions has had me pulling my hair out at times.
Yeah, I know exactly what you mean. For the first couple of months I basically tried to implement what I heard people talk about on the forums but without looking implementation details up. Narrow windows, and null move pruning and late move reductions and qsearch of course. Progress was relatively smooth and steady. But I didn't do much in terms of selectivity and when I ran out of ideas I thought I could just read up on all the specific extensions and reductions on the wiki and add some of them. But then nothing seemed to work. Maybe you can't add them one at a time but some of them have to go together? Or I did some fundamental error somewhere down the line and now my engine is too different? I couldn't figure out exactly why.

By now I've accepted that if I want to make progress with Leorik I shouldn't look too closely at what others are doing and be prepared to challenge what seems to be generally accepted. For example, why have both extensions and reductions? Why not have only reductions - which is an implicit extension of everything you don't reduce, right? Or when I look at how Stockfish get's to incredible depths so quickly... I don't know how they do that but certainly not by only pruning near the leafs. In the light of that calling it extended futility pruning because you start with the pruning at depth==2 instead of 1... well, seems like something from the 80s when search depth was much shallower. How about we *really* go and extend that?

I'm having more fun that way, my code is more aesthetically pleasing to me because there are less special cases and if I hit the wall at some point I can always go and look into NNUE or something. It's hobby, do whatever is fun to you! :)
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: Pedantic Developer's Log Stardate...

Post by JoAnnP38 »

lithander wrote: Wed Jan 25, 2023 1:21 pm
JoAnnP38 wrote: Wed Jan 25, 2023 6:29 am I thought that optimizing the evaluation would be the challenging part, but this entire process of tinkering with pruning, reductions, and extensions has had me pulling my hair out at times.
Yeah, I know exactly what you mean. For the first couple of months I basically tried to implement what I heard people talk about on the forums but without looking implementation details up. Narrow windows, and null move pruning and late move reductions and qsearch of course. Progress was relatively smooth and steady. But I didn't do much in terms of selectivity and when I ran out of ideas I thought I could just read up on all the specific extensions and reductions on the wiki and add some of them. But then nothing seemed to work. Maybe you can't add them one at a time but some of them have to go together? Or I did some fundamental error somewhere down the line and now my engine is too different? I couldn't figure out exactly why.
I still think there is black magic involved. I am almost ready to call in the local voodoo priestess! :D

I think the biggest thing I've learned is that reductions are best reserved for the null window search and if you get a cut-off or it appears to raise alpha then I do a full width search without reductions. That way I always have good or reliable information on my PV nodes. Of course, when I'm using MTD(f) search, I do a zero-width with reductions and follow it up with a zero-width without to verify cut-offs. I'm a fan of MTD(f).
lithander wrote: Wed Jan 25, 2023 1:21 pm By now I've accepted that if I want to make progress with Leorik I shouldn't look too closely at what others are doing and be prepared to challenge what seems to be generally accepted. For example, why have both extensions and reductions? Why not have only reductions - which is an implicit extension of everything you don't reduce, right? Or when I look at how Stockfish get's to incredible depths so quickly... I don't know how they do that but certainly not by only pruning near the leafs. In the light of that calling it extended futility pruning because you start with the pruning at depth==2 instead of 1... well, seems like something from the 80s when search depth was much shallower. How about we *really* go and extend that?
At this point I have looked at extensions both ways. I really need to get serious about setting up a test suite I can use to evaluate the benefit of these changes. Many of the changes may only benefit a minority of positions but without a good test I only have my hunches to lead me (which is probably a very bad idea.)