Progress on Rustic

[maksimKorzh]

Hi A short progress report on Rustic.

Because work has been very busy due to a massive system migration, I'm already at the computer for 10+ hours a day.

I haven't made a lot of progress on finishing the implementation of the XBoard protocol; I've implemented a few commands since the last update, but I still have some to go before I can start testing the engine in XBoard mode.

Therefore I have decided to just go ahead with creating a release for Rustic Alpha 1, so it can at least be tested by the CCRL and/or CEGT folks if they want to. Today I've installed a Linux virtual machine, and Rustic compiled without problems. Because I use MSYS2 on Windows, the build process is actually exactly the same. Actually, in the virtual machine, Rustic runs at the same speed as on Windows. I'm going to see how old a Debian version I can install and still have Rust 1.49 running, so the Linux executable will be able to run on as many systems as possible. These will be the releases:

Windows:
- 64-bit BMI2 (Haswell and up; IIRC)
- 64-bit POPCNT (Intel Nehalem)
- 64-bit Ancient (Intel Core2)
- 32-bit i586 (should basically run on any CPU created since 1995)

Linux:
- 64-bit BMI2
- 64-bit POPCNT
- 64-bit Ancient
- 32-bit: None. You'll have to compile it yourself. Most distributions are dropping 32-bit, and/or require workarounds.

Raspberry Pi (for picochess):
- Should run on any Raspberry Pi that runs the current official 32-bit Buster installation. (If the 64-bit installation becomes available, I'll also provide a 64-bit version.) I also have my own Picochess Buster image online that is smaller than the last official one, has many extra possibilities such as VNC and SSH access. It runs a bugfixed version of the last official Picochess 0.9n. (I use this myself, with my DGT Board.)

The link to the image (on Google Drive) is in the first post of this Google Group thread:
https://groups.google.com/g/picochess/c ... obYjmHBAAJ

Apple OSX:
- I would need help with this. I have no Mac (and will never have one), and I refuse to start hacking around with illegal OSX VM's. It should be possible to cross-compile from either Windows or Linux, but if I do, I can't test it. Then there's the Apple Silicon M1.

PS: I'm not abandoning XBoard. I just want to (finally) release the first version of this engine, and make a first start on the book/tutorial. After XBoard is finished and tested, I'll merge it into master.

Can't wait to read through the book/tutorial!

[Gabor Szots]

Therefore I have decided to just go ahead with creating a release for Rustic Alpha 1, so it can at least be tested by the CCRL and/or CEGT folks if they want to.

Not impossible. I have just compiled Alpha 1 successfully but I'm going to wait for a release. I spent some time learning Rust recently so I try to understand some of your code as a learning tool. Rust is very difficult for me at the moment.

[mvanthoor]

Therefore I have decided to just go ahead with creating a release for Rustic Alpha 1, so it can at least be tested by the CCRL and/or CEGT folks if they want to.

Not impossible. I have just compiled Alpha 1 successfully but I'm going to wait for a release. I spent some time learning Rust recently so I try to understand some of your code as a learning tool. Rust is very difficult for me at the moment.

I have just released Rustic Alpha 1.
Thank you for your patience and for looking into the progress of the engine. I would love to have this version tested by CCRL to establish a base rating. The engine is about as basic as you can get. I estimate the CCRL Blitz rating at +/- 1650.

Github release page

I provide binaries for:

- Windows (32-bit generic, 64-bit old, popcnt and bmi2.)
- Linux (compiled on Debian 8 stable; will run on Debian 9 and 10 Stable. 64-bit old, popcnt and bmi2.)
- Raspberry Pi (32-bit generic; should run on any Pi that runs Raspi-OS Buster.)

Next steps are going to be, in this order:

- Update / extend the compilation documentation.
- Get started with the outline of the book at https://rustic-chess.org/.
- Finish the XBoard protocol (about 75% done).
- Integrate the transposition table (written already; needs a bit of a code style update, and testing).
- Lots of other features in the chess engine
- And other projects that will take years and years as I build everything from scratch (engine, gui, DGT chess computer) as a programming pastime, just because I can.

With regard to Rust, I agree. Rust is not an easy language to get going with. The borrow-checker can be a real menace in the beginning, but after you get to grips with it, it will become your best friend, and REALLY speed up development (at go like "uh-uh...." if you do something that isn't going to work).

If you have any questions, I'd be happy to answer them. Just post them in this topic.

[unserializable]

I have just released Rustic Alpha 1.

Congratulations, Marcel! I went ahead and did a local pull and Linux compile with rustc 1.47.0 (18bf6b4f0 2020-10-07) on Ubuntu 20.04.1 LTS successfully.

Ran some quick 10s+0 games against Monchester -- at that time control Rustic has improved, in November it achieved 94.75% score (200 games), now 98.90% score (500 games +494/=1/-5). One of its losses was bit weird, as in very short 20 move game, should not have been time issue (Rustic's few losses on time occured on 80+ moves).

[pgn]
[Event "Computer Chess Game"]
[Date "2021.01.24"]
[Round "55"]
[White "Monchester 1.0.1-7-ga533423"]
[Black "Rustic Alpha 1"]
[Result "1-0"]
[TimeControl "10"]

1. Nf3 {-0,25/4} d5 {+0,05/5 0,1} 2. e3 {-0,34/4 0,1} Bd7 {+0,05/5 0,1} 3.
Nc3 {-0,17/4 0,1} Nf6 {+0,10/5 0,1} 4. Ne5 {-0,25/4 0,1} e6 {+0,30/5 0,1}
5. Nd3 {-0,28/4 0,1} b5 {+0,85/5 0,1} 6. h4 {-0,28/4 0,1} Bd6 {+1,20/5 0,1}
7. Rb1 {-0,31/4 0,1} b4 {+1,50/5 0,1} 8. Ne2 {-1,25/4 0,1} c5 {+1,55/5 0,1}
9. f4 {-1,17/4 0,1} Nc6 {+1,55/5 0,1} 10. Ng3 {-0,85/4 0,1} c4
{+1,95/5 0,1} 11. Nf2 {-0,97/4 0,1} Qa5 {+1,40/4 0,2} 12. Ra1 {-1,08/4 0,2}
O-O {+1,40/4 0,2} 13. Rh3 {-1,05/4 0,2} e5 {+2,10/4 0,2} 14. f5
{-1,17/4 0,2} e4 {+2,05/4 0,2} 15. Nh5 {-1,42/4 0,1} Nxh5 {+3,05/6 0,2} 16.
Qxh5 {-1,22/4 0,1} g6 {+3,05/5 0,2} 17. Qh6 {-1,77/4 0,2} b3 {+3,00/4 0,2}
18. f6 {+16,20/4 0,2} Qxd2+ {-1000,02/4 0,2} 19. Bxd2 {+1000,02/3 0,1} Bxh3
{-1000,01/4 0,2} 20. Qg7# {+1000,01/1 0,1}
{Xboard adjudication: Checkmate} 1-0
[/pgn]

[d]r4rk1/p2b1p1p/2nb2pQ/q2p1P2/1pp1p2P/4P2R/PPPP1NP1/R1B1KB2 b - - 1 17

Position where Rustic played 17. .. b3 with evaluation {+3,00/4 0,2} seems such that if indeed it searches to depth 4 as saved PGN indicates, it should have already avoided 17. .. b3 due to following 18. f6 ... 19. Qg7# threat because even if queen sacrifice moves mate behind search horizon, it would seem 17. .. b3 should at least already get negative score (on its next move it sees the definite mate, 18. .. Qxd2+ {-1000,02/4 0,2}).

Your plans are great as ever, much strength for the future work!

[mvanthoor]

Congratulations, Marcel! I went ahead and did a local pull and Linux compile with rustc 1.47.0 (18bf6b4f0 2020-10-07) on Ubuntu 20.04.1 LTS successfully.

...

Your plans are great as ever, much strength for the future work!

Thanks for testing (I assume you have Rust installed according to the instructions on the Rust website. If so, you can update to 1.49 with "rustup update". If you install using the website's instructions, Rust is installed in your home directory, and you don't need to wait for the distribution's repo to update.)

I've just checked the game above. It seems Rustic plays b4-b3 at ONLY depth 4:

Code: Select all

go
info score cp 310 depth 1 seldepth 4 time 0 nodes 32 nps 0 pv d7f5
info score cp 310 depth 2 seldepth 3 time 1 nodes 4625 nps 4625000 pv d7f5 h3h1
info score cp 310 depth 3 seldepth 7 time 5 nodes 22775 nps 4555000 pv d7f5 g2g4 f5e6
info score cp 300 depth 4 seldepth 5 time 45 nodes 212401 nps 4720022 pv b4b3 c2b3 c4b3 f1e2 b3a2
info score cp 270 depth 5 seldepth 7 time 189 nodes 925981 nps 4899370 pv d7f5 g2g4 f5e6 d2d4 b4b3
info score cp 250 depth 6 seldepth 7 time 848 nodes 3844728 nps 4533877 pv d7f5 g2g4 f5e6 d2d4 b4b3 c1d2 d6b4
info score cp 265 depth 7 seldepth 9 time 4425 nodes 20501905 nps 4633199 pv d7f5 g2g4 f5e6 d2d3 c4d3 c2d3 b4b3
info score cp 245 depth 8 seldepth 9 time 26025 nodes 118189447 nps 4541381 pv d7f5 g2g4 f5e6 d2d3 c4d3 c2d3 b4b3 c1d2 d6b4

On any other depth, it's solidly advocating Bd7xf5.

I think this is because the search and evaluation function are still very basic. Rustic starts at the beginning again at each new depth. It has no transposition table, history heuristics, or any kind of search extensions except check extension. It doesn't know about any danger, except mate and material loss, and only when it can clearly see it due to search depth. Somehow, it didn't see the mate at move 4, so b4b3 is a perfectly fine move. At the next depth, it switches back to Bd7xf5. Stuff like this will probably improve "all on its own" when Rustic gains more search and evaluation functionality.

Rustic's time management is OK, but basic as well; it can mostly do fine, except it sometimes overshoots at hyperfast time controls with no increment, like the one you used. (If there is no increment, and the game is not a clear win for one of the engines, especially if both are weaker engines, one of them at some point WILL overshoot; you can't keep shortening time usage indefinitely.) I always test at 60s+0.6s, and Rustic never loses on time. Obviously I'll look into improving time management later

The reason why Rustic has improved against your engine (and other engines as well) probably is because of the last enhancements it gained was an additional King PST for the endgame, so it can now mate a lone king using K+Q, K+R, and K+B+B. Previously, many of those games would be drawn because the King PST would keep the winning king at the edge of the board.

Thanks for testing. I appreciate any input.

With regard to my plans: I'm just glad I have my own chess engine out there. Now I have something to program on whenever I want to. First order of business will be to create a better compilation guide and start updating and outlining the book at https://rustic-chess.org/.

[unserializable]

Code: Select all

go
...
info score cp 300 depth 4 seldepth 5 time 45 nodes 212401 nps 4720022 pv b4b3 c2b3 c4b3 f1e2 b3a2
...

My first intuition on the curious selection, without familiarity with Rustic time management was that maybe when pre-allocated move time limit is reached, Rustic interrupts search without depth 4 being /fully/ searched, while still reporting search depth 4 --- in which case the move choice leading to mate would be unlikely to be consistently reproducible.

If this is indeed reproducible with 'go', assumedly searching without any time concerns, then more likely seems to be cut-off error in search or some woe raising from selective search (depth 4 PV is 5 plies, as is selective depth reported from 'go').

With regard to my plans: I'm just glad I have my own chess engine out there. Now I have something to program on whenever I want to. First order of business will be to create a better compilation guide and start updating and outlining the book at https://rustic-chess.org/.

Have you decided what communication protocol will you use for the book examples? From my brief encounters with UCI, it seems to be rather verbose due to statelessness, whereas most CECP protocol commands seem be able to fit on a width of a book page.

[mvanthoor]

[quote=unserializable post_id=880542 time=1611587116 user_id=127
My first intuition on the curious selection, without familiarity with Rustic time management was that maybe when pre-allocated move time limit is reached, Rustic interrupts search without depth 4 being /fully/ searched, while still reporting search depth 4 --- in which case the move choice leading to mate would be unlikely to be consistently reproducible.

If this is indeed reproducible with 'go', assumedly searching without any time concerns, then more likely seems to be cut-off error in search or some woe raising from selective search (depth 4 PV is 5 plies, as is selective depth reported from 'go').
[/quote]

I'll have to look into this, to see if I can find out more about it. At the moment, it doesn't have a huge priority, because this specific problem only occurs at depth 4; and yes, it occurs with "go" ("go infinite") as well. What this command does is just deepening the search:

1
1, 2
1, 2, 3
1, 2, 3, 4
1, 2, 3, 4, 5

It does do quiescence search, so it can actually search a 1-4 moves on top of the depth it is iterating to. At the moment I have no idea why Rustic decides to go for b4-b3 at depth 4.

The reason why depth = 4 and seldepth = 5 is either because of the quiescence search or the check extension. Both raise seldepth. There are no other extensions in the engine. I'll have a look into reporting this number (it has changed a few times), but if something in there changes, it's only the depth and seldepth report; not actual searching. What happens is the normal "if in check, depth++" and the normal quiescence search.

Have you decided what communication protocol will you use for the book examples? From my brief encounters with UCI, it seems to be rather verbose due to statelessness, whereas most CECP protocol commands seem be able to fit on a width of a book page.

Both. After I finished the XBoard-protocol, I'll also describe its implementation.

Personally, I found the UCI-protocol MUCH faster and easier to implement than XBoard because the specification is much shorter and the engine doesn't have to keep track about what it's doing. The GUI just screams "do this!", the engine does it and returns a result, and that's it.

In Rustic, both UCI and XBoard are implemented on top of the same infrastructure. For example:

position fen <fen> ==> fen_read(<fen>)
setboard <fen> ==> fen_read(<fen>)

When outputting information, the engine sends a "SearchSummary" to the communication module. This module can either be the XBoard or the UCI module (they share the same interface), and thus the SearchSummery is 'automatically' converted to either XBoard or UCI output.

There are many other examples where Rustic handles UCI and XBoard in the same way, with the same underlying code. You state that UCI is verbose with regard to command structure, and compred to XBoard, it is... but it requires the implementation of MUCH less commands. With the risk of angering some people, compared to UCI, I find XBoard laborious to implement completely. (I didn't say UCI is easier, or better, or more stable, or whatever; it is faster to implement, with less work. At least, according to my feelings.)

[mvanthoor]

... more about the weird b4b3 move ...

It seems I already found the problem.

In the search function, the comments actually state to do the check extension BEFORE the quiescence search... but the extension is actually AFTER the call to the qsearch. Must have been some oversight on my part, or a copy/paste error. Now Rustic can actually get into the situation that it does a qsearch while in check.

I've moved the in-check extension before the depth = 0 / qsearch call, so when in check the depth is increased by 1, and thus qsearch does not happen. The result of the search in this position is now as follows:

Code: Select all

go
info score cp 310 depth 1 seldepth 4 time 0 nodes 32 nps 0 pv d7f5
info score cp 310 depth 2 seldepth 3 time 1 nodes 4637 nps 4637000 pv d7f5 h3h1
info score cp 310 depth 3 seldepth 7 time 5 nodes 22939 nps 4587800 pv d7f5 g2g4 f5e6
info score cp 250 depth 4 seldepth 5 time 27 nodes 107233 nps 3971593 pv d7f5 g2g4 f5e6 d2d4
info score cp 260 depth 5 seldepth 7 time 161 nodes 708733 nps 4402068 pv d7f5 g2g4 f5e6 d2d3 b4b3 c1d2
info score cp 250 depth 6 seldepth 7 time 850 nodes 3499787 nps 4117396 pv d7f5 g2g4 f5e6 d2d4 b4b3 c1d2 d6b4
info score cp 255 depth 7 seldepth 9 time 4743 nodes 19958120 nps 4207911 pv d7f5 g2g4 f5e6 d2d3 b4b3 c1d2 d6b4 c2c3
info score cp 240 depth 8 seldepth 9 time 29455 nodes 121486668 nps 4124484 pv d7f5 g2g4 f5e6 h4h5 c4c3 h5g6 c3d2 c1d2 h7g6
info score cp 240 depth 9 seldepth 11 time 195385 nodes 806594589 nps 4128232 pv d6e5 d2d4 b4b3 e1e2 e5d4 c2b3 d4g7 h6g5 f7f6 g5f4 c4b3

So, b4b3 is not considered anymore. The engine is firmly stuck on Bd7xf5 until depth 9, where it switches to Bd6-e5. (Try to follow the variation, and then capture the bishop, after it goes Be5xd4... see what happens ) Bd6-e5 seems risky to me, as it leaves the f5 pawn in place. Nice fact... Stockfish 10 is stuck on Bd7xf5 until depth 24. However, Bd6-e5's evaluation is creeping up and up, as the second move in the list, and it takes over as the first move on depth 25.

It seems you found a bug / imperfection in Rustic again. Even after staring at the code for weeks to make sure there are no bugs, there's still a glaring error such as a comment by myself saying "do the check extension before qsearch" and then actually doing it after. I feel dumb

I'll run a quick 500 game test between Alpha 1 and this new version to see if there's actually a measurable difference. If there is, the next version will have a free ELO raise on top of what improvements I actually implement (if I do anything apart from the XBoard protocol, it'll probably be a transposition table only for that version).

[mvanthoor]

It seems that in a match of 1000 games of 10s+0.1s, the fixed version of Rustic is a bit weaker:

Code: Select all

Score of Rustic Alpha 1 vs Rustic QSearch Fix: 400 - 336 - 264 [0.532]
...      Rustic Alpha 1 playing White: 177 - 200 - 123  [0.477] 500
...      Rustic Alpha 1 playing Black: 223 - 136 - 141  [0.587] 500
...      White vs Black: 313 - 423 - 264  [0.445] 1000
Elo difference: 22.3 +/- 18.5, LOS: 99.1 %, DrawRatio: 26.4 %
1000 of 1000 games finished.

It seems the fix does help in specific positions where the engine could miss a mate due to getting into qsearch when in check, but overall, strength at hyperspeed time controls decreased. (I've verified a 50-50 balance with Rustic Alpha 1 playing 1000 games against a copy of itself. Alpha 1 came out with +7 ELO and a margin of +/- 20 ELO, so they can be considered equal.)

Tonight I'll try a longer time control of 1m+0.6s per game for 500 games between Alpha 1 and the fixed version, as that is the time control I'm going to use for testing. I'll also have a look into the default opening book of Cute Chess, as I saw a few double games...

[unserializable]

It seems I already found the problem.

Cool, sounds like the reason indeed has been found. Though it is also bit sad , if bugfixes continue at such pace, soon Monchester 1.0 will not be able to beat Rustic under any circumstances. I will also put it to play some more games for the night before the code gets updated .

Rustic's upcoming preference for Bd6-e5 bishops' activity on open diagonal instead of bishop blocked by own pawns seems like true grandmaster move.

EDIT: wow, from the statistics in your last post, Rustic really loves the black pieces