Progress on Rustic

[mvanthoor]

That would be more reasonable than trying to fight the language because race conditions like that are exactly what the borrow checker is designed to reject. Even if you find a suitable loophole in the language right now, expect that to be closed later because that's the point of Rust. Concurrent TT access without mutex is inherently "unsafe".

Yes; you either use Arc<Mutex<T>>, or you go unsafe. There is no in-between road and be "a little bit safe".

What's more, it's even platform dependent what can work because e.g. atomicity is different on 32 bit vs. 64 bit, and memory ordering is different on x86 vs. ARM so that the latter may require some additional memory barriers.

I'll see it when I get there. Right now I'm using Arc<Mutex<T>> because I want the engine to have safe code as much as possible until it just isn't practical anymore with regard to speed. Now it's not really a problem because there's only one search thread. (And yes, the Mutex does obviously slow the TT down, but because the TT is in the main thread and the search is in its own thread, that can't be helped.) I use unsafe to swap move structs in memory, and to create an unitialized move list; not doing this tanks the engine's speed by something like 70%, so using unsafe there is essential. (Fortunately in these cases, there's only one line of unsafe code, and it's very obvious what it does and why.)

The only 32-bit version of Rustic is the Raspberry Pi ARM-version, because the 64-bit OS is (as far as I know) still experimental.

I wonder if PicoChess, which is based on a _very_ old version of PyChess, will even run on Bullseye. Support for Python 2 has been dropped, PicoChess is incompatible with newer versions of PyChess, while the PyChess version it uses is incompatible with Python 3. Completely within the Python tradition of "Let's break the world like we see fit, and here's my middle finger."

I.e.... the only reason for me to compile a 32-bit version of Rustic is because I want it to run on the Raspberry Pi under PicoChess. I built my own image (on Buster) which I can use indefinitely until I get around to try and write a proper PicoChess replacement in Rust that _WON'T_ break on every other update of the language.

[mvanthoor]

Finally some update to post after about 4 months

Even though nothing much exciting has been going on, Rustic 4 has been undergoing a large refactoring effort to make the code more idiomatic Rust and less C-like in many places. Some of the libraries Rustic depends upon have been upgraded to the latest version. Steps have been made in implementing the last bits of the XBoard-protocol. (Which really is NOT fun because so many GUI's I've tried are buggy... ) This evening I implemented mate score reporting. It seems to work in XBoard, but it doesn't in Arena. (Arena also doesn't seem to support the extended thinking protocol.)

These things are still to be done before XBoard is completely done:
- Implement GameTime control (Rustic has been able to play games in MoveTime and Depth Mode for at least half a year)
- Implement "offer draw" just before a position with insufficient material or 50-move rule arises. (Now Rustic just immediately sends "result 1/2-1/2" and stops playing, but i think this is incorrect.)

To be honest, I think this is the first and last time I implement XBoard in an engine. Maybe I'll implement the other end in a GUI someday, but that will be it. Not because of the protocol itself per se, but because so many different (often conflicting) documents are floating around online, and because the GUI's I tested all have problems with some part of the protocol; except maybe for XBoard itself. Compared to the UCI protocol, this thing was exceedingly hard to get right.

Next steps after the last part of the XBoard protocol is done:
- Finish the refactoring
- Fix a bug in the Makefile (it outputs an error on Linux when not compiling on an RPi 4)
- Implement the Texel tuner
- Actually tune the evaluation

At that point Rustic 4 would finally be done. New things:
- Massive improvement of the code base
- XBoard implemented
- Tapered Evaluation using its own tuner (but Zurichess data)
- Estimated +300 Elo improvement over Rustic Alpha 3.

I'm planning to put Rustic's chess logic such as the board representation, move generator, etc... into a library called "librustic", so it can be used by other programs such as a GUI, or the PicoChess replacement I'm planning to someday write. These programs need things like a move generator and their own board to be able to validate input, and I'm NOT seeing myself write that all over again. (And certainly not in Javascript, should I choose to write the GUI in Electron+Frontend framework.)

So there has indeed been some progress.

Next up... implement MoveTime handling for XBoard.

[lithander]

Finally some update to post after about 4 months

Welcome back! I look forward to finally seeing Rustic 4!

To be honest, I think this is the first and last time I implement XBoard in an engine.

Why? Now that you have it working you can copy everything over to a new engine! At least that's what I did with Leorik. The whole engine part around the search is copy/pasted from MinimalChess. Took 2h top.

[mvanthoor]

Why? Now that you have it working you can copy everything over to a new engine! At least that's what I did with Leorik. The whole engine part around the search is copy/pasted from MinimalChess. Took 2h top.

Several reasons:

1. I'm not going to write another engine in Rust. If I do another engine, it will probably be in a Pascal-like language (FreePascal, Ada, etc...)
2. I don't see the need for the XBoard-protocol anymore, TBH. Arena is buggy with regard to support (or doesn't support some of the later features at all), other GUI's seem to do whatever the *** they please (only XBoard itself seems to implement XBoard according to the specs and I really don't like that GUI), and basically every new engine, with a small exception here or there is UCI only.
3. I implemented XBoard because I'm a completionist, and because I wanted an example in Rustic on how to load a different subsystem depending on the command-line parameters given.
4. I'm not intending to do anything with variants; and IF I ever do a variant, it will use the normal chess pieces on an 8x8 board, so I can actually play it with my DGT-board. UCI is enough for that. And if need be, I'll adapt / extend UCI for my own needs.

[jdart]

the GUI's I tested all have problems with some part of the protocol

Several GUIs also have interesting interpretations of or deviations from the UCI spec.

[mvanthoor]

the GUI's I tested all have problems with some part of the protocol

Several GUIs also have interesting interpretations of or deviations from the UCI spec.

Could you name one? Up until now, I've had no problems whatsoever in CuteChess, Arena, Fritz 11, Chessbase 12 (OK, they're both quite old, but still servicable enough for what I want to do), and Banksia. I've also tried Tarrasch and Shredder when I was still on Windows and that also seems to work correctly. I actually own an older version of Shredder Light because of the GUI. Maybe I should install the Linux version and see what it does.

[jdart]

For example, last I checked some GUIs do not send "ucinewgame."

[mvanthoor]

For example, last I checked some GUIs do not send "ucinewgame."

That could be problematic if an engine filters the last move from the incoming position command and executes this on the board; a new game could be started without the engine noticing this. You'd have to make an extra provision for it. In my case, the engine just sets up the board from the start (at least at the moment), so it's never an issue AFAIK. Never noticed any problem with this in every GUI I tested.

=====

With regard to refactoring: it's going well. Almost all of the custom ".as_string()" functions have been replaced. When I started this I didn't know a lot about Traits (its version of interfaces) in Rust. However, when you implement the Display trait, you get ".to_string()" for free, and you can just do "println!("{}", your_stuff);". Only two more to go: for the board itself, and the castling permissions. The latter have to wrapped into a new type to be able to implement Display, and because that type isn't a straight-up integer, I'll also have to implement the "Bit" family of traits (BitAnd, BitOr, etc..). This looks a lot like operator overloading in C++. To be able to implement Display for the board, I first need Display for the castling permissions.

Then, everything is as it should be in Rust, with all the C-isms removed for Rustic 4. One improvement could be to actually return Result<T, E> when reading a FEN-string, instead of bool, to indicate succes or failure. In Rust, Result<T,E> is the standard / idiom if you're doing something that can fail. The last improvement would be to create a new type for both Piece and Square, so they can't accidentally be mixed up when calling functions, but I don't know if I'm going to do this for version 4. This would be quite a bit of work. (But it would make the engine much more type safe, which is good if I want to turn its chess logic into a library in the future.)

[j.t.]

For example, last I checked some GUIs do not send "ucinewgame."

That could be problematic if an engine filters the last move from the incoming position command and executes this on the board; a new game could be started without the engine noticing this. You'd have to make an extra provision for it. In my case, the engine just sets up the board from the start (at least at the moment), so it's never an issue AFAIK. Never noticed any problem with this in every GUI I tested.

It could also cause reproducing issues, if you rely on clearing the hash table between games.

[mvanthoor]

Yesterday I have finally finished the (massive...) refactoring process and started the implementation of the last two XBoard things: handling of official FIDE-rule draws by insufficient material (which are _different_ from draws with insufficient material that can't force mate... I never knew that, TBH), and GameTime time management.

With regard to the draw rules: In practice, K+B vs K+N is a draw. But, it is a practical draw. A player can _NOT_ claim draw by insufficient material, because FIDE rules specifically state:

"The game is drawn if no mate can be achieved by any possible series of legal moves."

Therefore K+B vs. K+N is not draw, because black can achieve a mate through legal moves:

[fen]8/8/8/8/8/6nk/8/6BK w - - 0 1[/fen]

Yes, white has to actually assist in the mate by blocking his king with the bishop, so black can then deliver a checkmate... but it _IS_ a checkmate achievable through legal moves, so this is _NOT_ an official draw by insufficient material. It would have been, if the FIDE rule had stated "if a draw cannot be _forced_ by any possible series of legal moves."

I'll have to read this again tonight, but I also seem to remember that if you claim an inappropriate draw in XBoard, the GUI can forfeit your game. Therefore I'm only going to "offer draw" (IIRC, this is actually the way how it works with the fifty move rule: offer draw before making the move that reaches the position.) It will either be a draw because the other engine accepts (mine doesn't accept draws), the user interface adjudicates the game because I offered a draw, or the game will go on and run into the 50-move rule in the end.

To facilitate this i had to change my "draw_by_insufficient_material()" function into "sufficient_material_to_force_checkmate()", and add a function "draw_by_insufficient_material_rule()". It is this kind of stuff that makes XBoard so laborious to implement. You're trying to write a chess engine that plays chess as if its a human, including resigning, claiming and negotiating for draws. (Some of these functions can be disabled.) I had to add quite a bit of functionality to Rustic (including a state machine, and the capability to stop/start analysis on a position change, and that sort of thing) to _properly_ support the XBoard protocol.

Compared to UCI, it's massive:

Code: Select all

UCI Protocol: 392 LOC
XBoard Protocol: 543 LOC

UCI Handler: 93 LOC
XBoard Handler: 245 LOC

And then I'm not even including the mentioned functionality in the engine needed to support XBoard. This took way too much time and too much testing. In the end, I like the much smaller UCI-protocol better, including the philosophy of the omni-present arbiter: you just play, until the arbiter (GUI) says that the game is over. (With you as the user setting parameters for this, if desired).

After these two parts are done I can _finally_ start on the tuner, and that would be the last part of Rustic 4. Then I can _finally_ drop Alpha from the name, and implement Null Move and Static Null move for version 5.