List of bugfree, opensource Linux and MacOSX engines

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: List of bugfree, opensource Linux and MacOSX engines

Post by hgm »

Ras wrote: Thu Nov 12, 2020 6:51 pmThat might even be related to the time problem if the engine forfeits and Cutechess tries to ping it, to which the engine doesn't answer because it doesn't evaluate input during search.
OK, so this is just a misleading error message, and doesn't mean the engine has crashed. Just that it forfeited. So in principle it should be solvable by implementing a cold-turkey timeout that aborts the search.

It will be hard to do this in an elegant way, for a microMax-type design. (Which counts on the search itself for performing the game-level move by returning from the root before UnMake when that moves comes up.) I guess I could just save the best move in a global in the code section that prints the PV in the root (which gets executed when the root alpha increases). And after the search returns in the main program with the 'abort flag' set, add some code that would perform the thus saved move as if it was an input move. (The abort flag already exists for the purpose of terminating an analysis search.)

@Guenther - 2016 is pretty long ago. I don't remember it, and I cannot find any executable of that date on the computer I am on now. Typically I do use an extra number behind the letter to indicate a change in the package that does not affect the executable in an essential way. In my source repository I see 3 commits between the official version 5.0b and Sept 2016, but one of those was just the update of the manual, another one fixes a problem that only existed on some Linux distros with an unusual C compiler, and the third keeps track of some info on the position during search that is never used (but this might cause a minuscule slowdown).

These patches could be in 5.0b3, but it is more likely that 5.0b was compiled from the 5.0b source I had on Windows, with just the version number changed. And that this change was made because I updated the fmax.ini file in the package, by adding definitions for 2 new variants ('apothecary chess'). This would not affect any of the other variants.
User avatar
Ras
Posts: 2697
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: List of bugfree, opensource Linux and MacOSX engines

Post by Ras »

hgm wrote: Thu Nov 12, 2020 9:54 pmOK, so this is just a misleading error message, and doesn't mean the engine has crashed. Just that it forfeited.
I'm not sure - it might be what happens, but I didn't check the source code thoroughly to draw that conclusion. I just grep'ed for connection stall and found that comment.
Rasmus Althoff
https://www.ct800.net
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: List of bugfree, opensource Linux and MacOSX engines

Post by Guenther »

hgm wrote: Thu Nov 12, 2020 9:54 pm

@Guenther - 2016 is pretty long ago. I don't remember it, and I cannot find any executable of that date on the computer I am on now. Typically I do use an extra number behind the letter to indicate a change in the package that does not affect the executable in an essential way. In my source repository I see 3 commits between the official version 5.0b and Sept 2016, but one of those was just the update of the manual, another one fixes a problem that only existed on some Linux distros with an unusual C compiler, and the third keeps track of some info on the position during search that is never used (but this might cause a minuscule slowdown).

These patches could be in 5.0b3, but it is more likely that 5.0b was compiled from the 5.0b source I had on Windows, with just the version number changed. And that this change was made because I updated the fmax.ini file in the package, by adding definitions for 2 new variants ('apothecary chess'). This would not affect any of the other variants.
Thanks HG.

I also looked at the commit history and like you say there is practically no change for 'chess' in fmax since this Sept. 2016 'release'.
So there is no need to update from that 5.0b3 version for 'chess' at least.
https://rwbc-chess.de

[Trolls n'existent pas...]
User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: List of bugfree, opensource Linux and MacOSX engines

Post by hgm »

I have just pushed a new patch for Fairy-Max to my on-line repository. This should rigorously prevent any time losses, as it now will abort the search whenever it runs out of time. I set this 'cold-turkey timeout' at 3 times the nominal move time, which is now calculated as timeLeft/(movesLeft+2). The +2 in the denominator makes sure the factor 3 overrun will not cause flagging even when movesLeft = 1. And after the calculation I subtract 32 msec to allow for 2 clock ticks (which happen every 16ms on most systems). I tested it by artificially setting a very short cold-turkey timeout in a long TC game, so that every move gets aborted, and this worked fine. It just plays the last bestMove it found, (which could be from the aborted iteration, or the preceding one), after the abort.

If it forfeits now it can only be due to a genuine crash. But I would be very surprised if Fairy-Max could suffer from those. It seems way too smal to hide any bugs.
User avatar
mvanthoor
Posts: 1784
Joined: Wed Jul 03, 2019 4:42 pm
Location: Netherlands
Full name: Marcel Vanthoor

Re: List of bugfree, opensource Linux and MacOSX engines

Post by mvanthoor »

OliverBr wrote: Thu Nov 12, 2020 6:25 pm
mvanthoor wrote: Thu Nov 12, 2020 4:37 pm against Pulse 1.7.2 it can draw games it should have lost because Pulse has a bug in its repetition detection)
Oh, I like Pulse. Pulse 2.0.0 ist quite nice, but has a bug that it crashes, when any player has exactly time == 0.
Pulse 1.7.2 is OK, but for that bug regarding search repetitions.
Another weak but stable engine (at least on Windows) is the Tarrasch Toy Engine, from the Tarrasch GUI. (CCRL Blitz 1489 Elo).
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: List of bugfree, opensource Linux and MacOSX engines

Post by Guenther »

mvanthoor wrote: Fri Nov 13, 2020 1:50 pm
OliverBr wrote: Thu Nov 12, 2020 6:25 pm
mvanthoor wrote: Thu Nov 12, 2020 4:37 pm against Pulse 1.7.2 it can draw games it should have lost because Pulse has a bug in its repetition detection)
Oh, I like Pulse. Pulse 2.0.0 ist quite nice, but has a bug that it crashes, when any player has exactly time == 0.
Pulse 1.7.2 is OK, but for that bug regarding search repetitions.
Another weak but stable engine (at least on Windows) is the Tarrasch Toy Engine, from the Tarrasch GUI. (CCRL Blitz 1489 Elo).
Have you tried Pigeon or CDrill too?
(I guess you have also checked my XB/UCI chronology for entries rating classification C2-C3, which would be rougly 1500-1900 CCRL)
https://rwbc-chess.de

[Trolls n'existent pas...]
User avatar
mvanthoor
Posts: 1784
Joined: Wed Jul 03, 2019 4:42 pm
Location: Netherlands
Full name: Marcel Vanthoor

Re: List of bugfree, opensource Linux and MacOSX engines

Post by mvanthoor »

Guenther wrote: Fri Nov 13, 2020 2:25 pm Have you tried Pigeon or CDrill too?
(I guess you have also checked my XB/UCI chronology for entries rating classification C2-C3, which would be rougly 1500-1900 CCRL)
I haven't checked your site yet. I'll have a look at it.

I do have Pigeon 1.5.1 in my pool, and Cdrill 1800 has been installed for testing. I've been testing engines one-on-one against my own for something like 20-30 games to see if they can complete a match without issues. After I have 10-15 of them, I'll run 200 games against each engine. (Basically a gauntlet, split into engine matches of 200 games, because CuteChess can't stop tournaments.)

The one thing I'm still a bit worried about with regard to rating is under or over-performance.

As said, against Pulse 1.7.2, Rustic draws many games it should have lost, because if in a bad position, Rustic obviously goes for a repetition, which Pulse can't detect accurately. So either Rustic is over-performing, or Pulse is under-performing. The other case is TSCP: if TSCP can't create a connected past pawn, it'll probably draw or lose against Rustic. If it can create one or more passed pawns, it'll probably win. Because creating and pushing passed pawns is TSCP's specialty, it mostly succeeds in doing this, defeating Rustic, because my engine is too late to notice the threat and stop the pawn. (Which, if it HAD noticed it, it could have stopped it and drawn the game.)
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL
User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: List of bugfree, opensource Linux and MacOSX engines

Post by hgm »

This is why you test against a number of engines; they each have their own 'secret weapons', so most of your weaknesses will be exposed in the test. If you would only test against TSCP you would put way too much emphasis on passed Pawns, ignoring King Safety.

In my experience an engine is much like a boat. As long as it has holes under the water line, it will sink. Plugging some of the holes will not do much good; it will still sink. It just takes longer. When you finally plug the last hole, the rating will surge up.
OliverBr
Posts: 813
Joined: Tue Dec 18, 2007 9:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch

Re: List of bugfree, opensource Linux and MacOSX engines

Post by OliverBr »

Roland Chastain wrote: Wed Nov 11, 2020 12:32 pm Hello! I fixed the name issue for Hippocampe, and the issue of the missing promotion suffix. IIRC there is also a problem of illegal castling move (I am not sure, but I believe that the program doesn't check that the castling is not under attack): I will try to fix it later.

hippocampe.zip

Regards.

Roland
Hi! The addition with the name is great and works.
Still there is the castling issue..:

Code: Select all

Finished game 16 (OliPow 2.2.3 vs Hippocampe 0.4.2.0.1): 1-0 {Black makes an illegal move: e8g8}
Score of Hippocampe 0.4.2.0.1 vs OliPow 2.2.3: 0 - 1 - 0  [0.000] 1
Started game 33 of 1000 (Hippocampe 0.4.2.0.1 vs OliPow 2.2.3)
Finished game 21 (Hippocampe 0.4.2.0.1 vs OliPow 2.2.3): 0-1 {Black mates}
Score of Hippocampe 0.4.2.0.1 vs OliPow 2.2.3: 0 - 2 - 0  [0.000] 2
Started game 34 of 1000 (OliPow 2.2.3 vs Hippocampe 0.4.2.0.1)
Finished game 9 (Hippocampe 0.4.2.0.1 vs OliPow 2.2.3): 0-1 {White makes an illegal move: e1g1}
Score of Hippocampe 0.4.2.0.1 vs OliPow 2.2.3: 0 - 3 - 0  [0.000] 3
:)
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink
Terje
Posts: 347
Joined: Tue Nov 19, 2019 4:34 am
Location: https://github.com/TerjeKir/weiss
Full name: Terje Kirstihagen

Re: List of bugfree, opensource Linux and MacOSX engines

Post by Terje »

I like that this list is getting buggy/non-conforming programs fixed, good idea :)