List of bugfree, opensource Linux and MacOSX engines

[OliverBr]

I think GiuChess has the same issue as "ChessPuter":

Furthermore GuiChess is creating a huge file. The process wasn't killed (which can happen), but within short time a 600GB log file was created:

Code: Select all

-rw-r--r-- 1 ch ch 611572105216 Nov 10 17:02 xboard_cmnds.log

[OliverBr]

Fairy-Max 4.8 never really crashes, but it can fall into excessively long thinking (like 30 min in a bullet game) at the moment it switches off null move (in the root), and then tries to improve the search depth it finds in the hash entry for the root by 1 ply without null move. This of course makes it forfeit on time. I never bothered to fix that, because it almost exclusively happens in lost positions, so that no Elo would be gained by the fix.

I am not sure anymore if it actually crashed, but it forfeits on time and make illegal moves, this is probably why it didn't make it into the list.

Thank you for the link to Fairy-Max 5, I will test it.

Code: Select all

Finished game 40 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black makes an illegal move: f2f1}
Finished game 54 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black makes an illegal move: e2e1}
Finished game 93 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White makes an illegal move: d7d8}
Finished game 95 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White makes an illegal move: b7b8}
Finished game 181 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White makes an illegal move: h7h8}
Finished game 9 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White loses on time}
Finished game 34 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 66 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 121 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White loses on time}
Finished game 130 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 100 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 132 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 178 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 182 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black loses on time}
Finished game 185 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White loses on time}

EDIT: Ah, sorry, it crashed again after about 500 games and it's reproducible:

Code: Select all

Terminating process of engine Fairy-Max 4.8L(24)
Finished game 496 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black's connection stalls}

cutechess-cli 1.2.0 interrupts the tournament because of it.

Code: Select all

Terminating process of engine Fairy-Max 4.8L(24)
Finished game 496 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black's connection stalls}
Score of Fairy-Max 4.8L vs OliThink 5.9.0: 0 - 512 - 9  [0.009] 521
Finished game 536 (OliThink 5.9.0 vs Fairy-Max 4.8L): * {No result}
Score of Fairy-Max 4.8L vs OliThink 5.9.0: 0 - 512 - 9  [0.009] 521
Finished game 530 (OliThink 5.9.0 vs Fairy-Max 4.8L): * {No result}
Score of Fairy-Max 4.8L vs OliThink 5.9.0: 0 - 512 - 9  [0.009] 521
Finished game 520 (OliThink 5.9.0 vs Fairy-Max 4.8L): * {No result}
.....

[hgm]

EDIT: Ah, sorry, it crashed again after about 500 games and it's reproducible:

Code: Select all

Terminating process of engine Fairy-Max 4.8L(24)
Finished game 496 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black's connection stalls}

You have the game, so I can check?

[Madeleine Birchfield]

Hello
I checked a lot of open source engines, if they met the following criterias:

1) Open source (most are on git anyway)
2) Easy compile with make/cmake on both, Linux and MacOSX, with g++ and clang, telling their name on init.
3) Complete implementation of either UCI or CECP protocol
4) Stable over thousands of games
5) Bugfree in terms of timeouts and illegal moves and draw repitions.

From over 1000 engines I found, only 48 met all the criterias.
This is the list, the third column is the relative strength when playing h2h against OliThink 5.9.0 on very fast time control:

Code: Select all

cinnamon	uci	-690	2.3.3	https://github.com/gekomad/Cinnamon
chesley		xboard	-800	1.2.x	https://github.com/matthewgingell/chesley
claudia		uci	-1000	0.5.1	https://github.com/antoniogarro/Claudia
ct800 		uci 	-480	0.41	https://www.ct800.net/download.htm			(make..sh)
defenchess	uci	+600	2.3	https://github.com/cetincan0/Defenchess
demolito	uci	+450	.	https://github.com/lucasart/Demolito
delocto		uci	+10	0.6	https://github.com/moterink/Delocto
drosophila	uci	-210	1.6	https://github.com/gustafullberg/drosophila		(cmake)
ethereal 	uci	+700	12.55	https://github.com/AndyGrant/Ethereal
floyd		uci	-750	0.9	https://github.com/kervinck/floyd
fridolin3	uci	-200	3.1	https://sites.google.com/site/fridolinchess
fruit		uci	-180	2.1	https://github.com/Warpten/Fruit-2.1
glaurung	uci	-40	2.2	https://github.com/phenri/glaurung
googleplex	uci	-500	1.7	https://gitlab.com/tsoj/Googleplex_Starthinker		(cmake)
greko 		xb+uci	-150	2020.03	http://greko.su/index_en.html
greko-junior	xb+uci	-490	12.9	https://github.com/coderxlsn/greko-junior-chess
gully2		xboard	-860	2.16	https://github.com/borriss/gully
halogen		uci	-15	8	https://github.com/KierenP/Halogen
k2 		xb+uci	-80	0.96	https://github.com/serg-meus/k2
laser		uci	+500	1.8	https://github.com/jeffreyan11/laser-chess-engine
magic 		xb+uci 	-670	2	https://github.com/zzhangb4/MagiC-engine-chess
marvin		xb+uci	-220	5.0.0	https://github.com/bmdanielsson/marvin-chess
minnow		uci	-1250	150301	https://github.com/tm512/minnow
monolith	uci	+220	2	https://github.com/cimarronOST/Monolith
olithink5	xboard		5.9.0	https://github.com/olithink/OliThink		(/oliThink4 4.1.3: -700, /oliThink3 3.0.7: -950)
phalanx		xboard 	-180	XXV	https://github.com/student-t/Phalanx-XXIII
prophet4	xboard	-950	4	https://github.com/jswaff/prophet4
protector	xb+uci	+280	1.9.0	https://github.com/MichaelB7/Protector
purplehaze	xboard	-920	2.1.0	https://github.com/vinc/purplehaze
robocide	uci	-550	.	https://github.com/DanielWhite94/robocide
rodentII	uci	+80	0.9.68	https://github.com/nescitus/Rodent_II
sayuri		uci	-1000	18.5.23	https://github.com/MetalPhaeton/sayuri			(cmake)
sbchess		uci	-1100	.	https://github.com/DoubleEspresso/chess
scorpio 	xb+uci	-130	3.0.9	https://github.com/dshawul/Scorpio
senpai		uci	+40	2.0	https://github.com/B4dT0bi/senpai
snowy		uci	-850	0.2.5	https://github.com/JasonCreighton/snowy
stash		uci	-160	21.1	https://gitlab.com/mhouppin/stash-bot
stockfish 	uci	+920	11	https://github.com/official-stockfish/Stockfish
sungorus	uci	-510	1.4	https://sites.google.com/site/sungorus
taltos		xb+uci	-440	.	https://github.com/GBuella/Taltos
teki		uci	-600	.	https://github.com/Mk-Chan/Teki				(cmake)
texel		uci	+330	1.08	https://github.com/peterosterlund2/texel		(cmake)
togaII 		uci	-70	1.2.1a	https://github.com/zwegner/toga-mp
weiss 		uci	+250	1.1	https://github.com/TerjeKir/weiss
winter 		uci 	+300	0.8.16	https://github.com/rosenthj/Winter
wyldchess 	uci	-160	1.5	https://github.com/Mk-Chan/WyldChess
xiphos 		uci	+620	0.6.1	https://github.com/milostatarevic/xiphos
zevra2		uci	-370	2.1.2	https://github.com/sovaz1997/Zevra2		(/Zevra 1.8.6: -550)

Isn't latest Ethereal version 12.75 instead of 12.55?

[OliverBr]

EDIT: Ah, sorry, it crashed again after about 500 games and it's reproducible:

Code: Select all

Terminating process of engine Fairy-Max 4.8L(24)
Finished game 496 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black's connection stalls}

You have the game, so I can check?

I will start a new tournament and provide the pgn of the next incident.

[OliverBr]

Isn't latest Ethereal version 12.75 instead of 12.55?

The making of this list took a couple of weeks, so some engines may have newer versions.

[OliverBr]

Many engines indeed are not stable or do not handle fast time controls.

I am not clear why Arasan did not make your list? I routinely run tens of thousands of test games with it, commonly on Linux.

Arasan is very stable and bugfree, but unfortunately I am having problems to compile it in MacOSX and clang. I can provide every info/help if you want to change this:

1) make:

Code: Select all

make
clang: error: no input files
expr: syntax error
awk: syntax error at source line 1
 context is
	BEGIN{print CCVER >>>  >= <<< 
awk: illegal statement at source line 1
awk: illegal statement at source line 1
Makefile:186: *** GCC version is : versions <  not supported.  Stop.

2) cmake:

Code: Select all

cmake ..
-- The C compiler identification is AppleClang 10.0.0.10001044
-- The CXX compiler identification is AppleClang 10.0.0.10001044
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Configuring done
CMake Error at CMakeLists.txt:174 (add_executable):
  Cannot find source file:

    /Users/dreamlike/Downloads/Engines/arasan-chess/src/syzygy/src/tbprobe.c

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
  .hpp .hxx .in .txx


CMake Error at CMakeLists.txt:174 (add_executable):
  No SOURCES given to target: makeeco


CMake Error at CMakeLists.txt:160 (add_executable):
  No SOURCES given to target: ecocoder


CMake Error at CMakeLists.txt:146 (add_executable):
  No SOURCES given to target: makebook


CMake Error at CMakeLists.txt:132 (add_executable):
  No SOURCES given to target: playchess


CMake Error at CMakeLists.txt:118 (add_executable):
  No SOURCES given to target: pgnselect


CMake Error at CMakeLists.txt:104 (add_executable):
  No SOURCES given to target: tuner


CMake Error at CMakeLists.txt:89 (add_executable):
  No SOURCES given to target: arasanx-64


CMake Generate step failed.  Build files cannot be regenerated correctly.

[OliverBr]

I wonder what is wrong with Crafty to not be in your list.

I couldn't compile it on MacOSX.

[OliverBr]

Did you consider Shallow Blue 2.0 ? I have run thousands of games without issue.

True, I tested it in Linux and it looks stable. Unfortunately I can't compile it in MacOSX and clang:

Code: Select all

make
c++ -Wall -std=c++11 -O3 -march=native -flto -pthread -fno-exceptions -c -o obj/option.o src/option.cc
In file included from src/option.cc:1:
src/option.h:100:15: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
  std::string _value;
              ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
In file included from src/option.cc:1:
src/option.h:107:15: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
  std::string _type;
              ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
In file included from src/option.cc:1:
src/option.h:112:15: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
  std::string _defaultValue;
              ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
src/option.cc:22:17: error: no member named 'to_string' in namespace 'std'
    _value(std::to_string(value)),
           ~~~~~^
src/option.cc:24:24: error: no member named 'to_string' in namespace 'std'
    _defaultValue(std::to_string(value)),
                  ~~~~~^
src/option.cc:37:21: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
std::string Option::getValue() const {
                    ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
src/option.cc:41:21: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
std::string Option::getType() const {
                    ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
src/option.cc:45:21: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
std::string Option::getDefaultValue() const {
                    ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
src/option.cc:57:35: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
void Option::setValue(std::string value) {
                                  ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
In file included from src/option.cc:1:
In file included from src/option.h:4:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/map:442:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:15:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/iterator:417:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/__functional_base:19:
/Library/Developer/CommandLineTools/usr/include/c++/v1/utility:320:9: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
    _T1 first;
        ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/map:624:16: note: in instantiation of template class 'std::__1::pair<const std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>' requested here
    value_type __cc;
               ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:762:23: note: in instantiation of template class 'std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>' requested here
    __node_value_type __value_;
                      ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:1830:49: note: in instantiation of template class 'std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >, Option>, void *>' requested here
        destroy(static_cast<__node_pointer>(__nd->__left_));
                                                ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:1821:3: note: in instantiation of member function 'std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >, Option>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>,
      std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Option> > >::destroy' requested here
  destroy(__root());
  ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/map:805:28: note: in instantiation of member function 'std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >, Option>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>,
      std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Option> > >::~__tree' requested here
class _LIBCPP_TEMPLATE_VIS map
                           ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
In file included from src/option.cc:1:
In file included from src/option.h:4:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/map:442:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:15:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/iterator:417:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/__functional_base:19:
/Library/Developer/CommandLineTools/usr/include/c++/v1/utility:320:9: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
    _T1 first;
        ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/map:625:21: note: in instantiation of template class 'std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >,
      Option>' requested here
    __nc_value_type __nc;
                    ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:762:23: note: in instantiation of template class 'std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>' requested here
    __node_value_type __value_;
                      ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:1830:49: note: in instantiation of template class 'std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >, Option>, void *>' requested here
        destroy(static_cast<__node_pointer>(__nd->__left_));
                                                ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/__tree:1821:3: note: in instantiation of member function 'std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >, Option>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>,
      std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Option> > >::destroy' requested here
  destroy(__root());
  ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/map:805:28: note: in instantiation of member function 'std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >, Option>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >, Option>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>,
      std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Option> > >::~__tree' requested here
class _LIBCPP_TEMPLATE_VIS map
                           ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
11 errors generated.
make: *** [obj/option.o] Error 1

[OliverBr]

2) Easy compile with make/cmake on both, Linux and MacOSX, with g++ and clang, telling their name on init.

You should have make it clear in your title that you restricted your list to C/C++ engines. People looking at your list may think programs in Rust, D, C#, java, etc. are automatically buggy.

You are right. It is my fault that it doesn't get clear in the title. I have myself an Java engine which isn't buggy.
I am planing to do a second list for all not-C/C++ opensource engines like Java, Rust, Go or Haskell. Still they must run on both, Linux and MacOSX.

Perhaps an administrator can change the title to (easy to compile C/C++ engines)?!