Myracle GUI

[SSTATHIS]

just tried a small tournament at 8+0.08" and looking at the anomaly log for Donna, they seem like legit time losses,

Code: Select all

65423042> go wtime 286 btime 1736 winc 80 binc 80
65423201< info depth 1 currmove e2d1 currmovenumber 1
65423219< info depth 1 currmove e2e1 currmovenumber 2
65423244< info depth 1 currmove e2f1 currmovenumber 3
.
.
.
65759455< info depth 15 score cp -1261 nodes 1640643 nps 4897441 hashfull 18 time 335 pv e2d2 f4g4 d2e3 g4h4 e3d2 h4h3 d2e3 h3g4 e3e4 d3d2 e4e3 d2d1q e3e4 d1f3 e4e5 f3f5 e5d6 f5f2
65759474< info nodes 1640643 time 335
65759476< bestmove e2d2

from the log we can see that it took ~336 milliseconds from go to bestmove, but the go command says white has 286msec to think

it's possible to increase the timeout margin in Time control settings for that particular tc, like letting the engine overshoot more than the default 50msec, up to say 10 seconds

another possibility is to let the author of the engine know and wait for a fix

Thanks Martin!

[mar]

build 137 is up:
- fix: pipe character can now be input easily on spanish keyboard
- fix: engine unload task now actually waits for the engines to be unloaded instead of being returned to the scheduler instantly;
also engine pair + task are grouped to avoid violating the core budget
the result is that now there can be a small delay before a new game starts
- tournament hard stop and abort (and quit while tournaments are running) now use async unload too so all should be considerably faster now

[SSTATHIS]

With ponder ON (time is 3m+2s):
Lynx 1.11.0 many disconnects
slow chess 2.4 10% disconnects
Ethereal 14.31 HCE many disconnects
Chessnix 1.4 some forfeits
Houdini 6.0.3 Pro some disconnects and 1 time illegal move (in 12 games)

[mar]

unfortunately I don't have Houdini 6 pro so can't test (can't even get it anymore since it violated the Stockfish license IIRC)
perhaps if you could provide the .anomaly.log for the Houdini games so that I could investigate?

a quick observation for the other programs (only tried ponder on for hyperbullet):
- Lynx seems it can't always handle a stop quickly after go ponder (btw Cheng has a similar problem that I should fix at some point too)
- didn't observe any problems here with Slow chess 2.4
- only have Ethereal 13.75 here - the same problem as chessbrain, cannot handle immediate stop after go ponder
- chessnix 1.4: (note that the engine doesn't support pondering) - investigating the time forfeits, in all of them were the engine didn't output anything after go so they looked like legit timeouts
what's interesting is that with ponder off, I didn't observe any timeouts. the GUI will not ask an UCI engine to ponder if it doesn't support pondering, but the opponent pondering can change the pace of the game (opponent getting ponderhits now and then), again resulting in fast position/go after bestmove

a general note: it's impossible to check all versions of all engines under all conditions, but it's always possible to check the anomaly log
(*.anomaly.log in the tournaments folder)

sometimes the engines also just quits if e.g. NNUE paths are configured improperly, all this can be seen in the anomaly log, assuming the engine uses info string to let the GUI know that there was a problem

obviously the recent tests show that there are many engines out there that can't handle immediate ponderhit or stop after go, so I wonder what to do.

I'm not a fan of hacking the GUI to hide engine bugs, but this seems a prevalent problem (including my own engine),
I may have to add an option for that, I'm not sure if that is the right thing to do. would also have to run many more tests with that...

[SSTATHIS]

unfortunately I don't have Houdini 6 pro so can't test (can't even get it anymore since it violated the Stockfish license IIRC)
perhaps if you could provide the .anomaly.log for the Houdini games so that I could investigate?

a quick observation for the other programs (only tried ponder on for hyperbullet):
- Lynx seems it can't always handle a stop quickly after go ponder (btw Cheng has a similar problem that I should fix at some point too)
- didn't observe any problems here with Slow chess 2.4
- only have Ethereal 13.75 here - the same problem as chessbrain, cannot handle immediate stop after go ponder
- chessnix 1.4: (note that the engine doesn't support pondering) - investigating the time forfeits, in all of them were the engine didn't output anything after go so they looked like legit timeouts
what's interesting is that with ponder off, I didn't observe any timeouts. the GUI will not ask an UCI engine to ponder if it doesn't support pondering, but the opponent pondering can change the pace of the game (opponent getting ponderhits now and then), again resulting in fast position/go after bestmove

a general note: it's impossible to check all versions of all engines under all conditions, but it's always possible to check the anomaly log
(*.anomaly.log in the tournaments folder)

sometimes the engines also just quits if e.g. NNUE paths are configured improperly, all this can be seen in the anomaly log, assuming the engine uses info string to let the GUI know that there was a problem

obviously the recent tests show that there are many engines out there that can't handle immediate ponderhit or stop after go, so I wonder what to do.

I'm not a fan of hacking the GUI to hide engine bugs, but this seems a prevalent problem (including my own engine),
I may have to add an option for that, I'm not sure if that is the right thing to do. would also have to run many more tests with that...

I send you a pm

[mar]

build 138 is up:
- tournament limit is 10 million games per tournament now, number of max cycles increased to 10m too
- faster anomaly schedule filter with large schedule
- faster new tournament dialog schedule count preview (doesn't generate full schedule pairings, only counts)
- fix: update schedule after clear in new tournament dialog
- fix a rare GUI freeze bug when aborting a tour

note: had to update localizations for this build so apart from updating the binary it's necessary to update gui_data.zip too

[mar]

I send you a pm

got it, thanks

looking at the "disconnects" for Houdini, it again didn't handle ponderhit quickly after go ponder
furthermore, investigating the illegal move:

Code: Select all

5321411781> position startpos moves d2d4 g8f6 c2c4 e7e6 g1f3 b7b6 a2a3 c8b7 b1c3 d7d5 c4d5 f6d5
5321411784> go wtime 160342 btime 166681 winc 2000 binc 2000
5321530621< info multipv 1 depth 15 seldepth 31 score cp 43 time 120 nodes 371252 nps 3093000 tbhits 0 pv e2e3 b8d7 f1d3 d5c3 b2c3 f8e7 e3e4 e8g8 d1e2 c7c5 c1f4 b6b5 e1g1 c5c4 d3c2 d7b6 h2h3 h7h6 f4h2
5321701865< info multipv 1 depth 16 seldepth 35 score cp 32 time 291 nodes 834020 nps 2866000 tbhits 0 pv e2e3 d5c3 b2c3 f8e7 f1b5 c7c6 b5d3 e8g8 e1g1 c6c5 e3e4 b8c6 c1e3 c6a5 e3f4 a8c8 f1e1 c5c4 d3c2
.
.
.
5325150029< info depth 21
5325412844< info time 4001 nodes 10538227 nps 2633000 tbhits 0 hashfull 252
5325608469< info multipv 1 depth 21 seldepth 37 score cp 28 lowerbound time 4198 nodes 11051166 nps 2632000 tbhits 0 hashfull 261 pv d1c2
5325608524< info multipv 1 depth 21 seldepth 37 score cp 28 lowerbound time 4198 nodes 11051166 nps 2632000 tbhits 0 hashfull 261 pv d1c2
5325608550< bestmove d1c2 ponder d5c3
5325608595> position startpos moves d2d4 g8f6 c2c4 e7e6 g1f3 b7b6 a2a3 c8b7 b1c3 d7d5 c4d5 f6d5 d1c2 d5c3
5325608597> go wtime 158145 btime 166681 winc 2000 binc 2000 ponder
5325608722< info string Search running, command ignored

that's a bit odd - how is "search running" if it sent bestmove before, then it output b2b4 which was illegal in the position it was supposed to search that it ignored.

so again, I see nothing GUI-related in the logs... but I'll definitely think a bit more about how to handle pondering to accomodate for buggy programs, because they never expected commands to be sent this quickly...

[mar]

Code: Select all

# Name                        Points           %     Elo     +/-  W-L-D                                   1     2     3     4     5     6     7     8      
-----------------------------------------------------------------------------------------------------------------------------------------------------------
1 Stockfish 19                233314.5/250600  93.1  +452.1  2.0  +219476-3447=27677     [12 time losses]       68.3% 91.5% 94.6% 97.9% 99.6% 99.9% 100.0% 
2 Stockfish 18                215045.5/250600  85.8  +312.7  1.7  +198804-19313=32483     [2 time losses] 31.7%       84.1% 90.4% 95.8% 98.9% 99.8% 99.9%  
3 Cheng 4.49 (release avx2)   172674.0/250600  68.9  +138.2  1.4  +159253-64505=26842     [3 time losses] 8.5%  15.9%       70.8% 89.5% 98.2% 99.5% 99.9%  
4 Caligula 8.0                145984.5/250600  58.3   +57.9  1.3  +132275-90906=27419   [174 time losses] 5.4%  9.6%  29.2%       74.9% 91.2% 98.1% 99.4%  
5 isa_4_5_avx2                113160.5/250600  45.2   -33.8  1.3  +102946-127225=20429                    2.1%  4.2%  10.5% 25.1%       79.9% 94.9% 99.3%  
6 CT800_V1.46_x64              75350.0/250600  30.1  -146.6  1.4  +70330-170230=10040     [2 time losses] 0.4%  1.1%  1.8%  8.8%  20.1%       81.9% 96.3%  
7 Berserk 14                   36410.0/250600  14.5  -307.8  1.9  +33439-211219=5942                      0.1%  0.2%  0.5%  1.9%  5.1%  18.1%       75.9%  
8 Berserk 4.7.0                10461.0/250600   4.2  -544.4  3.3  +9041-238719=2840                       0.0%  0.1%  0.1%  0.6%  0.7%  3.7%  24.1%        

1002400/1002400 games, 8% draws, time control:1, book:avt-2026b.pgn, event: cool test

just for fun, nevermind the results or anything, since this is was 1 second per game sudden death with 0 timeout margin garbage time control.
I've read some rumors how "GUIs can't handle many games" - this is prove them wrong. 1 million games, peak GUI memory usage 270MB, no problems. and I bet many other GUIs can handle many games as well, just don't fall for what some con artists claim

[mar]

build 139 is up:
- fix anomaly filter not updating properly in some cases
- elo progression graph is limited to 16k data points (fixes sluggishness with large schedules)
- added new "Slow pondering (UCI)" option (=hack in tournament settings; this is a global flag and off by default
it "fixes" pondering issues for some engines that can't handle fast commands after go,
chessbrain however still won't work - checked and doesn't work with ponder on in cutechess either so I won't bother with it anymore

note: again the localizations have changed a bit so apart from binary it's necessary to copy gui_data.zip as well

[mar]

build 140 is up:
- labels that don't fit after horizontal stretch try to scale a bit more overall; this should make especially game overview fit more text
- cosmetic fix: material info is translated properly when switching languages
- fix hard stop/abort/app exit potential stability problems