cutechess-cli 0.8.0 released

[Dan Cooper]

1) At very fast tc, pretty much anything faster than 60 seconds, cutechess seems to instantiate too many concurrent games. I usually run with concurrency = 8, and after a few rounds I'll notice my CPU spike to 99% and there are 10+ games running. Then it doesn't start new games after the count gets under 8. It waits until all current games finish before starting another 8. I don't notice this type of behavior when running longer tc, such as 3M+2. I constantly see 8 instances running until there are less than 8 games remaining in the tournament.

Thanks for the report. Which OS do you use? And have you been able to reproduce this bug with concurrency values below 8? And how long (how many games) do you usually have to wait before you see this bug? I should be able to find out from analyzing the code how this could happen but it would help if the bug was easily reproducible.

I opened an issue about this at Github: https://github.com/cutechess/cutechess/issues/91
Unfortunately I could not reproduce it and could not figure out from the code how cutechess-cli could run more than the allowed number of concurrent games. Is it possible that the CPU spike is caused by engines refusing to terminate and continuing to use CPU when they're no longer needed? And it would be great if I could see console output (-debug flag is not needed) of one of these runs. From there it would be easy to see how many games are running at any given point in time.

I'm running on Windows 10. I usually don't have to wait longer than 5-10 minutes to see the problem. It happens when I use lower concurrency values also. I will try to get you some data later today.

[elcabesa]

this happen to mee too with concurrency 2 and tc of 6+0.05, I'll try to post some data

[elcabesa]

I launched a game with concurrency = 2

Code: Select all

C:\Users\elcab_000\Downloads\cutechess-cli-win64\cutechess-cli>cutechess-cli -engine conf="vajolet2 2.3.2" -engine conf="vajolet2 2.2" -recover -repeat  -concurrency 2 -openings file=8moves_v2.pgn -each tc=6+0.05 option.Hash=256 option.Threads=1 -rounds 64000   -ratinginterval 1 -pgnout test.pgn

Code: Select all

Score of vajolet2 2.3.2 vs vajolet2 2.2: 15 - 8 - 33  [0.562] 56
ELO difference: 43.66 +/- 58.60
Started game 58 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)
Finished game 58 (vajolet2 2.2 vs vajolet2 2.3.2): 1/2-1/2 {Draw by 3-fold repetition}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 15 - 8 - 34  [0.561] 57
ELO difference: 42.88 +/- 57.54
Started game 59 of 64000 (vajolet2 2.3.2 vs vajolet2 2.2)
Finished game 57 (vajolet2 2.3.2 vs vajolet2 2.2): 0-1 {Black mates}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 15 - 9 - 34  [0.552] 58
ELO difference: 36.07 +/- 57.85
Started game 60 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)
Finished game 59 (vajolet2 2.3.2 vs vajolet2 2.2): 1-0 {White mates}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 9 - 34  [0.559] 59
ELO difference: 41.42 +/- 58.03
Started game 61 of 64000 (vajolet2 2.3.2 vs vajolet2 2.2)
Finished game 60 (vajolet2 2.2 vs vajolet2 2.3.2): 1-0 {White mates}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 10 - 34  [0.550] 60
ELO difference: 34.86 +/- 58.25
Started game 62 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)
Finished game 61 (vajolet2 2.3.2 vs vajolet2 2.2): 1/2-1/2 {Draw by 3-fold repetition}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 10 - 35  [0.549] 61
ELO difference: 34.28 +/- 57.27
Started game 63 of 64000 (vajolet2 2.3.2 vs vajolet2 2.2)
Finished game 62 (vajolet2 2.2 vs vajolet2 2.3.2): 1/2-1/2 {Draw by 3-fold repetition}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 10 - 36  [0.548] 62
ELO difference: 33.73 +/- 56.32
Started game 64 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)
Finished game 63 (vajolet2 2.3.2 vs vajolet2 2.2): 0-1 {Black mates}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 11 - 36  [0.540] 63
ELO difference: 27.63 +/- 56.53
Started game 65 of 64000 (vajolet2 2.3.2 vs vajolet2 2.2)
Finished game 65 (vajolet2 2.3.2 vs vajolet2 2.2): 1/2-1/2 {Draw by 3-fold repetition}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 11 - 37  [0.539] 64
ELO difference: 27.20 +/- 55.62
Started game 66 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)
Finished game 64 (vajolet2 2.2 vs vajolet2 2.3.2): 1/2-1/2 {Draw by 3-fold repetition}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 16 - 11 - 38  [0.538] 65
ELO difference: 26.78 +/- 54.75
Started game 67 of 64000 (vajolet2 2.3.2 vs vajolet2 2.2)
Finished game 66 (vajolet2 2.2 vs vajolet2 2.3.2): 0-1 {Black mates}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 17 - 11 - 38  [0.545] 66
ELO difference: 31.67 +/- 54.90
Finished game 67 (vajolet2 2.3.2 vs vajolet2 2.2): 1-0 {White mates}
Score of vajolet2 2.3.2 vs vajolet2 2.2: 18 - 11 - 38  [0.552] 67
ELO difference: 36.43 +/- 55.03
Started game 68 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)
Started game 69 of 64000 (vajolet2 2.3.2 vs vajolet2 2.2)
Started game 70 of 64000 (vajolet2 2.2 vs vajolet2 2.3.2)

more the system is loaded and more it's easy to reproduce it. as you can see 3 games are started (games 68,69,70). it happened launching a search in arena ( heavy load for the PC)

[ilari]

more the system is loaded and more it's easy to reproduce it. as you can see 3 games are started (games 68,69,70). it happened launching a search in arena ( heavy load for the PC)

Thanks, that helps a lot! I'm pretty sure I can fix this quite easily now.

[Dan Cooper]

And it would be great if I could see console output (-debug flag is not needed) of one of these runs.

As it turns out, this is the cause of the problem. Remove -debug and games run normally.

[elcabesa]

let me know if you need some other info or test

[ilari]

let me know if you need some other info or test

I think I've fixed the bug. Can you test with this build: https://github.com/cutechess/cutechess/ ... -win64.zip

[elcabesa]

I'm trying cutechess with the same confioguration and with pu at 100%, it seems you solved the problem. I'll let it run a little more

[AndrewGrant]

I could really use a win32 compile, including whatever dlls are needed. I've made my own linux32 compile, but can't seem to make one for windows.

Still have some win32 machines in my testing cluster.

Thanks, Andrew Grant

[Dan Cooper]

I could really use a win32 compile, including whatever dlls are needed. I've made my own linux32 compile, but can't seem to make one for windows.

Still have some win32 machines in my testing cluster.

Thanks, Andrew Grant

https://www.dropbox.com/s/zrz4vfai9ncya ... li.7z?dl=0