Komodo 10.4 released

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9720
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Komodo 10.4 released

Post by Laskos » Wed Mar 15, 2017 1:12 pm

Laskos wrote:
lkaufman wrote:
A quick test at 12 ply on one thread (714 games) showed no changed games with Variety set to zero. So I suppose it must be due to timing variation, but then the difference between Variety 20 and Variety 0 is way too small, and the difference from Stockfish is implausibly large. I guess I'll have to investigate further.
Hash not cleared at ucinewgame?
I checked for time granularity at longer TC, 60''+0.1''. Komodo 1 thread Var=0 comes at 184/400 from 218/400 at 10''+0.1'', Stockfish at 1/400 (always the same position) from 6/400. So, time granularity has a very moderate effect on Komodo 10.4 1-thread randomization. Not clearing Hash between the games?

User avatar
Nordlandia
Posts: 2510
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Komodo 10.4 released

Post by Nordlandia » Wed Mar 15, 2017 2:56 pm

I bet Komodo 10.4 is capable to beat Houdini 4 Pro from this position ->

K10.4 is black obviously.

[d]r2q1r2/pbpnnpbk/1p1pp1pp/8/2BPPB2/2N2N2/PPPQ1PPP/3RR1K1 w - - 0 11

cma6
Posts: 145
Joined: Thu May 29, 2014 3:58 pm

Komodo 10.4 randomization

Post by cma6 » Wed Mar 15, 2017 2:59 pm

Kai:
What does "randomization" in this context mean? The definition of "Variety" is unclear. Since chess engines are already non-deterministic, e.g., may get different ranked evals on each run, what does Variety > 0 change?
Also, when one measures kN/s at start position, that is after how many seconds?
Thanks, CMA

mjlef
Posts: 1431
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 10.4 randomization

Post by mjlef » Wed Mar 15, 2017 3:45 pm

cma6 wrote:Kai:
What does "randomization" in this context mean? The definition of "Variety" is unclear. Since chess engines are already non-deterministic, e.g., may get different ranked evals on each run, what does Variety > 0 change?
Also, when one measures kN/s at start position, that is after how many seconds?
Thanks, CMA
Variety is described in more detail in the README file. At fixed depth and a single core, Komodo is 100% deterministic with Variety set to 0. That is assuming you restart Komodo between games to ensure all the various tables (Hash, eval, history....) start from the same place. With more than 1 thread, or in timed games, it is not as deterministic, since other things running on the computer rob different amounts of time on some positions, so Komodo has more or less CPU resources, and so can find another move.

For a general nps for one core, you can just start Komodo from the command line and type:

go depth 20<Enter>

look at the last few lines which will look like this:
info depth 20 time 8663 nodes 7956430 score cp 43 lowerbound nps 918369 hashfull 435 tbhits 0 pv e2e4 c7c5 g1f3
info time 8664 nodes 7956430 nps 918263 hashfull 435
bestmove e2e4 ponder c7c5

the number after nps (918369 in the example) is the nodes per second.

To see the nps with more cores type this before the go lin above:

setoption name Threads value x

where x is the number of threads you want to use.

mjlef
Posts: 1431
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Komodo 10.4 released

Post by mjlef » Wed Mar 15, 2017 3:47 pm

Laskos wrote:
Laskos wrote:
lkaufman wrote:
A quick test at 12 ply on one thread (714 games) showed no changed games with Variety set to zero. So I suppose it must be due to timing variation, but then the difference between Variety 20 and Variety 0 is way too small, and the difference from Stockfish is implausibly large. I guess I'll have to investigate further.
Hash not cleared at ucinewgame?
I checked for time granularity at longer TC, 60''+0.1''. Komodo 1 thread Var=0 comes at 184/400 from 218/400 at 10''+0.1'', Stockfish at 1/400 (always the same position) from 6/400. So, time granularity has a very moderate effect on Komodo 10.4 1-thread randomization. Not clearing Hash between the games?
hash clearing is a GUI/tester function. But there are other tables in Komodo not cleared by the Clear Hash function that could change the move choices, like history tables. Since the UCI spec does not specify when these tables need to be reinitialized, best solution is have the tester reload each engine between games. What tester do you use?

User avatar
Laskos
Posts: 9720
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Komodo 10.4 released

Post by Laskos » Wed Mar 15, 2017 4:04 pm

mjlef wrote:
Laskos wrote:
Laskos wrote:
lkaufman wrote:
A quick test at 12 ply on one thread (714 games) showed no changed games with Variety set to zero. So I suppose it must be due to timing variation, but then the difference between Variety 20 and Variety 0 is way too small, and the difference from Stockfish is implausibly large. I guess I'll have to investigate further.
Hash not cleared at ucinewgame?
I checked for time granularity at longer TC, 60''+0.1''. Komodo 1 thread Var=0 comes at 184/400 from 218/400 at 10''+0.1'', Stockfish at 1/400 (always the same position) from 6/400. So, time granularity has a very moderate effect on Komodo 10.4 1-thread randomization. Not clearing Hash between the games?
hash clearing is a GUI/tester function. But there are other tables in Komodo not cleared by the Clear Hash function that could change the move choices, like history tables. Since the UCI spec does not specify when these tables need to be reinitialized, best solution is have the tester reload each engine between games. What tester do you use?
Cutechess-Cli, doesn't reload, in a 2 engine match, the engines are loaded once for entire match.

User avatar
Laskos
Posts: 9720
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Komodo 10.4 randomization

Post by Laskos » Wed Mar 15, 2017 4:08 pm

cma6 wrote:Kai:
Also, when one measures kN/s at start position, that is after how many seconds?
Thanks, CMA
I assume the result was for a reasonable time, say 20-30 seconds with hundreds of MB or 1-2 GB hash. But you have to ask the person who tested Ryzen.

cma6
Posts: 145
Joined: Thu May 29, 2014 3:58 pm

Re: Komodo 10.4 released

Post by cma6 » Wed Mar 15, 2017 5:49 pm

Ron Langeveld wrote:Preliminary results I got are:
About 12.000 kN/s in start position with a Ryzen 1800X (HT)
versus 9.5000 kN/s with a 5960X Intel 8-core desktop (no-HT)

Ron:
Assuming you got these numbers from your GUI, after how many seconds did you take the readings?

Also, when will we see you in the Finals of ICCF WC?

User avatar
Guenther
Posts: 3211
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Komodo 10.4 released

Post by Guenther » Wed Mar 15, 2017 5:50 pm

Laskos wrote:Cutechess-Cli, doesn't reload, in a 2 engine match, the engines are loaded once for entire match.
Doesn't this option work?

Code: Select all

  restart=MODE	
	Set the restart mode to MODE which can be&#58;
			'auto'&#58; the engine decides whether to restart &#40;default&#41;
			'on'&#58; the engine is always restarted between games
			'off'&#58; the engine is never restarted between games
			Setting this option does not prevent engines from being
			restarted between rounds in a tournament featuring more
         than two engines.

User avatar
Laskos
Posts: 9720
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

laskos

Post by Laskos » Wed Mar 15, 2017 6:35 pm

Guenther wrote:
Laskos wrote:Cutechess-Cli, doesn't reload, in a 2 engine match, the engines are loaded once for entire match.
Doesn't this option work?

Code: Select all

  restart=MODE	
	Set the restart mode to MODE which can be&#58;
			'auto'&#58; the engine decides whether to restart &#40;default&#41;
			'on'&#58; the engine is always restarted between games
			'off'&#58; the engine is never restarted between games
			Setting this option does not prevent engines from being
			restarted between rounds in a tournament featuring more
         than two engines.
I actually use restart=off, as it seemed to me faster and more reliable in hyper-fast games.

Post Reply