Updated Dendrogram

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Laskos
Posts: 9323
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Updated Dendrogram

Post by Laskos » Sat Feb 02, 2013 9:08 am

Image

Measure: correlation
Distance on the horizontal axis to the common ancestor - degree of relatedness (smaller - more related).

User avatar
Rebel
Posts: 4542
Joined: Thu Aug 18, 2011 10:04 am

Re: Updated Dendrogram

Post by Rebel » Sat Feb 02, 2013 9:12 am

Ponder hits or similarity tester?

User avatar
Laskos
Posts: 9323
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Updated Dendrogram

Post by Laskos » Sat Feb 02, 2013 9:14 am

Rebel wrote:Ponder hits or similarity tester?
Similarity tester.

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Updated Dendrogram

Post by Adam Hair » Sat Feb 02, 2013 1:49 pm

ChessTiger's distance may be artificially high. It does not return a best move when the similarity tester sends the command. It sends it several hundred milliseconds later.

Shredder does not precisely obey the commands from the similarity tester. There is a lag before it starts thinking and a lag before it sends the best move. Both lags are 15-20 milliseconds. I do not think it has a large effect on the results.

Komodo has a lag before it starts thinking, about 90ms IIRC. It stops exactly when the command is received. That would make it seem more distant than it really is. When corrected for, it becomes a little closer to Rybka 3.

Stockfish does not obey the stop command precisely. It will not send a best move until it reaches depth 10, IIRC.

User avatar
Laskos
Posts: 9323
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Updated Dendrogram

Post by Laskos » Sat Feb 02, 2013 3:58 pm

Adam Hair wrote:ChessTiger's distance may be artificially high. It does not return a best move when the similarity tester sends the command. It sends it several hundred milliseconds later.

Shredder does not precisely obey the commands from the similarity tester. There is a lag before it starts thinking and a lag before it sends the best move. Both lags are 15-20 milliseconds. I do not think it has a large effect on the results.

Komodo has a lag before it starts thinking, about 90ms IIRC. It stops exactly when the command is received. That would make it seem more distant than it really is. When corrected for, it becomes a little closer to Rybka 3.

Stockfish does not obey the stop command precisely. It will not send a best move until it reaches depth 10, IIRC.
How do you correct Komodo? Add 90ms to Komodo time? I used a very rough adjustment for strength: scale 1 for weaker engines, scale 0.1 for super-engines. It's I think enough, because strength dependency is weak. Komodo with scale 0.1 still clustered apparently correctly, and it's only 10ms thinking time. Do you suggest using 100ms for Komodo?

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Updated Dendrogram

Post by Adam Hair » Sat Feb 02, 2013 5:28 pm

I think I found that Komodo grouped a little more closely to Rybka 3 when I increased the time for Komodo. How much closer? I do not remember, but I am certain that it would not make a lot of difference to the dendrogram.

Basically, I was trying to pass on to you ChessTiger does not run the test correctly, and then decided to mention the quirks that some of the other engines have. In all, I do not think the dendrogram would change much if all of the engines obeyed the similarity tester precisely.

Jouni
Posts: 1958
Joined: Wed Mar 08, 2006 7:15 pm

Re: Updated Dendrogram

Post by Jouni » Sat Feb 02, 2013 5:56 pm

How many ORIGINAL top program we really have? May be only 3 :o

1. Rybka + clones (Ippolit, Robbolit, Houdini, Critter, etc..)

2. Stockfish

3. Komodo
Jouni

User avatar
Laskos
Posts: 9323
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Updated Dendrogram

Post by Laskos » Sun Feb 03, 2013 10:07 am

Jouni wrote:How many ORIGINAL top program we really have? May be only 3 :o

1. Rybka + clones (Ippolit, Robbolit, Houdini, Critter, etc..)

2. Stockfish

3. Komodo
Interesting to note the clustering of engines which are claimed to be original around open sources (Fruit 2.1, Strelka 2, Ivanhoe) and around the RE of Houdini.

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: Updated Dendrogram

Post by Don » Sun Feb 03, 2013 12:44 pm

Adam Hair wrote:I think I found that Komodo grouped a little more closely to Rybka 3 when I increased the time for Komodo. How much closer? I do not remember, but I am certain that it would not make a lot of difference to the dendrogram.

Basically, I was trying to pass on to you ChessTiger does not run the test correctly, and then decided to mention the quirks that some of the other engines have. In all, I do not think the dendrogram would change much if all of the engines obeyed the similarity tester precisely.
Older versions of Komodo had a longer delay between thinking time startup. Did you measure this delay on all versions or a specific version?
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Updated Dendrogram

Post by Adam Hair » Sun Feb 03, 2013 1:22 pm

Don wrote:
Adam Hair wrote:I think I found that Komodo grouped a little more closely to Rybka 3 when I increased the time for Komodo. How much closer? I do not remember, but I am certain that it would not make a lot of difference to the dendrogram.

Basically, I was trying to pass on to you ChessTiger does not run the test correctly, and then decided to mention the quirks that some of the other engines have. In all, I do not think the dendrogram would change much if all of the engines obeyed the similarity tester precisely.
Older versions of Komodo had a longer delay between thinking time startup. Did you measure this delay on all versions or a specific version?
I am going by memory, which is not reliable in this case. I would have to go back over the logs I made. The versions would have been Komodo 3 and Komodo 4.

Post Reply