Similarity tests

[Adam Hair]

Hello Adam,

One thing more, you helped a lot regarding sim tool, now it's my turn
A little advise about how to run via sim tool, which you could not test
For example Bison, Ktulu, Chess Tiger etc..
You need to include all their files in sim test folder (during sim test process)

And I expect you can test them successfully too!

Hopes this helps too

Sedat

Have you confirmed that Ktulu, Chess Tiger, and the other engines are correctly obeying the commands sent to them by the sim tool?

During testing the mentioned engines, I could not see any problems...

Btw, have you managed to run them and I wonder what is your results?

Yes, I have run them. I have run hundreds of engines through the similarity test. What I have learned is that not all engines exactly obey the UCI and Winboard command protocols.

This morning, I created logs for Booot 5.2.0, Chess Tiger 2007.1, and Ktulu 8:

Code: Select all

Booot 5.2.0

1412764912.031 GUI->Adapter: ucinewgame
1412764912.031 Adapter->Engine: ucinewgame
1412764912.031 GUI->Adapter: isready
1412764912.031 Adapter->Engine: isready
1412764912.062 Engine->Adapter: readyok
1412764912.062 Adapter->GUI: readyok
1412764912.062 GUI->Adapter: position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 c8e6 d1d2 b8d7 f2f3 f8e7 g2g4 a8c8 h2h4 d7b6 e1c1 d8c7 c1b1 b6c4 f1c4 c7c4 h4h5 b7b5 a2a3 b5b4 a3b4 c4b4 g4g5 f6g8 h1g1 h7h6 g5h6 g7h6 c3d5 b4d2 d1d2 e6d5 d2d5 g8f6 d5a5 f6h5 a5a6 h5f6 b3a5 d6d5 a5c6 d5e4 c6e5 e4f3 e5f3 f6e4
1412764912.062 Adapter->Engine: position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 c8e6 d1d2 b8d7 f2f3 f8e7 g2g4 a8c8 h2h4 d7b6 e1c1 d8c7 c1b1 b6c4 f1c4 c7c4 h4h5 b7b5 a2a3 b5b4 a3b4 c4b4 g4g5 f6g8 h1g1 h7h6 g5h6 g7h6 c3d5 b4d2 d1d2 e6d5 d2d5 g8f6 d5a5 f6h5 a5a6 h5f6 b3a5 d6d5 a5c6 d5e4 c6e5 e4f3 e5f3 f6e4
1412764912.062 GUI->Adapter: go depth 50
1412764912.062 Adapter->Engine: go depth 50
1412764912.140 Engine->Adapter: info depth 7 time 16 nodes 17170 nps 1073000 score cp 121 pv a6h6 h8h6 e3h6 c8d8 f3e5 e4d2 b1c1 
1412764912.140 Adapter->GUI: info depth 7 time 16 nodes 17170 nps 1073000 score cp 121 pv a6h6 h8h6 e3h6 c8d8 f3e5 e4d2 b1c1 
1412764912.140 Engine->Adapter: bestmove a6h6
1412764912.140 Adapter->GUI: bestmove a6h6
1412764912.171 GUI->Adapter: stop
1412764912.171 Adapter->Engine: stop

Chess Tiger 2007.1

1412765028.484 GUI->Adapter: ucinewgame
1412765028.484 Adapter->Engine: ucinewgame
1412765028.484 GUI->Adapter: isready
1412765028.484 Adapter->Engine: isready
1412765028.484 Engine->Adapter: readyok
1412765028.484 Adapter->GUI: readyok
1412765028.484 GUI->Adapter: position startpos moves d2d4 g8f6 g1f3 g7g6 g2g3 f8g7 f1g2 e8g8 e1g1 d7d6 f1e1 b8c6 e2e4 e7e5 c2c3 c8d7
1412765028.484 Adapter->Engine: position startpos moves d2d4 g8f6 g1f3 g7g6 g2g3 f8g7 f1g2 e8g8 e1g1 d7d6 f1e1 b8c6 e2e4 e7e5 c2c3 c8d7
1412765028.484 GUI->Adapter: go depth 50
1412765028.484 Adapter->Engine: go depth 50
1412765028.609 GUI->Adapter: stop
1412765028.609 Adapter->Engine: stop
1412765029.234 Engine->Adapter: info depth 9 time 160 nodes 187872 nps 1174200 score cp 22 pv d4d5 c6e7 f3g5 h7h6 g5f3 c7c6 d5c6 d7c6 b1d2
1412765029.234 Adapter->GUI: info depth 9 time 160 nodes 187872 nps 1174200 score cp 22 pv d4d5 c6e7 f3g5 h7h6 g5f3 c7c6 d5c6 d7c6 b1d2
1412765029.234 Engine->Adapter: info currmove g1h1 currmovenumber 35 depth 9
1412765029.234 Adapter->GUI: info currmove g1h1 currmovenumber 35 depth 9
1412765029.234 Engine->Adapter: info depth 9 time 160 nodes 187872 nps 1174200 score cp 22 pv d4d5 c6e7 f3g5 h7h6 g5f3 c7c6 d5c6 d7c6 b1d2
1412765029.234 Adapter->GUI: info depth 9 time 160 nodes 187872 nps 1174200 score cp 22 pv d4d5 c6e7 f3g5 h7h6 g5f3 c7c6 d5c6 d7c6 b1d2
1412765029.234 Engine->Adapter: bestmove d4d5
1412765029.234 Adapter->GUI: bestmove d4d5

Ktulu 8

1412765245.187 GUI->Adapter: ucinewgame
1412765245.187 Adapter->Engine: ucinewgame
1412765245.187 GUI->Adapter: isready
1412765245.187 Adapter->Engine: isready
1412765245.187 Engine->Adapter: readyok
1412765245.187 Adapter->GUI: readyok
1412765245.187 GUI->Adapter: position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 c8e6 d1d2 b8d7 f2f3 f8e7 g2g4 a8c8 h2h4 d7b6 e1c1 d8c7 c1b1 b6c4 f1c4 c7c4 h4h5 b7b5 a2a3 b5b4 a3b4 c4b4 g4g5 f6g8 h1g1 h7h6 g5h6 g7h6 c3d5 b4d2 d1d2 e6d5 d2d5 g8f6 d5a5 f6h5 a5a6 h5f6 b3a5 d6d5 a5c6 d5e4 c6e5 e4f3 e5f3 f6e4
1412765245.187 Adapter->Engine: position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 c8e6 d1d2 b8d7 f2f3 f8e7 g2g4 a8c8 h2h4 d7b6 e1c1 d8c7 c1b1 b6c4 f1c4 c7c4 h4h5 b7b5 a2a3 b5b4 a3b4 c4b4 g4g5 f6g8 h1g1 h7h6 g5h6 g7h6 c3d5 b4d2 d1d2 e6d5 d2d5 g8f6 d5a5 f6h5 a5a6 h5f6 b3a5 d6d5 a5c6 d5e4 c6e5 e4f3 e5f3 f6e4
1412765245.187 GUI->Adapter: go depth 50
1412765245.187 Adapter->Engine: go depth 50
1412765245.203 Engine->Adapter: info depth 6 score cp 88 time 10 nodes 33195 nps 3319500 pv e3h6 e4c5 a6b6 c5e4 f3e5 c8a8 
1412765245.203 Adapter->GUI: info depth 6 score cp 88 time 10 nodes 33195 nps 3319500 pv e3h6 e4c5 a6b6 c5e4 f3e5 c8a8 
1412765245.203 Engine->Adapter: bestmove e3h6 ponder e4c5
1412765245.203 Adapter->GUI: bestmove e3h6 ponder e4c5
1412765245.296 GUI->Adapter: stop
1412765245.296 Adapter->Engine: stop

As you can see, Booot and Ktulu send their best move before receiving the command to stop, while Chess Tiger seems to keep searching after receiving the command to stop.

[Henk]

Hello Frank,

Thanks again...

Yes...in the past, all the things were very simple, no much clones etc...

But today, the situation is very different!
Really CompterChess is in BIG danger! As we see, a lot of non-original work!
Nowadays it's a fashion to release X engine based mainly on others work!!


About open letter, in my opinion:
It's a black 'shadow' in Computerchess!
Only 'Losers' can write a such funny letter, the real 'Gladiators' should not be in a such list !

And I hope to see another new open letter,
Where this time...all those chess friends will apologize to chess community !!!

For example,
I also studied from you (to be honest I copied:), from your ideas... believe me!)

When I say studied or copied,
I look in your conditions (when I was beginner) during 1997-1998 years
(Btw, still I keep your great rating list (40/10) in of my old CD archives)
And a few years later, I created a similar rating as yours...

So now...if we follow the open letter rules,
Maybe it's time to collect signatures...against me ))??
I mean SedatChess should be banned from ComputerChess too )))
What a joke hehe ??)))

Actually in reality, several times my work is tried to be down by:
-Web Attacks (a few times my site was down) from hackers !
-Attacks over my opening books (started firstly from Harry Schnapp)
And plus many more examples I can give...but this is other stories..


One thing more,
How many programmers are very honest ? I mean they tell us all the true in the ReadMe file ?
Look please Protector...Do you see any info that Protector is studied mainly from Stockfish ?!

Frank,Frank,Frank,
We should not be so naive....
If we follow your rules...many engines should not be as participants in FCT

About my new suggested rules,
I need you...really I need your support...we should be examples for rest new testers!
Otherwise we will make history mistakes...after several years people will smile to us!)

Btw, without to not mention I can't,
Without other ideas...almost nobody can create chess engine!
All those original ideas did not came from Mars !!))
And nobody can be on top by others work !!!
I believe in that...!!!

Hopes helps...

Best,
Sedat

It's your hobby. You can test whatever you like. Follow your own rules.

[But better not test all 150 old Skipper versions for that makes no sense, but nobody will stop you. First you have to steal these sources for they are private]

[Sedat Canbaz]

Hello again Frank,

This should be know too!

Really Mr. Robert Houdart's engine should be as participant in FCT

Because you like the realities, right ? ))

And as we see, he states us the true:
http://www.cruxis.com/chess/houdini.htm

Without many ideas and techniques from the open source chess engines Ippolit and Stockfish,
Houdini would not nearly be as strong as it is now.

And where Ippolit's author statements ?
- Without many ideas and techniques from Rybka,
Ippolit would not nearly be as strong as it is now.

If really Ippolit authors are very honest ??

Where is the statement by Critter's author (Richard Vida)
- Without many ideas and techniques from Rybka and from Houdini,
Critter would not nearly be as strong as it is now.

And with pleasure I can give you more examples...
Just let me know....))!!

A lot of realities you are missing my friend !!

And only banning Rybka and Houdini...I have no words to say for you, exception:
-Sad but true...!!


Best,
Sedat

[Sedat Canbaz]

Hello Adam,

Thanks...

Can you publish your sim test results by Ktulu and Chess Tiger ?

And then let's compare your results with mine....


Best,
Sedat

[kranium]

And as we see, he states us the true:
http://www.cruxis.com/chess/houdini.htm

Without many ideas and techniques from the open source chess engines Ippolit and Stockfish,
Houdini would not nearly be as strong as it is now.

Best,
Sedat

Houdini does NOT contain any Ippolit code.
Robert

http://talkchess.com/forum/viewtopic.ph ... at&start=8

http://www.open-chess.org/viewtopic.php ... 9&start=16
http://www.talkchess.com/forum/viewtopi ... 08&t=38052

[Sedat Canbaz]

And as we see, he states us the true:
http://www.cruxis.com/chess/houdini.htm

Without many ideas and techniques from the open source chess engines Ippolit and Stockfish,
Houdini would not nearly be as strong as it is now.

Best,
Sedat

Houdini does NOT contain any Ippolit code.
Robert

http://talkchess.com/forum/viewtopic.ph ... at&start=8

http://www.open-chess.org/viewtopic.php ... 9&start=16
http://www.talkchess.com/forum/viewtopi ... 08&t=38052

Thanks for the info

Not sure indeed... why he wrote like this

But when we look at his main site, he states that without IPPO (Rybka) and Stockfish, Houdini would not be so strong as it nowadays

In shortly,
I hope to see more pigeons to find home directly from 1000 km !)
Rybka, Houdini... managed to that, why not rest (bloody pigeons)?!


Best,
Sedat

[Frank Quisinsky]

Yes, only one example!
The reason I dislike Houdini!

Best
Frank

[Sedat Canbaz]

Dear Frank,

If we are looking for 100% original engines and 100% honest programmers,
- We should not be testers !! ))

Yes...I don't say that all are not honest, but maybe we will need to test maybe 50-100 engines based on 500 engines

Best,
Sedat

[Frank Quisinsky]

Sedat,

I hope of you ...
We need a bit against clone / derivative testing!

We have nothing so far ... my little list isn't strong enough but I try with a lot of additional information to make the FCT start point more interesting for the readers.

And in the past I made the same mistake.
I gave all the clones / derivative engines a good stage!
But each time with a bad conscience.

After my qualification tourney, perhaps end of the year is my new opening book ready for take off. All what I create here will be public, with stats. I have in my brain to start FCT2 with longer time controls and much harder conditions as I have in FCT1.

But I need all the games for my book idea, means all the FCT1 games. FCT2 will be run a long time with more hardware I think. Perhaps I buy a third system for testing.

Sedat, try to create a dendogram with all the stuff you find out.
Again, important is that you don't lost your interest in doing such things.

Protector:
Yes, I saw it ...
With a lot of big differents.
Sources are free available ... differents to older SF versions are to big. Can see a different style to older SF versions. Checked that ... Protector is fully OK for myself. Same with GullChess, sources are free, to big are the differents to IPPs. All is documented ... fully OK for myself.

Fire:
Will be replaced if Norman is ready with the new version.

Quazar is to old, Shredder is to old, Critter is to old ...
Spark too, but Spark is my favorite engine, no interest to replaced my favorite engine. Quazar is very important, complete other playing style. Most important for me is the style of an engine. Hiarcs have a nice style too, also old. Critter have a nice style, old too.

Only Shredder ... not happy with the style of Shredder. Very old engine. I am using Shredder because most of my stats are calculated with Shredder ELO 2.800. The reason I can't replaced Shredder.

During the qualification tourney I will thinking again and we can discuss about it.

Maybe we create the FCT2 rules together ... if you like.
You are invited!

Let us discuss about it during the FCT1 qualification tourney. In 10 days I have holiday and more time.

But after all ...
We can't make the World better and please thinking about it ...
We have to accept if others used clones / derivative engines. For derivative engines we have good positive arguments too (if a good readme file is available).

Best
Frank

[Sedat Canbaz]

Ok dear Frank,

Right now our game (discussion) is paused...

Later we can talk again...))

And thanks for your suggestion, meanwhile think please about my suggestion too !)

See you later my friend...


Best,
Sedat