georgerifkin wrote:According to your test, it seems that Komodo and Stockfish are clean, right?
And what's this tool: Sim03? Where can I find it?
And yet Don has admitted to getting ideas from the Ippolitts so again the Sim test proves nothing.
It is most likely a waste of time.
No
It is not a waste of time.
If you analyze a position with more than one engine then you prefer to choose engines that have bigger probability for their move choice to be different so the test can help people to choose engines to analyze.
georgerifkin wrote:According to your test, it seems that Komodo and Stockfish are clean, right?
And what's this tool: Sim03? Where can I find it?
And yet Don has admitted to getting ideas from the Ippolitts so again the Sim test proves nothing.
It is most likely a waste of time.
No
It is not a waste of time.
If you analyze a position with more than one engine then you prefer to choose engines that have bigger probability for their move choice to be different so the test can help people to choose engines to analyze.
Fair enough. I will amend my statement. It is most likely a waste of time for "clone or derivative detection."
georgerifkin wrote:According to your test, it seems that Komodo and Stockfish are clean, right?
And what's this tool: Sim03? Where can I find it?
And yet Don has admitted to getting ideas from the Ippolitts so again the Sim test proves nothing.
It is most likely a waste of time.
Maybe, but admit that if Critter wasn't there, all other engines are in their right places. I gave earlier other results in which it was pretty evident the Fruit 2.1 and Rybka 1.0 beta relation. A little strange to say it's fine when everything falls as expected, and to reject when we get some surprises. Then, there is Naum 4.2 problem with Strelka.
Uri Blass wrote:I believe that piece square table is only a small part of the evaluation.
there is also mobility evaluation and passed pawn evaluation and king safety evaluation.
I do not expect a big similiarity between 2 programs if they only share the same piece square table when they do not share other parts of the evaluation.
Uri, no need to "belive" or "expect/not expect"... Just try it!
I made such an experiment some time ago when there was a lengthy thread about the similarity tester. One poster (IIRC it was Milos) said that PSQT's alone are enough to made 2 engines look similar. I dont know whether this was just a wild guess or he made some research on this, but I wanted to prove or disprove this with my own tests.
On the other hand, I had a surprisingly low similarity result between two successive versions of Critter despite the same tables (and in fact, almost the whole evaluation). Very puzzling.
I was always very interested in knowing what are the parameters with highest impact on move selection (i.e. what his test measures). I had my hypothesis, which was material tables, based on the fact that R1 and Fruit were not so similar under this test, and the material tables were supposed to be biggest difference. So, I think I may try what you suggest.
georgerifkin wrote:According to your test, it seems that Komodo and Stockfish are clean, right?
And what's this tool: Sim03? Where can I find it?
And yet Don has admitted to getting ideas from the Ippolitts so again the Sim test proves nothing.
It is most likely a waste of time.
No
It is not a waste of time.
If you analyze a position with more than one engine then you prefer to choose engines that have bigger probability for their move choice to be different so the test can help people to choose engines to analyze.
georgerifkin wrote:According to your test, it seems that Komodo and Stockfish are clean, right?
And what's this tool: Sim03? Where can I find it?
And yet Don has admitted to getting ideas from the Ippolitts so again the Sim test proves nothing.
It is most likely a waste of time.
No
It is not a waste of time.
If you analyze a position with more than one engine then you prefer to choose engines that have bigger probability for their move choice to be different so the test can help people to choose engines to analyze.
Fair enough. I will amend my statement. It is most likely a waste of time for "clone or derivative detection."
It measures what it is supposed to measure and it does it very effectively. It detects engines that are likely to chose similar moves.
re-tuning my pst is on my todo list before the next release. If it goes anything like retuning my material values went, I expect quite different values. It will be interesting to see if this significantly affects the similarity metric with previous versions.
-Sam
rvida wrote:
Uri Blass wrote:I believe that piece square table is only a small part of the evaluation.
there is also mobility evaluation and passed pawn evaluation and king safety evaluation.
I do not expect a big similiarity between 2 programs if they only share the same piece square table when they do not share other parts of the evaluation.
Uri, no need to "belive" or "expect/not expect"... Just try it!
I made such an experiment some time ago when there was a lengthy thread about the similarity tester. One poster (IIRC it was Milos) said that PSQT's alone are enough to made 2 engines look similar. I dont know whether this was just a wild guess or he made some research on this, but I wanted to prove or disprove this with my own tests.
On the other hand, I had a surprisingly low similarity result between two successive versions of Critter despite the same tables (and in fact, almost the whole evaluation). Very puzzling.
2) Do you know what exactly is measured with sim03?
3) I will tell you a "secret". Houdini, Robbolito, Ippolit, etc. share same piece square tables. While the ones that Critter uses are not 100% identical but are very very close (differences are just because of rounding errors - Critter uses 1/256th of a pawn instead of 1/100 - for every practical purpose they can be called identical... they are working fine, and there are more reasonable ways to spend development time than to make them different just to make someone happy). I don't know about your programming skills but let's try an experiment: Take 2 different open source programs (let's say Crafty & Fruit) and force them to use the same exact PSQ tables. Now run them through sim03 and see the shocking result (and write a sensational post on a forum of your choice about one being a clone of the other...)
4) Sources of Critter are not top-secret. Although after version 0.42 I choose to go closed source, so far I have sent my sources to everyone who asked for them. Most of such requests concerned version 0.90, but a few people on this forum do have sources of v1.2 too (or the last beta before v1.2 release).
Richard
I think that this tool Sim03 doesn't prove anything. At such short time for move, the piece square table are indeed one of the most important factor for move selection in my opinion (with also mobility), so no wonder that two programs which share only the same PST can have an high similarity rate with this test.
Uri Blass wrote:I believe that piece square table is only a small part of the evaluation.
there is also mobility evaluation and passed pawn evaluation and king safety evaluation.
I do not expect a big similiarity between 2 programs if they only share the same piece square table when they do not share other parts of the evaluation.
Uri, no need to "belive" or "expect/not expect"... Just try it!
I made such an experiment some time ago when there was a lengthy thread about the similarity tester. One poster (IIRC it was Milos) said that PSQT's alone are enough to made 2 engines look similar. I dont know whether this was just a wild guess or he made some research on this, but I wanted to prove or disprove this with my own tests.
On the other hand, I had a surprisingly low similarity result between two successive versions of Critter despite the same tables (and in fact, almost the whole evaluation). Very puzzling.
I was always very interested in knowing what are the parameters with highest impact on move selection (i.e. what his test measures). I had my hypothesis, which was material tables, based on the fact that R1 and Fruit were not so similar under this test, and the material tables were supposed to be biggest difference. So, I think I may try what you suggest.
Miguel
Yes, but R1 and Fruit were still pretty similar, without Fruit even having material tables at all. I think there was no other engine closer to R1 than Fruit (at its time, not talking about Strelka & Co.), and the degree of similarity was close to that of Shredder 12 compared to Shredder 10.
