bob wrote:A direct challenge: how about listing exactly WHO I have accused of copying code? Vas/Rybka/ Houdart/Houdini. Because there is a TON of evidence showing both convincingly. The others would include El Chinito, Le Petite, Bionic Impakt, Voyager, and several other Crafty clones that were proven beyond any doubt. Who ELSE have I accused? Be waiting to see if you answer or run here...
Too funny.
But keep on with your insinuations and allegations towards other programmers, in the meantime some of us are working on solutions.
Rebel wrote:In addition I can tell the following, some programmer (who wants to remain unknown) has done the following experiment:
1. Take the Fruit 2.1 source and modify each of Fruit's EVAL equal to Rybka 1.0
2. Similarity detector reported an only 4% increase.
This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.
Try drastically changing null-move, or LMR threshold in Fruit, or a few very SIMPLE things like those. Those things greatly influence move selection.
Rebel wrote:This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.
I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.
Likely true. I had severe problems removing the traces of my own and only managed after the loss of 200-300 elo points. But I fail to see how this is good news for cloners.
Ed,
Could you try the similarity tool on the versions of Pro Deo that you removed the various parts of the evaluation. One thing that has not been done is to try to quantify how each part affects the move selection. It seems obvious that the parts that have the greatest effects on the strength would show the most difference in self-comparison. But I do not know this for sure.
Hi Adam,
I am sure you have seen The value of an evaluation function page. With the current ProDeo 1.74 version you then can modify the prodeo.eng file as follows:
Rebel wrote:In addition I can tell the following, some programmer (who wants to remain unknown) has done the following experiment:
1. Take the Fruit 2.1 source and modify each of Fruit's EVAL equal to Rybka 1.0
2. Similarity detector reported an only 4% increase.
This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.
Try drastically changing null-move, or LMR threshold in Fruit, or a few very SIMPLE things like those. Those things greatly influence move selection.
Adam Hair wrote:For what it is worth, here are the results of the test Miguel referred to. I included a pass with history pruning and null move turned off, at 100ms and 1s.
sim version 3
Key:
1) Fruit 2.1 (time: 100 ms scale: 1.0)
2) Fruit 2.1 (time: 1000 ms scale: 1.0)
3) Fruit 2.1_history_pruning_off (time: 100 ms scale: 1.0)
4) Fruit 2.1_history_pruning_off (time: 1000 ms scale: 1.0)
5) Fruit 2.1_historypruning_nullmove_off (time: 100 ms scale: 1.0)
6) Fruit 2.1_historypruning_nullmove_off (time: 1000 ms scale: 1.0)
7) Fruit 2.1_nullmove_off (time: 100 ms scale: 1.0)
8) Fruit 2.1_nullmove_off (time: 1000 ms scale: 1.0)
1 2 3 4 5 6 7 8
1. ----- 63.94 89.55 64.45 72.42 68.40 76.37 67.41
2. 63.94 ----- 61.97 81.66 57.42 69.65 58.85 72.78
3. 89.55 61.97 ----- 63.44 74.11 67.12 77.17 65.62
4. 64.45 81.66 63.44 ----- 58.47 71.35 59.86 73.89
5. 72.42 57.42 74.11 58.47 ----- 64.08 86.37 62.90
6. 68.40 69.65 67.12 71.35 64.08 ----- 64.86 81.51
7. 76.37 58.85 77.17 59.86 86.37 64.86 ----- 65.61
8. 67.41 72.78 65.62 73.89 62.90 81.51 65.61 -----
10x search time has the biggest effect on the move selections. As you can see, history pruning and null move off at 100ms and 1s are both still very similar (as based on earlier observations) to normal Fruit at 100ms and the largest difference is both off at 100ms and normal Fruit at 1s.
None of this is extraordinary, if I have understood everything I have read on these search techniques. Turning both off has the same effect on Elo as approximately halving the search time (according to other reports and some testing I've done). So, it is not surprising 10x increase in search time has a larger effect. What might be surprising (or maybe not ) is that move selection and Elo are not strongly coupled, at least in terms of intra-engine comparison. If turning off both history pruning and null move has the same effect on Fruit as it does on Crafty, then it is a 100+ Elo reduction. With a 1 second search per position, normal Fruit and both off Fruit share ~70% of the same moves (using the positions in the sim test). In context with all of the similarity testing I have done, this does not indicated a large difference.
And by large difference, I mean that ~70% indicates we are looking at related engines. So, even turning off null move and LMR in Fruit 2.1 does not remove the move selection characteristics (when judged in this manner).
Laskos wrote:
You seem to be very srupulous with regard to this test, but you yourself (and some others) are accusing directly or not a lot of authors unscrupulously. Generally speaking, this test is say >95% correct in positive detections, while your accusations are pretty random. Besides that, the engines will be labeled only as suspicious, and the inspection of the sources will establish the copying.
Kai
Who have I accused? Vas? Lots of supporting evidence. Houdart? Ditto. Beyond that, the only ones _I_ have accused have all been proven. El Chinito, Le Petite. etc... So please feel free to show me my "random accusations" that seem to be more imagination on your part than anything else. As a control experiment, I wonder what would happen on (say) the ponder hit data if human games were used? Should NOT produce suspicious behaviour, would you agree?
You have accused the engines like IvanHoe or even StockFish of possible cloning (or in need of investigation), besides Houdini (which if took something, took from the open domain, and that thing in the open domain was never proven anything). Do you have something better than hearsay on OpenChess, three lines of code and mob lynching which is so dear to you? Even for copyright violation, significant chunks of code must be found as copied.
How ponder hit works for human games? It would be great to use something analogous to sim on humans, you will see that it's much more reliable on correct positives than you imply.
After reading you and others in the same vein, and knowing that you have influence in ICGA, CSVN 60% tourney rule seems much more adequate than your views.
Kai
Ivanhoe is a robolito derivative. So it IS a "clone". Ditto for Firebird, Houdini, and all the other "kin". Houdini 1.5 or so was a PERFECT match with one of the robo's. You can find the specifics on open chess. And it was NOT "3 lines of code"...
I've not accused Stockfish of anything. If you think they chose the "magic right threshold" for CSVN, glad you are happy. I consider their "number" to be arbitrary and without any particular justification other than "it feels right."
bob wrote:A direct challenge: how about listing exactly WHO I have accused of copying code? Vas/Rybka/ Houdart/Houdini. Because there is a TON of evidence showing both convincingly. The others would include El Chinito, Le Petite, Bionic Impakt, Voyager, and several other Crafty clones that were proven beyond any doubt. Who ELSE have I accused? Be waiting to see if you answer or run here...
Too funny.
But keep on with your insinuations and allegations towards other programmers, in the meantime some of us are working on solutions.
Exactly what I thought. You elect to "run".
One of these days maybe you will stop with the distortions, exaggerations, etc? I write what I intend to write. I don't need you to interpret my comments in twisted ways and then claim I wrote what you twisted, rather than what I intended...
bob wrote:
Ivanhoe is a robolito derivative. So it IS a "clone". Ditto for Firebird, Houdini, and all the other "kin". Houdini 1.5 or so was a PERFECT match with one of the robo's. You can find the specifics on open chess. And it was NOT "3 lines of code"...
I've not accused Stockfish of anything. If you think they chose the "magic right threshold" for CSVN, glad you are happy. I consider their "number" to be arbitrary and without any particular justification other than "it feels right."
What do you know what Robbolito, IvanHoe, Ippolit are? Nobody knows for sure. Suddenly, after repeatedly stating that you did not study them, you are the best specialist in Ippos, more so, calling them collectively clones. I will post a thread "Crafty is a proven clone of Fruit and Belle", fine? About StockFish, you yourself stated that it needs to be investigated. Komodo has to be investigated? Could you explain why not, besides that Don in your silly appreciation is a good, old guy? Joker of HGM has to be investigated or it is too crappy even for you? Look guy, close your useless sources of the clone called Crafty, and leave us for good. CSVN "magic" 60% is MUCH better than your sick appreciation of who is good guy and who is not. Understood?
Laskos wrote:
You seem to be very srupulous with regard to this test, but you yourself (and some others) are accusing directly or not a lot of authors unscrupulously. Generally speaking, this test is say >95% correct in positive detections, while your accusations are pretty random. Besides that, the engines will be labeled only as suspicious, and the inspection of the sources will establish the copying.
Kai
Who have I accused? Vas? Lots of supporting evidence. Houdart? Ditto. Beyond that, the only ones _I_ have accused have all been proven. El Chinito, Le Petite. etc... So please feel free to show me my "random accusations" that seem to be more imagination on your part than anything else. As a control experiment, I wonder what would happen on (say) the ponder hit data if human games were used? Should NOT produce suspicious behaviour, would you agree?
You have accused the engines like IvanHoe or even StockFish of possible cloning (or in need of investigation), besides Houdini (which if took something, took from the open domain, and that thing in the open domain was never proven anything). Do you have something better than hearsay on OpenChess, three lines of code and mob lynching which is so dear to you? Even for copyright violation, significant chunks of code must be found as copied.
How ponder hit works for human games? It would be great to use something analogous to sim on humans, you will see that it's much more reliable on correct positives than you imply.
After reading you and others in the same vein, and knowing that you have influence in ICGA, CSVN 60% tourney rule seems much more adequate than your views.
Kai
Ivanhoe is a robolito derivative. So it IS a "clone". Ditto for Firebird, Houdini, and all the other "kin". Houdini 1.5 or so was a PERFECT match with one of the robo's. You can find the specifics on open chess. And it was NOT "3 lines of code"...
I've not accused Stockfish of anything. If you think they chose the "magic right threshold" for CSVN, glad you are happy. I consider their "number" to be arbitrary and without any particular justification other than "it feels right."
Ed did his own testing with the sim utility and then he asked me my opinion about a 60% threshold (he may have asked others also). I informed him of my findings and my belief that 5 standard deviations, which is approximately 59.5% for my data, represented a significant amount of correlation in move selection. This belief is based on statistics, the totality of my data, and on Mark Watkins' RE efforts. I feel that my input helped to solidify the 60% figure that the CSVN is using.
Laskos wrote:
You seem to be very srupulous with regard to this test, but you yourself (and some others) are accusing directly or not a lot of authors unscrupulously. Generally speaking, this test is say >95% correct in positive detections, while your accusations are pretty random. Besides that, the engines will be labeled only as suspicious, and the inspection of the sources will establish the copying.
Kai
Who have I accused? Vas? Lots of supporting evidence. Houdart? Ditto. Beyond that, the only ones _I_ have accused have all been proven. El Chinito, Le Petite. etc... So please feel free to show me my "random accusations" that seem to be more imagination on your part than anything else. As a control experiment, I wonder what would happen on (say) the ponder hit data if human games were used? Should NOT produce suspicious behaviour, would you agree?
You have accused the engines like IvanHoe or even StockFish of possible cloning (or in need of investigation), besides Houdini (which if took something, took from the open domain, and that thing in the open domain was never proven anything). Do you have something better than hearsay on OpenChess, three lines of code and mob lynching which is so dear to you? Even for copyright violation, significant chunks of code must be found as copied.
How ponder hit works for human games? It would be great to use something analogous to sim on humans, you will see that it's much more reliable on correct positives than you imply.
After reading you and others in the same vein, and knowing that you have influence in ICGA, CSVN 60% tourney rule seems much more adequate than your views.
Kai
Ivanhoe is a robolito derivative. So it IS a "clone". Ditto for Firebird, Houdini, and all the other "kin". Houdini 1.5 or so was a PERFECT match with one of the robo's. You can find the specifics on open chess. And it was NOT "3 lines of code"...
I've not accused Stockfish of anything. If you think they chose the "magic right threshold" for CSVN, glad you are happy. I consider their "number" to be arbitrary and without any particular justification other than "it feels right."
After you asked something in this thread, Adam was gracious enough to provide real data, not guesswork. He has done a thorough analysis with 3 digit number engines (not 6 or 7). You should read it. You are coming very late to the discussion, because many of the questions you ask have been debated, investigated, and answered with real experiments. Before you criticize, you should familiarize yourself with the topic and the data. The 60% (with the set he used) represents 5 standard deviations.
bob wrote:A direct challenge: how about listing exactly WHO I have accused of copying code? Vas/Rybka/ Houdart/Houdini. Because there is a TON of evidence showing both convincingly. The others would include El Chinito, Le Petite, Bionic Impakt, Voyager, and several other Crafty clones that were proven beyond any doubt. Who ELSE have I accused? Be waiting to see if you answer or run here...
Too funny.
But keep on with your insinuations and allegations towards other programmers, in the meantime some of us are working on solutions.
Exactly what I thought. You elect to "run".
One of these days maybe you will stop with the distortions, exaggerations, etc? I write what I intend to write. I don't need you to interpret my comments in twisted ways and then claim I wrote what you twisted, rather than what I intended...
