World Computer Chess Championship ?

[Rebel]

This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.

I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.

Likely true. I had severe problems removing the traces of my own and only managed after the loss of 200-300 elo points. But I fail to see how this is good news for cloners.

[Uri Blass]

This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.

I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.

Likely true. I had severe problems removing the traces of my own and only managed after the loss of 200-300 elo points. But I fail to see how this is good news for cloners.

I wondered if you tried only changing the evaluation because it is possible to try other changes like changing the order of moves in the move generator.

I also wonder if there is a significant increase in similiarity with significantly longer time control(100 times slower than the time control that you usually use).

I suspect that in some positions there is only one best move that all programs find but part of the programs need a long search to find the best move so if the time control is long enough you can find a significant increase in similiarity.

If it is the case then maybe removing the relevant positions from the set of positions can help to improve the similiarity tool.

[mar]

But I fail to see how this is good news for cloners.

Simply: StockFish (or any other super strong open source engine) -300 or -400 elo should be still far above average which would IMHO satisfy a cloner, at least for version 0.1 or 1.0 beta or whatever The most important thing is that it will fool the similary test, I am pretty sure.

So the general recipe for "similarity safe" cloning is to take a strong engine, strip eval (material only) and voila, much weaker but already "clean". Then rewrite eval from scratch, that should do the trick.

[Laskos]

This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.

I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.

Martin

True, but one will lose several hundred Elo. Smarter guys could do even better, try ro adapt several sources to their engine. False negatives are always a danger, but do you have better tools? How do you go after a lobotomized Stockfish? At least it cannot win a WCCC. This thread is already full of pretty arbitrary accusations, if such a mob will decide the originality of an engine in competitions, it would be very much worse than the simple 60% at CSVN. In the future they will have to use random, secret positions with a bit different, pretty universal threshold.

Kai

[Jeroen]

The first and most important issue to move forward in computer chess again, instead of running backwards, is to sideline the ICGA and organize a real World Championship. The ICGA obviously does an extremely bad job regarding the PR of computer chess and its development, which should be its first and most important mission. The simplest is not to try to have the ICGA change its mind (which is pointless anyway), but to sideline them and come up with an alternative.

[Laskos]

This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.

I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.

Likely true. I had severe problems removing the traces of my own and only managed after the loss of 200-300 elo points. But I fail to see how this is good news for cloners.

I wondered if you tried only changing the evaluation because it is possible to try other changes like changing the order of moves in the move generator.

I also wonder if there is a significant increase in similiarity with significantly longer time control(100 times slower than the time control that you usually use).

I suspect that in some positions there is only one best move that all programs find but part of the programs need a long search to find the best move so if the time control is long enough you can find a significant increase in similiarity.

If it is the case then maybe removing the relevant positions from the set of positions can help to improve the similiarity tool.

It is done, Don's positions are filtered so that they are mostly positions on which not all engines agree. But random positions would work as well, the similarity window is somewhat smaller, therefore maybe some 15,000 positions are needed instead of 8,000.

As for increase of similarity at longer time controls, it is there, but a very slow function, some maybe 2% for a factor of 10. I wrote that 60% seems a good threshold for 100ms time on one decent core. It would be maybe some 62% for 1s time.

Kai

[mar]

True, but one will lose several hundred Elo. Smarter guys could do even better, try ro adapt several sources to their engine. False negatives are always a danger, but do you have better tools? How do you go after a lobotomized Stockfish? At least it cannot win a WCCC. This thread is already full of pretty arbitrary accusations, if such a mob will decide the originality of an engine in competitions, it would be very much worse than the simple 60% at CSVN. In the future they will have to use random, secret positions with a bit different, pretty universal threshold.

Kai

No I don't have anything better, in fact I appreciate Adam's work on similarity tests a lot. It's fully automated and is next best thing to full RE.
The whole point is that these similarity tests are heavily influenced by eval (which is logical as you can't run similarity test on tactical positions with one best move for example),
so they basically compare evals (and PSQ tables). That's the problem.
AFAIK There's no oracle that you would feed with a binary and it would output either "clone" or "clean".

[Laskos]

True, but one will lose several hundred Elo. Smarter guys could do even better, try ro adapt several sources to their engine. False negatives are always a danger, but do you have better tools? How do you go after a lobotomized Stockfish? At least it cannot win a WCCC. This thread is already full of pretty arbitrary accusations, if such a mob will decide the originality of an engine in competitions, it would be very much worse than the simple 60% at CSVN. In the future they will have to use random, secret positions with a bit different, pretty universal threshold.

Kai

No I don't have anything better, in fact I appreciate Adam's work on similarity tests a lot. It's fully automated and is next best thing to full RE.
The whole point is that these similarity tests are heavily influenced by eval (which is logical as you can't run similarity test on tactical positions with one best move for example),
so they basically compare evals (and PSQ tables). That's the problem.
AFAIK There's no oracle that you would feed with a binary and it would output either "clone" or "clean".

Mostly eval but some search too, as for example increasing the time control shows. Something similar to the CSVN approach would seem the most reasonable, after reading tons of wild clone wars and accusations threads here. I don't think the jury can RE every participating or every existing engine, and I don't think the RE is the magic tool. Besides that, one is fascinated by finding 2-3 lines of identical code, while it may or may not be relevant. The sim and ponder-hit tool checks almost the whole eval and some search.

Kai

[Adam Hair]

This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.

I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.

Likely true. I had severe problems removing the traces of my own and only managed after the loss of 200-300 elo points. But I fail to see how this is good news for cloners.

Ed,

Could you try the similarity tool on the versions of Pro Deo that you removed the various parts of the evaluation. One thing that has not been done is to try to quantify how each part affects the move selection. It seems obvious that the parts that have the greatest effects on the strength would show the most difference in self-comparison. But I do not know this for sure.

Adam

[Rebel]

This is BAD NEWS for cloners who think they can take an existing source code modify all eval values (some even use multiplication) and think they can get away with it. Playing style is hard to remove from an engine.

I dare to disagree Ed. Playing style is determined by the eval in the first place. So I would say it's actually good news for them.
If I would take for example Stockfish and would heavily lobotomize it's eval (losing several hundred elo), I bet it would appear as "crystal clear" in the similary test. And yet much stronger than vast majority of other engines.

Likely true. I had severe problems removing the traces of my own and only managed after the loss of 200-300 elo points. But I fail to see how this is good news for cloners.

Ed,

Could you try the similarity tool on the versions of Pro Deo that you removed the various parts of the evaluation. One thing that has not been done is to try to quantify how each part affects the move selection. It seems obvious that the parts that have the greatest effects on the strength would show the most difference in self-comparison. But I do not know this for sure.

Hi Adam,

I am sure you have seen The value of an evaluation function page. With the current ProDeo 1.74 version you then can modify the prodeo.eng file as follows:

Code: Select all

[King Safety = 0]            // remove king safety

[Mobility = 0]               // remove mobility
[Minimum Knight Mobility = 0] 
[Minimum Bishop Mobility = 0]  
[Knight Mobility = 0]         
[Bishop Mobility = 0]             

[Passed Pawns = 0]           // remove passed pawn eval

These seems to be the dominant components of EVAL. There is another parameter to remove the PST evaluation:

Code: Select all

[Piece Square = 0]            // 0 to cancel

But you need the latest version for that. Lemme know if you want to have it.