Open letter to chess programmers

[pijl]

Dear programmers

Some time has past since the crisis in computer chess broke out and the damage it has done is clearer than ever. Only a few computer chess tournaments have survived and those that still exist have trouble attracting sufficient participants.

This is certainly also true for the tournaments organized by the CSVN. The decision to allow Rybka to participate in the CSVN tournament in 2011 was followed by an open letter to the CSVN members by concerned former participants. This resulted in a further decline in the number of participants of CSVN tournaments, eventually resulting in the resignation of the CSVN board.

Now there is a new CSVN board where two of the members have the special task in reforming the organization of the programmer's events: Marcel van Kervinck (author of Rookie) and Richard Pijl (author of the Baron). In this letter we want to explain the plans of revitalizing the CSVN tournaments. The result is an attempt to reunite the programmer communities again into a tournament format.

The first principle we adopted is that the rules we create for the tournaments should be enforceable. It is of no use to have rules if there is no way to detect if a rule has been broken.

The second principle is that when rules are created, they will have to be enforced. Again, it is of no use to have rules if participants are not checked whether they comply to the rules or not.

And the third principle is that as much as possible should be done to prevent the participation of a rule-breaker. This means checking the registration details and previous history of a participant as well as taking a 'fingerprint' of the engine that has been registered.

Finally, the whole process should be light-weight enough for both the participants and the tournament organisation.

How will process look like?

First we ask a programmer who wants to register for a CSVN tournament to send in the fingerprint of his program. This consists of the preferred move of the program on 10,000 positions with a 1 second search time. The positions are supplied by the tournament organization and are designed to identify the 'character' of a program. This fingerprint will be compared with a database of engine fingerprints that will be compiled by the CSVN tournament organization. Consolidated results of this comparison will be published.

When the fingerprint of a program is found to be close to the fingerprint of another program or when there is a well-founded other reason to doubt the origin of the program, the tournament organization will start an inquiry where the programmer is obliged to cooperate. Otherwise his entry will not be accepted.

When the tournament is started another verification will be done with selected (possibly all) participants. You could probably compare this to a doping test. A relatively small subset of the 10,000 fingerprint positions will be required to be executed on the tournament machine, with the engine used during the tournament. The sole purpose of this test is to verify whether the fingerprint that was submitted with the registration matches the program that is actually playing.

Is this an air-tight method for detecting clones? No.

Does it mean that cheating can no longer occur? No again.

So why do it?

Because by putting a method in place to screen the entries before the tournament we will have made a step in improving the originality of the participating programs. And we will improve this further in the next events.

What do we request from you? First of all that you register for our tournaments, but we also value constructive feedback on our tournament rules, the process described above and anything else that we can use to improve the CSVN tournaments.

What will we do next? We feel there is room for more improvement: In the programmer's tournament but also in organizing other activities. We do have ideas of our own, but we also want to know what you would like us to do.

The CSVN site has been improved and also contains a forum. Please make use of that to discuss the CSVN activities. And you could start by providing us your feedback on this letter.
We hope to see you in one of the CSVN events soon,

Marcel van Kervinck
Richard Pijl

[bob]

Dear programmers

Some time has past since the crisis in computer chess broke out and the damage it has done is clearer than ever. Only a few computer chess tournaments have survived and those that still exist have trouble attracting sufficient participants.

This is certainly also true for the tournaments organized by the CSVN. The decision to allow Rybka to participate in the CSVN tournament in 2011 was followed by an open letter to the CSVN members by concerned former participants. This resulted in a further decline in the number of participants of CSVN tournaments, eventually resulting in the resignation of the CSVN board.

Now there is a new CSVN board where two of the members have the special task in reforming the organization of the programmer's events: Marcel van Kervinck (author of Rookie) and Richard Pijl (author of the Baron). In this letter we want to explain the plans of revitalizing the CSVN tournaments. The result is an attempt to reunite the programmer communities again into a tournament format.

The first principle we adopted is that the rules we create for the tournaments should be enforceable. It is of no use to have rules if there is no way to detect if a rule has been broken.

The second principle is that when rules are created, they will have to be enforced. Again, it is of no use to have rules if participants are not checked whether they comply to the rules or not.

And the third principle is that as much as possible should be done to prevent the participation of a rule-breaker. This means checking the registration details and previous history of a participant as well as taking a 'fingerprint' of the engine that has been registered.

Finally, the whole process should be light-weight enough for both the participants and the tournament organisation.

How will process look like?

First we ask a programmer who wants to register for a CSVN tournament to send in the fingerprint of his program. This consists of the preferred move of the program on 10,000 positions with a 1 second search time. The positions are supplied by the tournament organization and are designed to identify the 'character' of a program. This fingerprint will be compared with a database of engine fingerprints that will be compiled by the CSVN tournament organization. Consolidated results of this comparison will be published.

When the fingerprint of a program is found to be close to the fingerprint of another program or when there is a well-founded other reason to doubt the origin of the program, the tournament organization will start an inquiry where the programmer is obliged to cooperate. Otherwise his entry will not be accepted.

When the tournament is started another verification will be done with selected (possibly all) participants. You could probably compare this to a doping test. A relatively small subset of the 10,000 fingerprint positions will be required to be executed on the tournament machine, with the engine used during the tournament. The sole purpose of this test is to verify whether the fingerprint that was submitted with the registration matches the program that is actually playing.

Is this an air-tight method for detecting clones? No.

Does it mean that cheating can no longer occur? No again.

So why do it?

Because by putting a method in place to screen the entries before the tournament we will have made a step in improving the originality of the participating programs. And we will improve this further in the next events.

What do we request from you? First of all that you register for our tournaments, but we also value constructive feedback on our tournament rules, the process described above and anything else that we can use to improve the CSVN tournaments.

What will we do next? We feel there is room for more improvement: In the programmer's tournament but also in organizing other activities. We do have ideas of our own, but we also want to know what you would like us to do.

The CSVN site has been improved and also contains a forum. Please make use of that to discuss the CSVN activities. And you could start by providing us your feedback on this letter.
We hope to see you in one of the CSVN events soon,

Marcel van Kervinck
Richard Pijl

Very first step ought to be to ask EVERYONE to run the 10K positions. I'll be happy to. Then some analysis can point out whether this approach will be reliable or not. For example, it would be interesting if Crafty comes up looking similar to another engine, knowing where Crafty came from.

Tell me where to locate the positions and I'll run 'em.

BTW, what is the expected feedback? Just "best moves" or scores + moves or complete log or what...

[michiguel]

Dear programmers

Some time has past since the crisis in computer chess broke out and the damage it has done is clearer than ever. Only a few computer chess tournaments have survived and those that still exist have trouble attracting sufficient participants.

This is certainly also true for the tournaments organized by the CSVN. The decision to allow Rybka to participate in the CSVN tournament in 2011 was followed by an open letter to the CSVN members by concerned former participants. This resulted in a further decline in the number of participants of CSVN tournaments, eventually resulting in the resignation of the CSVN board.

Now there is a new CSVN board where two of the members have the special task in reforming the organization of the programmer's events: Marcel van Kervinck (author of Rookie) and Richard Pijl (author of the Baron). In this letter we want to explain the plans of revitalizing the CSVN tournaments. The result is an attempt to reunite the programmer communities again into a tournament format.

The first principle we adopted is that the rules we create for the tournaments should be enforceable. It is of no use to have rules if there is no way to detect if a rule has been broken.

The second principle is that when rules are created, they will have to be enforced. Again, it is of no use to have rules if participants are not checked whether they comply to the rules or not.

And the third principle is that as much as possible should be done to prevent the participation of a rule-breaker. This means checking the registration details and previous history of a participant as well as taking a 'fingerprint' of the engine that has been registered.

Finally, the whole process should be light-weight enough for both the participants and the tournament organisation.

How will process look like?

First we ask a programmer who wants to register for a CSVN tournament to send in the fingerprint of his program. This consists of the preferred move of the program on 10,000 positions with a 1 second search time. The positions are supplied by the tournament organization and are designed to identify the 'character' of a program. This fingerprint will be compared with a database of engine fingerprints that will be compiled by the CSVN tournament organization. Consolidated results of this comparison will be published.

When the fingerprint of a program is found to be close to the fingerprint of another program or when there is a well-founded other reason to doubt the origin of the program, the tournament organization will start an inquiry where the programmer is obliged to cooperate. Otherwise his entry will not be accepted.

When the tournament is started another verification will be done with selected (possibly all) participants. You could probably compare this to a doping test. A relatively small subset of the 10,000 fingerprint positions will be required to be executed on the tournament machine, with the engine used during the tournament. The sole purpose of this test is to verify whether the fingerprint that was submitted with the registration matches the program that is actually playing.

Is this an air-tight method for detecting clones? No.

Does it mean that cheating can no longer occur? No again.

So why do it?

Because by putting a method in place to screen the entries before the tournament we will have made a step in improving the originality of the participating programs. And we will improve this further in the next events.

What do we request from you? First of all that you register for our tournaments, but we also value constructive feedback on our tournament rules, the process described above and anything else that we can use to improve the CSVN tournaments.

What will we do next? We feel there is room for more improvement: In the programmer's tournament but also in organizing other activities. We do have ideas of our own, but we also want to know what you would like us to do.

The CSVN site has been improved and also contains a forum. Please make use of that to discuss the CSVN activities. And you could start by providing us your feedback on this letter.
We hope to see you in one of the CSVN events soon,

Marcel van Kervinck
Richard Pijl

Kudos for trying to do something to keep this activity alive!

I tried to register but I keep getting
"Your session has expired. Please log in again."

Miguel

[michiguel]

Dear programmers

Some time has past since the crisis in computer chess broke out and the damage it has done is clearer than ever. Only a few computer chess tournaments have survived and those that still exist have trouble attracting sufficient participants.

This is certainly also true for the tournaments organized by the CSVN. The decision to allow Rybka to participate in the CSVN tournament in 2011 was followed by an open letter to the CSVN members by concerned former participants. This resulted in a further decline in the number of participants of CSVN tournaments, eventually resulting in the resignation of the CSVN board.

Now there is a new CSVN board where two of the members have the special task in reforming the organization of the programmer's events: Marcel van Kervinck (author of Rookie) and Richard Pijl (author of the Baron). In this letter we want to explain the plans of revitalizing the CSVN tournaments. The result is an attempt to reunite the programmer communities again into a tournament format.

The first principle we adopted is that the rules we create for the tournaments should be enforceable. It is of no use to have rules if there is no way to detect if a rule has been broken.

The second principle is that when rules are created, they will have to be enforced. Again, it is of no use to have rules if participants are not checked whether they comply to the rules or not.

And the third principle is that as much as possible should be done to prevent the participation of a rule-breaker. This means checking the registration details and previous history of a participant as well as taking a 'fingerprint' of the engine that has been registered.

Finally, the whole process should be light-weight enough for both the participants and the tournament organisation.

How will process look like?

First we ask a programmer who wants to register for a CSVN tournament to send in the fingerprint of his program. This consists of the preferred move of the program on 10,000 positions with a 1 second search time. The positions are supplied by the tournament organization and are designed to identify the 'character' of a program. This fingerprint will be compared with a database of engine fingerprints that will be compiled by the CSVN tournament organization. Consolidated results of this comparison will be published.

When the fingerprint of a program is found to be close to the fingerprint of another program or when there is a well-founded other reason to doubt the origin of the program, the tournament organization will start an inquiry where the programmer is obliged to cooperate. Otherwise his entry will not be accepted.

When the tournament is started another verification will be done with selected (possibly all) participants. You could probably compare this to a doping test. A relatively small subset of the 10,000 fingerprint positions will be required to be executed on the tournament machine, with the engine used during the tournament. The sole purpose of this test is to verify whether the fingerprint that was submitted with the registration matches the program that is actually playing.

Is this an air-tight method for detecting clones? No.

Does it mean that cheating can no longer occur? No again.

So why do it?

Because by putting a method in place to screen the entries before the tournament we will have made a step in improving the originality of the participating programs. And we will improve this further in the next events.

What do we request from you? First of all that you register for our tournaments, but we also value constructive feedback on our tournament rules, the process described above and anything else that we can use to improve the CSVN tournaments.

What will we do next? We feel there is room for more improvement: In the programmer's tournament but also in organizing other activities. We do have ideas of our own, but we also want to know what you would like us to do.

The CSVN site has been improved and also contains a forum. Please make use of that to discuss the CSVN activities. And you could start by providing us your feedback on this letter.
We hope to see you in one of the CSVN events soon,

Marcel van Kervinck
Richard Pijl

Kudos for trying to do something to keep this activity alive!

I tried to register but I keep getting
"Your session has expired. Please log in again."

Miguel

I succeeded. There was an issue with cookies.

Miguel

[pijl]

The instructions on how to run the test can be found here.
Unfortunately the python script is not working on Windows, and we're working on an alternative for it.

[pijl]

Great!

[bob]

The instructions on how to run the test can be found here.
Unfortunately the python script is not working on Windows, and we're working on an alternative for it.

Unfortunately? You mean "commonly"?

Hell, my AT&T U-verse CISCO stuff is windows-based. Has to be rebooted every month or so or it gets too slow to use.

"Windows is NOT the answer. Windows? is the question. NO is the answer." -- Linus Torvalds...

BTW, what about hardware? IE should it be on the fastest hardware? What about parallel search? One thread or as many as possible?

[bob]

the "import subprocess" produces an error on my cluster.

Works just fine on my macbook... with 2 cores it will run a while, but I will let it go...

[lucasart]

The first principle we adopted is that the rules we create for the tournaments should be enforceable. It is of no use to have rules if there is no way to detect if a rule has been broken.

The second principle is that when rules are created, they will have to be enforced. Again, it is of no use to have rules if participants are not checked whether they comply to the rules or not.

And the third principle is that as much as possible should be done to prevent the participation of a rule-breaker. This means checking the registration details and previous history of a participant as well as taking a 'fingerprint' of the engine that has been registered.

Finally, the whole process should be light-weight enough for both the participants and the tournament organisation.

How will process look like?

First we ask a programmer who wants to register for a CSVN tournament to send in the fingerprint of his program. This consists of the preferred move of the program on 10,000 positions with a 1 second search time. The positions are supplied by the tournament organization and are designed to identify the 'character' of a program. This fingerprint will be compared with a database of engine fingerprints that will be compiled by the CSVN tournament organization. Consolidated results of this comparison will be published.

When the fingerprint of a program is found to be close to the fingerprint of another program or when there is a well-founded other reason to doubt the origin of the program, the tournament organization will start an inquiry where the programmer is obliged to cooperate. Otherwise his entry will not be accepted.

When the tournament is started another verification will be done with selected (possibly all) participants. You could probably compare this to a doping test. A relatively small subset of the 10,000 fingerprint positions will be required to be executed on the tournament machine, with the engine used during the tournament. The sole purpose of this test is to verify whether the fingerprint that was submitted with the registration matches the program that is actually playing.

Is this an air-tight method for detecting clones? No.

Does it mean that cheating can no longer occur? No again.

So why do it?

Because by putting a method in place to screen the entries before the tournament we will have made a step in improving the originality of the participating programs. And we will improve this further in the next events.

What do we request from you? First of all that you register for our tournaments, but we also value constructive feedback on our tournament rules, the process described above and anything else that we can use to improve the CSVN tournaments.

What will we do next? We feel there is room for more improvement: In the programmer's tournament but also in organizing other activities. We do have ideas of our own, but we also want to know what you would like us to do.

The CSVN site has been improved and also contains a forum. Please make use of that to discuss the CSVN activities. And you could start by providing us your feedback on this letter.
We hope to see you in one of the CSVN events soon,

Marcel van Kervinck
Richard Pijl

That fingerprint thing is a pretty stupid idea. The only way to make sure that only original engines enter your tournament is to accept only open source engines. Open source means that you can check, and if you're not a programmer, you can (supposedly) rely on the community to do it.

[jdart]

I don't have any problem submitting to this procedure. As Bob noted though its validity as a clone detection mechanism is currently unproven. A relatively small amount of copying could still be enough to constitute a license violation, and make the program unoriginal by most definitions. Personally I think it is a reasonable tradeoff to have a transparent and rapid testing method vs. trying to track down every possible instance of unoriginality. But some further testing might make it more acceptable. E.g. do the various Ippolit or Robbolito family engines test as similar with this method?

--Jon