Murka 3.0 released

[Laskos]

Positions are chosen such as to not have a single best move.

I thought they were hundreds of random positions.

No, they are well filtered neutral positions, which have no clear best move, and unrelated engines often disagree on. There are positions on which even perfect engines can choose different moves, as shown by TBs.

[Steve Maughan]

(...)Positions are chosen such as to not have a single best move. No, as engines get stronger, they will not converge to have accidentally higher similarity.

Fascinating! A cool test.

In the case of Murka 3 I'd be interested to see what has been copied.

It would also be interesting to see what happens if you take a basic engine and inflate one aspect of the evaluation function and see how this impacts the similarity. Ed's work on the impact of the different evaluation terms may be an interesting place to start:

http://www.top-5000.nl/eval.htm

I may try this with Maverick (once it starts to play chess).

Steve

[Laskos]

Positions are chosen such as to not have a single best move.

I thought they were hundreds of random positions.

No, they are well filtered neutral positions, which have no clear best move, and unrelated engines often disagree on. There are positions on which even perfect engines can choose different moves, as shown by TBs.

Who filtered them?

Isn't this Sim by Don? In this case Don filtered them, 8,000+ positions.

[velmarin]

In the case of Murka 3 I'd be interested to see what has been copied.
Steve

If you are very interested, murka born on this date: 31st May 2010
In Inmortal forum, Russian section, google translate, is read without registration.
http://immortalchess.net/forum/showthre ... ight=murka

All three versions and their code.
http://igorkorshunov.narod.ru/Murka_1_0.rar
http://igorkorshunov.narod.ru/Murka_2.rar
http://igorkorshunov.narod.ru/Murka_3.rar

What the author says in his first post.

Code: Select all

 31st May 2010 16:44
Murka / Murka
Decided to write a new engine, as Sources old absolutely terrible. yet made ​​a generator rotating bitbordah and perft for verification. Speed ​​turned about comparable to what it was in the old engine. Already you can download the source code and executable files: Murka 1.0 . We must bear in mind that this is not the engine , but only the computer perft to preset positions. Hopefully our craftsmen will help than they can. in global terms we would like to go somewhere on the level of arrows , and then we will see. within the next few plans to make the simple support for UCI / Xboard , simple formatting (material, the value of the fields), alpha-beta (sort: take-killers, others), EF-taking. While like all ... 

A great author, and a very good engine ..

[velmarin]

I would not be surprised if any author filtered out 8000 positions and it turned out that his own engine is original.

If anyone can do that, you could call him Houdini or Rybka 5 6 for example.
That effort for nothing ...

Postdata:
Indeed, in this forum Don and explain how them out or change them.

[Laskos]

Positions are chosen such as to not have a single best move.

I thought they were hundreds of random positions.

No, they are well filtered neutral positions, which have no clear best move, and unrelated engines often disagree on. There are positions on which even perfect engines can choose different moves, as shown by TBs.

Who filtered them?

Isn't this Sim by Don? In this case Don filtered them, 8,000+ positions.

I wonder how much time he required to do that. And what purpose? Could have been spent on Komodo development.

BTW, I would not be surprised if any author filtered out 8000 positions and it turned out that his own engine is original.

Don is very interested in many aspects of game statistics and such. Look up in other threads, he is now working on a tool to detect human cheaters. As for him purposely filtering positions to show that Komodo is original, I doubt he did it, and I doubt it's doable. He simply filtered quiet, neutral positions as to have the most disagreement between varied engines.

[velmarin]

Don is very interested in many aspects of game statistics and such. Look up in other threads, he is now working on a tool to detect human cheaters. As for him purposely filtering positions to show that Komodo is original, I doubt he did it, and I doubt it's doable. He simply filtered quiet, neutral positions as to have the most disagreement between varied engines.

No one can doubt that, or Don,
or anyone else.

It is almost impossible and absurd work also.

That tool relase takes several years, there have been several versions of Komodo.
Also anyone with any knowledge can change positions, or give different search times, or other parameters.

It sounds absurd.

[IWB]

Though the similarity test seemingly has not produced any false positives,

Hi Adam,

That is a bold statement. "We do not know of false positives" would be better!

Anyhow, I don't like the test at all as it is used for a witch hunt. Just have a look at this thread. Even if the source is available anyone not knowing anything about programming can use it to throw stones. But that is something you already mentioned.

I personally do not think that this tool is good to prove anything. If there IS something like "perfect play" it might give a 100% similarity. Similarity is true but not saying that the two programs are identical. So, the better the programs get the more similar they might be ...

Bye
Ingo

[Steve Maughan]

Hi Ingo,

Your reaction was my first reaction. But I'm now intrigued.

Up to now I have had little interest in detecting clones. Simply because clones have zero interest for me. But this seems interesting.

If the positions are truly random with multiple "good" or reasonable moves then it is suspicious if there are a high percentage of common choices. Since the test is done with little time per position I really think it's a test of cloned evaluation functions. I cannot see how search plays too much of a role (I could be wrong).

My one point of skepticism is maybe the test is simply detecting similar styles and not just clones. So I could create an engine which likes to throw it's pieces towards the enemy's king. This (surely) increases the likelihood the engine will have a higher common hit rate with other aggressive engines. But I remain open minded.

Steve

[IWB]

...
If the positions are truly random with multiple "good" or reasonable moves then it is suspicious if there are a high percentage of common choices. Since the test is done with little time per position I really think it's a test of cloned evaluation functions....

Yes, I have no good argument against that, just, I can see other possibilities and do not like pointing to someone because of "circumstances"* (hard to translate, in German we talk if "Indizien")
Or in other words: Even a high possibility is no prove! (And in this case it is not even a high possibility because of common sense but just because of a tool saying so!)

Bye
Ingo

*Especialy not in this case as the source is available. It is a bit like having a video of a crime but judging on a drunken witness