MEA and temere.epd

[Rebel]

MEA is a tool (written by Ferdinand Mosca) that analyses an EPD position set in STS-style assigning bonus points pre-defined in each EPD record.

temere.epd (Latin for random) is a 4975 position set created from a much bigger random EPD collection in an intelligent way with as goal to produce a reasonable reliable ranking list of engines with an estimated error bar of -25/+25 elo at fast time controls like 100ms, 250ms, 500ms etc.

Examples

....

http://rebel13.nl/misc/mea.html

[Norm Pollock]

Is this an error?

1r3rk1/3qbppp/p1npb3/1p1Np3/4P2P/2PBN3/PP3PP1/R2QK2R w KQ - bm a4; c0 "Nf5=10, a4=9, Qf3=9, Bc2=8";
1r3rk1/3qbppp/p1npb3/1p1Np3/4P2P/2PBN3/PP3PP1/R2QK2R w KQ - bm a4; c0 "Nf5=10, a4=3, Qf3=1, Bc2=0";

[Alayan]

If you get the EPD position set big enough and the evaluation quality of the moves high enough, this could get really close to measuring playing strength. The results are already quite good.

One thing such an approach would require is to score more moves. An engine should lose some points compared to the maximum when playing an inaccuracy, but it should lose a lot when playing an outright blunder.

I could imagine that with enough refinements, such an approach could begin giving good insights on a patch quality.

EDIT : I think it would make more sense to compare with FastGM bullet ratings. Much lower error bars than CCRL.

[Rebel]

Is this an error?

1r3rk1/3qbppp/p1npb3/1p1Np3/4P2P/2PBN3/PP3PP1/R2QK2R w KQ - bm a4; c0 "Nf5=10, a4=9, Qf3=9, Bc2=8";
1r3rk1/3qbppp/p1npb3/1p1Np3/4P2P/2PBN3/PP3PP1/R2QK2R w KQ - bm a4; c0 "Nf5=10, a4=3, Qf3=1, Bc2=0";

Ah, a double after all, odd...

[Rebel]

If you get the EPD position set big enough and the evaluation quality of the moves high enough, this could get really close to measuring playing strength. The results are already quite good.

One thing such an approach would require is to score more moves. An engine should lose some points compared to the maximum when playing an inaccuracy, but it should lose a lot when playing an outright blunder.

I could imagine that with enough refinements, such an approach could begin giving good insights on a patch quality.

I am not so optimistic as you Doing this too long. But who knows? I am now creating creating a second temere set, we will see. Maybe this just was a lucky shot.

EDIT : I think it would make more sense to compare with FastGM bullet ratings. Much lower error bars than CCRL.

The thing with FastGM is that contrary to CCRL FastGM only test top-engines. But there is an easy fix, use the FastGM elo's in the "set CCRL=" option, then open "new.html" with a text editor and replace "CCRL" into "FastGM", save the file.

[abulmo2]

I guess many active chesss engine developers use linux instead of windows here. Is there a linux port of this software suite?

[Rebel]

I guess many active chesss engine developers use linux instead of windows here. Is there a linux port of this software suite?

Unfortunately my compiler doesn't support Linux.

[xr_a_y]

I guess many active chesss engine developers use linux instead of windows here. Is there a linux port of this software suite?

As the .edp is given in the zip file, it is easy to run it by yourself in your own tool.

[xr_a_y]

I guess many active chesss engine developers use linux instead of windows here. Is there a linux port of this software suite?

As the .edp is given in the zip file, it is easy to run it by yourself in your own tool.

Not using given scores for alternatives moves, Minic @1sec per position is finding 3147 best moves (the are 4975 fen in the file). Isn't that too much good results for Minic ?

[Rebel]

I guess many active chesss engine developers use linux instead of windows here. Is there a linux port of this software suite?

What I understood from a previous discussion is that MEA.exe (which is a Python application) won't run under Wine, the rest of the package (mrl.exe and csv.exe) does. Maybe one should install Python first?

https://www.google.nl/search?sourceid=n ... 6271258004