MEA and temere.epd

[Rebel]

Not using given scores for alternatives moves, Minic @1sec per position is finding 3147 best moves (the are 4975 fen in the file). Isn't that too much good results for Minic ?

Sorry to come back with the same question, but it is normal that Minic 1sec search on 1 thread is performing 3127/4975 best move only ? Isn't that breaking the +-/25 elo estimate ?

Not necessarily.

Maybe your PC is faster than mine?

Also (as I understood) you ran the set outside the MEA system.

I have Minic 2.07 running now.

Have some 4975 seconds patience

[xr_a_y]

Not using given scores for alternatives moves, Minic @1sec per position is finding 3147 best moves (the are 4975 fen in the file). Isn't that too much good results for Minic ?

Sorry to come back with the same question, but it is normal that Minic 1sec search on 1 thread is performing 3127/4975 best move only ? Isn't that breaking the +-/25 elo estimate ?

Not necessarily.

Maybe your PC is faster than mine?

Also (as I understood) you ran the set outside the MEA system.

I have Minic 2.07 running now.

Have some 4975 seconds patience

Cool, thanks !

[Rebel]

Code: Select all

    EPD  : epd\temere.epd
    Time : 1000ms
                                                     Solving     Max    Total   Time   Hash          
    Engine           Score   Used Time Found   Pos     Time     Score    Rate    ms     Mb  Cpu  CCRL
 1  Minic 2.07       77853  01:25:22.3  2683  4975  00:06:06.6  149250  0.522   1000   128    1  2900

Would place Minic at rank 20 after Naum.

[xr_a_y]

Code: Select all

    EPD  : epd\temere.epd
    Time : 1000ms
                                                     Solving     Max    Total   Time   Hash          
    Engine           Score   Used Time Found   Pos     Time     Score    Rate    ms     Mb  Cpu  CCRL
 1  Minic 2.07       77853  01:25:22.3  2683  4975  00:06:06.6  149250  0.522   1000   128    1  2900

Would place Minic at rank 20 after Naum.

Which is OK with the CCRL order.

[Rebel]

Code: Select all

    EPD  : epd\temere.epd
    Time : 1000ms
                                                     Solving     Max    Total   Time   Hash          
    Engine           Score   Used Time Found   Pos     Time     Score    Rate    ms     Mb  Cpu  CCRL
 1  Minic 2.07       77853  01:25:22.3  2683  4975  00:06:06.6  149250  0.522   1000   128    1  2900

Would place Minic at rank 20 after Naum.

Which is OK with the CCRL order.

Yes, thing is with this util that it is an unexplored area and one thing I already learned that time controls like 100ms, 250ms and 500ms don't work well and with 1000ms things get better. Next thing to do is split the EPD into x parts and divide the run over x threads. That creares room to increase the time control in order (or more modest, in the hope) to get more reliable results.

[xr_a_y]

Just ran MEA on my hardware, Minic dev 1000ms per position 1 thread on temere.epd

Code: Select all

36142 / 49750 (0.726)

[pohl4711]

I created another set, 5000 random positions from a 9.7 million EPD database. The pattern remains the same.

http://rebel13.nl/mea/dc1-100ms.html
http://rebel13.nl/mea/dc1-250ms.html
http://rebel13.nl/mea/dc1-500ms.html

I will create 2 more of 5000 and if the pattern remains the same we end up with 20,000 positions which maybe (emphasis added) can serve as a second opinion for engine changes. 5000 positions at 100ms takes 10 minutes, 20,000 (just) 40-45 minutes.

This tool is really great, to compare NeuralNets of the same size very fast, when the thinkingtime is set to 1 ms and Slowmover Option is set to 0. When doing so, lc0 only calculates the board-position with its net, without using any MCTS.
But for this, it would be great to have more than the temere.epd with nearly 5000 positions.
Do you have any idea, when you will release the other 2 or 3 epd-sets, which you are developing?!?

[Rebel]

http://rebel13.nl/mea-sets.7z

3 sets of 10,000 positions each.

[pohl4711]

http://rebel13.nl/mea-sets.7z

3 sets of 10,000 positions each.

Great! Thanx!

One question: Are there double positions in these sets (old temere.epd, and set 2,3,4)?!?


Update: I used this online-tool on a merged set of all 4 sets:
http://textmechanic.com/text-tools/basi ... ate-lines/
And 34844 lines were remaining (out of 34975). So, there are 131 doubles in the 4 files. I deleted these doubles, but 131 out of 34975 is only 0.37%, that is acceptable, of course.

[Dann Corbit]

After stripping spurious e.p. from temere.epd I get the same count as you.
There are, in addition, duplicate EPD records with different move choices, embodied in this set:

Code: Select all

1r2r1k1/p4pp1/2p5/b1BqN1P1/P2P1P2/4Q1Rp/2b1P2P/5RK1 b - -
1r3rk1/3qbppp/p1npb3/1p1Np3/4P2P/2PBN3/PP3PP1/R2QK2R w KQ -
1r4rk/pp1qb2p/2n2p2/2pnp1pb/P1N5/2PP1NPP/1P3PB1/2BRRQK1 w - -
2br4/6k1/5p2/4p2B/1pR4p/1P3P1P/5KP1/8 b - -
2r2rk1/pbq2p1p/1p1bpp2/3p4/3P4/2PB1P2/PP3QPP/R3RNK1 b - -
2r3k1/pp3p1p/4R3/7p/1P1n4/P4P2/3B2PP/R5K1 b - -
2r3k1/r3p2p/2p1P1p1/p1P2p2/7P/1R4P1/P4P2/3R2K1 b - -
2r5/R1n3RP/P1kp4/2p2p1r/2P5/6P1/6K1/8 w - -
2r5/p1q2k1p/1pr1pn2/5pp1/2BR4/PP2PP2/4QP1P/3R2K1 w - -
2rr2k1/pb2q2p/1pp1ppp1/8/3P4/1Q3B1P/PP2NNP1/2bR2K1 w - -
3r1b2/kpq2p2/p1p1nNnp/P4N2/R7/2Q3PB/1P2PP1K/8 b - -
3r2k1/1b2bpq1/p3p3/1p5Q/5B1P/1PN5/1PP4P/2K1R3 b - -
3r4/1q4k1/p1np2pp/b1p2p2/2P1p3/1PP1P2P/4QPPB/1NR3K1 w - -
3r4/2p4p/kp3bp1/p1n5/3Nn3/4BB2/PP2PP1P/3RK3 b - -
3rr1k1/3q1pbp/4pnp1/1pp3B1/7P/1NPP2P1/1P2QP2/R3R1K1 b - -
4r1k1/2q2pp1/p3b2p/p2p4/P1rR4/Q1P1P2P/4NPP1/1R4K1 b - -
5k2/7p/3p4/3P1b2/2P1q3/5p1P/5Q2/5B1K w - -
5r2/p3k3/b6p/P1p1p1bP/2PpP1P1/1R4P1/4B1K1/4B3 w - -
5rk1/pp1qbppp/2r1p3/3pP3/3P4/2RQ1N1P/PP3PP1/2R3K1 w - -
6k1/2nrr1q1/1p1n1b2/1Pp1p1p1/R1P1PpP1/R2P1N1Q/1B3N2/5K2 b - -
6k1/6p1/6Bp/p1q4P/3R4/1b6/4N3/2B1K3 b - -
6k1/B1q2pbp/6p1/4pbP1/p4P1P/P1P5/1P2RQBK/3r4 w - -
6k1/pp3pb1/2ppqnpp/8/3P2P1/2N1QN1P/PPP2PK1/8 w - -
7k/p5b1/1Pb1p2p/n3P3/2P5/P2BqN1Q/1B4K1/1R6 b - -
8/1q1b1pk1/1r1p1n2/2p1p1p1/4P3/2PP1PB1/B3Q1P1/3R2K1 b - -
8/5knp/2p5/p1b1Pp2/P1P2P2/3N4/8/3K4 w - -
8/5p2/1brpk3/pN2p1p1/2P1Pp2/2K2P1P/1P4P1/3R4 w - -
8/5p2/3p2pk/3P4/4rP2/1r1p2Pp/1P1R1K1P/1R6 w - -
8/6k1/5pP1/5P2/1p1b2P1/1P6/PnK1B3/8 w - -
8/6pk/4pn1p/P1q2p2/8/1N3P2/1P2Q1P1/3K4 b - -
8/N2b4/1Bpbkpp1/P6p/2P1KP1P/6P1/8/8 w - -
b3r1k1/2pQ1ppp/8/4qN2/1p4P1/6P1/1PP1PP2/5RK1 b - -
q7/r3b1pk/2p2p1p/2p1p3/2P1P2P/P2R2P1/5PK1/2BQ4 b - -
r1bqr1k1/1p1n2bp/2p1p1p1/3p1p2/p1PPn3/BPN1P1PN/PQ3PBP/2RR2K1 w - -
r1r3k1/5pp1/1pN4p/1Pb5/4p3/4P1P1/2R2PP1/3R2K1 b - -
r2br1k1/1p1q1ppn/2p3b1/3pB3/N2P2P1/P3PP2/1P4B1/2RQR1K1 b - -
r2q1r1k/pp2ppbp/1nnp2p1/2p5/P1B1PP2/2NPBR1P/1PP3P1/RQ4K1 w - -
r2q1rk1/1bp2ppp/3p4/p3P3/1p1B1P2/1P2Q3/PP4PP/3R1RK1 b - -
r2qk2r/pp1bbppp/2n1p1n1/3pP3/3P4/4BN2/PP2BPPP/RN1Q1RK1 w kq -
r2r2k1/5pp1/1ppNp1b1/3nP2p/1q1P4/6PP/1P1Q1PN1/R1R3K1 w - -
r4k2/1ppq1pp1/2npb2r/p6p/2P4P/1PN2BP1/P2Q1P2/R3R1K1 b - -
r4k2/5ppp/p3pb2/1pN5/1P1P4/P3PPP1/7P/4B1K1 w - -
r4rk1/1pqb1p1p/p2ppp1b/4n3/P3P3/1NNB4/1PP2PPP/R2Q1RK1 w - -
r4rk1/p2bn1pp/3q1n2/1p1pNp2/2pP1P2/2N1P3/PP4PP/1BRQ1R1K w - -
r5k1/p3ppbp/bp4p1/q1p1n3/3P1B2/2P1P3/P2Q1PPP/1R2K2R w K -
r7/2p5/3b4/PNpPpp1k/2P2q2/2Q2P2/5KR1/8 b - -