Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Cheng 4.39 ucielo 1500 : 597 195 60.5 64 94.5
2 Cheese 2.1 ucielo 1500 : 366 132 53.0 64 82.8
3 Rybka v2.3.2a ucielo 1500 : 300 122 50.0 64 78.1
4 Fruit reloaded v3.21 ucielo 1500 : 300 122 50.0 64 78.1
5 Amyan 1.72 ucielo 1500 : 231 116 46.5 64 72.7
6 Houdini 3 ucielo 1500 : 76 105 37.5 64 58.6
7 Ufim v8.02 ucielo 1500 : 76 102 37.5 64 58.6
8 Rhetoric 1.4.3 ucielo 1500 : 44 102 35.5 64 55.5
9 MadChess 2.2 ucielo 1500 : -80 100 27.5 64 43.0
10 Discocheck 5.2 ucielo 1500 : -111 100 25.5 64 39.8
11 Deuterium v2019.2.37.71 ucielo 1500 : -111 105 25.5 64 39.8
12 Stockfish 260819 ucielo 1500 : -126 108 24.5 64 38.3
13 Rodent IV 021 ucielo 1500 : -150 105 23.0 64 35.9
14 Arasan 21.3 ucielo 1500 : -267 109 16.0 64 25.0
15 DanaSah 7.9 engine_opp ucielo 1500 : -276 113 15.5 64 24.2
16 CT800 V1.34 ucielo 1500 : -410 137 9.0 64 14.1
17 Hiarcs 14 ucielo 1500 : -461 153 7.0 64 10.9
Updated my TOPSIS ranking, by testing at (1s/pos) these new engines with 5k positions from human games with a rating of 1450 to 1550.
Test result table:
Code: Select all
UCI_Elo 1500 engine test results on FIDE Elo 1500
Test positions are taken from players with FIDE Elo 1450 to 1550
Engine Total Match High Low HACD LACD HEMSE
Ufim v8.02 UCI_Elo 1500 5000 2168 1363 1469 422 361 4737056
Arasan 21.3 UCI_Elo 1500 5000 1641 1167 2192 495 452 6708957
CT800 V1.34 UCI_Elo 1500 5000 1642 1149 2209 333 807 10656012
DanaSah 7.9 human_opp UCI_Elo 1500 5000 1679 1202 2119 461 396 6041935
Cheng 4.39 UCI_Elo 1500 5000 2068 1466 1466 474 300 5016436
Discocheck 5.2 UCI_Elo 1500 5000 1948 1308 1744 381 443 5505496
Amyan 1.72 UCI_Elo 1500 5000 1803 1218 1979 419 561 7648687
MadChess 2.2 UCI_Elo 1500 5000 1693 1240 2067 424 552 7680236
Cheese 2.1 UCI_Elo 1500 5000 2123 1486 1391 404 210 3546737
Rhetoric 1.4.3 UCI_Elo 1500 5000 1853 1349 1798 357 461 5719087
Stockfish 10 5000 2244 2340 416 355 46 3238666
Arminius 2017-01-01 5000 2216 1827 957 420 68 3236474
Rybka v2.3.2a UCI_Elo 1500 5000 2128 1368 1504 428 269 4128324
Rodent IV 021 UCI_Elo 1500 5000 2013 1270 1717 410 537 6605086
DanaSah 7.9 engine_opp ucielo 1500 5000 1730 1214 2056 425 390 5673680
Stockfish 260819 UCI_Elo 1500 5000 1337 1252 2411 421 436 6484532
Houdini 3 UCI_Elo 1500 5000 1722 1290 1988 446 236 4121787
Hiarcs 14 UCI_Elo 1500 5000 1795 1107 2098 376 695 8627186
Deuterium v2019.2.37.71 UCI_Elo 1500 5000 1896 1297 1807 433 311 4515235
::Legend::
Total: Number of test positions from human games.
Match: Count of pos, where engine and human move are the same.
High : Count of pos, where engine move is stronger than human move.
Low : Count of pos, where engine move is weaker than human move.
HACD : High Average Centipawn Difference, or diff between engine move score
and human move score where engine move is stronger than human move
according to Stockfish dev 2019.04.16
LACD : Low Average Centipawn Difference, or diff between engine move score
and human move score where engine move is weaker than human move.
HEMSE: Human and Engine MSE or Sum((HumanScore - EngineScore)^2)/total, smaller is better.
TOPSIS:
Apply 3 criteria with corresponding weight.
Match, w=0.1, maximize
LACD, w=0.4, maximize
HEMSE, w=0.5, minimize
Code: Select all
TOPSIS (mnorm=vector, wnorm=sum) - Solution:
ALT./CRIT. Match (max) W.0.1 LACD (max) W.0.4 HEMSE (min) W.0.5 Rank
------------------------------------ ------------------- ------------------ ------------------- ------
Ufim v8.02 UCI_Elo 1500 2168 361 4.73706e+06 5
Arasan 21.3 UCI_Elo 1500 1641 452 6.70896e+06 12
CT800 V1.34 UCI_Elo 1500 1642 807 1.0656e+07 13
DanaSah 7.9 human_opp UCI_Elo 1500 1679 396 6.04194e+06 14
Cheng 4.39 UCI_Elo 1500 2068 300 5.01644e+06 17
Discocheck 5.2 UCI_Elo 1500 1948 443 5.5055e+06 3
Amyan 1.72 UCI_Elo 1500 1803 561 7.64869e+06 6
MadChess 2.2 UCI_Elo 1500 1693 552 7.68024e+06 8
Cheese 2.1 UCI_Elo 1500 2123 210 3.54674e+06 15
Rhetoric 1.4.3 UCI_Elo 1500 1853 461 5.71909e+06 2
Stockfish 10 2244 46 3.23867e+06 19
Arminius 2017-01-01 2216 68 3.23647e+06 18
Rybka v2.3.2a UCI_Elo 1500 2128 269 4.12832e+06 10
Rodent IV 021 UCI_Elo 1500 2013 537 6.60509e+06 1
DanaSah 7.9 engine_opp ucielo 1500 1730 390 5.67368e+06 9
Stockfish 260819 UCI_Elo 1500 1337 436 6.48453e+06 11
Houdini 3 UCI_Elo 1500 1722 236 4.12179e+06 16
Hiarcs 14 UCI_Elo 1500 1795 695 8.62719e+06 4
Deuterium v2019.2.37.71 UCI_Elo 1500 1896 311 4.51524e+06 7
TOPSIS ref:
https://en.wikipedia.org/wiki/TOPSIS
https://scikit-criteria.readthedocs.io/ ... start.html