UCI_Elo

[Ferdy]

Does anyone know of what is the base engine and rating of UCI_Elo?

Collected some engines with UCI_Elo support. Here is the result for UCI_Elo 2300, TC 1m+1s on i7 3.4ghz PC, format is RR and Gauntlet for Lc0 since it was added later. The anchor is CT800 with CCRL40/40 rating of around 2300.

UCI_Elo 2300

Code: Select all

   # PLAYER                              :  RATING  ERROR  POINTS  PLAYED   (%)
   1 Cheng 4.39 ucielo 2300              :  3187.9  173.9    95.0     102    93
   2 Fruit reloaded v3.21 ucielo 2300    :  2945.0  141.5    78.5     102    77
   3 Amyan 1.72 ucielo 2300              :  2832.9  134.8    68.0     103    66
   4 Cheese 2.0 ucielo 2300              :  2823.7  135.1    66.5     102    65
   5 Lc0 0.21.2 w48x5 blas               :  2789.5  119.1   154.0     243    63
   6 Rhetoric 1.4.3 ucielo 2300          :  2726.9  129.6    56.0     102    55
   7 Discocheck 5.2 ucielo 2300          :  2652.3  129.2    48.0     102    47
   8 Arasan 21.3 ucielo 2300             :  2628.2  126.3    45.5     102    45
   9 Wasp 3.60 ucielo 2300               :  2535.3  125.1    36.5     102    36
  10 CT800 V1.34 ucielo 2300             :  2300.0   ----    19.0     102    19
  11 D2019.2.37.53 ucielo 2300           :  2159.2  143.1    11.5     102    11
  12 Hiarcs 14 ucielo 2300               :  1983.5  188.0     4.5     102     4
  

Added Lc0 48x5 on blas to see where it stands against UCI_Elo 2300 engines.
I hope more engine authors would support UCI_Elo, this would allow user to select engine opponent easily to suit their strength.


And the UCI_Elo 2000, ordo was run at 2000 average rating.

Code: Select all

   # PLAYER                        :  RATING  ERROR  POINTS  PLAYED   (%)
   1 Cheng 4.39 ucielo 2000        :  2633.9  178.2    74.5      80    93
   2 Cheese 2.0 ucielo 2000        :  2508.2  146.9    69.5      80    87
   3 Amyan 1.72 ucielo 2000        :  2410.2  128.8    65.0      80    81
   4 Rhetoric 1.4.3 ucielo 2000    :  2171.5  108.8    52.5      80    66
   5 Ufim v8.02 ucielo 2000        :  2008.2  101.1    42.5      80    53
   6 Discocheck 5.2 ucielo 2000    :  1977.8  100.2    40.5      80    51
   7 MadChess 2.2 ucielo 2000      :  1817.3  102.8    29.5      80    37
   8 Arasan 21.3 ucielo 2000       :  1759.7  100.1    25.5      80    32
   9 CT800 V1.34 ucielo 2000       :  1624.1  112.6    16.5      80    21
  10 D2019.2.37.53 ucielo 2000     :  1607.9  111.7    15.5      80    19
  11 Hiarcs 14 ucielo 2000         :  1481.1  130.0     8.5      80    11

[Kotlov]

UCI_Elo is senseless parameter, IMHO.

[Patrice Duhamel]

Any advice to tune engine parameters to correspond to the right UCI_Elo value ?

In Cheese I have 2 parameters, one to make very small pauses every N nodes,
and another one to add randomness to the evaluation under 2000 ELO.

[Paloma]

I'm assuming that with uci_ELO 2300

Code: Select all

Cheng 4.39  3187 Elo
and
Hiarcs 14   1983  "

achieved ?

That must be wrong, or do I take this the wrong way?

[Ferdy]

Any advice to tune engine parameters to correspond to the right UCI_Elo value ?

In Cheese I have 2 parameters, one to make very small pauses every N nodes,
and another one to add randomness to the evaluation under 2000 ELO.

If we cannot find a reference engine and corresponding UCI_Elo value we might as well use the CCRL as a reference. CCRL is better compared to other rating lists in this regard because it has tested lots of weaker programs. Perhaps we can use single processor at CCRL 40/4, as a base for UCI_Elo.

I did try to scale Deuterium's UCI_Elo rating by running a self-test on games with nodes limit. nodes500 vs nodes1000, then nodes1000 vs nodes1500 and so on with increasing nodes.Then run Ordo anchoring one engine, n4000 at 1500 (can be any value as a start), then do regression to get the formula for rating given nodes. It is logarithmic. The x-axis is the nodes, the y is the rating.



For Deuterium

Code: Select all

rating = 297.12 x Ln(nodes) - 976.7

The formula can be adjusted to match the UCI_Elo.

Now we don't know exactly what is UCI_Elo 1000?, 1500? 2000?, 2300? Do We? So what I did is gather these engines having UCI_Elo support and run some tests, to have an idea of the result.

My initial general formula to scale the rating thru nodes is:

Code: Select all

rating = 297.12 x Ln(nodes) - 976.7

The user input is rating when selecting UCI_Elo, so we need to know how many nodes are we going to search to match the UCI_Elo requested by the user. To get the nodes given rating.
r = a * Ln(nodes) - b
a and b are constants

a*Ln(nodes) = r+b
Ln(nodes) = (r+b)/a
y = (r+b)/a
Ln(nodes) = y

Eliminate Ln
nodes = e^y
where e^y is e raised to power y

Given a rating 1500, I can get nodes by
r = 1500
a = 297.12
b = 976.7
y = (r+b)/a
y=(1500+976.7)/297.12 = 8.34
nodes = e^y
e = 2.718
nodes = 2.718^8.34 = 4184

So when I use that formula and user selects UCI_Elo rating, I will just calculate the nodes and use it to limit the search, in this particular case I will limit Deuterium at 4184 nodes.

I created my first test of UCI_elo 1500 together with other engines having UCI_elo support. The result was that Deuterium performed higher than elo 1500. So I adjusted my formula, now it is.

Code: Select all

nodes = e ^ ((uciElo + 200) / 297.12)

was

Code: Select all

nodes = e ^ ((uciElo + 976.7) / 297.12)

Repeat the tournament of UCI_Elo 1500 but now using the new formula

Code: Select all

nodes = e ^ ((uciElo + 200) / 297.12)

my node limit now is
nodes = e^(5.72) = 305

The result of this tournament is much better compared to the previous one. It still needs adjustment though.

Code: Select all

   # PLAYER                        :  RATING  POINTS  PLAYED   (%)
   1 Cheese 2.0 ucielo 1500        :  2354.0   193.0     200    97
   2 Cheng 4.39 ucielo 1500        :  2197.4   183.0     200    92
   3 Amyan 1.72 ucielo 1500        :  1738.5   145.5     200    73
   4 Ufim v8.02 ucielo 1500        :  1586.7   125.0     200    63
   5 Rhetoric 1.4.3 ucielo 1500    :  1422.4    97.0     200    49
   6 MadChess 2.2 ucielo 1500      :  1384.4    90.0     200    45
   7 D2019.2.37.53 ucielo 1500     :  1384.4    90.0     200    45
   8 Discocheck 5.2 ucielo 1500    :  1325.2    79.0     200    40
   9 Arasan 21.3 ucielo 1500       :  1142.5    47.0     200    24
  10 CT800 V1.34 ucielo 1500       :  1066.7    35.5     200    18
  11 Hiarcs 14 ucielo 1500         :   897.7    15.0     200     8

[MikeB]

Any advice to tune engine parameters to correspond to the right UCI_Elo value ?

In Cheese I have 2 parameters, one to make very small pauses every N nodes,
and another one to add randomness to the evaluation under 2000 ELO.

If we cannot find a reference engine and corresponding UCI_Elo value we might as well use the CCRL as a reference. CCRL is better compared to other rating lists in this regard because it has tested lots of weaker programs. Perhaps we can use single processor at CCRL 40/4, as a base for UCI_Elo.

I did try to scale Deuterium's UCI_Elo rating by running a self-test on games with nodes limit. nodes500 vs nodes1000, then nodes1000 vs nodes1500 and so on with increasing nodes.Then run Ordo anchoring one engine, n4000 at 1500 (can be any value as a start), then do regression to get the formula for rating given nodes. It is logarithmic. The x-axis is the nodes, the y is the rating.



For Deuterium

Code: Select all

rating = 297.12 x Ln(nodes) - 976.7

The formula can be adjusted to match the UCI_Elo.

Now we don't know exactly what is UCI_Elo 1000?, 1500? 2000?, 2300? Do We? So what I did is gather these engines having UCI_Elo support and run some tests, to have an idea of the result.

My initial general formula to scale the rating thru nodes is:

Code: Select all

rating = 297.12 x Ln(nodes) - 976.7

The user input is rating when selecting UCI_Elo, so we need to know how many nodes are we going to search to match the UCI_Elo requested by the user. To get the nodes given rating.
r = a * Ln(nodes) - b
a and b are constants

a*Ln(nodes) = r+b
Ln(nodes) = (r+b)/a
y = (r+b)/a
Ln(nodes) = y

Eliminate Ln
nodes = e^y
where e^y is e raised to power y

Given a rating 1500, I can get nodes by
r = 1500
a = 297.12
b = 976.7
y = (r+b)/a
y=(1500+976.7)/297.12 = 8.34
nodes = e^y
e = 2.718
nodes = 2.718^8.34 = 4184

So when I use that formula and user selects UCI_Elo rating, I will just calculate the nodes and use it to limit the search, in this particular case I will limit Deuterium at 4184 nodes.

I created my first test of UCI_elo 1500 together with other engines having UCI_elo support. The result was that Deuterium performed higher than elo 1500. So I adjusted my formula, now it is.

Code: Select all

nodes = e ^ ((uciElo + 200) / 297.12)

was

Code: Select all

nodes = e ^ ((uciElo + 976.7) / 297.12)

Repeat the tournament of UCI_Elo 1500 but now using the new formula

Code: Select all

nodes = e ^ ((uciElo + 200) / 297.12)

my node limit now is
nodes = e^(5.72) = 305

The result of this tournament is much better compared to the previous one. It still needs adjustment though.

Code: Select all

   # PLAYER                        :  RATING  POINTS  PLAYED   (%)
   1 Cheese 2.0 ucielo 1500        :  2354.0   193.0     200    97
   2 Cheng 4.39 ucielo 1500        :  2197.4   183.0     200    92
   3 Amyan 1.72 ucielo 1500        :  1738.5   145.5     200    73
   4 Ufim v8.02 ucielo 1500        :  1586.7   125.0     200    63
   5 Rhetoric 1.4.3 ucielo 1500    :  1422.4    97.0     200    49
   6 MadChess 2.2 ucielo 1500      :  1384.4    90.0     200    45
   7 D2019.2.37.53 ucielo 1500     :  1384.4    90.0     200    45
   8 Discocheck 5.2 ucielo 1500    :  1325.2    79.0     200    40
   9 Arasan 21.3 ucielo 1500       :  1142.5    47.0     200    24
  10 CT800 V1.34 ucielo 1500       :  1066.7    35.5     200    18
  11 Hiarcs 14 ucielo 1500         :   897.7    15.0     200     8

McCain also has UCI-ELo support.

The latest binaries can be found here. My method is different - I reduce the NPS . Level 1500 is around 50 nps.

https://github.com/MichaelB7/Stockfish/releases/tag/X4

[Ferdy]

McCain also has UCI-ELo support.

The latest binaries can be found here. My method is different - I reduce the NPS . Level 1500 is around 50 nps.

https://github.com/MichaelB7/Stockfish/releases/tag/X4

After typing uci, I cannot find UCI_Elo and UCI_LimitStrength.

Code: Select all

McCain X4 by M. Byrne and scores of others...
uci
id name McCain X4
id author M. Byrne and scores of others...

option name Debug Log File type string default
option name Clear_Hash type button
option name Use_Book_1 type check default false
option name Book_File_1 type string default
option name Best_Move_1 type check default false
option name Book_Depth_1 type spin default 127 min 1 max 127
option name Use_Book_2 type check default false
option name Book_File_2 type string default
option name Best_Move_2 type check default false
option name Book_Depth_2 type spin default 127 min 1 max 127
option name Use_Book_3 type check default false
option name Book_File_3 type string default
option name Best_Move_3 type check default true
option name Book_Depth_3 type spin default 127 min 1 max 127
option name Use_Book_4 type check default false
option name Book_File_4 type string default
option name Best_Move_4 type check default true
option name Book_Depth_4 type spin default 127 min 1 max 127
option name W_Contempt type spin default 22 min -150 max 150
option name B_Contempt type spin default 2 min -150 max 150
option name Analysis_Contempt type combo default Off var White var Black var Bot
h var Off
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 30 min 0 max 5000
option name Minimum Thinking Time type spin default 20 min 0 max 5000
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 131072
option name Ponder type check default false
option name 7 Man Probing type check default false
option name BruteForce type check default false
option name Clean Search type check default false
option name Dynamic_Contempt type check default true
option name FastPlay type check default false
option name Minimal_Output type check default false
option name No_Null_Moves type check default false
option name Play_By_Elo type check default false
option name Engine_Elo type spin default 1200 min 1200 max 2850
option name Engine_Level type combo default None var World_Champion var Super_GM
 var GM var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var
 Class_A var Class_B var Class_C var Class_D var Boris var Novice var None
option name Score_Output type combo default ScorPct-GUI var CentiPawn var ScorPc
t var ScorPct-GUI
option name DC_Slider type spin default 65 min -180 max 180
option name MultiPV type spin default 1 min 1 max 256
option name Bench_KNPS type spin default 1500 min 500 max 5000
option name Jekyll_&_Hyde type spin default 0 min 0 max 15
option name Tactical type spin default 0 min 0 max 8
option name Variety type spin default 0 min 0 max 15
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name SyzygyPath type string default
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
uciok

[Ferdy]

According to protocol.

* = UCI_Elo, type spin
The engine can limit its strength in Elo within this interval.
If UCI_LimitStrength is set to false, this value should be ignored.
If UCI_LimitStrength is set to true, the engine should play with this specific strength.

Looks like this is it https://en.wikipedia.org/wiki/Elo_rating_system.

[MikeB]

McCain also has UCI-ELo support.

The latest binaries can be found here. My method is different - I reduce the NPS . Level 1500 is around 50 nps.

https://github.com/MichaelB7/Stockfish/releases/tag/X4

After typing uci, I cannot find UCI_Elo and UCI_LimitStrength.

Code: Select all

McCain X4 by M. Byrne and scores of others...
uci
id name McCain X4
id author M. Byrne and scores of others...

option name Debug Log File type string default
option name Clear_Hash type button
option name Use_Book_1 type check default false
option name Book_File_1 type string default
option name Best_Move_1 type check default false
option name Book_Depth_1 type spin default 127 min 1 max 127
option name Use_Book_2 type check default false
option name Book_File_2 type string default
option name Best_Move_2 type check default false
option name Book_Depth_2 type spin default 127 min 1 max 127
option name Use_Book_3 type check default false
option name Book_File_3 type string default
option name Best_Move_3 type check default true
option name Book_Depth_3 type spin default 127 min 1 max 127
option name Use_Book_4 type check default false
option name Book_File_4 type string default
option name Best_Move_4 type check default true
option name Book_Depth_4 type spin default 127 min 1 max 127
option name W_Contempt type spin default 22 min -150 max 150
option name B_Contempt type spin default 2 min -150 max 150
option name Analysis_Contempt type combo default Off var White var Black var Bot
h var Off
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 30 min 0 max 5000
option name Minimum Thinking Time type spin default 20 min 0 max 5000
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 131072
option name Ponder type check default false
option name 7 Man Probing type check default false
option name BruteForce type check default false
option name Clean Search type check default false
option name Dynamic_Contempt type check default true
option name FastPlay type check default false
option name Minimal_Output type check default false
option name No_Null_Moves type check default false
option name Play_By_Elo type check default false
option name Engine_Elo type spin default 1200 min 1200 max 2850
option name Engine_Level type combo default None var World_Champion var Super_GM
 var GM var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var
 Class_A var Class_B var Class_C var Class_D var Boris var Novice var None
option name Score_Output type combo default ScorPct-GUI var CentiPawn var ScorPc
t var ScorPct-GUI
option name DC_Slider type spin default 65 min -180 max 180
option name MultiPV type spin default 1 min 1 max 256
option name Bench_KNPS type spin default 1500 min 500 max 5000
option name Jekyll_&_Hyde type spin default 0 min 0 max 15
option name Tactical type spin default 0 min 0 max 8
option name Variety type spin default 0 min 0 max 15
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name SyzygyPath type string default
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
uciok

To select the Elo, check the Play By Elo box, and the select the Engine ELo. It should work. I should look into it for the next release if it's causing you an issue.

[Ferdy]

McCain also has UCI-ELo support.

The latest binaries can be found here. My method is different - I reduce the NPS . Level 1500 is around 50 nps.

https://github.com/MichaelB7/Stockfish/releases/tag/X4

After typing uci, I cannot find UCI_Elo and UCI_LimitStrength.

Code: Select all

McCain X4 by M. Byrne and scores of others...
uci
id name McCain X4
id author M. Byrne and scores of others...

option name Debug Log File type string default
option name Clear_Hash type button
option name Use_Book_1 type check default false
option name Book_File_1 type string default
option name Best_Move_1 type check default false
option name Book_Depth_1 type spin default 127 min 1 max 127
option name Use_Book_2 type check default false
option name Book_File_2 type string default
option name Best_Move_2 type check default false
option name Book_Depth_2 type spin default 127 min 1 max 127
option name Use_Book_3 type check default false
option name Book_File_3 type string default
option name Best_Move_3 type check default true
option name Book_Depth_3 type spin default 127 min 1 max 127
option name Use_Book_4 type check default false
option name Book_File_4 type string default
option name Best_Move_4 type check default true
option name Book_Depth_4 type spin default 127 min 1 max 127
option name W_Contempt type spin default 22 min -150 max 150
option name B_Contempt type spin default 2 min -150 max 150
option name Analysis_Contempt type combo default Off var White var Black var Bot
h var Off
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 30 min 0 max 5000
option name Minimum Thinking Time type spin default 20 min 0 max 5000
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 131072
option name Ponder type check default false
option name 7 Man Probing type check default false
option name BruteForce type check default false
option name Clean Search type check default false
option name Dynamic_Contempt type check default true
option name FastPlay type check default false
option name Minimal_Output type check default false
option name No_Null_Moves type check default false
option name Play_By_Elo type check default false
option name Engine_Elo type spin default 1200 min 1200 max 2850
option name Engine_Level type combo default None var World_Champion var Super_GM
 var GM var Deep_Thought var SIM var Cray_Blitz var IM var Master var Expert var
 Class_A var Class_B var Class_C var Class_D var Boris var Novice var None
option name Score_Output type combo default ScorPct-GUI var CentiPawn var ScorPc
t var ScorPct-GUI
option name DC_Slider type spin default 65 min -180 max 180
option name MultiPV type spin default 1 min 1 max 256
option name Bench_KNPS type spin default 1500 min 500 max 5000
option name Jekyll_&_Hyde type spin default 0 min 0 max 15
option name Tactical type spin default 0 min 0 max 8
option name Variety type spin default 0 min 0 max 15
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name SyzygyPath type string default
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
uciok

To select the Elo, check the Play By Elo box, and the select the Engine ELo. It should work. I should look into it for the next release if it's causing you an issue.

I am only interested on UCI_Elo and UCI_LimitStrength options of the engine.