best way to determine elos of a group
Moderators: hgm, Rebel, chrisw
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: best way to determine elos of a group
@Daniel,
How do you calculate the error bars?
I seem to recall BayesElo does it by default by assuming that the elos of the other engines are fixed. But then of course the error bar does not represent the variance of the absolute elo.
The correct way to do it is to invert the Hessian of the log likelihood (you have to delete the row and column corresponding to the anchor engine).
Ordo does it by simulation, which is also correct.
How do you calculate the error bars?
I seem to recall BayesElo does it by default by assuming that the elos of the other engines are fixed. But then of course the error bar does not represent the variance of the absolute elo.
The correct way to do it is to invert the Hessian of the log likelihood (you have to delete the row and column corresponding to the anchor engine).
Ordo does it by simulation, which is also correct.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: best way to determine elos of a group
That is what do. Here it is in codeMichel wrote: ↑Tue Jul 30, 2019 6:29 pm @Daniel,
How do you calculate the error bars?
I seem to recall BayesElo does it by default by assuming that the elos of the other engines are fixed. But then of course the error bar does not represent the variance of the absolute elo.
The correct way to do it is to invert the Hessian of the log likelihood (you have to delete the row and column corresponding to the anchor engine).
Code: Select all
816 //variance
817 static void calc_variance() {
818 double sd;
819 covariance.assign(RSIZE * RSIZE,0.0);
820 for(unsigned i = 0;i < n_players;i++) {
821 sd = covariance[pair(i,i)] = (-1 / hessian[pair(i,i)]);
822 sd = sqrt(sd);
823 players[i].deltau = 1.96 * sd;
824 players[i].deltal = 1.96 * sd;
825 }
826 }
Ordo does it by simulation, which is also correct.
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: best way to determine elos of a group
Do I read it correctly that you only invert the diagonal of the Hessian matrix? In principle this is wrong (it amounts to assuming that the other elo's are fixed). You have to invert the full matrix (minus the row/column corresponding to the anchor engine), and then take the diagonal elements.Daniel Shawul wrote: ↑Tue Jul 30, 2019 7:04 pmThat is what do. Here it is in codeMichel wrote: ↑Tue Jul 30, 2019 6:29 pm @Daniel,
How do you calculate the error bars?
I seem to recall BayesElo does it by default by assuming that the elos of the other engines are fixed. But then of course the error bar does not represent the variance of the absolute elo.
The correct way to do it is to invert the Hessian of the log likelihood (you have to delete the row and column corresponding to the anchor engine).I thought that is what bayeselo does too. I will run the ratings with bayeselo but last time i checked my rating tool gives same results as bayeselo when draws are ignored.Code: Select all
816 //variance 817 static void calc_variance() { 818 double sd; 819 covariance.assign(RSIZE * RSIZE,0.0); 820 for(unsigned i = 0;i < n_players;i++) { 821 sd = covariance[pair(i,i)] = (-1 / hessian[pair(i,i)]); 822 sd = sqrt(sd); 823 players[i].deltau = 1.96 * sd; 824 players[i].deltal = 1.96 * sd; 825 } 826 }
Ordo does it by simulation, which is also correct.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: best way to determine elos of a group
Ah...I see, I do use the same assumption as bayeselo for calculating error bars. However I do the full hessian matrix inversion
when calculating likelihood of superiority (LOS). I am a bit rusty with the code since it has been too long.
when calculating likelihood of superiority (LOS). I am a bit rusty with the code since it has been too long.
Code: Select all
827 //covariance = inverse of information matrix (negative hessian)
828 static void calc_covariance() {
829 unsigned i;
830 for(i = 0;i < RSIZE * RSIZE;i++)
831 covariance[i] = -hessian[i];
832
833 //lu decompose with partial pivoting
834 vector<int> pivot(n_players);
835 Doolittle_LU(&covariance[0],&pivot[0],n_players);
836
837 //construct A matrix
838 vector<double> A(RSIZE * RSIZE,-1.0 / n_players);
839 for(i = 0;i < n_players;i++)
840 A[pair(i,i)] += 1;
841
842 //solve for AC
843 vector<double> AC(RSIZE * RSIZE,0.0);
844 for(i = 0;i < n_players;i++) {
845 int Index = i * n_players;
846 Doolittle_Solve(&covariance[0], &A[Index], &pivot[0],&AC[Index],n_players);
847 }
848 //multiply AC by A to get covariance matrix
849 Mul_Transpose(&AC[0],&A[0],&covariance[0],n_players);
850
851 for(unsigned i = 0;i < n_players;i++) {
852 for(unsigned j = 0;j < n_players;j++) {
853 print("%d %d. %3d",covariance[pair(i,j)]);
854 }
855 print("\n");
856 }
857 }
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: best way to determine elos of a group
This appears to be more or less correct. I assume that somewhere you delete a row and column from the Hessian matrix (the full Hessian matrix is singular)?Daniel Shawul wrote: ↑Tue Jul 30, 2019 7:28 pm Ah...I see, I do use the same assumption as bayeselo for calculating error bars. However I do the full hessian matrix inversion
when calculating likelihood of superiority (LOS). I am a bit rusty with the code since it has been too long.
Code: Select all
827 //covariance = inverse of information matrix (negative hessian) 828 static void calc_covariance() { 829 unsigned i; 830 for(i = 0;i < RSIZE * RSIZE;i++) 831 covariance[i] = -hessian[i]; 832 833 //lu decompose with partial pivoting 834 vector<int> pivot(n_players); 835 Doolittle_LU(&covariance[0],&pivot[0],n_players); 836 837 //construct A matrix 838 vector<double> A(RSIZE * RSIZE,-1.0 / n_players); 839 for(i = 0;i < n_players;i++) 840 A[pair(i,i)] += 1; 841 842 //solve for AC 843 vector<double> AC(RSIZE * RSIZE,0.0); 844 for(i = 0;i < n_players;i++) { 845 int Index = i * n_players; 846 Doolittle_Solve(&covariance[0], &A[Index], &pivot[0],&AC[Index],n_players); 847 } 848 //multiply AC by A to get covariance matrix 849 Mul_Transpose(&AC[0],&A[0],&covariance[0],n_players); 850 851 for(unsigned i = 0;i < n_players;i++) { 852 for(unsigned j = 0;j < n_players;j++) { 853 print("%d %d. %3d",covariance[pair(i,j)]); 854 } 855 print("\n"); 856 } 857 }
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 201
- Joined: Thu Jun 06, 2019 8:05 pm
- Full name: Percival Tiglao
Re: best way to determine elos of a group
Hmm, I looked over the data and items aren't obvious to me.Daniel Shawul wrote: ↑Tue Jul 30, 2019 5:24 pm Here is the full ratings table with error bars:
http://scorpiozero.ddns.net/scorpiozero ... atings.txt
My guess is:1 ID-146 557 46 46 200 55.8% 504 35.5%
Rank -- ID -- Elo -- ?? -- ?? -- Number of Matches -- Win Percent? -- Elo-lower bound? -- ??
I'm probably very wrong with my above guesses.
-
- Posts: 1968
- Joined: Wed Jul 13, 2011 9:04 pm
- Location: Madrid, Spain.
Re: best way to determine elos of a group
Hello:
Score = (wins + 0.5*draws)/games = (wins + 0.5*draws)/(wins + draws + loses).
Error margins in Elo (Bayeselo), probably up to 1.96-sigma (95% confidence) or 2-sigma (~ 95.45% confidence) if a normal distribution is used somewhere, which I do not know.
Daniel will know the answer, of course.
Regards from Spain.
Ajedrecista.
I would say:dragontamer5788 wrote: ↑Tue Jul 30, 2019 8:07 pmHmm, I looked over the data and items aren't obvious to me.Daniel Shawul wrote: ↑Tue Jul 30, 2019 5:24 pm Here is the full ratings table with error bars:
http://scorpiozero.ddns.net/scorpiozero ... atings.txt
My guess is:1 ID-146 557 46 46 200 55.8% 504 35.5%
Rank -- ID -- Elo -- ?? -- ?? -- Number of Matches -- Win Percent? -- Elo-lower bound? -- ??
I'm probably very wrong with my above guesses.
Code: Select all
Rank -- ID -- Elo -- Upper error margin -- Lower error margin -- Score (%) -- Average opponents' Elo -- Draws (%)
Error margins in Elo (Bayeselo), probably up to 1.96-sigma (95% confidence) or 2-sigma (~ 95.45% confidence) if a normal distribution is used somewhere, which I do not know.
Daniel will know the answer, of course.
Regards from Spain.
Ajedrecista.
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: best way to determine elos of a group
Yes, I think i take care of that during partial pivoting iirc.Michel wrote: ↑Tue Jul 30, 2019 7:57 pmThis appears to be more or less correct. I assume that somewhere you delete a row and column from the Hessian matrix (the full Hessian matrix is singular)?Daniel Shawul wrote: ↑Tue Jul 30, 2019 7:28 pm Ah...I see, I do use the same assumption as bayeselo for calculating error bars. However I do the full hessian matrix inversion
when calculating likelihood of superiority (LOS). I am a bit rusty with the code since it has been too long.
Code: Select all
827 //covariance = inverse of information matrix (negative hessian) 828 static void calc_covariance() { 829 unsigned i; 830 for(i = 0;i < RSIZE * RSIZE;i++) 831 covariance[i] = -hessian[i]; 832 833 //lu decompose with partial pivoting 834 vector<int> pivot(n_players); 835 Doolittle_LU(&covariance[0],&pivot[0],n_players); 836 837 //construct A matrix 838 vector<double> A(RSIZE * RSIZE,-1.0 / n_players); 839 for(i = 0;i < n_players;i++) 840 A[pair(i,i)] += 1; 841 842 //solve for AC 843 vector<double> AC(RSIZE * RSIZE,0.0); 844 for(i = 0;i < n_players;i++) { 845 int Index = i * n_players; 846 Doolittle_Solve(&covariance[0], &A[Index], &pivot[0],&AC[Index],n_players); 847 } 848 //multiply AC by A to get covariance matrix 849 Mul_Transpose(&AC[0],&A[0],&covariance[0],n_players); 850 851 for(unsigned i = 0;i < n_players;i++) { 852 for(unsigned j = 0;j < n_players;j++) { 853 print("%d %d. %3d",covariance[pair(i,j)]); 854 } 855 print("\n"); 856 } 857 }
I checked bayeselo code now and it seems there is an option to calculate the deltas from either the variance or covariance matrix.
The default is to use the diagonal of the covariance matrix which is why you thought it does not have that option.
Code: Select all
variance ........ compute intervals with the diagonal of the Hessian
covariance ...... compute intervals with the full Hessian
Default delta calculated with inverse of diagonal of hessian
Code: Select all
Rank Name Elo + - games score oppo. draws
1 scorpio-146 415 40 40 200 56% 379 36%
2 scorpio-147 383 56 56 100 45% 415 34%
3 scorpio-145 374 40 40 200 50% 375 38%
4 scorpio-134 347 39 39 200 55% 317 43%
5 scorpio-144 335 39 39 200 48% 350 40%
6 scorpio-141 329 39 39 200 52% 315 38%
7 scorpio-143 326 40 40 200 49% 330 35%
8 scorpio-142 325 40 40 200 50% 327 37%
9 scorpio-133 319 40 40 200 51% 310 37%
10 scorpio-135 315 39 39 200 48% 327 43%
11 scorpio-137 311 40 40 200 52% 298 34%
12 scorpio-136 308 39 39 200 49% 313 37%
13 scorpio-139 308 39 39 200 52% 297 37%
14 scorpio-140 306 40 40 200 48% 318 34%
15 scorpio-138 288 40 40 200 47% 309 37%
16 scorpio-131 283 39 39 200 56% 247 39%
17 scorpio-132 273 40 40 200 46% 301 36%
18 scorpio-125 243 40 40 200 55% 210 32%
19 scorpio-128 224 39 39 200 51% 217 37%
20 scorpio-129 222 39 39 200 50% 222 40%
21 scorpio-130 221 39 39 200 45% 253 41%
22 scorpio-124 218 40 40 200 50% 218 34%
23 scorpio-127 212 39 39 200 50% 213 39%
24 scorpio-121 203 40 40 200 57% 154 35%
25 scorpio-126 201 40 40 200 46% 227 35%
26 scorpio-123 193 39 39 200 51% 185 38%
27 scorpio-118 172 39 39 200 54% 149 41%
28 scorpio-120 156 39 39 200 47% 176 38%
29 scorpio-122 152 39 39 200 43% 198 38%
30 scorpio-117 150 54 54 101 47% 170 42%
31 scorpio-119 149 39 39 200 48% 164 42%
32 scorpio-17 62 38 38 200 56% 24 50%
33 scorpio-16 38 38 38 200 50% 40 50%
34 scorpio-14 35 39 39 200 55% 5 41%
35 scorpio-15 18 38 38 200 47% 36 47%
36 scorpio-18 10 38 38 200 47% 27 50%
37 scorpio-115 10 39 39 200 53% -11 38%
38 scorpio-101 8 38 38 200 56% -25 48%
39 scorpio-116 8 55 55 101 50% 11 40%
40 scorpio-11 6 27 27 400 52% -8 42%
41 scorpio-21 6 38 38 200 53% -11 49%
42 scorpio-10 3 27 27 400 51% -2 44%
43 scorpio-99 0 39 39 200 55% -28 43%
44 scorpio-20 -6 38 38 200 49% 0 50%
45 scorpio-19 -7 37 37 200 49% 2 52%
46 scorpio-8 -7 28 28 400 52% -17 41%
47 scorpio-13 -8 38 38 200 48% 8 46%
48 scorpio-86 -11 39 39 200 53% -30 41%
49 scorpio-9 -11 27 27 400 49% -2 43%
50 scorpio-22 -17 37 37 200 50% -16 55%
51 scorpio-12 -20 31 31 300 47% 2 45%
52 scorpio-109 -21 39 39 200 55% -53 39%
53 scorpio-104 -23 39 39 200 55% -52 39%
54 scorpio-7 -24 27 27 400 51% -28 45%
55 scorpio-112 -24 39 39 200 52% -37 43%
56 scorpio-102 -24 38 38 200 49% -18 48%
57 scorpio-100 -25 38 38 200 45% 4 48%
58 scorpio-87 -27 38 38 200 49% -21 46%
59 scorpio-97 -27 39 39 200 53% -45 42%
60 scorpio-114 -30 40 40 200 48% -15 36%
61 scorpio-98 -30 39 39 200 48% -14 42%
62 scorpio-88 -30 38 38 200 51% -34 50%
63 scorpio-55 -32 38 38 200 53% -50 48%
64 scorpio-85 -33 39 39 200 52% -43 42%
65 scorpio-56 -35 38 38 200 51% -41 48%
66 scorpio-111 -35 39 39 200 51% -37 40%
67 scorpio-107 -35 39 39 200 54% -60 43%
68 scorpio-37 -36 37 37 200 51% -44 60%
69 scorpio-24 -37 38 38 200 54% -61 51%
70 scorpio-47 -38 37 37 200 54% -63 52%
71 scorpio-23 -39 37 37 200 48% -27 53%
72 scorpio-113 -39 39 39 200 48% -27 42%
73 scorpio-90 -40 39 39 200 51% -43 43%
74 scorpio-33 -40 38 38 200 54% -64 48%
75 scorpio-89 -42 38 38 200 49% -35 47%
76 scorpio-38 -43 36 36 200 50% -42 62%
77 scorpio-103 -44 39 39 200 47% -23 42%
78 scorpio-36 -44 37 37 200 51% -48 56%
79 scorpio-91 -44 38 38 200 51% -47 45%
80 scorpio-39 -47 37 37 200 50% -47 59%
81 scorpio-6 -48 27 27 400 49% -45 46%
82 scorpio-48 -48 38 38 200 52% -60 46%
83 scorpio-40 -50 37 37 200 53% -67 54%
84 scorpio-43 -50 37 37 200 53% -66 55%
85 scorpio-57 -50 38 38 200 52% -60 51%
86 scorpio-110 -51 39 39 200 47% -28 37%
87 scorpio-34 -51 37 37 200 50% -50 59%
88 scorpio-1 -53 29 29 400 56% -96 30%
89 scorpio-108 -54 38 38 200 46% -28 48%
90 scorpio-92 -55 38 38 200 50% -54 47%
91 scorpio-35 -60 37 37 200 48% -48 60%
92 scorpio-105 -60 38 38 200 47% -44 45%
93 scorpio-95 -61 38 38 200 51% -66 50%
94 scorpio-96 -61 39 39 200 48% -44 43%
95 scorpio-93 -63 39 39 200 50% -62 40%
96 scorpio-44 -64 37 37 200 50% -62 54%
97 scorpio-4 -64 28 28 400 50% -66 39%
98 scorpio-54 -65 37 37 200 49% -58 54%
99 scorpio-106 -65 39 39 200 48% -48 39%
100 scorpio-5 -66 28 28 400 49% -56 42%
101 scorpio-3 -66 28 28 400 51% -72 39%
102 scorpio-42 -68 37 37 200 50% -69 57%
103 scorpio-94 -70 38 38 200 49% -62 45%
104 scorpio-31 -73 37 37 200 52% -84 57%
105 scorpio-59 -74 38 38 200 56% -112 46%
106 scorpio-45 -74 37 37 200 50% -71 57%
107 scorpio-84 -75 39 39 200 49% -70 44%
108 scorpio-32 -77 38 38 200 47% -57 49%
109 scorpio-46 -78 37 37 200 46% -56 57%
110 scorpio-2 -80 28 28 400 47% -59 38%
111 scorpio-49 -83 38 38 200 51% -87 49%
112 scorpio-53 -83 38 38 200 50% -86 49%
113 scorpio-25 -84 38 38 200 47% -65 50%
114 scorpio-58 -86 38 38 200 46% -62 46%
115 scorpio-41 -87 37 37 200 45% -59 54%
116 scorpio-62 -90 39 39 200 58% -138 39%
117 scorpio-30 -91 37 37 200 49% -85 55%
118 scorpio-26 -93 38 38 200 52% -106 50%
119 scorpio-29 -96 38 38 200 54% -118 47%
120 scorpio-52 -107 38 38 200 50% -104 51%
121 scorpio-83 -107 38 38 200 53% -123 47%
122 scorpio-0 -111 42 42 200 42% -53 24%
123 scorpio-51 -125 37 37 200 49% -116 55%
124 scorpio-50 -125 37 37 200 46% -104 54%
125 scorpio-27 -128 38 38 200 48% -119 50%
126 scorpio-63 -134 39 39 200 49% -131 41%
127 scorpio-60 -137 38 38 200 45% -108 50%
128 scorpio-61 -141 39 39 200 46% -114 44%
129 scorpio-71 -141 39 39 200 53% -157 38%
130 scorpio-28 -144 38 38 200 45% -112 47%
131 scorpio-66 -150 38 38 200 52% -162 44%
132 scorpio-72 -155 39 39 200 52% -164 41%
133 scorpio-65 -156 38 38 200 51% -161 48%
134 scorpio-70 -159 38 38 200 51% -165 48%
135 scorpio-67 -168 39 39 200 49% -160 40%
136 scorpio-68 -171 39 39 200 51% -178 42%
137 scorpio-82 -171 39 39 200 46% -146 45%
138 scorpio-64 -172 39 39 200 46% -145 43%
139 scorpio-76 -179 39 39 200 54% -206 39%
140 scorpio-81 -185 39 39 200 53% -205 42%
141 scorpio-73 -186 38 38 200 48% -176 46%
142 scorpio-69 -188 38 38 200 46% -165 47%
143 scorpio-74 -196 39 39 200 50% -196 38%
144 scorpio-79 -203 37 37 200 55% -233 52%
145 scorpio-75 -205 39 39 200 47% -188 39%
146 scorpio-77 -207 40 40 200 50% -203 35%
147 scorpio-78 -228 39 39 200 46% -205 44%
148 scorpio-80 -238 38 38 200 43% -194 47%
This makes much more sense indeed and captures the problem with the "incremental elo" approach. Good catch!
Code: Select all
Rank Name Elo + - games score oppo. draws
1 scorpio-146 415 632 632 200 56% 379 36%
2 scorpio-147 383 635 635 100 45% 415 34%
3 scorpio-145 374 630 630 200 50% 375 38%
4 scorpio-134 347 605 605 200 55% 317 43%
5 scorpio-144 335 627 627 200 48% 350 40%
6 scorpio-141 329 621 621 200 52% 315 38%
7 scorpio-143 326 625 625 200 49% 330 35%
8 scorpio-142 325 623 623 200 50% 327 37%
9 scorpio-133 319 603 603 200 51% 310 37%
10 scorpio-135 315 607 607 200 48% 327 43%
11 scorpio-137 311 611 611 200 52% 298 34%
12 scorpio-136 308 609 609 200 49% 313 37%
13 scorpio-139 308 616 616 200 52% 297 37%
14 scorpio-140 306 618 618 200 48% 318 34%
15 scorpio-138 288 614 614 200 47% 309 37%
16 scorpio-131 283 599 599 200 56% 247 39%
17 scorpio-132 273 601 601 200 46% 301 36%
18 scorpio-125 243 587 587 200 55% 210 32%
19 scorpio-128 224 593 593 200 51% 217 37%
20 scorpio-129 222 595 595 200 50% 222 40%
21 scorpio-130 221 597 597 200 45% 253 41%
22 scorpio-124 218 585 585 200 50% 218 34%
23 scorpio-127 212 591 591 200 50% 213 39%
24 scorpio-121 203 580 580 200 57% 154 35%
25 scorpio-126 201 589 589 200 46% 227 35%
26 scorpio-123 193 584 584 200 51% 185 38%
27 scorpio-118 172 575 575 200 54% 149 41%
28 scorpio-120 156 578 578 200 47% 176 38%
29 scorpio-122 152 582 582 200 43% 198 38%
30 scorpio-117 150 574 574 101 47% 170 42%
31 scorpio-119 149 577 577 200 48% 164 42%
32 scorpio-17 62 339 339 200 56% 24 50%
33 scorpio-16 38 342 342 200 50% 40 50%
34 scorpio-14 35 349 349 200 55% 5 41%
35 scorpio-15 18 345 345 200 47% 36 47%
36 scorpio-18 10 336 336 200 47% 27 50%
37 scorpio-115 10 298 298 200 53% -11 38%
38 scorpio-101 8 263 263 200 56% -25 48%
39 scorpio-116 8 301 301 101 50% 11 40%
40 scorpio-11 6 357 357 400 52% -8 42%
41 scorpio-21 6 326 326 200 53% -11 49%
42 scorpio-10 3 359 359 400 51% -2 44%
43 scorpio-99 0 259 259 200 55% -28 43%
44 scorpio-20 -6 330 330 200 49% 0 50%
45 scorpio-19 -7 333 333 200 49% 2 52%
46 scorpio-8 -7 363 363 400 52% -17 41%
47 scorpio-13 -8 352 352 200 48% 8 46%
48 scorpio-86 -11 239 239 200 53% -30 41%
49 scorpio-9 -11 361 361 400 49% -2 43%
50 scorpio-22 -17 323 323 200 50% -16 55%
51 scorpio-12 -20 356 356 300 47% 2 45%
52 scorpio-109 -21 282 282 200 55% -53 39%
53 scorpio-104 -23 270 270 200 55% -52 39%
54 scorpio-7 -24 365 365 400 51% -28 45%
55 scorpio-112 -24 290 290 200 52% -37 43%
56 scorpio-102 -24 265 265 200 49% -18 48%
57 scorpio-100 -25 261 261 200 45% 4 48%
58 scorpio-87 -27 240 240 200 49% -21 46%
59 scorpio-97 -27 255 255 200 53% -45 42%
60 scorpio-114 -30 295 295 200 48% -15 36%
61 scorpio-98 -30 257 257 200 48% -14 42%
62 scorpio-88 -30 242 242 200 51% -34 50%
63 scorpio-55 -32 247 247 200 53% -50 48%
64 scorpio-85 -33 238 238 200 52% -43 42%
65 scorpio-56 -35 245 245 200 51% -41 48%
66 scorpio-111 -35 287 287 200 51% -37 40%
67 scorpio-107 -35 277 277 200 54% -60 43%
68 scorpio-37 -36 282 282 200 51% -44 60%
69 scorpio-24 -37 317 317 200 54% -61 51%
70 scorpio-47 -38 260 260 200 54% -63 52%
71 scorpio-23 -39 320 320 200 48% -27 53%
72 scorpio-113 -39 292 292 200 48% -27 42%
73 scorpio-90 -40 244 244 200 51% -43 43%
74 scorpio-33 -40 292 292 200 54% -64 48%
75 scorpio-89 -42 243 243 200 49% -35 47%
76 scorpio-38 -43 280 280 200 50% -42 62%
77 scorpio-103 -44 267 267 200 47% -23 42%
78 scorpio-36 -44 284 284 200 51% -48 56%
79 scorpio-91 -44 245 245 200 51% -47 45%
80 scorpio-39 -47 277 277 200 50% -47 59%
81 scorpio-6 -48 367 367 400 49% -45 46%
82 scorpio-48 -48 259 259 200 52% -60 46%
83 scorpio-40 -50 275 275 200 53% -67 54%
84 scorpio-43 -50 268 268 200 53% -66 55%
85 scorpio-57 -50 244 244 200 52% -60 51%
86 scorpio-110 -51 284 284 200 47% -28 37%
87 scorpio-34 -51 289 289 200 50% -50 59%
88 scorpio-1 -53 376 376 400 56% -96 30%
89 scorpio-108 -54 279 279 200 46% -28 48%
90 scorpio-92 -55 247 247 200 50% -54 47%
91 scorpio-35 -60 287 287 200 48% -48 60%
92 scorpio-105 -60 272 272 200 47% -44 45%
93 scorpio-95 -61 252 252 200 51% -66 50%
94 scorpio-96 -61 253 253 200 48% -44 43%
95 scorpio-93 -63 248 248 200 50% -62 40%
96 scorpio-44 -64 266 266 200 50% -62 54%
97 scorpio-4 -64 370 370 400 50% -66 39%
98 scorpio-54 -65 248 248 200 49% -58 54%
99 scorpio-106 -65 274 274 200 48% -48 39%
100 scorpio-5 -66 368 368 400 49% -56 42%
101 scorpio-3 -66 372 372 400 51% -72 39%
102 scorpio-42 -68 271 271 200 50% -69 57%
103 scorpio-94 -70 250 250 200 49% -62 45%
104 scorpio-31 -73 297 297 200 52% -84 57%
105 scorpio-59 -74 242 242 200 56% -112 46%
106 scorpio-45 -74 264 264 200 50% -71 57%
107 scorpio-84 -75 237 237 200 49% -70 44%
108 scorpio-32 -77 295 295 200 47% -57 49%
109 scorpio-46 -78 262 262 200 46% -56 57%
110 scorpio-2 -80 374 374 400 47% -59 38%
111 scorpio-49 -83 257 257 200 51% -87 49%
112 scorpio-53 -83 250 250 200 50% -86 49%
113 scorpio-25 -84 314 314 200 47% -65 50%
114 scorpio-58 -86 243 243 200 46% -62 46%
115 scorpio-41 -87 273 273 200 45% -59 54%
116 scorpio-62 -90 238 238 200 58% -138 39%
117 scorpio-30 -91 300 300 200 49% -85 55%
118 scorpio-26 -93 311 311 200 52% -106 50%
119 scorpio-29 -96 303 303 200 54% -118 47%
120 scorpio-52 -107 251 251 200 50% -104 51%
121 scorpio-83 -107 237 237 200 53% -123 47%
122 scorpio-0 -111 379 379 200 42% -53 24%
123 scorpio-51 -125 253 253 200 49% -116 55%
124 scorpio-50 -125 255 255 200 46% -104 54%
125 scorpio-27 -128 309 309 200 48% -119 50%
126 scorpio-63 -134 237 237 200 49% -131 41%
127 scorpio-60 -137 240 240 200 45% -108 50%
128 scorpio-61 -141 239 239 200 46% -114 44%
129 scorpio-71 -141 233 233 200 53% -157 38%
130 scorpio-28 -144 306 306 200 45% -112 47%
131 scorpio-66 -150 235 235 200 52% -162 44%
132 scorpio-72 -155 233 233 200 52% -164 41%
133 scorpio-65 -156 236 236 200 51% -161 48%
134 scorpio-70 -159 233 233 200 51% -165 48%
135 scorpio-67 -168 234 234 200 49% -160 40%
136 scorpio-68 -171 234 234 200 51% -178 42%
137 scorpio-82 -171 236 236 200 46% -146 45%
138 scorpio-64 -172 236 236 200 46% -145 43%
139 scorpio-76 -179 233 233 200 54% -206 39%
140 scorpio-81 -185 235 235 200 53% -205 42%
141 scorpio-73 -186 233 233 200 48% -176 46%
142 scorpio-69 -188 234 234 200 46% -165 47%
143 scorpio-74 -196 233 233 200 50% -196 38%
144 scorpio-79 -203 234 234 200 55% -233 52%
145 scorpio-75 -205 233 233 200 47% -188 39%
146 scorpio-77 -207 233 233 200 50% -203 35%
147 scorpio-78 -228 234 234 200 46% -205 44%
148 scorpio-80 -238 234 234 200 43% -194 47%
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: best way to determine elos of a group
There are two more options for computing intervals
With exactdist
jointdist takes a lot of time. Will post again if it finishes.
Code: Select all
jointdist [p] ... compute intervals from joint distribution
exactdist [p] ... compute intervals assuming exact opponent Elos
Code: Select all
Rank Name Elo + - games score oppo. draws
1 scorpio-146 415 40 40 200 56% 379 36%
2 scorpio-147 383 56 56 100 45% 415 34%
3 scorpio-145 374 40 40 200 50% 375 38%
4 scorpio-134 347 39 38 200 55% 317 43%
5 scorpio-144 335 39 39 200 48% 350 40%
6 scorpio-141 329 39 39 200 52% 315 38%
7 scorpio-143 326 40 40 200 49% 330 35%
8 scorpio-142 325 40 39 200 50% 327 37%
9 scorpio-133 319 40 39 200 51% 310 37%
10 scorpio-135 315 39 39 200 48% 327 43%
11 scorpio-137 311 40 40 200 52% 298 34%
12 scorpio-136 308 39 39 200 49% 313 37%
13 scorpio-139 308 39 39 200 52% 297 37%
14 scorpio-140 306 40 40 200 48% 318 34%
15 scorpio-138 288 39 40 200 47% 309 37%
16 scorpio-131 283 40 39 200 56% 247 39%
17 scorpio-132 273 40 40 200 46% 301 36%
18 scorpio-125 243 40 40 200 55% 210 32%
19 scorpio-128 224 39 39 200 51% 217 37%
20 scorpio-129 222 39 39 200 50% 222 40%
21 scorpio-130 221 39 39 200 45% 253 41%
22 scorpio-124 218 40 40 200 50% 218 34%
23 scorpio-127 212 39 39 200 50% 213 39%
24 scorpio-121 203 40 40 200 57% 154 35%
25 scorpio-126 201 40 40 200 46% 227 35%
26 scorpio-123 193 39 39 200 51% 185 38%
27 scorpio-118 172 39 39 200 54% 149 41%
28 scorpio-120 156 39 39 200 47% 176 38%
29 scorpio-122 152 39 40 200 43% 198 38%
30 scorpio-117 150 54 54 101 47% 170 42%
31 scorpio-119 149 39 39 200 48% 164 42%
32 scorpio-17 62 38 38 200 56% 24 50%
33 scorpio-16 38 38 38 200 50% 40 50%
34 scorpio-14 35 39 39 200 55% 5 41%
35 scorpio-15 18 38 38 200 47% 36 47%
36 scorpio-18 10 38 38 200 47% 27 50%
37 scorpio-115 10 39 39 200 53% -11 38%
38 scorpio-101 8 38 38 200 56% -25 48%
39 scorpio-116 8 55 55 101 50% 11 40%
40 scorpio-11 6 28 28 400 52% -8 42%
41 scorpio-21 6 38 38 200 53% -11 49%
42 scorpio-10 3 27 27 400 51% -2 44%
43 scorpio-99 0 39 38 200 55% -28 43%
44 scorpio-20 -6 38 38 200 49% 0 50%
45 scorpio-19 -7 37 37 200 49% 2 52%
46 scorpio-8 -7 28 28 400 52% -17 41%
47 scorpio-13 -8 38 38 200 48% 8 46%
48 scorpio-86 -11 39 39 200 53% -30 41%
49 scorpio-9 -11 27 27 400 49% -2 43%
50 scorpio-22 -17 37 37 200 50% -16 55%
51 scorpio-12 -20 31 31 300 47% 2 45%
52 scorpio-109 -21 39 39 200 55% -53 39%
53 scorpio-104 -23 39 39 200 55% -52 39%
54 scorpio-7 -24 27 27 400 51% -28 45%
55 scorpio-112 -24 39 38 200 52% -37 43%
56 scorpio-102 -24 38 38 200 49% -18 48%
57 scorpio-100 -25 38 38 200 45% 4 48%
58 scorpio-87 -27 38 38 200 49% -21 46%
59 scorpio-97 -27 39 39 200 53% -45 42%
60 scorpio-114 -30 39 40 200 48% -15 36%
61 scorpio-98 -30 39 39 200 48% -14 42%
62 scorpio-88 -30 38 38 200 51% -34 50%
63 scorpio-55 -32 38 38 200 53% -50 48%
64 scorpio-85 -33 39 39 200 52% -43 42%
65 scorpio-56 -35 38 38 200 51% -41 48%
66 scorpio-111 -35 39 39 200 51% -37 40%
67 scorpio-107 -35 39 39 200 54% -60 43%
68 scorpio-37 -36 37 37 200 51% -44 60%
69 scorpio-24 -37 38 37 200 54% -61 51%
70 scorpio-47 -38 38 37 200 54% -63 52%
71 scorpio-23 -39 37 37 200 48% -27 53%
72 scorpio-113 -39 39 39 200 48% -27 42%
73 scorpio-90 -40 39 39 200 51% -43 43%
74 scorpio-33 -40 38 38 200 54% -64 48%
75 scorpio-89 -42 38 38 200 49% -35 47%
76 scorpio-38 -43 36 36 200 50% -42 62%
77 scorpio-103 -44 39 39 200 47% -23 42%
78 scorpio-36 -44 37 37 200 51% -48 56%
79 scorpio-91 -44 38 38 200 51% -47 45%
80 scorpio-39 -47 37 37 200 50% -47 59%
81 scorpio-6 -48 27 27 400 49% -45 46%
82 scorpio-48 -48 38 38 200 52% -60 46%
83 scorpio-40 -50 37 37 200 53% -67 54%
84 scorpio-43 -50 37 37 200 53% -66 55%
85 scorpio-57 -50 38 38 200 52% -60 51%
86 scorpio-110 -51 39 40 200 47% -28 37%
87 scorpio-34 -51 37 37 200 50% -50 59%
88 scorpio-1 -53 29 29 400 56% -96 30%
89 scorpio-108 -54 38 38 200 46% -28 48%
90 scorpio-92 -55 38 38 200 50% -54 47%
91 scorpio-35 -60 37 37 200 48% -48 60%
92 scorpio-105 -60 38 38 200 47% -44 45%
93 scorpio-95 -61 38 38 200 51% -66 50%
94 scorpio-96 -61 38 39 200 48% -44 43%
95 scorpio-93 -63 39 39 200 50% -62 40%
96 scorpio-44 -64 37 37 200 50% -62 54%
97 scorpio-4 -64 28 28 400 50% -66 39%
98 scorpio-54 -65 37 37 200 49% -58 54%
99 scorpio-106 -65 39 39 200 48% -48 39%
100 scorpio-5 -66 28 28 400 49% -56 42%
101 scorpio-3 -66 28 28 400 51% -72 39%
102 scorpio-42 -68 37 37 200 50% -69 57%
103 scorpio-94 -70 38 38 200 49% -62 45%
104 scorpio-31 -73 37 37 200 52% -84 57%
105 scorpio-59 -74 38 38 200 56% -112 46%
106 scorpio-45 -74 37 37 200 50% -71 57%
107 scorpio-84 -75 38 38 200 49% -70 44%
108 scorpio-32 -77 38 38 200 47% -57 49%
109 scorpio-46 -78 37 37 200 46% -56 57%
110 scorpio-2 -80 28 28 400 47% -59 38%
111 scorpio-49 -83 38 38 200 51% -87 49%
112 scorpio-53 -83 38 38 200 50% -86 49%
113 scorpio-25 -84 38 38 200 47% -65 50%
114 scorpio-58 -86 38 38 200 46% -62 46%
115 scorpio-41 -87 37 37 200 45% -59 54%
116 scorpio-62 -90 39 39 200 58% -138 39%
117 scorpio-30 -91 37 37 200 49% -85 55%
118 scorpio-26 -93 38 38 200 52% -106 50%
119 scorpio-29 -96 38 38 200 54% -118 47%
120 scorpio-52 -107 38 38 200 50% -104 51%
121 scorpio-83 -107 38 38 200 53% -123 47%
122 scorpio-0 -111 41 42 200 42% -53 24%
123 scorpio-51 -125 37 37 200 49% -116 55%
124 scorpio-50 -125 37 37 200 46% -104 54%
125 scorpio-27 -128 38 38 200 48% -119 50%
126 scorpio-63 -134 39 39 200 49% -131 41%
127 scorpio-60 -137 38 38 200 45% -108 50%
128 scorpio-61 -141 38 39 200 46% -114 44%
129 scorpio-71 -141 39 39 200 53% -157 38%
130 scorpio-28 -144 38 38 200 45% -112 47%
131 scorpio-66 -150 38 38 200 52% -162 44%
132 scorpio-72 -155 39 39 200 52% -164 41%
133 scorpio-65 -156 38 38 200 51% -161 48%
134 scorpio-70 -159 38 38 200 51% -165 48%
135 scorpio-67 -168 39 39 200 49% -160 40%
136 scorpio-68 -171 39 39 200 51% -178 42%
137 scorpio-82 -171 38 39 200 46% -146 45%
138 scorpio-64 -172 38 39 200 46% -145 43%
139 scorpio-76 -179 39 39 200 54% -206 39%
140 scorpio-81 -185 39 39 200 53% -205 42%
141 scorpio-73 -186 38 38 200 48% -176 46%
142 scorpio-69 -188 38 38 200 46% -165 47%
143 scorpio-74 -196 39 39 200 50% -196 38%
144 scorpio-79 -203 37 37 200 55% -233 52%
145 scorpio-75 -205 39 39 200 47% -188 39%
146 scorpio-77 -207 40 40 200 50% -203 35%
147 scorpio-78 -228 38 39 200 46% -205 44%
148 scorpio-80 -238 38 38 200 43% -194 47%
-
- Posts: 201
- Joined: Thu Jun 06, 2019 8:05 pm
- Full name: Percival Tiglao
Re: best way to determine elos of a group
So if I'm reading this correctly: #147 finished with 383 Elo (+/- 56) and played 100 matches. But all of its matches were against #146 (the only Elo score of 415 in the entire chart). That seems like a recipe for poor statistics.2 scorpio-147 383 56 56 100 45% 415 34%
How did you determine which networks would match up against other networks? I would expect a tournament design to start off with random matches, followed up by maybe "focused" matches with a particular partner to maybe solidify the scores.
EDIT: A score of +/- 600 Elo means that your estimated probability-of-win is somewhere between 3% and 97%. Obviously, these error bounds are huge. I would expect better error-bounds after 80,000 games.
Last edited by dragontamer5788 on Tue Jul 30, 2019 10:26 pm, edited 1 time in total.