I am revising the program to include this in the error calculation.Dirt wrote:So, if a player has a winning position and makes a mistake that results in an even score, it doesn't count as an error?Ferdy wrote:1. When the search score of the position is already outside of +/-2 pawns as analyzed by the engine, this position is not included.
SF6 moves comparison: Event Vugar Gashimov Mem 2015
Moderator: Ras
-
Ferdy
- Posts: 4851
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
-
Ferdy
- Posts: 4851
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
I can just output the epd given a threshold of say 1/2 or more pawn worst than the best.Vinvin wrote:Is it possible to get a list of the 20 biggest blunders ?
-
Dirt
- Posts: 2851
- Joined: Wed Mar 08, 2006 10:01 pm
- Location: Irvine, CA, USA
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
Good. It just seemed like an odd choice.Ferdy wrote:I am revising the program to include this in the error calculation.
Deasil is the right way to go.
-
Ferdy
- Posts: 4851
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
Tournament and players compared. Just a quick 1sec/pos, see also the notes. Sample epd files are attached.
If move is not best, epd positions are saved per player and per amount of error. These blunder positions saved are in append mode, so be sure to delete those if you run same epd. The summary is append mode also, the output below is an example. There is also a log output, for checking, this is overwrite mode, be sure to backup if you want to review previous results.
Sample blunder @ 3 pawns or more.
[d]br2qr1k/6bp/1N1Pp1p1/2Q1np2/4p3/2B1P3/4BPPP/2R2RK1 b - -
In the game he played e5d7.
Download:
http://www.mediafire.com/download/xqfte ... orV8.2.rar
If move is not best, epd positions are saved per player and per amount of error. These blunder positions saved are in append mode, so be sure to delete those if you run same epd. The summary is append mode also, the output below is an example. There is also a log output, for checking, this is overwrite mode, be sure to backup if you want to review previous results.
Code: Select all
Moves Error calculation based from Stockfish 6 64 POPCNT and players from
Event 2nd Shamkir Chess 2015, after round 5
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine id name Stockfish 6 64 POPCNT
Hash 128 MB, Threads 1, MoveTime 1.0 sec/pos
Rank Players AveDepth TotalPos AveError Err_3.0P Err_2.0P Err_1.0P Err_0.5P W L D Games Score SRate
1 Anand, Viswanathan 18 166 3.97 0 0 1 2 1 0 4 5 3.0 60.0%
2 Mamedov, Rauf 17 174 4.05 0 0 0 2 0 1 4 5 2.0 40.0%
3 Carlsen, Magnus 18 171 4.26 0 0 0 1 3 0 2 5 4.0 80.0%
4 So, Wesley 16 147 5.25 0 0 2 4 3 1 1 5 3.5 70.0%
5 Caruana, Fabiano 18 192 5.93 0 0 0 5 0 1 4 5 2.0 40.0%
6 Mamedyarov, Shakhriyar 17 213 6.95 1 0 1 5 1 1 3 5 2.5 50.0%
7 Vachier-Lagrave, Maxime 18 159 7.30 1 0 0 3 0 1 4 5 2.0 40.0%
8 Kramnik, Vladimir 17 173 8.86 0 0 0 5 1 1 3 5 2.5 50.0%
9 Giri, Anish 16 154 10.71 0 1 1 2 0 1 4 5 2.0 40.0%
10 Adams, Michael 16 134 16.30 0 0 1 4 0 2 3 5 1.5 30.0%
Overall Average Error: 7.36Code: Select all
Moves Error calculation based from Stockfish 6 64 POPCNT and players from
Event New York 1927, after round 1
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine id name Stockfish 6 64 POPCNT
Hash 128 MB, Threads 1, MoveTime 1.0 sec/pos
Rank Players AveDepth TotalPos AveError Err_3.0P Err_2.0P Err_1.0P Err_0.5P W L D Games Score SRate
1 Capablanca, Jose Raul 17 147 4.74 0 0 0 2 3 0 2 5 4.0 80.0%
2 Nimzowitsch, Aaron 19 149 9.87 0 1 0 4 3 1 1 5 3.5 70.0%
3 Spielmann, Rudolf 17 155 13.86 0 0 3 8 0 2 3 5 1.5 30.0%
4 Vidmar, Milan Sr 17 119 14.83 0 1 0 11 0 1 4 5 2.0 40.0%
5 Alekhine, Alexander 17 166 16.51 0 0 2 9 1 1 3 5 2.5 50.0%
6 Marshall, Frank James 18 147 28.48 1 1 2 9 0 2 3 5 1.5 30.0%
Overall Average Error: 14.71Code: Select all
Legend:
Rank - ranking based on low average error
AveDepth - average depth reached by the engine in analyzing those positions
AveError - average difference between the engine and player score, in centipawn (1p = 100)
Err_3.0P - count of positions where player has blundered by 3 or more pawns
Err_2.0P - count of positions where player has blundered by 2 to 2.9 pawns
Err_1.0P - count of positions where player has blundered by 1 to 1.9 pawns
Err_0.5P - count of positions where player has blundered by 0.5 to 0.9 pawns
Notes:
1. Analysis of positions start at move 12.
2. If best score of engine is already above 3 pawns and the score of player move
is also above 3 pawns then error for this position is set to zero. The more cases
like this only proves that that player is able to maintain its advantage and this
will minimize its average error.
On the other hand if the best score of engine is already below 3 pawns, no further analysis
will be done, and this position will not be included in error calculation.
3. When a player move is a blunder that brings down the score below 3 pawns worst,
error for this position is set to 6 pawns in the calculation. The reason being once
the opponent found the refutation, the succeeding positions will no longer be evaluated.
It is just fair to penalize this heavily to also reflect on the average error which
should be high for players that have more game loses or fatal blunders.br2qr1k/6bp/1N1Pp1p1/2Q1np2/4p3/2B1P3/4BPPP/2R2RK1 b - - fmvn 26; sm "e5d7"; c3 "Result 1-0"; c2 "Carlsen, Magnus / Mamedyarov, Shakhriyar"; c1 "Round 2"; c0 "Event 2nd Shamkir Chess 2015"; hmvc 10; c5 1; c4 1648;Mamedyarov, Shakhriyar_blunder3P.epd
[d]br2qr1k/6bp/1N1Pp1p1/2Q1np2/4p3/2B1P3/4BPPP/2R2RK1 b - -
In the game he played e5d7.
Download:
http://www.mediafire.com/download/xqfte ... orV8.2.rar
-
Ferdy
- Posts: 4851
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
After round 7 with improvement on determining move average error, see note 5.
New is column EECnt for Enginer Error Count, these are position counts
where player move score is better than the engine.
New is column EECnt for Enginer Error Count, these are position counts
where player move score is better than the engine.
Code: Select all
Move Error Evaluator v8.5
Moves Error calculation based from Stockfish 180415 64 POPCNT and players from
Event Vugar Gashimov Mem 2015, after round 7
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine id name Stockfish 180415 64 POPCNT
Hash 128 MB, Threads 1, MoveTime 1.0 sec/pos
Rank Players AveDepth TotalPos AveError E3.0P E2.0P E1.0P E0.5P EECnt EERate W L D Games Score SRate
1 Anand,V 18 223 8.40 0 0 2 6 24 10.76% 2 0 5 7 4.5 64.3%
2 Carlsen,M 20 235 8.47 0 0 2 2 16 6.81% 4 0 3 7 5.5 78.6%
3 Mamedyarov,S 18 260 10.77 1 0 3 2 23 8.85% 1 1 5 7 3.5 50.0%
4 Mamedov,Rau 18 212 11.00 0 0 1 1 18 8.49% 0 1 6 7 3.0 42.9%
5 Caruana,F 20 270 13.71 0 0 0 8 23 8.52% 2 1 4 7 4.0 57.1%
6 Vachier Lagrave,M 20 241 14.88 1 0 1 3 22 9.13% 0 1 6 7 3.0 42.9%
7 Giri,A 18 214 20.18 0 1 0 6 11 5.14% 0 1 6 7 3.0 42.9%
8 So,W 18 208 22.63 0 0 1 8 15 7.21% 3 2 2 7 4.0 57.1%
9 Adams,Mi 18 215 26.31 0 0 4 6 16 7.44% 0 3 4 7 2.0 28.6%
10 Kramnik,V 18 238 30.85 0 0 3 9 23 9.66% 1 3 3 7 2.5 35.7%
Overall Average Error: 16.72
Legend:
Rank - ranking based on low average error
AveDepth - average depth reached by the engine in analyzing those positions
AveError - average difference between the engine and player score, in centipawn (1p = 100)
E3.0P - count of positions where player has blundered by 3 or more pawns
E2.0P - count of positions where player has blundered by 2 to 2.9 pawns
E1.0P - count of positions where player has blundered by 1 to 1.9 pawns
E0.5P - count of positions where player has blundered by 0.5 to 0.9 pawns
EECnt - count of positions where player move score is better than engine move score
EERate - 100*EECnt/TotalPos or the percentage of the Engine Error Counts
Notes:
1. Analysis of positions start at move 12.
2. If best score of engine is already above 3 pawns and the score of player move
is also above 3 pawns then error for this position is set to zero. The more cases
like this only proves that that player is able to maintain its advantage and this
will minimize its average error.
On the other hand if the best score of engine is already below minus 3 pawns, no further
analysis will be done, and this position will not be included in error calculation.
3. When a player move is a blunder that brings down the score below minus 3 pawns,
error for this position is set to 6 pawns in the calculation. The reason being once
the opponent found the refutation, the succeeding positions will no longer be evaluated.
It is just fair to penalize this heavily to also reflect on the average error which
should be high for players that have more game loses or fatal blunders.
4. move_error = engine_score - player_score.
engine_score = score returned by the engine after searching for a given amount of time.
player_score = score returned by the engine after searching the player move for a given amount of time.
AveError = sum of [move_error] / totalPos.
5. Error is increased if the score of position after player move has dropped to below zero.
6x error if (score <= -200 cp), 5x error if (score <= -150 cp), 4x error if (score <= -100 cp),
3x error if (score <= -50 cp), and 2x error if (score < 0 cp).
The idea is only those errors which will result to inferior position (below zero score) should be penalized more.
If the increased error penalty is more than 3 pawns, error is set to 3 pawns.
6. Regarding EECnt or Engine error counts, this is a counter check on engine where the human move
is sometimes better than the engine move, this is because the analysis time given to the engine is
shorter and also perhaps the position is complicated that bigger depths are required to determine
correct assessment of the position.-
Ferdy
- Posts: 4851
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
Comparison New York 1927.
Code: Select all
Move Error Evaluator v8.5
Moves Error calculation based from Stockfish 180415 64 POPCNT and players from
Event New York 1927, after round 1
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Engine id name Stockfish 180415 64 POPCNT
Hash 128 MB, Threads 1, MoveTime 1.0 sec/pos
Rank Players AveDepth TotalPos AveError E3.0P E2.0P E1.0P E0.5P EECnt EERate W L D Games Score SRate
1 Capablanca, Jose Raul 18 147 6.54 0 0 0 5 11 7.48% 3 0 2 5 4.0 80.0%
2 Nimzowitsch, Aaron 20 148 11.59 0 0 1 3 11 7.43% 3 1 1 5 3.5 70.0%
3 Alekhine, Alexander 18 166 25.31 0 0 2 11 9 5.42% 1 1 3 5 2.5 50.0%
4 Spielmann, Rudolf 19 155 35.63 0 0 3 9 8 5.16% 0 2 3 5 1.5 30.0%
5 Marshall, Frank James 19 145 44.25 1 0 3 7 11 7.59% 0 2 3 5 1.5 30.0%
6 Vidmar, Milan Sr 18 119 44.32 0 1 2 8 6 5.04% 0 1 4 5 2.0 40.0%
Overall Average Error: 27.94-
Astatos
- Posts: 18
- Joined: Thu Apr 10, 2014 5:20 pm
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
So, in other words, you proved that your method(s) prove(s) nothing 
-
Vinvin
- Posts: 5308
- Joined: Thu Mar 09, 2006 9:40 am
- Full name: Vincent Lejeune
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
Please add blunder by 0.2 to 0.49 !Ferdy wrote:Code: Select all
Err_3.0P - count of positions where player has blundered by 3 or more pawns Err_2.0P - count of positions where player has blundered by 2 to 2.9 pawns Err_1.0P - count of positions where player has blundered by 1 to 1.9 pawns Err_0.5P - count of positions where player has blundered by 0.5 to 0.9 pawns
-
Ferdy
- Posts: 4851
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
Here it is, also with new info on dangerous error counts, DECnt. This is the count of positions where players' position goes below zero after a move. This feature will give us info how many times a player is in trouble.Vinvin wrote:Please add blunder by 0.2 to 0.49 !Ferdy wrote:Code: Select all
Err_3.0P - count of positions where player has blundered by 3 or more pawns Err_2.0P - count of positions where player has blundered by 2 to 2.9 pawns Err_1.0P - count of positions where player has blundered by 1 to 1.9 pawns Err_0.5P - count of positions where player has blundered by 0.5 to 0.9 pawns
Sample table sorted on less DECnt, from sr5_less_fatal_error.txt.
Code: Select all
Players AveDepth TotalPos AveError DECnt E300CP E200CP E100CP E50CP E20CP EECnt EERate W L D Games Score SRate
------------------------ -------- -------- -------- ------ ------ ------ ------ ------ ------ ------ ------ --- --- --- ----- ----- -----
Carlsen, Magnus 13 288 9.54 25 0 0 1 5 26 29 10.07 5 0 4 9 7 77.8
So, Wesley 12 284 15.2 42 0 0 2 10 25 29 10.21 3 2 4 9 5 55.6
Anand, Viswanathan 12 280 12.76 47 0 0 3 8 32 28 10.0 3 0 6 9 6 66.7
Mamedov, Rauf 12 262 16.29 48 0 0 2 6 32 27 10.31 0 2 7 9 3.5 38.9
Giri, Anish 12 299 19.96 51 0 0 6 11 28 31 10.37 0 2 7 9 3.5 38.9
Mamedyarov, Shakhriyar 12 307 14.65 52 1 0 2 8 32 31 10.1 1 2 6 9 4 44.4
Vachier-Lagrave, Maxime 13 310 17.3 59 0 1 2 8 22 34 10.97 0 2 7 9 3.5 38.9
Caruana, Fabiano 13 327 19.09 59 0 0 3 13 37 30 9.17 2 1 6 9 5 55.6
Kramnik, Vladimir 12 337 26.97 71 0 0 3 14 33 32 9.5 2 3 4 9 4 44.4
Adams, Michael 12 258 25.31 73 1 0 1 10 36 22 8.53 1 3 5 9 3.5 38.9 Code: Select all
Players AveDep TotalPos AveError DECnt E300CP E200CP E100CP E50CP E20CP EECnt EERate W L D Games Score SRate
------------------------ ------ -------- -------- ------ ------ ------ ------ ------ ------ ------ ------ --- --- --- ----- ----- -----
Carlsen, Magnus 13 288 9.54 25 0 0 1 5 26 29 10.07 5 0 4 9 7 77.8
Anand, Viswanathan 12 280 12.76 47 0 0 3 8 32 28 10.0 3 0 6 9 6 66.7
Mamedyarov, Shakhriyar 12 307 14.65 52 1 0 2 8 32 31 10.1 1 2 6 9 4 44.4
So, Wesley 12 284 15.2 42 0 0 2 10 25 29 10.21 3 2 4 9 5 55.6
Mamedov, Rauf 12 262 16.29 48 0 0 2 6 32 27 10.31 0 2 7 9 3.5 38.9
Vachier-Lagrave, Maxime 13 310 17.3 59 0 1 2 8 22 34 10.97 0 2 7 9 3.5 38.9
Caruana, Fabiano 13 327 19.09 59 0 0 3 13 37 30 9.17 2 1 6 9 5 55.6
Giri, Anish 12 299 19.96 51 0 0 6 11 28 31 10.37 0 2 7 9 3.5 38.9
Adams, Michael 12 258 25.31 73 1 0 1 10 36 22 8.53 1 3 5 9 3.5 38.9
Kramnik, Vladimir 12 337 26.97 71 0 0 3 14 33 32 9.5 2 3 4 9 4 44.4 I will give more analysis time later.
Standings, sorted by score and num wins. Last 3 ranking is not accurate. Nice output from sqlite cli.
Code: Select all
Players AveDepth TotalPos AveError DECnt E300CP E200CP E100CP E50CP E20CP EECnt EERate W L D Games Score SRate
------------------------ -------- -------- -------- ------ ------ ------ ------ ------ ------ ------ ------ --- --- --- ----- ----- -----
Carlsen, Magnus 13 288 9.54 25 0 0 1 5 26 29 10.07 5 0 4 9 7 77.8
Anand, Viswanathan 12 280 12.76 47 0 0 3 8 32 28 10.0 3 0 6 9 6 66.7
So, Wesley 12 284 15.2 42 0 0 2 10 25 29 10.21 3 2 4 9 5 55.6
Caruana, Fabiano 13 327 19.09 59 0 0 3 13 37 30 9.17 2 1 6 9 5 55.6
Kramnik, Vladimir 12 337 26.97 71 0 0 3 14 33 32 9.5 2 3 4 9 4 44.4
Mamedyarov, Shakhriyar 12 307 14.65 52 1 0 2 8 32 31 10.1 1 2 6 9 4 44.4
Adams, Michael 12 258 25.31 73 1 0 1 10 36 22 8.53 1 3 5 9 3.5 38.9
Mamedov, Rauf 12 262 16.29 48 0 0 2 6 32 27 10.31 0 2 7 9 3.5 38.9
Vachier-Lagrave, Maxime 13 310 17.3 59 0 1 2 8 22 34 10.97 0 2 7 9 3.5 38.9
Giri, Anish 12 299 19.96 51 0 0 6 11 28 31 10.37 0 2 7 9 3.5 38.9 -
Guenther
- Posts: 4718
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: SF6 moves comparison: Event Vugar Gashimov Mem 2015
Hi Ferdinand,
You should also exclude one time repetition moves, because Humans just play them to gain time to reach the time control and have more time to calculate the winning line.
If the weaker side really has nothing better then to repeat too, there is nothing lost for the better side!
Guenther
You should also exclude one time repetition moves, because Humans just play them to gain time to reach the time control and have more time to calculate the winning line.
If the weaker side really has nothing better then to repeat too, there is nothing lost for the better side!
Guenther