One billion random games

[sje]

One billion random games:

Code: Select all

0.153051   checkmate
0.193435   fiftymoves
0.56713   insufficient
0.0251883   repetition
0.0611956   stalemate

mean length: 334.354
limit: 1000000000
usage: 185920
frequency: 5378.64
period: 0.00018592

[sje]

The above run was done using a single thread on a 3.2 GHz Intel Core i3. It took 52 hours, 38 minutes, and 40 seconds. A trillion game run with the same configuration would take about 5 years and 11 months.

As mentioned earlier, the ordering priority is: checkmate/stalemate, fiftymoves, insufficient, and finally repetition.

[Mark]

One billion random games:

Code: Select all

0.153051   checkmate
0.193435   fiftymoves
0.56713   insufficient
0.0251883   repetition
0.0611956   stalemate

mean length: 334.354
limit: 1000000000
usage: 185920
frequency: 5378.64
period: 0.00018592

Interesting data, Steven. Thanks for posting. I'm surprised that the checkmate percentage was so high for random moves!

[Ajedrecista]

Hello Mark:

One billion random games:

Code: Select all

0.153051   checkmate
0.193435   fiftymoves
0.56713   insufficient
0.0251883   repetition
0.0611956   stalemate

mean length: 334.354
limit: 1000000000
usage: 185920
frequency: 5378.64
period: 0.00018592

Interesting data, Steven. Thanks for posting. I'm surprised that the checkmate percentage was so high for random moves!

This experiment was done almost two years ago: the results were similar than now. Here is the link:

One billion random games

That thread contains something more than the raw numbers.

Regards from Spain.

Ajedrecista.

[S.Taylor]

One billion random games:

Code: Select all

0.153051   checkmate
0.193435   fiftymoves
0.56713   insufficient
0.0251883   repetition
0.0611956   stalemate

mean length: 334.354
limit: 1000000000
usage: 185920
frequency: 5378.64
period: 0.00018592

That's quite alot!
Maybe if i can get them all in a book, or in convenient computer setup, i can start analysing them all (as a human, move by move). That would be a bit of a marathon! Then i might understand a bit about chess!

[Mark]

Hello Mark:

One billion random games:

Code: Select all

0.153051   checkmate
0.193435   fiftymoves
0.56713   insufficient
0.0251883   repetition
0.0611956   stalemate

mean length: 334.354
limit: 1000000000
usage: 185920
frequency: 5378.64
period: 0.00018592

Interesting data, Steven. Thanks for posting. I'm surprised that the checkmate percentage was so high for random moves!

This experiment was done almost two years ago: the results were similar than now. Here is the link:

One billion random games

That thread contains something more than the raw numbers.

Regards from Spain.

Ajedrecista.

Thanks for that link! I didn't remember that old thread. One of the posts in that thread mentioned "max plies = 886" for a run of 1 million random move games. Wonder what the max plies is for 1 billion games?

[sje]

Thanks for that link! I didn't remember that old thread. One of the posts in that thread mentioned "max plies = 886" for a run of 1 million random move games. Wonder what the max plies is for 1 billion games?

Different runs will give different results. I don't recall any random game length greater than 1,000 ply.

[hgm]

I did a follow-up on this old study, to answer some of the questions it raised. To this end I wrote my own random game generator. I took the short-cut of abolishing castling and e.p. capture; this did not have a significant effect on the statistics of the purely random play, and I could not imagine that the matters I was curious about would be different in a chess variant that doesn't have castling and e.p. than in orthodox chess. The generatorplays a million games in a few minutes (single-threaded).

Then I started playing with classifying the moves by (attacker, victim), and possibly promotion choice, and giving each class its own probability 'weight', rather than selecting all moves with equal probabilit. As expected reducing the probability for under-promotion leads to stronger play (i.e. the player that does it wins more than he loses against a player that under-promotes with equal probability). It also increases the number of checkmates, from 15% to 18%. So I just took under-promotion out of the generator.

Increasing the weight of captures makes the random player much stronger. Which was of course expected: against a purely random player the chances that you will be recaptured are very low, so even Q x protected P usually gains you a Pawn. Not so obvious was whether this would remain true if the opponent would also be biased towards captures. Because then he might nearly always recapture, if the piece was protected, so that a good outcome is by no means nearly certain anymore.

Yet I did not really find an optimum weight, when I bias all captures equally compared to non-captures. Even with a bias as large as 30, you would slightly lose against a player that biases captures by 31. My implementation limited the total weight of all moves in a position to 4096, so I could not afford weights much more than 100. But it pretty much looks like that it is best to never try a non-capture if any capture is available. The checkmate probability goes down a lot because of this, though, to about 5%, and stayed around that value for my later optimizations.

Then I started differentiating the weights by attacker, which had a lot larger effect than differentiating by victim. This is not really what I expected: I had expected it to be a huge benefit to put a preference on capturing a Queen. Instead there was a huge benefit preferring capture with a Pawn. Again the situation seemed to be that when capture with a Pawn is possible, you should not consider any other capture. (And certainly no non-capture.) I ended up setting the weight of P x Any to 250 (the maximum my implementation could afford), and the other captures in the range 30-100, where all non-captures still had weight 1.

For capturing with other pieces it was also beneficial to give captures with weaker pieces a larger weight. I ended up with N=75, B=60, R=40. For capturing with the Queen the situation was more complex: here it really mattered what you captured with it. Wile the optimum that was not differentiated in this respect was about the same as that for a Rook, a significant improvement was optained by setting capturing of Pawns much lower. I ended up with 6 for this, which definitely worked better than 8 or 2. Q x P is a bad idea against an aggressive opponent (i.e. one that will recapture when he can). Funny enough there was no such effect for capture of Pawns by other pieces. You have of course to capture the Pawns at some point, to avoid their promotion, and since Rooks are initially trapped behind their own Pawns, they are ideal pieces for this; the R x P capture weight can be high without very quickly leading to sacrifice of the Rook. Knights and Bishops did also not benefit from being made more reluctant to capture Pawns, however.

To my surprise it turned out good to give higher weight to Q x R and Q x B (I ended up with 80). I had naively expected these H x L captures to be too risky. But I guess the point is that if Q can capture R or B, there is a 50% that the R or B can capture that Q, and with an aggressive opponent will most likely do so on the next turn. Because the biased random play cannot be smart enough to move that Queen away (and even if it did, it could easily turn out to be attacked in the new location as well). So you either capture, hoping your attacker was unprotected, or you will be captured. Probably for the same reason it was also advantageous to up the weight of equal captures BxB (100), RxR and QxQ (both 60). OTOH, there is no such reason for QxN, because the Knight will certainly not be attacking you back, and it was advantageous to lower the weight of such captures to 25.

Because we only consider legal moves, captures with the King should always be safe. Still it did not have any benefits to increase the weight of captures with the King to above 50. After a lot of trial and error I arrived at the following weight table, on which it was not easy to improve in a way that would be obvious in 1 million games:

Code: Select all

attacker           victim
         -    P    N    B    R    Q
   P     1  400  250  250  250  400
   N     1   75  150   75   75   75
   B     1   60   60  100   60  100
   R     1   40   40   40   60  100
   Q     1    6   25   80   80  100
   K     1   50   50   50   50   50
   

[hgm]

My tentative conclusion is that these random games have no reality value; I tried them from several positions with material imbalance, and it even fails to properly judge the value of those imbalances. For Queen, Rook and Knight odds the results reasonably match the classical piece values. But a Knight is completely underrated, giving only half as much advantage as a Bishop. While Pawns are absurdly overrated, a Pawn advantage causing a larger win rate than a Knight advantage. I am not sure what causes this. Of course Pawns move irreversibly forward, so in contrast to the other pieces they do not aimlessly 'diffuse' over the board, but drive relentlessly forward. They get not easily blocked when pieces they attack see no reason to move away. They just eat their way towards promotion.

So I guess the concept of being under attack or being protected is very basic to chess, and awareness of it during the move-selection process is an essential ingredient for generating games that have any 'reality value' (so they could be used as rollouts in MCTS to estimate the evaluation.) Probability weights must be differentiated by whether the destination square is attacked by an opponent or not, and by which type of opponent. And by similar information on the from-square. Preferably it should even know which other pieces get protected by the move.

[smcracraft]

Nice! Thanks for this data science study!