The Superfinal

[arjuntemurnikar]

Avoid all this mess...
let the engines play the openings!

We take away the endings via TBs...
We take away the openings..
Why?

These powerful engines are capable of playing the whole game and beating any human, no matter what opening

It's pretentious IMO that humans claim to know better than 3200 ELO engines

Some may say it will be boring...no way!
I think it would fascinating to see what openings lines they chose...
from my experience we'd see a good variety, Ruy Lopez, Queens Gambit, Sicilian, etc.

Some may say 'too many duplicate games',
but several LTC matches have been run with 100s of games and 0 duplicates

I say let the engines play!

That has chances. It would be a game changer for some. Some programs have little opening knowledge and more midgame and endgame knowledge. They don't encode the opening knowledge, because they can use a book and it speeds up the program. However, some of us have opening knowledge in the engines. We would gain some (at least temporary) advantage.

You know, the more I think about it, the more I like it. A no book tournament. Norman, I think this might be the best idea you have ever had.

Although this may be the fairest option, it doesn't bode well for a tournament that is being watched by spectators. There will be duplicate openings at least up till 6-10 ply... I have tried this before. Many engines have preferences in openings. For example SF will keep playing the Ruy Lopez and Komodo always starts with 1. d4. This will lead to lesser variety (though no duplicate full games because chess is just so vast).

A spectator driven tournament has slightly different objectives than say a private tournament. A spectator-driven tournament needs some sort of attractive quality to keep people hooked. If games are 90% boring draws or 90% d4 openings, people will get bored and stop watching. A private tournament that is not watched live can afford more boring draws in search of more fairness and more objective result.

A spectator driven tournament is event-oriented. A non-spectator-driven tournament is result-oriented. Once you understand this critical difference, you will realize that Nelson and co. did the best possible job with the openings (given limited resources).

Does that mean that previous editions of TCEC, where the book was of an acknowledged worse quality, were more event-oriented, and achieved bigger success?

The most important thing in top-level games is to ensure fair playing conditions, as fair as possible. And a bigger score right after book hands an indubitable competitive advantage to the weaker engine.

Chess is not like any other sport. People watching chess are highbrows, I do not know if white collars. In order to enjoy something, you have to understand it. Football fens understand the primitive joy of circenses, but even there a necessary prerequisite is absolutely fair conditions, both teams start with 11 men. If one wins or there are too many draws, is a matter of the strength of the competitors. And, I have to say this, entertainments does not depend in the least upon the final result, but mainly upon the quality of games. When you are lucky, you will also have a win.

Chess fens, on the other hand, must also understand what is going on in order to be happy. And when you understand what is going on, the first thing would be fair playing conditions. I am certain you would not like SF to be unjustly penalised in this final, but this is exactly what happens. If all openings were fully even, Komodo still would not have gotten a win. How has that changed the attraction of the event? In no way at all. Fischer 1972 was much more impressive and entertaining than many outwardly more competitive events. And also the repeated encounters of Karpov and Kasparov that witnessed so many draws.

Was the 6-0 whitewash of Fischer over Taimanov and Larsen dull to watch? Or the +5 -3 =48 score of the 1984 Karpov-Kasparov match? No tension there, no enetertainment? It is exactly the opposite. So that entertainment does not depend at all on the draw ratio, but rather on the quality of the games themselves. Fair conditions, on the other hand, are an absolute must, and you should ensure them, as long as you are able to do so.

Now, I know many still think 60cps advantage is not enough to win...

I think the points that you keep missing are:

1. Nelson has explained very lucidly that he did not arbitarily choose bad openings. There was lots of testing done. And even if SF shows +0.60, it does not mean all other engines show the same. Komodo may show +0.30, Houdini +0.35 so this is by no means a bad opening. Both engines have a lot to prove here, which in turn makes the game interesting.

2. Even if the openings were bad, it does not make the quality of games any less. If you define the "quality" of games as number of mistakes made during the game, then by no means are any of the TCEC games from stage 4 onwards of any less quality than the best human games ever played. Perhaps they are of low quality for you Lyudmil, because you seem to be better than all of them.

3. Yes, draws are indeed boring. They are by no means less educational. You can learn a lot from draws, but most draws are boring in terms of entertainment value. (There are of course those draws that have lot of struggle and imbalance and they are not boring... but most draws are.) How many Kasparov-Karpov draws do you hear often getting talked about? How many draws in chess are actually annotated in books or media? Very few. Remember the last Anand-Carlsen WC? People started getting frantic when the tournament started with some really stale draws. When people go to watch any sport (chess is no exception), it is natural that they want to see a result. If you go to a ManUtd-ManCity match, spend all that money and free weekend time, and you see no goals scored and a 0-0 draw, aren't you going to be disappointed? I bet you are! Those that went to see the WC live were obviously disappointed when games ended too quickly in draws. I myself stayed awake all night (2am in my timezone) to watch the first game, and I was very pissed when it ended in a draw. I think you probably don't watch TCEC live and that is why you feel this way because if you did watch live, you wouldn't be complaining.

The TCEC openings strive to bring the best entertainment value, without compromising on fairness as much as possible. This is exactly in line with what TCEC is all about -- 24/7 live high quality games between the best chess engines on very strong hardware and long time controls.

If you want to see draw chess, you should go elsewhere.

Oh, there were many memorable human draws that were discussed and overdiscussed for long periods of time, just do not want to post now any links. Draw does not mean boring or lacking in importance.

You mention entertainment all the time: it is not all about entertainment. Actually, chess is not an entertainment game at all. Wikipedia and others say it is an intellectual game and, as far as I know, entertainment and intellectual quality are not synonymous. Sports, football, baseball, etc. might be synonymous to entertainment. When you have to understand a lot, you can not possibly entertain yourself, you can enjoy it, but not entertain yourself. You can entertain yourself when you do not need to understand a lot, like watching football games.

Who said that TCEC is about entertainment? I thought its main goal was to pick up the best chess playing software, provided that hardware conditions are equal. Maybe Martin can say if it is more about entertainment or about quality chess. In any case, when you have to choose the best, one prerequisite is to have fair conditions. And as big opening advantage, even with reversed colours, already favours by some margin the weaker engine, one can not say the conditions are fair.

Man united and Man city also start with 11 men each side. I guess if you leave the weaker side with 11 men and the stronger with 10, and then do the opposite, the stronger side getting 11 and the weaker 10, quite likely both sides could win their games with 11 if the difference in strength is not too big. When it is 11-11 in both games, chances are the stronger side will achieve a better result, as it is not supposed to lose a game almost by default. So that fair conditions are a sine qua non. (do I look like a wise person now)

Any live event needs to have an entertainment appeal or it is doomed. If you want to see boring draws, you can pick up a database from CCRL, extract all draws and watch them at your leisure. TCEC has an entertainment element whether you like it or not. Again, as I explained, this does not degrade the quality in anyway. These are orthogonal terms.

If both sides play the same opening, this is 100% fair. It does not favor the weak side or anything like that. That part of your logic is flawed. We have seen countless games where one side wins the slightly favorable opening with one color and draws the reverse. None of the openings are predecided in anyway. There is always uncertainty. Anything can happen.

[Lyudmil Tsvetkov]

In the future if engines search deeper 0.6 positions that are in theory win for white are going to be mate in 80 and not 0.6 so I see no reason to assume that 0.6 is going to mean a winning position.

It may be interesting to know if 0.6 is practically a sure win at the top level
so it may be interesting to see if engines can convert all the scores of more than 0.6 after a deep search.

For example 1.d4 h5 and all the opening possibilities after 2 plies when deep search by stockfish shows more than 0.6 score at depth 30

Maybe 0.6 means more in the opening relative to middle game or endgame when it is often not enough to win based on experience but we need some evidence for it and if 0.6 is better in the opening maybe it is better simply to increase the evaluation when there are many pieces on the board.

You raise an interesting point, Uri.

I don't know if evaluation should be increased with many pieces on the board, but it is almost certain that 0.6 opening advantage is better than 0.6 edge later on. You simply have more material and other resources to further increase the advantage. The difference should not be so drastic though, maybe very subtle. If 0.45cps edge in the early opening is already a win with right evaluation and perfect search, on the verge of middlegame and endgame, with half material present, you might already need 55cps to win, 65cps in rich endgames. As the game draws near to its end, in simple endings with 2 or less pieces per side even 75cps edge might not be sufficient for a win.

That is why I have always been a very ardent fan of scaling down different types of endings, because the winning chances simply sharply decrease there. For example, you can safely scale almost all endings with 2 and less pieces per side and equal material and just 3vs 2 pawns on f-h files. One pawn edge here means nothing.

[arjuntemurnikar]

If 0.45cps edge in the early opening is already a win with right evaluation and perfect search

You clearly don't understand even the basics of computer chess. With perfect search, you have no need for evaluation. The engine will simply display a MATE-IN-X.

There is no right evaluation, and there is no perfect search.

[arjuntemurnikar]

Since you keep on complaining without actually reading the description that is plainly and lucidly written out and is available on the TCEC website, I have copy-pasted it for your convinience here:

Who said that TCEC is about entertainment? I thought its main goal was to pick up the best chess playing software, provided that hardware conditions are equal. Maybe Martin can say if it is more about entertainment or about quality chess. In any case, when you have to choose the best, one prerequisite is to have fair conditions. And as big opening advantage, even with reversed colours, already favours by some margin the weaker engine, one can not say the conditions are fair.

Season 6 opening book notes from Nelson Hernandez (nick: Cato the Younger)

TCEC is a website intended to showcase computer chess programs and provide visitors with chess games of extraordinarily high quality on top-flight hardware at long time controls, eventually producing a Seasonal 'champion'.

There are a great number of ways one might structure a sequential-stage tournament with these intentions. Veteran observers of TCEC have no doubt noticed evolutions from one season to the next in various details. Martin's chief guiding principle in all these changes has been 'entertainment first'.

With respect to openings in particular, numerous individuals have advanced or supported different proposals in the past. Some favor no opening books at all ('let the engines play!') and others favor opening books of limited or unlimited size ('let the authors match wits!'). Some favor openings being selected from an extremely short opening book of two or three moves. Some favor thematic openings, i.e. playing an opening system repeatedly during a Stage.

All of the above approaches could result in an interesting competition. However, Martin decided that the best way to promote entertainment, showcase the engines, present a fair challenge to all programs and impose the least administrative burden on everyone involved was for all games to start from pre-selected opening positions that were produced in a GM-level game in the past.

Obviously the implementation details of such a plan are a paramount concern. On one hand you do not want one-sided opening positions where the same color will almost always win. On the other hand you do not want extremely draw-prone opening positions, particularly in Stages when very evenly-matched, high-quality programs are playing. What you are looking for are hundreds of openings that are consistently in-between. Further, in a single round-robin (as in Stages 1 and 2) the positions need to be very similar in terms of the degree to which each color is advantaged. Since the engines are getting to play only one side of a position fairness is critical. Your white player will be justifiably upset if he is presented with an opening position where black is objectively much better.

To find these hundreds of opening positions with a high degree of quantitative precision you need lots of empirical and evaluative data, i.e. you need many millions of games in your database and you need deep comparative evaluations across different high-quality engines. There is simply no other way to do it reliably. You have a classic 'Big Data' problem.

Martin invited me to tackle this problem last November and I took it on with enthusiasm. I had one major component already in hand-a world-class private database with tens of millions of human and engine games, all carefully de-duplicated and verified for ending outcomes and adjusted as necessary, which has been a daily work in progress of mine since 2004 and been contributed to by many other collectors and individuals over the years.

The second component, developing a list of candidate opening positions, turned out to be easy because it was already done. Adam Hair had an eight-move book compiled from Norm Pollock's database of GM-level games. After some refinement provided by Dariusz Orzechowski and Lucas Braesch, the book contained 44,624 unique positions achieved in GM-level games throughout history. Eight moves seemed ideal to me because it is long enough to span a much fuller range of ECOs than six or fewer moves. Nine or ten moves, it seemed to me, would be taking too much liberty away from the engines.

A third component was getting deep evaluations for the candidate positions. Adam Hair and I developed these evaluations from Houdini 4, Stockfish DD and Komodo TCECr for all of the candidate positions in two months, which was all the time we had to complete the task. For Houdini and Komodo we went to 24 plies and for Stockfish we went to 28 plies (which was comparable to the others in terms of time spent per position).

Perhaps the key idea that emerged from this project was the recognition that there are four quantifiable properties to all opening positions:

Frequency (or commonness) - how often has the candidate position been seen in human and engine games?

Play-balance - how closely does the candidate position align to the relative advantage seen in the opening position in empirical and evaluative terms?

Draw-tendency - how often has the candidate position resulted in a draw in human and engine databases relative to each of those databases?

Dissonance - to what degree do leading engines disagree in deep evaluations of the position?

My opening selection process was based on quantifying these four dimensions and converting the resulting analysis into percentile-ranks. In other words, every position was boiled down to four simple numbers on a 0-100 scale. I then assigned varying weights to each of these numbers for each Stage.

In all Stages no position was selected that had not been seen at least 100 times in human and engine games; in Stage 1 this number is at least 300 times. I did not choose very rare openings because the metrics for play balance and draw-tendency would have been unreliable.

As already mentioned play-balance is critically important in Stages 1 and 2 because they are both single round-robin. Every game in those Stages will slightly favor white as is seen in the opening position. Play-balance remains quite important through the other Stages, but not as much, because engines will get to play both sides of each position.

Attempting to reduce the draw-rate is not important in Stage 1 because engines of widely varying strength will be competing resulting in many decisive games. From Stage 2 onward all openings selected have below-average draw-rates. Our theory, which was demonstrated in the Superfinal last Season, is that you can knock down draw-rates by as much as 15-20% by selecting balanced openings that have historically produced low draw-rates.

Dissonance is irrelevant to my opening selection until Stage 4 and the Superfinal. In those Stages I purposely picked positions where the leading engines displayed sharp evaluation disagreements while not being heavily unbalanced in favor of one color. I am hoping that this will both further depress the draw-rate as well as lead to some really wild and exciting games. (Or it could lead to some big names getting steamrolled because they flat-out do not understand the starting position, in which case it will be very revealing!)

Q&A

Q: Can you provide us with a list of what openings will be played in each Stage?
A: I could, but Martin prefers the element of surprise and so do I.

Q: Will you know what openings any two engines will play before a game starts?
A: No. For each Stage I have provided Martin with a pgn file containing all the opening positions that will be played. His tournament GUI selects the openings at random, never repeating one that has already been played, apart from cases where engines are playing both sides of the same opening.

Q: You say you compared evaluations across the engines. How could you do this when engines have different ways of evaluating positions and scale differently?
A: Adam Hair did a linear regression comparing the 44,000+ evaluations and concluded that Stockfish DD is about 1.88x higher than Houdini 4 while Komodo TCECr is 1.18x higher. We adjusted their evaluations accordingly to get a better impression of evaluation dissonance. Correlations were quite high after we did this.

Q: Is there any theme or opening system that will be seen more often than others?
A: Not by design. It is interesting, though, that 1.e4 openings are more predominant in Stage 1 and 2 but 1.d4 openings achieve parity after that. This outcome was happenstance.

Q: Any gambits or exotic openings in store?
A: Gambits, or positions that evaluate poorly but have compensation in human contests will be seldom seen. I did not go looking for them. As for exotic positions, different people will see different things in a position. I picked positions entirely on the basis of their known, objective numerical properties, not my impressions of what I saw on the board, which would be very subjective and unreliable.

Q: Would you be so kind as to send me your database?
A: No.

My thanks to my collaborators, Adam Hair and Anson Williams, whose assistance and input were essential, and also to Martin Thoresen, whose encouragement and trust that we could pull it all off in time for this Season's launch certainly helped us complete this ambitious project.

I hope now it is clear.

[Lyudmil Tsvetkov]

Since you keep on complaining without actually reading the description that is plainly and lucidly written out and is available on the TCEC website, I have copy-pasted it for your convinience here:

Who said that TCEC is about entertainment? I thought its main goal was to pick up the best chess playing software, provided that hardware conditions are equal. Maybe Martin can say if it is more about entertainment or about quality chess. In any case, when you have to choose the best, one prerequisite is to have fair conditions. And as big opening advantage, even with reversed colours, already favours by some margin the weaker engine, one can not say the conditions are fair.

Season 6 opening book notes from Nelson Hernandez (nick: Cato the Younger)

TCEC is a website intended to showcase computer chess programs and provide visitors with chess games of extraordinarily high quality on top-flight hardware at long time controls, eventually producing a Seasonal 'champion'.

There are a great number of ways one might structure a sequential-stage tournament with these intentions. Veteran observers of TCEC have no doubt noticed evolutions from one season to the next in various details. Martin's chief guiding principle in all these changes has been 'entertainment first'.

With respect to openings in particular, numerous individuals have advanced or supported different proposals in the past. Some favor no opening books at all ('let the engines play!') and others favor opening books of limited or unlimited size ('let the authors match wits!'). Some favor openings being selected from an extremely short opening book of two or three moves. Some favor thematic openings, i.e. playing an opening system repeatedly during a Stage.

All of the above approaches could result in an interesting competition. However, Martin decided that the best way to promote entertainment, showcase the engines, present a fair challenge to all programs and impose the least administrative burden on everyone involved was for all games to start from pre-selected opening positions that were produced in a GM-level game in the past.

Obviously the implementation details of such a plan are a paramount concern. On one hand you do not want one-sided opening positions where the same color will almost always win. On the other hand you do not want extremely draw-prone opening positions, particularly in Stages when very evenly-matched, high-quality programs are playing. What you are looking for are hundreds of openings that are consistently in-between. Further, in a single round-robin (as in Stages 1 and 2) the positions need to be very similar in terms of the degree to which each color is advantaged. Since the engines are getting to play only one side of a position fairness is critical. Your white player will be justifiably upset if he is presented with an opening position where black is objectively much better.

To find these hundreds of opening positions with a high degree of quantitative precision you need lots of empirical and evaluative data, i.e. you need many millions of games in your database and you need deep comparative evaluations across different high-quality engines. There is simply no other way to do it reliably. You have a classic 'Big Data' problem.

Martin invited me to tackle this problem last November and I took it on with enthusiasm. I had one major component already in hand-a world-class private database with tens of millions of human and engine games, all carefully de-duplicated and verified for ending outcomes and adjusted as necessary, which has been a daily work in progress of mine since 2004 and been contributed to by many other collectors and individuals over the years.

The second component, developing a list of candidate opening positions, turned out to be easy because it was already done. Adam Hair had an eight-move book compiled from Norm Pollock's database of GM-level games. After some refinement provided by Dariusz Orzechowski and Lucas Braesch, the book contained 44,624 unique positions achieved in GM-level games throughout history. Eight moves seemed ideal to me because it is long enough to span a much fuller range of ECOs than six or fewer moves. Nine or ten moves, it seemed to me, would be taking too much liberty away from the engines.

A third component was getting deep evaluations for the candidate positions. Adam Hair and I developed these evaluations from Houdini 4, Stockfish DD and Komodo TCECr for all of the candidate positions in two months, which was all the time we had to complete the task. For Houdini and Komodo we went to 24 plies and for Stockfish we went to 28 plies (which was comparable to the others in terms of time spent per position).

Perhaps the key idea that emerged from this project was the recognition that there are four quantifiable properties to all opening positions:

Frequency (or commonness) - how often has the candidate position been seen in human and engine games?

Play-balance - how closely does the candidate position align to the relative advantage seen in the opening position in empirical and evaluative terms?

Draw-tendency - how often has the candidate position resulted in a draw in human and engine databases relative to each of those databases?

Dissonance - to what degree do leading engines disagree in deep evaluations of the position?

My opening selection process was based on quantifying these four dimensions and converting the resulting analysis into percentile-ranks. In other words, every position was boiled down to four simple numbers on a 0-100 scale. I then assigned varying weights to each of these numbers for each Stage.

In all Stages no position was selected that had not been seen at least 100 times in human and engine games; in Stage 1 this number is at least 300 times. I did not choose very rare openings because the metrics for play balance and draw-tendency would have been unreliable.

As already mentioned play-balance is critically important in Stages 1 and 2 because they are both single round-robin. Every game in those Stages will slightly favor white as is seen in the opening position. Play-balance remains quite important through the other Stages, but not as much, because engines will get to play both sides of each position.

Attempting to reduce the draw-rate is not important in Stage 1 because engines of widely varying strength will be competing resulting in many decisive games. From Stage 2 onward all openings selected have below-average draw-rates. Our theory, which was demonstrated in the Superfinal last Season, is that you can knock down draw-rates by as much as 15-20% by selecting balanced openings that have historically produced low draw-rates.

Dissonance is irrelevant to my opening selection until Stage 4 and the Superfinal. In those Stages I purposely picked positions where the leading engines displayed sharp evaluation disagreements while not being heavily unbalanced in favor of one color. I am hoping that this will both further depress the draw-rate as well as lead to some really wild and exciting games. (Or it could lead to some big names getting steamrolled because they flat-out do not understand the starting position, in which case it will be very revealing!)

Q&A

Q: Can you provide us with a list of what openings will be played in each Stage?
A: I could, but Martin prefers the element of surprise and so do I.

Q: Will you know what openings any two engines will play before a game starts?
A: No. For each Stage I have provided Martin with a pgn file containing all the opening positions that will be played. His tournament GUI selects the openings at random, never repeating one that has already been played, apart from cases where engines are playing both sides of the same opening.

Q: You say you compared evaluations across the engines. How could you do this when engines have different ways of evaluating positions and scale differently?
A: Adam Hair did a linear regression comparing the 44,000+ evaluations and concluded that Stockfish DD is about 1.88x higher than Houdini 4 while Komodo TCECr is 1.18x higher. We adjusted their evaluations accordingly to get a better impression of evaluation dissonance. Correlations were quite high after we did this.

Q: Is there any theme or opening system that will be seen more often than others?
A: Not by design. It is interesting, though, that 1.e4 openings are more predominant in Stage 1 and 2 but 1.d4 openings achieve parity after that. This outcome was happenstance.

Q: Any gambits or exotic openings in store?
A: Gambits, or positions that evaluate poorly but have compensation in human contests will be seldom seen. I did not go looking for them. As for exotic positions, different people will see different things in a position. I picked positions entirely on the basis of their known, objective numerical properties, not my impressions of what I saw on the board, which would be very subjective and unreliable.

Q: Would you be so kind as to send me your database?
A: No.

My thanks to my collaborators, Adam Hair and Anson Williams, whose assistance and input were essential, and also to Martin Thoresen, whose encouragement and trust that we could pull it all off in time for this Season's launch certainly helped us complete this ambitious project.

I hope now it is clear.

TCEC is a website intended to showcase computer chess programs and provide visitors with chess games of extraordinarily high quality on top-flight hardware at long time controls, eventually producing a Seasonal 'champion'.


Did you read this one, which is written first?

This practically means that the primary intention is precision measurement of engine strength. Everyone wants entertainment. But it is not possible always. You can not have it both ways: you either prefer the serious part, or the entertainment. In case you prefer the entertainment, you could very well forget about claims that this tournament is actually an official World championship, which it actually is.

I am sure Martin understands this very well. And also other persons on the team. And, as really TCEC is but much much above the 7-9 rounds official world championship that was last staged in Yokohama, even in the domain of book usage, because the book is unified, I do not see why we should argue more. As it currently stands, TCEC is about choosing the best chess software entity on Earth. If this is the case, you simply can not compromise on fair conditions. No one compromises with fair conditions in a world championship.

The book team did what they could, they improved on the older book substantially. Should we have stayed with the older book, as it provided more fun? And Nelson and co would of course have removed dubious openings, had they been able to check them with bigger depths and newer engine versions. They would have replaced them with other, more balanced openings, but in no way less interesting. So it is about limited resources. And about constant evolution. TCEC is improving with every season and I am sure it will continue to do so.

But about the world championship and the entertainment part: you really can not have it both ways. I remember when I used to play AnMon and when I am playing SF now. AnMon was an entertainment partner, it makes many mistakes, and I also make many mistakes, so I can just take it easy. When it comes to SF, it is really difficult to entertain yourself because the challenge is different. You need a lot of effort to understand every single move the engine plays and effort is no fun in any way. You really can not have it both ways.

[Lyudmil Tsvetkov]

Congratulations to the Stockfish team!

Stockfish has just won the TCEC superfinal, becoming the official world champion for the first time in its history.

Congratulations to Marco, Tord, Joona and everybody involved in this wonderful, as they call it, international project.

SF dominated throughout the match and could have even scored more convincingly. What is the SF secret? In the first place, it is its striking depth, 8-12 plies on the average more than other engines. However, it is not just depth. SF had similar depths and still failed to win previous TCEC editions. It is also the eval, however imperfect it might be. SF improved here and I already believe its eval might be a bit better than most of the top engines. And of course, what mattered very much in this final, was the last-minute Joona SMP patch that added the handful of elos necessary to outclass Komodo in the final. When the strength of engines is almost equal, a handful of elo matters very much.

So that everything helped for this amazing success. And of course, it was Gary Linscott and the framework. Without the framework, SF would not be here now. So that all hardware contributors to the framework deserve a big portion of this success.

I hope we will be treated soon to a new SF 5 that will top all the rating lists. I think SF 5 should be the version that played in the final, not only because there are currently some possible regressions, but mainly because people would like to associate the new version with the memorable event.

I intended to comment a bit more, but now it already does not make sense any more.

I hope I did not say something Marco should have said first, just wanted to congratulate.

Congratulations again.

[Modern Times]

Indeed a brilliant effort, and reminds me that I must set aside some CPU cores to help this project.

[Henk]

In Holland we actually only have one railway company. They don't repair much. Some bad connections stay forever it seems.

[ernest]

becoming the official world champion for the first time in its history.

unofficial !...

[Lyudmil Tsvetkov]

becoming the official world champion for the first time in its history.

unofficial !...

Of course it is official.
Unofficial are Deep Junior Yokohama and Hiarcs Yokohama.