correlation between FRC and normal chess

[Uri Blass]

CCRL 40/4
first number is FRC rating second number is same time control normal chess

It seems that
2200 FRC is equivalent to 2250 normal chess
2700 FRC is equivalent to 2750 normal chess
3400 FRC is equivalent to 3250 normal chess

It seems that we get closer to perfect chess in normal chess so I guess
that it may be better for programmers to test their program today only in FRC and not in normal chess because FRC is more sensitive for changes and it is practically impossible to make a significant improvement in FRC without making a significant improvement in normal chess.



Houdini4 64 bit 3412 3239
Houdini 3 64-bit 3359 3212
Stockfish DD 64-bit 3355 3225
Critter 1.6a 64-bit 3289 3158
Houdini 2.0 64-bit 3272 3198(I assume houdini2.0c is the same)
Critter 1.2 64 bit 3237 3116
Stockfish 3 64-bit 3230 3124
Stockfish 2.1.1 64-bit 3189 3088
Stockfish 2.2.2 64-bit 3186 3111
Rybka 4.1 64-bit 3164 3099
Critter 1.01 64-bit 3163 3068
Stockfish 2.0.1 64-bit 3139 3086
Stockfish 1.8 64-bit 3130 3056
Chiron 2 64-bit 3115 3042
Stockfish 1.9.1 64-bit 3112 3051
Critter 0.90 64-bit 3106 3045
Rybka 3 64-bit 3088 3077
Chiron 1.5 64-bit 3043 2987
Critter 0.80 64-bit 3040 2988
Naum 4.2 64-bit 3038 2984
Shredder 12 3023 2950
Naum 4 64-bit 2987 2946
Chiron 1.1a 64-bit 2985 2964
Spike 1.4 Leiden 2941 2920
Stockfish 1.5.1 64-bit 2941 2910
Hiarcs 13.2 2925 2928
Hiarcs 13.1 2899 2912
Deep Sjeng WC2008 64-bit 2877 2840
Shredder 11 2877 2846
Naum 3.1 64-bit 2856 2861
Deep Sjeng 3.0 64-bit 2853 2838
Naum 3 64-bit 2832 2853
Hiarcs 12.1 2827 2831
Hiarcs 12 2822 2833
Hiarcs Paderborn 2007 2822 2817
EXchess 7.18b 64-bit 2817 2764
Glaurung 2.2 64-bit 2794 2799
Hiarcs 11.1 2776 2803
Naum 2.2 64-bit 2771 2787
Shredder 10 2770 2754
Hiarcs 11 2768 2792
Fruit 051103 2760 2787
Hiarcs 11.2 2760 2804
GNU Chess 5.50 64-bit 2756 2763
Tornado 4.88 64-bit 2753 2797
Fruit 2.3 2752 2784
Hiarcs X54 2745 2748
Octochess 5190 64-bit 2736 2770
Glaurung 2.1 64-bit 2729 2781
Bright 0.4a 2721 2778
Fruit 2.2.1 2716 2751
Spike 1.2 Turin 2711 2729
Frenzee 3.5.19 64-bit 2710 2764
Glaurung 2.0.1 64-bit 2696 2741
EXchess 6.50b 64-bit 2688 2671
Tornado 4.4 64-bit 2688 2722
Naum 2.1 2678 2751
Glaurung 1.2.1 2636 2652
Deep Sjeng 2.5 2611 2666
Nebula 2.0 64-bit 2606 2644
Daydreamer 1.75 64-bit 2604 2674
Frenzee Feb08 64-bit 2568 2695
Movei 00.8.438 2568 2628
GNU Chess 5.07.173b 64-bit 2551 2598
Pharaon 3.5.1 2545 2605
The Baron 2.23 2545 2558
Hermann 2.8 64-bit 2535 2530
Jonny 2.83 2527 2553
Amyan 1.72 2526 2596
Hamsters 0.7.1 2515 2582
Hermann 2.5 64-bit 2512 2514
Hamsters 0.6 2493 2563
Ufim 8.02 2487 2538
DanaSah 4.66 2475 2532
DanaSah 5.00 2472 2513
Movei 00.8.383 2472 2533
The Baron 1.7.0 2448 2460
Hamsters 0.5 2426 2487
Nebula 1.0 64-bit 2415 2461
LittleThought 1.052 64-bit 2402 2452
Hermann 2.0 2397 2388
Hamsters 0.4 2379 2450
Hussar 0.4 2338 2422
Patzer 3.80 2310 2378
TJchess 1.01 2288 2309
Aice 0.99.2 2257 2311
Ayito 0.2.994 2254 2292
Chispa 4.0.3 2174 2200
Hussar 0.2 2169 2224
BigLion 2.23x 1879 1989

[lucasart]

It seems that we get closer to perfect chess in normal chess so I guess
that it may be better for programmers to test their program today only in FRC and not in normal chess because FRC is more sensitive for changes and it is practically impossible to make a significant improvement in FRC without making a significant improvement in normal chess.

Totally agreed. Also we could use extremely shallow books in FRC while retaining many positions. Was already tested and approved in SF, although people don't use it in practice, because fishtest does not enable FRC castling by sending -variant fisherrandom to cutechess-cli.

[Modern Times]

The anchor point is not necessarily the same for both lists, that would need checking. You could shift the ratings on one of the lists yourself to give an engine of your choice the same rating on both, and then take another look.

[Modern Times]

.........I guess that it may be better for programmers to test their program today only in FRC and not in normal chess because FRC is more sensitive for changes and it is practically impossible to make a significant improvement in FRC without making a significant improvement in normal chess.

Yes I agree, but many programmers don't feel the need to add FRC support to their engines. Komodo is a notable example.

[Edmund]

[Modern Times]

CCRL 40/4
first number is FRC rating second number is same time control normal chess

It seems that
2200 FRC is equivalent to 2250 normal chess
2700 FRC is equivalent to 2750 normal chess
3400 FRC is equivalent to 3250 normal chess

Looking at this, it reminds me that I think what we did with the FRC list was to shift everything down a bit - maybe by the 50 Elo number - because the ratings for the top end were so high. The graph shows that too. Personally I really prefer bayeselo zero based ratings that get rid of the problem of what base values you set for your list.

[Lyudmil Tsvetkov]

CCRL 40/4
first number is FRC rating second number is same time control normal chess

It seems that
2200 FRC is equivalent to 2250 normal chess
2700 FRC is equivalent to 2750 normal chess
3400 FRC is equivalent to 3250 normal chess

It seems that we get closer to perfect chess in normal chess so I guess
that it may be better for programmers to test their program today only in FRC and not in normal chess because FRC is more sensitive for changes and it is practically impossible to make a significant improvement in FRC without making a significant improvement in normal chess.

You get closer to perfect chess, but not that close at all.
In SF you can get some 20 elo more only by tuning the current already much tuned, and even recently, psqt, all of them. And another 30 elo from tuning the weak pawns (backward, isolated, etc.) And this is not on the scale where a patch that passes SPRT 0;6 after 40000 games with some +200/250 wins more is considered to have gained approximately 2 elo, but realsitically only about 0.2 elo, as different regression tests show. There are too many eval improvements that could pass LTC only after 2000-3000 games, but you have to look for them and tune them. A patch that passes after 2000 games SPRT 0;6 I think could realistically add 4-5 elo.

3 very simple psqt examples:
- rook on 7th has been simplified, but who knows when was the last time when they checked if the values are OK; 13cps bonus in mg is too low, it might not have been in the past, but now is. changing this in psqt should not be difficult at all
- very low penalties for doubled and isolated pawns on a and h files; no oe knows why those pawns should be less weak on edge files when it is actually quite the opposite. In any case the edge file penalties should be increased
- knight psqt changed, but still not sufficiently; very nice bonus for 6th rank, but surprisingly missing for edge a and h files, even penalty there. Is that one reason why I see so many lost SF games where the opponent places a knight on a6/h6, which SF thinks is perfectly fine?

And so on, and so forth, I will not continue, but the point is you can still improve a lot, especially when you get rid of some stereotypes inherited from much older code, for example that a/h file deserves much higher penalty/lower bonus almost automatically in all cases.

Concerning FRC, Houdini has always performed and still performs significantly better than SF if compared with standard chess, although the gap gets recently much narrower. What is the reason for that? I think SF has too much standard-specific knowledge, too much things that work only in a specific environment, probably because they have been tuned so that a patchwork of elements becomes a single whole. Houdini, on the other hand, has more general knowledge, maybe less complex, but better tuned general knowledge with fewer elements that would be only standard chess-specific.