ZeroFish - getting the best of 2 worlds

[Rebel]

ZeroFish is an experimental program that tries to combine the best of Stockfish and Lc0 in a number of experiments.

Experiment 1 - it's widely assumed that Stockfish is better in tactics than Lc0, we setup a crude MEA system:

....

http://rebel13.nl/prodeo/zerofish.html

[matejst]

Ed,

I went to your site to read about ZeroFish, then continued (re)reading other articles (still interested in ERL) and finally stumbled over the Speedy Rating list. You were so amiable to test almost all engines so far, so I would ask you, if it is possible, to test the one of these hybrid NN engines -- StockFish NN or StockFinn.

While we are at it: I find that the NN validated your ideas about evaluation and big data. There was a lot of Elo (and not only Elo) to find there, but eventually another way was chosen to incorporated previous experience/knowledge into modern engines.

[Rebel]

Ed,

I went to your site to read about ZeroFish, then continued (re)reading other articles (still interested in ERL) and finally stumbled over the Speedy Rating list. You were so amiable to test almost all engines so far, so I would ask you, if it is possible, to test the one of these hybrid NN engines -- StockFish NN or StockFinn.

While we are at it: I find that the NN validated your ideas about evaluation and big data. There was a lot of Elo (and not only Elo) to find there, but eventually another way was chosen to incorporated previous experience/knowledge into modern engines.

I could try.

[matejst]

I could try.

Thanks, Ed.

[Leo]

Zerofish seems like a great idea.

[matejst]

Zerofish seems like a great idea.

Indeed it is. A similar idea was tested before, but here it makes even more sense. It could be a great analysis tool, to study and enjoy chess.

[Rebel]

I could try.

Thanks, Ed.

StockFiNN 0.1 1000ms

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 1  Stockfish 11     324287  11:30:00.8  23191  40000  3242  400000  81.07%   1000   128    1     0
 2  StockfiNN 01     322894  11:30:58.1  23357  40000  3228  400000  80.72%   1000   128    1     0
 3  Komodo 14        317110  11:09:21.4  22398  40000  3171  400000  79.28%   1000   128    1     0
 4  Houdini 6.03     314596  11:27:54.8  22218  40000  3146  400000  78.65%   1000   128    1     0
 5  SlowChess 2.2    313139  10:30:56.4  21860  40000  3131  400000  78.28%   1000   128    1     0
 6  Ethereal 12.25   312493  11:30:24.1  21959  40000  3124  400000  78.12%   1000   128    1     0
 7  rofChade 2.3     309642  11:24:20.2  21587  40000  3096  400000  77.41%   1000   128    1    41
 8  SlowChess 2.1    309590  10:31:56.2  21595  40000  3096  400000  77.40%   1000   128    1     0
 9  Ethereal 12      309088  11:30:20.9  21840  40000  3090  400000  77.27%   1000   128    1     0
10  Laser 1.7        308178  11:29:30.5  21320  40000  3081  400000  77.04%   1000   128    1     0
11  Schooner 2.2     306808  10:22:01.9  21335  40000  3068  400000  76.70%   1000   128    1     0
12  Xiphos 0.6       306796  10:03:55.9  21333  40000  3068  400000  76.70%   1000   128    1     0
13  RubiChess 1.7.2  305900  11:26:03.9  21215  40000  3059  400000  76.48%   1000   128    1     0
14  Booot 6.4        305082  12:56:19.6  21267  40000  3050  400000  76.27%   1000   128    1   124
15  Defenchess 2.2   303754  10:12:00.7  21208  40000  3037  400000  75.94%   1000   128    1    41
16  Andscacs 0.95    301560  12:10:30.8  20745  40000  3015  400000  75.39%   1005   128    1    41
17  Shredder 13      300619  11:28:47.0  20946  40000  3006  400000  75.15%   1000   128    1     0
18  Wasp 4.00        299925  11:45:50.1  20727  40000  2999  400000  74.98%   1000   128    1     0
19  Senpai 2         299279  11:36:50.5  20615  40000  2992  400000  74.82%   1000   128    1    42
20  Arasan 22        298663  11:27:32.7  20460  40000  2986  400000  74.67%   1000   128    1     0
21  ice 4.0          297995  11:49:51.1  20767  40000  2980  400000  74.50%   1000   128    1     0
22  Critter 1.6      297939  11:38:55.1  20654  40000  2979  400000  74.48%   1000   128    1    22
23  Fire 7.1         296996  11:26:35.7  20356  40000  2970  400000  74.25%   1000   128    1     0
24  Vajolet 2.8      296269  11:25:51.6  20327  40000  2962  400000  74.07%   1000   128    1     0
25  Demolito 200514  295686  10:53:58.5  20429  40000  2956  400000  73.92%   1000   128    1     0
26  igel 2.50        294457  11:29:26.8  20369  40000  2944  400000  73.61%   1000   128    1     0
27  Wasp 3.75        293908  11:44:35.3  20203  40000  2939  400000  73.48%   1000   128    1     0
28  Gogobello 2.2    293824  11:44:24.1  20296  40000  2938  400000  73.46%   1000   128    1    41
29  igel 2.40        293056  11:28:38.5  20261  40000  2930  400000  73.26%   1000   128    1    41
30  Nemorino 5.00    291649  10:05:27.4  19882  40000  2916  400000  72.91%   1000   128    1    41
31  Rodent 4         291647  11:45:03.3  19774  40000  2916  400000  72.91%   1000   128    1     0
32  Winter 0.8       291596  10:02:03.0  20057  40000  2916  400000  72.90%   1000   128    1    41
33  Hannibal 1.7     291446  11:39:04.6  19985  40000  2914  400000  72.86%   1000   128    1     0
34  Combusken 1.2    291389  12:25:31.9  19902  40000  2914  400000  72.85%   1000   128    1    41
35  Texel 1.7        291164  11:27:40.3  20064  40000  2911  400000  72.79%   1000   128    1     0
36  Monolith 2       290807  11:31:27.5  19972  40000  2908  400000  72.70%   1000   128    1     0
37  Topple 0.7.5     290649  12:13:57.0  19696  40000  2906  400000  72.66%   1000   128    1     0
38  Deuterium 2019   290578  11:31:42.7  19819  40000  2905  400000  72.64%   1000   128    1     0
39  Chiron 4         290250  11:46:17.8  19978  40000  2902  400000  72.56%   1000   128    1     0
40  Godel 7.0        290222  12:52:57.9  19758  40000  2902  400000  72.56%   1000   128    1    42
41  Amoeba 3.1       289712  09:40:26.2  19682  40000  2897  400000  72.43%   1000   128    1     0
42  Minic 2.32       289394  11:01:03.5  19880  40000  2894  400000  72.35%   1000   128    1     0
43  FabChess 1.15    289296  11:50:40.2  19849  40000  2892  400000  72.32%   1000   128    1    41
44  sting-sf-21      288904  11:31:56.3  19905  40000  2889  400000  72.23%   1000   128    1     0
45  Marvin 3.60      288732  10:48:52.8  19848  40000  2887  400000  72.18%   1000   128    1    41
46  Minic 2.25       287730  10:54:48.6  19780  40000  2877  400000  71.93%   1000   128    1     0
47  Bobcat 8         287527  10:54:11.9  19486  40000  2875  400000  71.88%   1000   128    1     0
48  Invictus r305    285355  10:16:08.6  19227  40000  2853  400000  71.34%   1000   128    1     0
49  Counter 3.5      285137  11:25:20.1  19329  40000  2851  400000  71.28%   1000   128    1     0
50  Weiss 1.0        284693  11:23:14.3  19241  40000  2846  400000  71.17%   1000   128    1     0
51  Asymptote 0.8    284305  12:19:43.9  19245  40000  2843  400000  71.08%   1000   128    1    41
52  Weiss 0.10       281037  11:06:09.2  18927  40000  2810  400000  70.26%   1000   128    1     0
53  Tucano 8.0       280426  04:46:55.0  18777  40000  2804  400000  70.11%   1000   128    1     0
54  RuyDos 1.1.11    278666  10:25:44.5  18567  40000  2786  400000  69.67%   1000   128    1     0
55  Cheese 2.1       277252  11:28:33.3  18763  40000  2772  400000  69.31%   1000   128    1     0
56  GreKo 2020.03    276482  11:29:35.7  18694  40000  2764  400000  69.12%   1000   128    1     0
57  Fruit 2.3        275618  11:30:55.1  18270  40000  2756  400000  68.90%   1000   128    1     0
58  ProDeo 2.2       273793  11:29:46.3  18263  40000  2738  400000  68.45%   1000   128    1     0
59  Benjamin         271801  11:29:55.8  17985  40000  2718  400000  67.95%   1000   128    1     0
60  Fridolin 3.10    271357  11:40:42.9  18030  40000  2713  400000  67.84%   1000   128    1     0
61  Fruit 2.1        270830  11:29:45.7  17982  40000  2708  400000  67.71%   1000   128    1     0
62  Devel 3.0.0b     269433  12:22:53.0  17986  40000  2694  400000  67.36%   1000   128    1    65
63  Orion 0.6        268120  11:31:45.3  17895  40000  2681  400000  67.03%   1000   128    1    41
64  Ruffian 2        265592  11:13:01.7  17740  40000  2656  400000  66.40%   1000   128    1     0
65  Stash 18.0       253693  11:10:20.4  16848  40000  2536  400000  63.42%   1000   128    1     0
66  CT800 1.40       253449  10:45:20.4  16533  40000  2534  400000  63.36%   1000   128    1     0
67  Stash 17.0       250114  11:11:49.7  16549  40000  2501  400000  62.53%   1000   128    1     0
68  Stash 16.0       249149  11:10:56.4  16461  40000  2491  400000  62.29%   1000   128    1     0
69  Ares 005-2       243380  04:42:01.1  15937  40000  2433  400000  60.84%   1000   128    1     0
70  FoxSee 3.3.3     240244  11:08:18.3  15505  40000  2402  400000  60.06%   1000   128    1     0

                                    Created with MEA
                                          by
                                       Ferdinand
                                         Mosca

StockFiNN 0.1 4000ms

Code: Select all

    EPD  : epd\lc1.epd
    Time : 4000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 1  StockfiNN 01     335716  44:50:30.7  24660  40000  3357  400000  83.93%   4000   128    1     0
 2  Stockfish 11     333316  44:49:32.7  24467  40000  3333  400000  83.33%   4000   128    1     0
 3  Komodo 14        326627  44:27:14.3  23558  40000  3266  400000  81.66%   4000   128    1     0
 4  Houdini 6.03     323896  44:47:34.1  23189  40000  3238  400000  80.97%   4000   128    1     0
 5  SlowChess 2.2    322132  41:55:28.0  22885  40000  3221  400000  80.53%   4000   128    1     0
 6  Ethereal 12      321795  44:44:14.7  22957  40000  3218  400000  80.45%   4000   128    1     0
 7  rofChade 2.3     318859  44:39:33.9  22599  40000  3188  400000  79.71%   4000   128    1    41
 8  SlowChess 2.1    318320  41:59:55.0  22470  40000  3183  400000  79.58%   4000   128    1     0
 9  Xiphos 0.6       317879  43:21:42.3  22555  40000  3178  400000  79.47%   4000   128    1     0
10  Schooner 2.2     317565  43:41:15.3  22562  40000  3175  400000  79.39%   4000   128    1     0
11  Booot 6.4        316479  47:12:45.4  22523  40000  3164  400000  79.12%   4000   128    1     2
12  RubiChess 1.7.2  314933  44:43:11.4  22096  40000  3149  400000  78.73%   4000   128    1     0
13  Laser 1.7        314814  44:45:07.0  22345  40000  3148  400000  78.70%   4000   128    1     0
14  Defenchess 2.2   312030  38:52:42.1  22110  40000  3120  400000  78.01%   4000   128    1    41
15  Andscacs 0.95    311822  45:24:58.7  21794  40000  3118  400000  77.96%   4000   128    1    41
16  ice 4.0          309829  44:52:39.6  21658  40000  3098  400000  77.46%   4000   128    1     0
17  Shredder 13      309807  44:48:21.9  21994  40000  3098  400000  77.45%   4000   128    1     0
18  Arasan 22        309362  44:45:31.2  21516  40000  3093  400000  77.34%   4000   128    1     0
19  Fire 7.1         308708  44:42:14.0  21560  40000  3087  400000  77.18%   4000   128    1     0
20  Senpai 2         307960  44:42:47.2  21479  40000  3079  400000  76.99%   4000   128    1    42
21  Vajolet 2.8      307941  45:11:43.1  21536  40000  3079  400000  76.99%   4000   128    1     0
22  Gogobello 2.2    307173  45:07:39.2  21400  40000  3071  400000  76.79%   4000   128    1    41
23  Critter 1.6      306779  44:56:44.7  21528  40000  3067  400000  76.69%   4000   128    1     2
24  Demolito 200514  306093  44:12:20.5  21277  40000  3060  400000  76.52%   4000   128    1     0
25  Wasp 4.00        305798  45:02:22.4  21558  40000  3058  400000  76.45%   4000   128    1     0
26  igel 2.40        305711  44:41:01.8  21288  40000  3057  400000  76.43%   4000   128    1    41
27  Winter 0.8       303798  43:20:18.5  21335  40000  3038  400000  75.95%   4000   128    1    41
28  Minic 2.32       303662  41:28:25.6  21042  40000  3036  400000  75.92%   4000   128    1     0
29  Texel 1.7        303519  44:44:52.5  21036  40000  3035  400000  75.88%   4000   128    1     0
30  Nemorino 5.00    302844  43:23:07.1  21009  40000  3028  400000  75.71%   4000   128    1    41
31  Wasp 3.75        301668  45:01:39.0  21049  40000  3016  400000  75.42%   4000   128    1     0
32  FabChess 1.15    301637  44:48:29.5  20899  40000  3016  400000  75.41%   4000   128    1    41
33  Rodent 4         301144  44:55:36.6  20719  40000  3011  400000  75.29%   4000   128    1     0
34  Topple 0.7.5     300807  45:21:48.1  20742  40000  3008  400000  75.20%   4000   128    1     0
35  Chiron 4         300637  45:00:28.3  20953  40000  3006  400000  75.16%   4000   128    1     0
36  Amoeba 3.1       300288  36:57:29.1  20750  40000  3002  400000  75.07%   4000   128    1     0
37  Marvin 3.60      300080  44:10:55.7  20986  40000  3000  400000  75.02%   4000   128    1    41
38  Monolith 2       299874  44:44:18.9  20911  40000  2998  400000  74.97%   4000   128    1     0
39  Combusken 1.2    299766  45:28:35.3  20880  40000  2997  400000  74.94%   4000   128    1    41
40  sting-sf-21      297202  44:48:36.1  20697  40000  2972  400000  74.30%   4000   128    1     0
41  Deuterium 2019   297072  44:41:19.1  20712  40000  2970  400000  74.27%   4000   128    1     0
42  Godel 7.0        296809  45:19:39.9  20640  40000  2968  400000  74.20%   4000   128    1    42
43  Bobcat 8         294708  42:35:45.9  20387  40000  2947  400000  73.68%   4000   128    1     0
44  Tucano 8.0       294410  31:43:42.9  20135  40000  2944  400000  73.60%   4000   128    1     0
45  Asymptote 0.8    294116  45:21:11.1  20110  40000  2941  400000  73.53%   4000   128    1    41
46  Weiss 1.0        293699  44:43:10.2  20111  40000  2936  400000  73.42%   4000   128    1     0
47  Invictus r305    292937  40:07:31.4  19986  40000  2929  400000  73.23%   4000   128    1     0
48  Hannibal 1.7     291930  42:39:15.1  20365  40000  2919  400000  72.98%   4000   128    1     0
49  Counter 3.5      291767  44:40:43.3  20239  40000  2917  400000  72.94%   4000   128    1     0
50  Weiss 0.10       289536  44:25:53.2  19821  40000  2895  400000  72.38%   4000   128    1     0
51  RuyDos 1.1.11    288440  43:41:56.9  19632  40000  2884  400000  72.11%   4000   128    1     0
52  Cheese 2.1       286202  44:44:56.2  19579  40000  2862  400000  71.55%   4000   128    1     0
53  Fruit 2.3        285653  44:43:17.1  19219  40000  2856  400000  71.41%   4000   128    1     0
54  GreKo 2020.03    285588  44:40:38.2  19566  40000  2856  400000  71.40%   4000   128    1     0
55  ProDeo 2.2       282592  44:47:12.1  18903  40000  2826  400000  70.65%   4000   128    1     0
56  Fridolin 3.10    281514  44:53:55.1  18984  40000  2815  400000  70.38%   4000   128    1     0
57  Benjamin         280598  44:47:19.4  18740  40000  2806  400000  70.15%   4000   128    1     0
58  Fruit 2.1        280170  44:41:52.0  18825  40000  2801  400000  70.04%   4000   128    1     0
59  Devel 3.0.0b     279037  45:29:21.4  18849  40000  2790  400000  69.76%   4000   128    1    46
60  Ruffian 2        277825  44:31:05.7  18629  40000  2778  400000  69.46%   4000   128    1     0
61  Orion 0.6        277732  44:45:12.5  18797  40000  2777  400000  69.43%   4000   128    1    41
62  Ares 005-2       261984  49:44:18.2  17472  40000  2620  400000  65.50%   4000   128    1     0
63  Stash 16.0       261931  44:34:09.0  17370  40000  2619  400000  65.48%   4000   128    1     0
64  CT800 1.40       261194  44:00:47.7  17252  40000  2612  400000  65.30%   4000   128    1     0
65  FoxSee 3.3.3     249604  44:15:53.8  16313  40000  2496  400000  62.40%   4000   128    1     0

                                    Created with MEA
                                          by
                                       Ferdinand
                                         Mosca

Wild guess - I don't expect the same exceptional good result when testing StochFiNN 0.1 with the SF (sf1.epd) analysis, but we will see, I have it running

[Rebel]

StockFiNN 0.1 1000ms SF1.EPD (the Stockfish anlysis)

Code: Select all

    EPD  : epd\sf1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 1  Stockfish 11     336059  11:29:37.8  24143  40000  3360  400000  84.01%   1000   128    1     0
 2  Komodo 14        319226  11:09:29.0  22311  40000  3192  400000  79.81%   1000   128    1     0
 3  StockfiNN 01     317901  11:30:53.1  22348  40000  3179  400000  79.48%   1000   128    1     0
 4  Houdini 6.03     316790  11:28:13.0  22228  40000  3168  400000  79.20%   1000   128    1     0
 5  SlowChess 2.2    310267  10:30:32.8  21613  40000  3102  400000  77.57%   1000   128    1     0
 6  Ethereal 12      309903  11:30:46.0  21512  40000  3099  400000  77.48%   1000   128    1     0
 7  Defenchess 2.2   306114  10:11:24.2  21153  40000  3061  400000  76.53%   1000   128    1    39
 8  rofChade 2.3     305616  11:25:15.0  21126  40000  3056  400000  76.40%   1000   128    1    39
 9  Xiphos 0.6       304989  10:03:40.2  21051  40000  3050  400000  76.25%   1000   128    1     0
10  Laser 1.7        304526  11:28:48.8  21014  40000  3045  400000  76.13%   1000   128    1     0
11  SlowChess 2.1    304347  10:31:13.3  21018  40000  3043  400000  76.09%   1000   128    1     0
12  Booot 6.4        304313  12:54:41.9  20968  40000  3043  400000  76.08%   1000   128    1   153
13  RubiChess-1.7.2  303175  11:25:39.8  20977  40000  3031  400000  75.79%   1000   128    1     0
14  Schooner 2.2     302004  10:19:22.6  20800  40000  3020  400000  75.50%   1000   128    1     0
15  Andscacs 0.95    300213  12:10:08.1  20814  40000  3002  400000  75.05%   1005   128    1    39
16  Shredder 13      299610  11:29:20.3  20610  40000  2996  400000  74.90%   1000   128    1     0
17  Demolito 200514  296246  10:54:32.7  20277  40000  2962  400000  74.06%   1000   128    1     0
18  Wasp 4.00        295032  11:45:56.8  20189  40000  2950  400000  73.76%   1000   128    1     0
19  Fire 7.1         294904  11:25:51.4  20082  40000  2949  400000  73.73%   1000   128    1     0
20  ice 4.0          293707  11:44:34.3  20274  40000  2937  400000  73.43%   1000   128    1     0
21  Wasp 3.75        293471  11:45:23.9  19838  40000  2934  400000  73.37%   1000   128    1     0
22  Vajolet 2.8      293281  11:25:55.1  19868  40000  2932  400000  73.32%   1000   128    1     0
23  Arasan 22        291556  11:27:52.6  20013  40000  2915  400000  72.89%   1000   128    1     0
24  Gogobello 2.2    289938  11:45:41.7  19724  40000  2899  400000  72.48%   1000   128    1    39
25  igel 2.40        287689  11:27:51.2  19567  40000  2876  400000  71.92%   1000   128    1    39
26  Deuterium 2019   286625  11:32:54.5  19382  40000  2866  400000  71.66%   1000   128    1     0
27  Monolith 2       285643  11:30:49.3  19385  40000  2856  400000  71.41%   1000   128    1     0
28  Rodent 4         285176  11:45:37.0  19332  40000  2851  400000  71.29%   1000   128    1     0
29  Marvin 3.60      283132  10:51:18.7  19221  40000  2831  400000  70.78%   1000   128    1    39
30  Minic 2.25       282445  10:55:58.8  19188  40000  2824  400000  70.61%   1000   128    1     0
31  FabChess 1.15    282281  11:48:51.9  19162  40000  2822  400000  70.57%   1000   128    1    39
32  Amoeba 3.1       281918  09:40:41.7  18965  40000  2819  400000  70.48%   1000   128    1     0
33  Counter 3.5      276142  11:25:15.0  18592  40000  2761  400000  69.04%   1000   128    1     0
34  Asymptote 0.8    274789  12:15:41.4  18507  40000  2748  400000  68.70%   1000   128    1    39
35  Weiss 0.10       272105  11:07:28.2  18224  40000  2721  400000  68.03%   1000   128    1     0
36  Fruit 2.3        267475  11:29:21.9  17797  40000  2674  400000  66.87%   1000   128    1     0
37  Fruit 2.1        261216  11:30:13.4  17261  40000  2612  400000  65.30%   1000   128    1     0
38  Ruffian 2        259370  11:12:59.3  17236  40000  2593  400000  64.84%   1000   128    1     0

                                    Created with MEA
                                          by
                                       Ferdinand
                                         Mosca

Making more sense now.

[matejst]

Thank you, Ed, once again.

BTW, Joerg Oster made two very fast compiles for StockFish NNUE and StockFiNN for older computers, like my laptop. I am very interested to compare their evaluation, and the evaluation of Winter. I am sure NNs could be used just to improve parts of the evaluation, keeping the best of both worlds, in a manner (dis)similar to what ZeroFish tried to achieve.

[Rebel]

Well, you can install SF-NN into the MEA1 folder, Winter in the MEA2 folder and modify the zerofish.ini file accordingly, should work.