Leelafish with distilled network

[Eduard]

The distilled networks are good fast on CPU, and plays positionally very good but bad in tactical positions and in endgames.
Also I used the new Leelafish with one of these networks. For my test I used here the distilled network 11258 112x9 and as
AuxEngine "Corchess".

Setting AuxEngine:
AuxEngineDepth=18
AuxEngineFollowPvDepth=8
All others default

All following positions are played by Lc0 (distilled networks) games on my PC (i3 Core Duo 2x2,4 GHz).
You can see the difference now between Leelafish (11258 112x9 + Corchess) and Lc0 with network 112x9.

In following position Lc0 played wrong and losed after 54. Rxb3 Qxb3 55. Qxf5+! To find is here this variant:

[d]8/6p1/4Qpk1/p4n1p/2P2P2/1r1R2PP/2q1N1K1/8 w - - 0 1

Analysis by Leelafish v0.21.0-fish-v0.3:

54.Txb3 Dxb3 55.De8+ Kh6 56.g4 Sh4+ 57.Kh2 hxg4 58.hxg4 Kh7 59.Dh5+ Kg8
+/- (1.09) Tiefe: 9/16 00:00:13 1kN
54.Txb3 Dxb3 55.Dxf5+ Kxf5 56.Sd4+ Ke4 57.Sxb3 a4 58.Sc5+ Kd4 59.Sxa4 Kxc4 60.Sb6+ Kd4 61.Kf3 Kc5
+- (2.26) Tiefe: 10/16 00:00:16 1kN
54.Txb3 Dxb3 55.Dxf5+ Kxf5 56.Sd4+ Ke4 57.Sxb3 a4 58.Sc5+ Kd4 59.Sxa4 Kxc4 60.Sb6+ Kd3 61.Kf3 Kd4 62.Sc8
+- (2.42) Tiefe: 10/17 00:00:17 2kN

Analysis by Lc0 v0.21.0-rc1 w112x9:

54.Txb3 Dxb3 55.De8+ Kh6 56.g4 hxg4 57.hxg4 Se3+ 58.Kf3 Sxc4+ 59.Kf2 Db6+ 60.Kg2 Se3+ 61.Kf3 Db7+ 62.Kxe3
+- (2.02) Tiefe: 12/20 00:00:57 9kN
54.Txb3 Dxb3 55.Dxf5+ Kxf5 56.Sd4+ Ke4 57.Sxb3 a4 58.Sc5+ Kd4 59.Sxa4 Kxc4 60.Sb2+ Kd5 61.Kf3 Kd4 62.Sa4 Kd5
+- (2.18) Tiefe: 12/20 00:01:02 9kN
54.Txb3 Dxb3 55.Dxf5+ Kxf5 56.Sd4+ Ke4 57.Sxb3 a4 58.Sc5+ Kd4 59.Sxa4 Kxc4 60.Sb2+ Kd5 61.Kf3 Ke6 62.Ke4 f5+
+- (2.48) Tiefe: 12/20 00:01:07 10kN

Leelafish is much faster!

In following position Lc0 played wrong 18...c4? and after 19. e4! game was lost:

[d]r5k1/1b1Nbpp1/np2rn1p/1Bpp4/8/2N1P1B1/PP3PPP/R2R2K1 b - - 0 1

Analysis by Leelafish v0.21.0-fish-v0.3:

18...c4 19.e4 Sxe4 20.Sxe4 Txe4 21.Sxb6 Td8 22.Lxa6 Lxa6 23.Sxd5 Lc5 24.Sc7 Txd1+ 25.Txd1 Lb7 26.Kf1 h5 27.f3 Td4
+/= (0.58) Tiefe: 9/24 00:01:11 9kN
18...Se8 19.e4 d4 20.Sd5 Ld8 21.a4 h5 22.f3 h4 23.Lf4 g5 24.Ld2 Td6 25.Se5 Sac7 26.Sxc7 Sxc7 27.Sxf7
+/- (0.82) Tiefe: 9/24 00:01:13 9kN
18...Se8 19.e4 d4 20.Sd5 Ld8 21.a4 h5 22.f3 h4 23.Lf4 g5 24.Ld2 Td6 25.Se5 Sac7 26.Sxc7 Sxc7 27.Lc4
+/- (0.83) Tiefe: 9/25 00:01:15 9kN

Analysis by Lc0 v0.21.0-rc1 w112x9:

18...c4 19.b3 Sxd7 20.Lxd7 Lf6 21.Tac1 Sc5 22.Lxe6 fxe6 23.bxc4 dxc4 24.Tc2 Lxc3 25.Txc3 b5 26.a3 Se4 27.Tcc1 Sxg3 28.hxg3
+/- (0.73) Tiefe: 11/24 00:04:55 47kN
18...c4 19.b3 Sxd7 20.Lxd7 Lf6 21.Tac1 Sc5 22.Lxe6 fxe6 23.bxc4 dxc4 24.Tc2 Lxc3 25.Txc3 b5 26.a3 Se4 27.Tcc1 Sxg3 28.hxg3
+/- (0.73) Tiefe: 11/24 00:05:00 48kN

Network 11258 112x9 is totally blind.

Following position is difficult. After 25...Re5? white can play 26. Qxe5! and after 25...Rc5? white can play 26. Nf6+! (not Bb2).
Is not easy for a small network only on CPU:

[d]2r2bk1/1b1q2pp/p2pr3/1p1N3Q/4P3/P4P2/2P2RPP/2BR2K1 b - - 0 1

Analysis by Leelafish v0.21.0-fish-v0.3:

25...Te5 26.Dxe5 dxe5 27.Sf6+ gxf6 28.Txd7 Lc6 29.Td3 Lc5 30.Le3 Le7 31.Tc3 Kf7
+/- (0.82) Tiefe: 7/15 00:00:17 1kN
25...Df7 26.Dxf7+ Kxf7 27.Se3 d5 28.Sxd5 Lxd5 29.exd5 Tee8 30.d6 Ted8
+/- (1.26) Tiefe: 7/15 00:00:17 1kN
25...Df7 26.Dxf7+ Kxf7 27.Se3 d5 28.Sxd5 Lxd5 29.exd5 Tee8 30.d6 Ted8
+/- (1.26) Tiefe: 7/16 00:00:18 1kN
25...Df7 26.Dxf7+ Kxf7 27.Se3 d5 28.Sxd5 Lxd5 29.exd5 Tee8 30.d6 Ted8
+/- (1.24) Tiefe: 7/17 00:00:20 2kN
25...Df7 26.Dxf7+ Kxf7 27.Se3 Tee8 28.Te2 d5 29.exd5 Lc5 30.Kf1 a5 31.d6
+/- (1.20) Tiefe: 7/17 00:00:25 2kN

Analysis by Lc0 v0.21.0-rc1 w112x9:

25...Te5 26.Dxe5 dxe5 27.Sf6+ gxf6 28.Txd7 Lc5 29.Txb7 Lxf2+ 30.Kxf2 Txc2+ 31.Kg3 Txc1 32.Tb6 Kf7 33.Txa6 Kg6 34.Ta5
+/- (1.21) Tiefe: 7/18 00:00:29 3kN
25...Df7 26.Dxf7+ Kxf7 27.Lb2 Lxd5 28.Txd5 g6 29.Td3 Le7 30.Kf1
+/- (1.15) Tiefe: 7/18 00:00:32 3kN
25...Tc5 26.Lb2 Lxd5 27.Txd5 Dc8 28.Txc5 dxc5 29.Dd5 Dc6 30.c4 Dxd5 31.exd5
+/- (1.10) Tiefe: 7/18 00:00:47 6kN
25...Df7 26.Dxf7+ Kxf7 27.Lb2 Tee8 28.g4 Lxd5 29.Txd5 Tc4 30.Td3 Tec8 31.c3 Le7
+/- (1.14) Tiefe: 7/18 00:01:06 10kN
25...Tc5 26.Lb2 Lxd5 27.Txd5 Dc6 28.Df5 Te8 29.Txc5 dxc5 30.Td2 c4 31.Td7 Te7
+/- (1.12) Tiefe: 7/18 00:01:07 10kN
25...Df7 26.Dxf7+ Kxf7 27.Lb2 Tee8 28.h4 Lxd5 29.Txd5 Tc4 30.c3 Te5 31.Tfd2
+/- (1.19) Tiefe: 7/18 00:01:29 14kN

Leelafish finds much faster the right path.

In fallowing endgame Lc0 played in a game wrong 55. Kxf4? and losed fast.

[d]3k4/1R6/7p/p4P2/4Pr2/1P2K2p/8/8 w - - 0 1

Analysis by Leelafish v0.21.0-fish-v0.3:

55.Kxf4 h2 56.f6 Ke8 57.Tb8+ Kf7 58.Tb7+ Kxf6 59.Tb6+ Kg7 60.Tb7+ Kf8 61.Tb8+ Ke7 62.Tb7+ Kd8 63.b4 h1D 64.bxa5 Dc1+ 65.Ke5
-/+ (-1.10) Tiefe: 11/23 00:01:00 11kN, tb=1
55.Tb8+ Kc7 56.Tb5 Th4 57.Tc5+ Kd6 58.Tc1 h2 59.Th1 Ke5 60.Kf3 Tf4+ 61.Kg3 Kxe4 62.Te1+ Kxf5 63.Ta1 Tb4 64.Txa5+ Ke4 65.Kxh2 Txb3
-/+ (-1.23) Tiefe: 11/23 00:01:01 11kN, tb=2
55.Tb8+ Kc7 56.Tb5 Th4 57.Tc5+ Kd6 58.Tc1 h2 59.Th1 Ke5 60.Kf3 Tf4+ 61.Kg3 Kxe4 62.Te1+ Kxf5 63.Ta1 Tb4 64.Txa5+ Kg6 65.Kxh2 Txb3
-/+ (-1.22) Tiefe: 11/23 00:01:06 13kN, tb=5

Analysis by Lc0 v0.21.0-rc1 w112x9:

55.Kxf4 h2 56.f6 Ke8 57.Tb8+ Kf7 58.Tb7+ Kxf6 59.Tb6+ Kg7 60.Tb7+ Kf8 61.Tb8+ Ke7 62.Tb7+ Ke8 63.Tb8+ Kd7 64.Tb7+ Kc6 65.Th7 h1D 66.Ta7 Kb6 67.Tg7
-/+ (-1.38) Tiefe: 13/26 00:03:14 57kN, tb=41
55.Tb8+ Kc7 56.Tb5 h2 57.Tc5+ Kd6 58.Td5+ Ke7 59.f6+ Txf6 60.Th5 Tb6 61.Txh2 Txb3+ 62.Kd4 Tb6 63.Kc5 Te6 64.Kd5 Ta6 65.Kc5 a4 66.Kb4 a3 67.Ta2
-/+ (-1.30) Tiefe: 13/26 00:03:18 60kN, tb=45

Leelafish is much stronger, and with distilled networks good fast on CPUs. Great work!

[Leo]

The dynamic duo. This is really interesting. The 2 engines melding together to become the most powerful chess entity.

[Eduard]

I only test on CPU. Leelafish with the network 11258 96x8 seems to be better than Rybka 4.1 x64. In a short blitz test at level 5+3, Leelafish won +3 (+7=9-4). 10 variants were played from a short book (3 to 5 moves).

GUI=Fritz 15, ponder OFF
CPU=i3 core duo 2,4 GHz with 4 Threads.

Leelafish:
Network 11258 96x8
Hash=384MB
Threads=3

AuxEngine=Cfish with 128 MB hash, Threads=1
AuxEngineDepth=16
AuxEngineFollowPvDepth=6

Rybka 4.1 x64 single core Threads=2, hash 128 MB

[jorose]

Interesting, have you tried turning off the main engine and seeing how the auxilliary engine faires against Rybka?

[Eduard]

I do not understand what you mean?

[Eduard]

Sorry, I understand. No. But I will do that and then report.

[Guenther]

I do not understand what you mean?

He means, it is no wonder that LC0 + Corfish are better than Rybka 4.1, because Corfish alone would be already much better than Rybka 4.1
on your hardware condition.
Practically the question is how much LC0 weakens it vs. Rybka 4.1 ;-)

IMHO this hybrid approach only makes sense, if the hybrid is guaranteed to better than each single instance alone.

BTW what is the meaning of this:

Rybka 4.1 x64 single core Threads=2, hash 128 MB

Does this mean this is 2 threads with hyperthreading, otherwise I don't understand the 'single core' in the quote?

[Eduard]

Yes, hyperthreading. You are right Guenther! Cfish is a very fast Engine, and clear better than Rybka. Should also win with only one thread. Rybka is a little bit stronger than Lc0 on my CPU (currently leads 5-3 on timecontroll 40/40). That here was only a short test. I wanted to see how Leelafish plays. I'm later more interested in how Leelafish works in analysis mode? That could be interesting!

[jorose]

I think it would be interesting to find out where the break even point is between Leelafish being stronger than both Leela and the auxiliary engine as a function of the auxiliary engine and the specific Leela. Ie for a fixed choice of Leela, what are the weakest engines that result in a Leelafish stronger than Leela and what are the strongest auxiliary engines where Leelafish is still stronger than the auxiliary.

I imagine that it is not dependent on just playing strength, but also the strengths and weaknesses of the auxiliary engine. It could also be that the specific auxiliary engine is not so important and Leela just needs a good framework to deal with tactics or perhaps something to fight its own biases.

[Eduard]

Interesting, have you tried turning off the main engine and seeing how the auxilliary engine faires against Rybka?

Cfish only on 1/2 Core, Rybka 4.1 x64 on 1 Core:

After 8 games I finish. Cfish leads 7.5-0.5. That's too clear!