Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
stavros
Posts: 158
Joined: Tue Dec 02, 2014 12:29 am

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by stavros » Sat Feb 01, 2020 7:43 pm

Eduard wrote:
Sat Feb 01, 2020 12:54 pm
Poor Stockfish, a disaster.

really? lets compare cpu with gpu

https://www.sp-cc.de/lc0-testing.htm
cpu at 7.5MN/sec vs rtx 2060

with <400$ (same price as rtx 2060) i got ryzen 3800 8 core with 26MN/sec=3.45x faster than 7.5MN/sec in test
now lets test...

https://openbenchmarking.org/showdown/pts/stockfish
thump of rule :16c+sf=1 rtx 2080 ti+lco

User avatar
Eduard
Posts: 282
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by Eduard » Sat Feb 01, 2020 7:47 pm

He plays with the mobile RTX 2060 and slow speed. Lc0 Ratio is 1.3.

stavros
Posts: 158
Joined: Tue Dec 02, 2014 12:29 am

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by stavros » Sat Feb 01, 2020 7:55 pm

Eduard wrote:
Sat Feb 01, 2020 7:47 pm
He plays with the mobile RTX 2060 and slow speed. Lc0 Ratio is 1.3.
laptop? then i dont know laptops cant keep heavy tests,maybe mwyoung test is better?

mwyoung
Posts: 2115
Joined: Wed May 12, 2010 8:00 pm

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by mwyoung » Sat Feb 01, 2020 7:58 pm

stavros wrote:
Sat Feb 01, 2020 7:43 pm
Eduard wrote:
Sat Feb 01, 2020 12:54 pm
Poor Stockfish, a disaster.

really? lets compare cpu with gpu

https://www.sp-cc.de/lc0-testing.htm
cpu at 7.5MN/sec vs rtx 2060

with <400$ (same price as rtx 2060) i got ryzen 3800 8 core with 26MN/sec=3.45x faster than 7.5MN/sec in test
now lets test...

https://openbenchmarking.org/showdown/pts/stockfish
thump of rule :16c+sf=1 rtx 2080 ti+lco
Currently running test on that exact hardware.
Professing themselves to be wise, they became fools,
take on me. Foes 0.

Exa65536
Posts: 7
Joined: Sun Nov 25, 2018 4:28 am
Full name: Alexa Stevens

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by Exa65536 » Sun Feb 02, 2020 3:23 am

Tuning via self-play, at least in the past, has tended to produce a race to the bottom of lower CPUCT. Since the opponent is nearly a clone (aside from search settings) they're very unlikely to make moves that aren't on the nets radar at all, and so it's more important to get higher depth on the high-policy possibilities than it is to explore the possible surprises.

So, if you're genuinely playing against another instance of leela, narrower search is likely to be better, but past some point that'll stop translating into better performance vs other opponents.

pohl4711
Posts: 1300
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by pohl4711 » Sun Feb 02, 2020 11:37 am

Laskos wrote:
Sat Feb 01, 2020 12:29 pm
pohl4711 wrote:
Sat Feb 01, 2020 11:26 am

The first 45 games are played and at this point, it looks very promising.
Lc0 0.23.2kl t40-1541 (20x256) (kl= Kiudee with Laskos change CPuct=1.900) is at 62% vs. Stockfish 191210 (final result of Kiudee setting without Laskos CPuct-change was 57%), which would mean around +35 Elo more and a real destruction of Stockfish.
But 45 games does not mean a really reliable result - all can still change. We have to wait some days more, but the result is very good so far, so I let the test go on...
Thank you very much! I abandoned the tests against Stockfish, and am trying to tune several CPuct related parameters in self-games. Kiudee will come with a new, continued to longer TC global fit, but still not close to your TC and npm.
93 games played, now. "Only" 58.1% score, which is only +8 Elo better, than Kiudee-setting without CPuct-change to 1.900
Stay tuned!

mwyoung
Posts: 2115
Joined: Wed May 12, 2010 8:00 pm

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by mwyoung » Sun Feb 02, 2020 1:41 pm

So far Kiudee is not showing itself to be so perky in a real world test. But the games continue...


Hardware 2950x, RTX 2080 ti

58 of 200 played.

Blitz 0m+10s 2020

Stockfish 290120 64 POPCNT - Lc0 v0.23.2+git.c8d9095,Kiudee 15.0 - 14.0 +2/=26/-1 51.72%
Stockfish 290120 64 POPCNT - Lc0 v0.23.2+git.c8d9095 15.0 - 13.0 +3/=24/-1 53.57%

Stockfish 250120
4Gb HT
32 threads
Contempt = 0

Lc0 23.2 62183 - Kiudee
CS = 2000000
CPuct=2.147
Fpu=0.443
PolicyTemperature=1.607
CPuctBase=18368
CPuctFactor=2.815

Lc0 23.2 62183 - Custom
CS = 2000000
CPuct=3.5
Fpu=0.443
PolicyTemperature=1.75
CPuctBase=18368
CPuctFactor=2.815

live: https://www.youtube.com/watch?v=S5VfFmbTifc
Professing themselves to be wise, they became fools,
take on me. Foes 0.

User avatar
Laskos
Posts: 10314
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by Laskos » Sun Feb 02, 2020 2:42 pm

pohl4711 wrote:
Sun Feb 02, 2020 11:37 am
Laskos wrote:
Sat Feb 01, 2020 12:29 pm
pohl4711 wrote:
Sat Feb 01, 2020 11:26 am

The first 45 games are played and at this point, it looks very promising.
Lc0 0.23.2kl t40-1541 (20x256) (kl= Kiudee with Laskos change CPuct=1.900) is at 62% vs. Stockfish 191210 (final result of Kiudee setting without Laskos CPuct-change was 57%), which would mean around +35 Elo more and a real destruction of Stockfish.
But 45 games does not mean a really reliable result - all can still change. We have to wait some days more, but the result is very good so far, so I let the test go on...
Thank you very much! I abandoned the tests against Stockfish, and am trying to tune several CPuct related parameters in self-games. Kiudee will come with a new, continued to longer TC global fit, but still not close to your TC and npm.
93 games played, now. "Only" 58.1% score, which is only +8 Elo better, than Kiudee-setting without CPuct-change to 1.900
Stay tuned!
Thanks! Still, things can happen.

Hugo
Posts: 782
Joined: Tue Dec 01, 2009 10:10 am

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by Hugo » Mon Feb 03, 2020 6:01 am

Hi all

i tried the kuidee-mod settings in a match:

SF11 12cpu, contempt=0 against Leelenstein 13.2 default, ponder on=-17Elo for LS_13.2def

Code: Select all

LS_13.2, Blitz 10m+2s  2020

                                
1   Stockfish 11 64 BMI2-12cpu  +31/-21/=148 52.50%  105.0/200
2   Lc0,v0.23.2+git.c8d9095     +21/-31/=148 47.50%   95.0/200

SF11 12cpu, contempt=0 against Leelenstein 13.2 kiudee-mod, ponder on=+9Elo for LS_13.2-kiudee-mod

Code: Select all

kiudee-mod, LS_13.2, Blitz 10m+2s  2020

                                
1   Lc0,v0.23.2+git.c8d9095     +31/-26/=143 51.25%  102.5/200
2   Stockfish 11 64 BMI2-12cpu  +26/-31/=143 48.75%   97.5/200

so it seems there is a little benefit for LS_13.2 with the kiudee-mod

C.K.

pohl4711
Posts: 1300
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Crazy good LTC Kiudee "mod" setting just by adjusting CPuct

Post by pohl4711 » Mon Feb 03, 2020 9:00 am

Laskos wrote:
Sun Feb 02, 2020 2:42 pm
pohl4711 wrote:
Sun Feb 02, 2020 11:37 am
Laskos wrote:
Sat Feb 01, 2020 12:29 pm
pohl4711 wrote:
Sat Feb 01, 2020 11:26 am

The first 45 games are played and at this point, it looks very promising.
Lc0 0.23.2kl t40-1541 (20x256) (kl= Kiudee with Laskos change CPuct=1.900) is at 62% vs. Stockfish 191210 (final result of Kiudee setting without Laskos CPuct-change was 57%), which would mean around +35 Elo more and a real destruction of Stockfish.
But 45 games does not mean a really reliable result - all can still change. We have to wait some days more, but the result is very good so far, so I let the test go on...
Thank you very much! I abandoned the tests against Stockfish, and am trying to tune several CPuct related parameters in self-games. Kiudee will come with a new, continued to longer TC global fit, but still not close to your TC and npm.
93 games played, now. "Only" 58.1% score, which is only +8 Elo better, than Kiudee-setting without CPuct-change to 1.900
Stay tuned!
Thanks! Still, things can happen.
Sorry to say, that the result is disappointing, right now. After 150 games of 300, the score is only 55%, which is 2% weaker, than kiudee. So, I will abort that testrun. And will test Leelenstein 13.2 with Kiudee right now.

Post Reply