Page 3 of 4
Re: Texel tuning Zurichess quiet like generator
Posted: Sat Aug 10, 2019 6:49 pm
by xr_a_y
File is available here for some time :
https://serv.x-ray.fr/chess/CCRL-quiet- ... edp.tar.gz (my home server)
Here is also a python script to randomly split it, because it is maybe to huge to be used ...
Code: Select all
import random
import sys
def split_huge_file(file,out1,out2,percentage=0.75,seed=42):
random.seed(seed)
with open(file, 'r') as fin, \
open(out1, 'w') as foutBig, \
open(out2, 'w') as foutSmall:
for line in fin:
r = random.random()
if r < percentage:
foutBig.write(line)
else:
foutSmall.write(line)
split_huge_file(sys.argv[1], sys.argv[2], sys.argv[3], float(sys.argv[4]), int(sys.argv[5]))
Re: Texel tuning Zurichess quiet like generator
Posted: Sat Aug 10, 2019 7:17 pm
by fabianVDW
zenpawn wrote: ↑Sat Aug 10, 2019 5:40 pm
fabianVDW wrote: ↑Sat Aug 10, 2019 4:31 pm
Lichess Quiet( no illegal positions there) ) k: 1.0887, Error: 0.06516 Residual: 0.25526, STC Tests: +244 =261 -203 +20.14 +/- 25.61 So that seems to be a success
I will start LTC regression tests for the tuned params of Lichess Quiet against Version 1.9 and Version 1.9.2 (private version)
Is "Lichess Quiet" the second link from Jon above
Yes, it is
(perhaps with non-quiet positions removed?) or something else?
No, I don't remove non-quiets. I replace every position with the leaf of their q-search (which for quiet positions, is the position itself).
Re: Texel tuning Zurichess quiet like generator
Posted: Sat Aug 10, 2019 9:06 pm
by xr_a_y
xr_a_y wrote: ↑Sat Aug 10, 2019 6:49 pm
File is available here for some time :
https://serv.x-ray.fr/chess/CCRL-quiet- ... edp.tar.gz (my home server)
Here is also a python script to randomly split it, because it is maybe to huge to be used ...
Code: Select all
import random
import sys
def split_huge_file(file,out1,out2,percentage=0.75,seed=42):
random.seed(seed)
with open(file, 'r') as fin, \
open(out1, 'w') as foutBig, \
open(out2, 'w') as foutSmall:
for line in fin:
r = random.random()
if r < percentage:
foutBig.write(line)
else:
foutSmall.write(line)
split_huge_file(sys.argv[1], sys.argv[2], sys.argv[3], float(sys.argv[4]), int(sys.argv[5]))
The file has been updated, 35millions positions (instead of 38), removing duplicates.
Re: Texel tuning Zurichess quiet like generator
Posted: Sat Aug 10, 2019 9:45 pm
by fabianVDW
Thanks Vivien, I will download tomorrow
Re: Texel tuning Zurichess quiet like generator
Posted: Sun Aug 11, 2019 1:58 pm
by fabianVDW
fabianVDW wrote: ↑Sat Aug 10, 2019 4:31 pm
Lichess Quiet( no illegal positions there) ) k: 1.0887, Error: 0.06516 Residual: 0.25526, STC Tests: +244 =261 -203 +20.14 +/- 25.61 So that seems to be a success
I will start LTC regression tests for the tuned params of Lichess Quiet against Version 1.9 and Version 1.9.2 (private version)
Fabi
LTC tests passed. +289 =363 -211 +31.49 +/- 23.26 ELO. Nice.
Will soon test again with part of the CCRL data.
Fabi
Re: Texel tuning Zurichess quiet like generator
Posted: Sun Aug 11, 2019 2:50 pm
by zenpawn
fabianVDW wrote: ↑Sun Aug 11, 2019 1:58 pm
LTC tests passed. +289 =363 -211 +31.49 +/- 23.26 ELO. Nice.
Will soon test again with part of the CCRL data.
Fabi
Very nice indeed. Would you mind sharing your modified version of lichess-quiet?
Re: Texel tuning Zurichess quiet like generator
Posted: Sun Aug 11, 2019 4:11 pm
by fabianVDW
zenpawn wrote: ↑Sun Aug 11, 2019 2:50 pm
fabianVDW wrote: ↑Sun Aug 11, 2019 1:58 pm
LTC tests passed. +289 =363 -211 +31.49 +/- 23.26 ELO. Nice.
Will soon test again with part of the CCRL data.
Fabi
Very nice indeed. Would you mind sharing your modified version of lichess-quiet?
https://drive.google.com/file/d/1ZtR1My ... sp=sharing File is there.
I am afraid It is not your typical epd encoding, it has historical (dumb) reasons why it is not. It should still be pretty easy to parse just split every line at "|" and you can extract FEN and Game Result.
Fabi
Re: Texel tuning Zurichess quiet like generator
Posted: Sun Aug 11, 2019 9:46 pm
by zenpawn
fabianVDW wrote: ↑Sun Aug 11, 2019 4:11 pm
I am afraid It is not your typical epd encoding, it has historical (dumb) reasons why it is not. It should still be pretty easy to parse just split every line at "|" and you can extract FEN and Game Result.
Thank you. I did a few search and replace operations, and now it's all set.
Re: Texel tuning Zurichess quiet like generator
Posted: Mon Aug 12, 2019 8:40 pm
by fabianVDW
Tried now with 20% of CCRL data (around 7mil positions):
k: 0.6089, Error: 0.09379 Residual: 0.30625 STC Tests: Cancelled early +38 =46 -106 -130.1 +/-52.7 Elo
Data seems to be too badly labelled.
Re: Texel tuning Zurichess quiet like generator
Posted: Mon Aug 12, 2019 10:17 pm
by xr_a_y
fabianVDW wrote: ↑Mon Aug 12, 2019 8:40 pm
Tried now with 20% of CCRL data (around 7mil positions):
k: 0.6089, Error: 0.09379 Residual: 0.30625 STC Tests: Cancelled early +38 =46 -106 -130.1 +/-52.7 Elo
Data seems to be too badly labelled.
ok thanks for testing, these are only >2800 elo games. The result shall be quite reliable.