Page 3 of 10
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 9:26 am
by Dann Corbit
I disagree. Every evaluation is worthwhile. And the deeper the search and the stronger the engine, the more worthwhile it is. There is nothing fundamentally different from an opening position and any other mostly closed position. And a 36 ply search by stockfish or komodo is every bit as good as a GM evaluation.
Yes, computers can be very wrong, even offering stupid moves after a long search. But GMs can also be wrong after a good long think.
Classical openings can have blunders in them (I have found many) but they get played a lot because they get played by rote.
It is true that a GM can give you a clear explanation of a positional move, but deep computer searches can also provide insight.
Look at the complaints about Ed's best move choices. And yet these moves were made by great chess players.
GM opinions have great vale.
Statistical analysis of prior outcomes have great value.
Computer analysis of a position has great value.
We can ignore any of the above, at our peril.
By the way, I have a practice in my database.
If after a minimal search (36 plies for an opening) the move agrees with a dominant human choice, I call it good for now.
If there is a disagreement, I double the time, again and again until there is a certain outcome.
Sometimes, it is necessary to analyze the forward moves, but eventually the truth will start to float up.
I also value stats, analyzing to great depth when the actual outcomes do not appear to agree.
The same for an adamant GM opinion.
Now, I add thousands of positions every day and compute power is not unlimited.
But I think that every kind of information is valuable.
IMO YMMV
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 9:37 am
by Rebel
http://rebel13.nl/3-moves.html
3 moves available now.
But note this (and 2 and 4 moves as well) will change as I am working on a bigger database.
Stay tuned.
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 9:55 am
by Rebel
Guenther wrote: ↑Mon Jun 10, 2019 9:28 pm
[d]r1bqkb1r/pppppppp/2n2n2/8/2PP4/8/PP2PPPP/RNBQKBNR w KQkq
Code: Select all
r1bqkb1r/pppppppp/2n2n2/8/2PP4/8/PP2PPPP/RNBQKBNR w KQkq - c0 "Nc3=53, Nf3=46";
3. d5 / 3.g3 are surely as good - still 0 points? What about 3. Bg5? /3. Bf4?
New database gives - Nf3=74, Nc3=21, d5=2, g3=0, a3=0, e3=0, f3=0, Bg5=0, Bf4=0, f4=0
Not much I can do with the points since they come from the Polyglot weight.
Code: Select all
Opening Book : books\mb-2017.bin
Positions : 812.054
Book Weight Score Depth Learn
g1f3 74.91% 0 0 0
b1c3 21.11% 0 0 0
d4d5 2.92% 0 0 0
g2g3 0.35% 0 0 0
a2a3 0.22% 0 0 0
e2e3 0.13% 0 0 0
f2f3 0.13% 0 0 0
c1g5 0.08% 0 0 0
c1f4 0.08% 0 0 0
f2f4 0.04% 0 0 0
Guenther wrote: ↑Mon Jun 10, 2019 9:28 pm
[d]r1bqkbnr/pp1ppppp/2n5/2p5/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq
Code: Select all
r1bqkbnr/pp1ppppp/2n5/2p5/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq - c0 "Nf3=100";
again 3. g3 and 3. e4 are as good - still 0 points
New database gives - g3=64, Nf3=30, e3=2, e4=0, d3=0, f4=0, a3=0, b3=0, h3=0, Rb1=0
Code: Select all
Opening Book : books\mb-2017.bin
Positions : 812.054
Book Weight Score Depth Learn
g2g3 64.60% 0 0 0
g1f3 30.99% 0 0 0
e2e3 2.39% 0 0 0
e2e4 0.88% 0 0 0
d2d3 0.53% 0 0 0
f2f4 0.28% 0 0 0
a2a3 0.21% 0 0 0
b2b3 0.05% 0 0 0
h2h3 0.03% 0 0 0
a1b1 0.01% 0 0 0
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 10:03 am
by Guenther
Rebel wrote: ↑Tue Jun 11, 2019 9:55 am
Guenther wrote: ↑Mon Jun 10, 2019 9:28 pm
[d]r1bqkb1r/pppppppp/2n2n2/8/2PP4/8/PP2PPPP/RNBQKBNR w KQkq
Code: Select all
r1bqkb1r/pppppppp/2n2n2/8/2PP4/8/PP2PPPP/RNBQKBNR w KQkq - c0 "Nc3=53, Nf3=46";
3. d5 / 3.g3 are surely as good - still 0 points? What about 3. Bg5? /3. Bf4?
New database gives - Nf3=74, Nc3=21, d5=2, g3=0, a3=0, e3=0, f3=0, Bg5=0, Bf4=0, f4=0
Not much I can do with the points since they come from the Polyglot weight.
Code: Select all
Opening Book : books\mb-2017.bin
Positions : 812.054
Book Weight Score Depth Learn
g1f3 74.91% 0 0 0
b1c3 21.11% 0 0 0
d4d5 2.92% 0 0 0
g2g3 0.35% 0 0 0
a2a3 0.22% 0 0 0
e2e3 0.13% 0 0 0
f2f3 0.13% 0 0 0
c1g5 0.08% 0 0 0
c1f4 0.08% 0 0 0
f2f4 0.04% 0 0 0
Guenther wrote: ↑Mon Jun 10, 2019 9:28 pm
[d]r1bqkbnr/pp1ppppp/2n5/2p5/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq
Code: Select all
r1bqkbnr/pp1ppppp/2n5/2p5/2P5/2N5/PP1PPPPP/R1BQKBNR w KQkq - c0 "Nf3=100";
again 3. g3 and 3. e4 are as good - still 0 points
New database gives - g3=64, Nf3=30, e3=2, e4=0, d3=0, f4=0, a3=0, b3=0, h3=0, Rb1=0
Code: Select all
Opening Book : books\mb-2017.bin
Positions : 812.054
Book Weight Score Depth Learn
g2g3 64.60% 0 0 0
g1f3 30.99% 0 0 0
e2e3 2.39% 0 0 0
e2e4 0.88% 0 0 0
d2d3 0.53% 0 0 0
f2f4 0.28% 0 0 0
a2a3 0.21% 0 0 0
b2b3 0.05% 0 0 0
h2h3 0.03% 0 0 0
a1b1 0.01% 0 0 0
Well, an opening test requires lots of manual work. Just adding PG weights, which say nothing about quality, but frequencey will not help.
No billions of games will change this, as long as you don't rework the scoring system manually...
As long as the game is in such an early stage (2-4 moves) there are much too much equal moves which should be rewarded similar
and not fundamentally different.
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 11:14 am
by Rebel
Guenther wrote: ↑Tue Jun 11, 2019 10:03 amWell, an opening test requires lots of manual work. Just adding PG weights, which say nothing about quality, but frequencey will not help. No billions of games will change this, as long as you don't rework the scoring system manually...
As long as the game is in such an early stage (2-4 moves) there are much too much equal moves which should be rewarded similarand not fundamentally different.
Regarding manually -
#1. You never get it perfect due to the volume. The new 2-moves.epd increased in size from 404 to 6728 positions.
#2. Computer analysis doesn't help unless you search to very deep depths (© Dann) and once again volume is a problem.
Manually is not an option, doing nothing also not. The system is not perfect (never will) yet (like in eng-eng matches and SIM03) volume is the key to weed out randomness, in this case volume is the key to weed out incompleteness.
Too early to give up already.
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 6:21 pm
by Dann Corbit
I think you could create a very good statistics based book by using the lichees data.
It's the crowd wisdom metaphor.
True, you can't get a perfect book jut via statistics.
But you can't get a perfect book any other way either.
If you can build a perfect book then you can solve chess.
We're not there yet.
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 6:35 pm
by JohnWoe
Sapeli has opening knowledge as it has a pawn formation array. I want Sapeli to have a pawn in the middle always. I think in chess960 that strategy works too.
Plus I want Sapeli to develop minor pieces ASAP.
That's what strong chess players do anyway.
60 second search from standard position. Only e4/d4
Code: Select all
./sapeli -search 60000
info depth 0 nodes 40 time 0 nps 0 score cp 35 pv e2e4
info depth 1 nodes 301 time 1 nps 301000 score cp 0 pv d2d4
info depth 2 nodes 2822 time 5 nps 564400 score cp 21 pv d2d4
info depth 3 nodes 5319 time 9 nps 591000 score cp 9 pv e2e4
info depth 4 nodes 26592 time 45 nps 590933 score cp 10 pv d2d4
info depth 5 nodes 88664 time 114 nps 777754 score cp 5 pv e2e4
info depth 6 nodes 243127 time 259 nps 938714 score cp 13 pv d2d4
info depth 7 nodes 695450 time 527 nps 1319639 score cp 6 pv e2e4
info depth 8 nodes 1683896 time 961 nps 1752233 score cp 9 pv e2e4
info depth 9 nodes 4225397 time 2159 nps 1957108 score cp 12 pv e2e4
info depth 10 nodes 13683801 time 6190 nps 2210630 score cp 14 pv e2e4
info depth 11 nodes 47208226 time 20830 nps 2266357 score cp 11 pv e2e4
info depth 12 nodes 133528893 time 60001 nps 2225444 score cp 11 pv e2e4
chess960 opening analysis:
1. Nde3 // Ok move
Code: Select all
exclude: none best +tail
dep score nodes time (not shown: tbhits knps seldep)
11 +0,02 154,0M 1:13.12 Nde3
10 +0,08 24,2M 0:11.93 Nde3
9 +0,09 8,96M 0:04.73 Nde3
8 +0,15 4,42M 0:02.50 Nde3
7 +0,11 1,49M 0:01.04 Nde3
6 +0,19 707486 0:00.62 Nfe3
5 +0,01 577098 0:00.55 e4
4 +0,31 35087 0:00.07 Nfe3
3 +0,14 5403 0:00.01 Nfe3
2 +0,35 2134 0:00.01 Nfe3
1 0.00 305 0:00.00 b3
0 #
[d]bbrnknrq/pppppppp/8/8/8/8/PPPPPPPP/BBRNKNRQ w GCgc - 0 1
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 7:17 pm
by Rebel
Dann Corbit wrote: ↑Tue Jun 11, 2019 9:26 am
Every evaluation is worthwhile.
Yep.
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 7:29 pm
by Guenther
Dann Corbit wrote: ↑Tue Jun 11, 2019 6:21 pm
I think you could create a very good statistics based book by using the lichees data.
It's the crowd wisdom metaphor.
True, you can't get a perfect book jut via statistics.
But you can't get a perfect book any other way either.
If you can build a perfect book then you can solve chess.
We're not there yet.
I don't know, whom you want to answer Dann (I guess Ed), but we are not anymore in a thread about building
an opening book, but about constructing an opening test for engines. (OKE)
With the current methodology it fails miserably. I repeat it again, the idea is good, but
1. you need to start later in the opening
2. you have to carefully check the scoring system, which cannot be done automatically
3. you need to decide whom to trust (SF LC0 Komodo Houdini, all can fail here too, moreover
it would introduce bias trusting one of them, it seems inevitable to consult opening encyclopedies
based on decades of human knowledge, crosschecked by all of them and then add scoring intelligently,
not black/white or 99:1, if the average result will be the same)
Re: OKE - Opening Knowledge Engines
Posted: Tue Jun 11, 2019 7:56 pm
by Rebel
Dann Corbit wrote: ↑Tue Jun 11, 2019 6:21 pm
I think you could create a very good statistics based book by using the lichees data.
It's the crowd wisdom metaphor.
True, you can't get a perfect book jut via statistics.
But you can't get a perfect book any other way either.
If you can build a perfect book then you can solve chess.
We're not there yet.
I think the massive Lichess stuff perhaps is good for NN engines. Or to extract games on elo. Else I see no usefulness. But perhaps you might want to elaborate.