Page 2 of 5
Re: OKE - one year after
Posted: Tue Sep 08, 2020 8:31 pm
by Rebel
Terje wrote: ↑Tue Sep 08, 2020 5:33 pm
As expected Weiss is no opening master
Would be interesting to see normal minic on there for comparison.
Done.
the 6-moves.html is fun, the NNUE version third place from the top, the normal version third place from the bottom.
Re: OKE - one year after
Posted: Tue Sep 08, 2020 9:29 pm
by Alayan
What's the reference used to judge opening move quality ? I see "2, 3 and 4-moves.epd are created from 2600+ elo rated human players.", is this related to what moves are considered good/bad ?
The score range of engines is very small compared to the minimum/maximum possible scores.
Re: OKE - one year after
Posted: Tue Sep 08, 2020 9:50 pm
by Rebel
Alayan wrote: ↑Tue Sep 08, 2020 9:29 pm
What's the reference used to judge opening move quality ? I see "2, 3 and 4-moves.epd are created from 2600+ elo rated human players.", is this related to what moves are considered good/bad ?
The assumption is that 2600+ elo rated players seldom blunder the first 6 moves, let alone in the first 2-3 moves.
Alayan wrote: ↑Tue Sep 08, 2020 9:29 pm
The score range of engines is very small compared to the minimum/maximum possible scores.
True.
Re: OKE - one year after
Posted: Tue Sep 08, 2020 10:07 pm
by Alayan
Rebel wrote: ↑Tue Sep 08, 2020 9:50 pm
Alayan wrote: ↑Tue Sep 08, 2020 9:29 pm
What's the reference used to judge opening move quality ? I see "2, 3 and 4-moves.epd are created from 2600+ elo rated human players.", is this related to what moves are considered good/bad ?
The assumption is that 2600+ elo rated players seldom blunder the first 6 moves, let alone in the first 2-3 moves.
They don't blunder, but then neither do engines.
In the end, the challenge is to find moves that lead into positions where the most complications exists for your opponent while conceding the least possible, so as to maximize the expected score.
Human GMs are rather good at doing choices that maximize their expected score, but there is a strong specialization bias - try to go into lines you know very well and avoid lines your opponent knows very well.
The expected score of a line might very well be different for 3000+ engines than for 2600+ humans. The expected score of a line is also a function of the rating difference between the two playing entities - the optimal line vs a stronger opponent is drawish, while it is more forceful and imbalanced vs a weaker opponent. In this regard, there is often no absolute best move in balanced drawn positions.
Openings that are objectively worse might also get selected by a human GM because they offer a higher chance to get a win that's needed to get a good tournament placement.
In short, I'm skeptical about the methodology.
Re: OKE - one year after
Posted: Wed Sep 09, 2020 12:46 am
by Dann Corbit
Alayan wrote: ↑Tue Sep 08, 2020 10:07 pm
In short, I'm skeptical about the methodology.
It is a good question, and perhaps a mix would be better.
I guess that the bottom line is, "The proof of the pudding is in the eating."
Which is to say, when we try it does it work or not?
Re: OKE - one year after
Posted: Wed Sep 09, 2020 12:49 am
by Rebel
Alayan wrote: ↑Tue Sep 08, 2020 10:07 pm
Rebel wrote: ↑Tue Sep 08, 2020 9:50 pm
Alayan wrote: ↑Tue Sep 08, 2020 9:29 pm
What's the reference used to judge opening move quality ? I see "2, 3 and 4-moves.epd are created from 2600+ elo rated human players.", is this related to what moves are considered good/bad ?
The assumption is that 2600+ elo rated players seldom blunder the first 6 moves, let alone in the first 2-3 moves.
They don't blunder, but then neither do engines.
They do, they don't give away a piece, but they don't understand the strategy or concept behind (certain) openings and don't play the right moves, humans on that level do.
Alayan wrote: ↑Tue Sep 08, 2020 10:07 pm
In the end, the challenge is to find moves that lead into positions where the most complications exists for your opponent while conceding the least possible, so as to maximize the expected score.
Human GMs are rather good at doing choices that maximize their expected score, but there is a strong specialization bias - try to go into lines you know very well and avoid lines your opponent knows very well.
The expected score of a line might very well be different for 3000+ engines than for 2600+ humans. The expected score of a line is also a function of the rating difference between the two playing entities - the optimal line vs a stronger opponent is drawish, while it is more forceful and imbalanced vs a weaker opponent. In this regard, there is often no absolute best move in balanced drawn positions.
Openings that are objectively worse might also get selected by a human GM because they offer a higher chance to get a win that's needed to get a good tournament placement.
In short, I'm skeptical about the methodology.
The only thing I am skeptical about is the lack of volume. Besides, there is no alternative, you can't use comp games to test how well comps play the early opening, one needs human games. Besides (2) - Certain top-engines score badly on 2-3-4 movers, for instance Komodo, take a 50-100 lower rated engine that is higher placed than Komodo and let it play cutechess with 2-3-4 movers, Komodo might lose. Try Ethereal (CCRL 3374) vs Stockfish 10 (CCRL 3491), I bet you Ethereal will do good on 2-3 movers. I did some tests in the past and that is what it showed.
Re: OKE - one year after
Posted: Wed Sep 09, 2020 10:56 am
by Rebel
Re: OKE - one year after
Posted: Wed Sep 09, 2020 12:57 pm
by Jouni
Soon EVERY engine uses Vieri net and start with 3400 ELO..
Re: OKE - one year after
Posted: Wed Sep 09, 2020 2:10 pm
by xr_a_y
Jouni wrote: ↑Wed Sep 09, 2020 12:57 pm
Soon EVERY engine uses Vieri net and start with 3400 ELO..
Well, Minic and Igel were already 3100-3200 engines. So the "every" is maybe a little too optimistic.
But indeed, as shown by the MinicNNUE+sv net versus SF12 similiraty test (=>86% versus 36% for standard Minic eval), evaluation is a very large part of strength here.
Re: OKE - one year after
Posted: Wed Sep 09, 2020 3:02 pm
by voffka
Jouni wrote: ↑Wed Sep 09, 2020 12:57 pm
Soon EVERY engine uses Vieri net and start with 3400 ELO..
Igel 2.7.0 does not use SV nets, so it is weaker even with NNUE. I am curios btw about the simtest for 2.7.0 and for 2.6.0 (between each) as well as simtext between SF12/Igel and Lc0/Igel
