Out of the Kai 450 positions, 99 not found by SF

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
zullil
Posts: 5593
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Out of the Kai 450 positions, 99 not found by SF

Post by zullil » Thu Aug 15, 2019 3:04 pm

Ovyron wrote:
Thu Aug 15, 2019 2:53 pm
But in your world view, it's not chess at all, as at the base, there's two colors, white and black, you have to be one of the colors to play. If you're not one of the colors, you're not playing chess, and if you're both colors, you're not playing chess. So what are you doing?
Preparing for a match against the Almighty. Very long time controls, not 12/2. And no blunders. :P

User avatar
Ovyron
Posts: 2488
Joined: Tue Jul 03, 2007 2:30 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Ovyron » Thu Aug 15, 2019 3:09 pm

zullil wrote:
Thu Aug 15, 2019 3:04 pm
Preparing for a match against the Almighty.
Who is the Almighty, and how well it plays anyway? Is it capable of defeating Stockfish at Depth 10? Because a common theme is that computer chess programmers can't defeat their own inventions once they become strong enough.
Great spirits have always encountered violent opposition from mediocre minds.

jp
Posts: 748
Joined: Mon Apr 23, 2018 5:54 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by jp » Thu Aug 15, 2019 3:10 pm

Ovyron wrote:
Thu Aug 15, 2019 2:40 pm
And yet I just beat this guy yesterday and that gave me the IM norm...
Is this ICCF? How many norms is this and how many do you need?

User avatar
Ovyron
Posts: 2488
Joined: Tue Jul 03, 2007 2:30 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Ovyron » Thu Aug 15, 2019 3:17 pm

jp wrote:
Thu Aug 15, 2019 3:10 pm
Is this ICCF? How many norms is this and how many do you need?
Yes, the ICCF. You don't need any norms (unless you want to play in special events that require them). The norms are CCE (Correspondence Chess Expert), CCM (Correspondence Chess Master), IM (International Master), LIM (Lady International Master), SIM (Senior International Master), GM (International Grandmaster) and LGM (Lady International Grandmaster.)
Great spirits have always encountered violent opposition from mediocre minds.

User avatar
Master Om
Posts: 282
Joined: Wed Nov 24, 2010 9:57 am
Location: INDIA

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Master Om » Thu Aug 15, 2019 3:59 pm

Ovyron wrote:
Thu Aug 15, 2019 2:40 pm
zullil wrote:
Thu Aug 15, 2019 11:43 am
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:
And yet I just beat this guy yesterday and that gave me the IM norm...


1-0

So, in practice, finding the best moves is very workable :)

That's why I don't get why people would analyze chess positions for the sake of it, as best moves depend on what's the opponent's level. I have been able to calibrate, and play the moves that defeat someone, after playing several games with them, just by knowing how they play (...I haven't been able to do that by just looking at their already played games.) If with all your analysis, you're not beating someone, you're wasting your time.
Black Played awful kings Indian IMHO. Started well But couldn't cope up.
Always Expect the Unexpected

zullil
Posts: 5593
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Out of the Kai 450 positions, 99 not found by SF

Post by zullil » Thu Aug 15, 2019 4:34 pm

Ovyron wrote:
Thu Aug 15, 2019 2:40 pm
zullil wrote:
Thu Aug 15, 2019 11:43 am
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:
And yet I just beat this guy yesterday and that gave me the IM norm...
Congratulations. I had you mistaken for someone else. Someone with ICCF ID 340114.

Zenmastur
Posts: 412
Joined: Sat May 31, 2014 6:28 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Zenmastur » Thu Aug 15, 2019 5:56 pm

zullil wrote:
Thu Aug 15, 2019 11:43 am
Ovyron wrote:
Thu Aug 15, 2019 5:39 am
zullil wrote:
Tue Aug 13, 2019 10:39 am
What does "best move" or "right move" even mean here? Surely there are many moves in most of these positions that are equal with correct play---likely all leading to draws.
The "best move" is the one that makes it the hardest for the opponent to find the "correct play", which in some positions may turn out to be extremely hard, or even impossible (I've been on the receiving end.) This depends on the opponent, though at least, something that works for strong opponents works for weaker opponents.
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:
I'm not so sure he is wrong.

I analyzed Lc0 w2ins against SF in the TCEC superfinale and one of the things that I noticed about the games that Lco won (also some of the draws) was that relatively deep Multi-PV analysis of SF's alternatives showed a large number of moves had almost the same score (usually a bad score). In the same games if you looked at Lc0's positions there was almost always one move that was MUCH better than the alternative. So, Lc0 had easy choices to make and SF had very difficult ones. I saw this over and over in the games Lc0 won in the super-finale.

Regards,

Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.

User avatar
Master Om
Posts: 282
Joined: Wed Nov 24, 2010 9:57 am
Location: INDIA

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Master Om » Thu Aug 15, 2019 6:52 pm

Zenmastur wrote:
Thu Aug 15, 2019 5:56 pm
zullil wrote:
Thu Aug 15, 2019 11:43 am
Ovyron wrote:
Thu Aug 15, 2019 5:39 am
zullil wrote:
Tue Aug 13, 2019 10:39 am
What does "best move" or "right move" even mean here? Surely there are many moves in most of these positions that are equal with correct play---likely all leading to draws.
The "best move" is the one that makes it the hardest for the opponent to find the "correct play", which in some positions may turn out to be extremely hard, or even impossible (I've been on the receiving end.) This depends on the opponent, though at least, something that works for strong opponents works for weaker opponents.
"The" best move is the "one" that makes finding "the" correct play most difficult. Sounds awfully unworkable to me. :wink:
I'm not so sure he is wrong.

I analyzed Lc0 w2ins against SF in the TCEC superfinale and one of the things that I noticed about the games that Lco won (also some of the draws) was that relatively deep Multi-PV analysis of SF's alternatives showed a large number of moves had almost the same score (usually a bad score). In the same games if you looked at Lc0's positions there was almost always one move that was MUCH better than the alternative. So, Lc0 had easy choices to make and SF had very difficult ones. I saw this over and over in the games Lc0 won in the super-finale.

Regards,

Zenmastur
The difference is the time frame. SF needs more time than Alpha 0. Thats the only difference betwen AB and NN engines. NN are somewhat faster.
Always Expect the Unexpected

Zenmastur
Posts: 412
Joined: Sat May 31, 2014 6:28 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Zenmastur » Thu Aug 15, 2019 7:06 pm

Laskos wrote:
Tue Aug 13, 2019 6:16 am
Uri Blass wrote:
Tue Aug 13, 2019 3:46 am
I think that if we claim that stockfish does not find the right move then we need some evidence that the move we claim to be better is really the better move(for example some tree that if we go forward and backward in the tree stockfish can learn to play the right move).
... I guess Stockfish of today has very little idea about the outcome (the finality) of a quiet balanced position in openings or midgames. ...
You are absolutely right! Stockfish has trouble making plans in the opening and early middlegame.
However, "IF" you have good analysis technique, you can improve it's performance significantly. And above all else you do need strong verification that the best move is actually best.
I am against picking the solutions based on some engine analysis in these positional test suites. I checked to see when exactly I built Openings200, it was built almost 3 years ago. It could have been easily proven junk by Lc0 when it arrived, and that would have probably happened if I had built the positional suite by engine analysis (that happened to STS suite, in fact). To my surprise, my approach was somehow validated by Lc0, it comes in Openings200 (revised or not) far ahead of regular eval AB engines.
I'm not against using computer analysis provided it meets certain criterion. The primary one being that the positions evaluation must eventually lead to a decided advantage for one side or the other. If you can't “prove” this with an AB type engine (NN engines are too weak tactically to do this I think) then the position should be excluded or put into a separate category.

There are enough instances where this is possible even when the positions appears to be drawn with “normal” analysis techniques to make this a workable solution. It does, however, require extensive analysis of the positions involved.

Laskos wrote:
Tue Aug 13, 2019 9:02 am

While about my positional test-suites I had great doubts myself, about positional superiority of Lc0 on a reasonable GPU with a strong net, I have few doubts. Maybe I don't understand what "positional" means. What is obvious, from games, test-suites, is that Lc0 is clearly weaker tactically compared to not only Stockfish, but even to much weaker modern AB engines with a regular eval. It is again obvious to me that to be the strongest engine on my PC from regular openings, Lc0 compensates by its very strong "positional" play, maybe in my wrong understanding of the notion of "positional", as some sort of conjugate of "tactical". I am curious, do you have some confidence that a strong Lc0 is superior "positionally" to a strong regular eval AB engine?

That's why I was pretty happy that Lc0 comes far atop in my dubious positional suites. And after all, is it common sense to say: databases of human games are wrong, Lc0 is not that strong positionally so a validation by it means nothing, and the only way to check for correctness is to go back and forth from each position with Stockfish for hours on end? My common sense would tell me that it is a wrong methodology for what I understand to be a positional test suite.
Depending solely on Lc0 for validation is risky. It's tactical weakness allows for errors to creep into the analysis. I would think that ideally you would want SF and Lc0 to agree on a move. Lc0 will provide the “strategic” insight while SF will ensure that the plan/move is “tactically” sound. I don't believe that “normal” analysis using an AB engine is sufficient by itself to guarantee tactical soundness. If you want to use these positions as a “test” suite then you need to go to “extraordinary” measures to see that no unforeseen tactical motifs rear their ugly heads. I leave the definition of “extraordinary” up to you, but note that it should be better than just turning the engine on and letting it run for a while on a given position. At the very least, after a “normal” analysis is done a multi-PV=2 (or more) should be done to the same depth to verify that there is truly only one “best” move in the position. Having multiple “best” moves dilutes the value of the position even if you know all of them. (e.g. a position that has 36 moves and 12 of them are “best” is all most worthless)
zullil wrote:
Tue Aug 13, 2019 10:39 am

What does "best move" or "right move" even mean here? Surely there are many moves in most of these positions that are equal with correct play---likely all leading to draws.
A VERY good fundamental question. In most “truly” even middle-game positions there can be many “equal” moves. So, there IS NO “BEST” move. I would think that a position that “looks” even under normal analysis but is “in fact” won or at least highly favorable to one side is what is needed.
And forgive me, but in my opinion relying on statistics from completed games, especially human games, is like relying on noise created by cosmic radiation. My guess is that 95% of games that contain one of these positions are decided by significant blunders made subsequent to the positions.


True enough!

Laskos wrote:
Tue Aug 13, 2019 12:32 pm

… Are there many bottlenecks with one single best move? What is the usual dominant in regular games, many equivalent best moves? Maybe the oddities occurring in databases do mean something close to bottlenecks? We do have 6 and even 7 men tablebases, and although I haven't sit to analyze them, there are both of these cases, and interesting enough, an engine like Stockfish by itself (no TBs), with much of hardcoded endgame knowledge, often finds solutions (bm) even in these bottlenecks, often showing itself as a good estimator.
I spent a week or so looking at SF analysis of very long EGTB mates. The “raw” number of moves for the losing side seems to be epicyclic with both amplitude (# of moves) and length of cycle length (# of moves between the min and max # of moves) varying in a bounded semi-random fashion.

Uri Blass wrote:
Wed Aug 14, 2019 2:51 am

I think that testing the level of engines even in year 2000 should not be done at short time control because people could give the computer hours for analysis at home in order to find novelties.

I wonder if there is a top engine that suggested 2.Nc3 after 1.Nf3 d5 when you gave it one hour to search and note that using engines to find novelties does not always mean choosing the top moves of chess engines.

Note that I used chess engines in correspondence games from the opening stage and did not trust opening theory when I played correspondence chess(I do not play correspondence chess today).
Yes, opening theory is terrible compared to deep analysis with AB engines. GM games are all most worthless in correspondence chess. I used them when I first started but found that there are so many errors they are more of a hindrance than anything else. Even using ICCF games as guides is dangerous. Many contain errors and can't be trusted unless you examine every move in a given line of play in depth.


Regards,

Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.

User avatar
Ovyron
Posts: 2488
Joined: Tue Jul 03, 2007 2:30 am

Re: Out of the Kai 450 positions, 99 not found by SF

Post by Ovyron » Fri Aug 16, 2019 10:45 am

Zenmastur wrote:
Thu Aug 15, 2019 7:06 pm
A VERY good fundamental question. In most “truly” even middle-game positions there can be many “equal” moves. So, there IS NO “BEST” move. I would think that a position that “looks” even under normal analysis but is “in fact” won or at least highly favorable to one side is what is needed.
For any chess position there's ALWAYS a move that is better than another, by some metric. I already suggested some metric (say, with one move it's easier to beat Stockfish Depth 10 than with another; as the level increases, the metric changes, and the best move changes, but it's still clearly better than another one.)

Even for positions that trivially transpose with best play, there's clearly a better move than another. There's actually different schools of thought on this, mainly, whether you should keep your opponent's choices sharp (they have no better thing that following your best line, any other moves lead to disaster for them), or versatile (your mainline allows the opponent different choices, and if they don't follow your mainline, you gain a bit of an edge.) For years I had concluded "versatile" was better, as when I did that, my opponents would rarely go for the best moves, allowing me to gain small advantages that accumulated until they were overwhelming and I won.

It turns out those were really weak players I could have beaten with anything anyway, once I was with a strong player, they taught me what is best is to leave things versatile for yourself. Basically, you assume that your own analysis is wrong, that beyond the horizon of what you can analyze, your opponent has a surprise that refutes your mainline, and it's a refutation that you can't see. So you need a best second line in case this happens, that is decent, and then, if your analysis is actually wrong and your opponent is actually superior, you have a decent plan B to fall in, and if you didn't and your mainline fell, you'd be in big trouble.

Rarely do different moves in positions allow you the same number of backup plans in case your opponent refutes your mainline, so it's clear the one that would leave you best even if most mainlines were refuted is a better move than the rest. And for all positions this is true, so this is the BEST move in them.
Great spirits have always encountered violent opposition from mediocre minds.

Post Reply