STS rating v13.1 for Lc0 0.21.2 with nodes = 1

[Laskos]

I just want to say that I think the distinction between "tactical" and "positional" problems is rather arbitrary and not so useful, because in real chess games good positional moves are found by tactical details. For example, let's say that rook on the 7th rank is usually good (with whatever conditions you want to specify). One engine may find some odd-looking move that after a deep search results in getting a rook to the 7th rank because preventing it loses material. Is this tactical or positional? I wonder if there is some set of problems taken from high level human games where the right move is difficult but 90% or so agreed upon as best, without distinguishing between tactical and positional problems? That might be a test with some predictive power for elo ratings.

I am not sure I would agree, more so seeing the playing of this weirdo called Leela. We do know what a "tactical" problem or puzzle is, don't we? In fact so much emphasis in this forum is about some tactical puzzles, that to me it became a clear, albeit often a bit obnoxious topic. We can find positions which per se don't pose any tactical complications, and aren't they, brushing aside elementary tactics, the majority of in-game chess positions? I do not know a strong human's perspective on that, maybe there are few "quiet" moves for humans, and even a strong human is wary of some hidden tactics move upon move. But this concept that "tactical" and "positional" are hard to separate came to me with the top regular AB engines, where I can clearly see that what I call "positional" strength is due to deeper search and deeper tactics. In case of regular engines, "positional" strength came mostly as a side effect of deeper, tactically accurate search. But Leela doesn't play this game. Leela can easily miss a three-mover shot, but in real games that is not what usually happens.

If we know what "tactics" is, then we know that WAC suite is a very tactical test-suite. I trimmed it from 300 to 145 positions which have a unique, game-changing solution. Komodo solves all of them in under 5 seconds/position, and the vast majority of them at depths 1-12 (125/145 solved), literally in 1-30 milliseconds.

Code: Select all

Engine: Komodo 13.02 64-bit (192 MB)
by Don Dailey, Larry Kaufman, Mark Lefler

1      sec    ->       142/145 
2      sec    ->       143/145 
3      sec    ->       143/145 
4      sec    ->       144/145 
5      sec    ->       145/145 

  n/s: 7.202.528  
  TotTime: 2:33m    SolTime: 11s
  Ply: 0   Positions:145   Avg Nodes:       0   Branching = 0.00
  Ply: 1   Positions:113   Avg Nodes:    1956   Branching = 0.00
  Ply: 2   Positions: 97   Avg Nodes:    4834   Branching = 2.47
  Ply: 3   Positions: 82   Avg Nodes:    6860   Branching = 1.42
  Ply: 4   Positions: 74   Avg Nodes:    9254   Branching = 1.35
  Ply: 5   Positions: 66   Avg Nodes:   14131   Branching = 1.53
  Ply: 6   Positions: 57   Avg Nodes:   19534   Branching = 1.38
  Ply: 7   Positions: 50   Avg Nodes:   26732   Branching = 1.37
  Ply: 8   Positions: 43   Avg Nodes:   36925   Branching = 1.38
  Ply: 9   Positions: 39   Avg Nodes:   50848   Branching = 1.38
  Ply:10   Positions: 32   Avg Nodes:   91416   Branching = 1.80
  Ply:11   Positions: 26   Avg Nodes:  134044   Branching = 1.47
  Ply:12   Positions: 20   Avg Nodes:  235666   Branching = 1.76
  Ply:13   Positions: 16   Avg Nodes:  372846   Branching = 1.58
  Ply:14   Positions: 12   Avg Nodes:  529047   Branching = 1.42
  Ply:15   Positions: 10   Avg Nodes:  794787   Branching = 1.50
  Ply:16   Positions:  7   Avg Nodes: 1289637   Branching = 1.62
  Ply:17   Positions:  6   Avg Nodes: 1792729   Branching = 1.39
  Ply:18   Positions:  5   Avg Nodes: 2959558   Branching = 1.65
  Ply:19   Positions:  4   Avg Nodes: 4611414   Branching = 1.56
  Ply:20   Positions:  4   Avg Nodes: 5795014   Branching = 1.26
  Ply:21   Positions:  3   Avg Nodes: 9113716   Branching = 1.57
  Ply:22   Positions:  2   Avg Nodes: 4529585   Branching = 0.50
  Ply:23   Positions:  1   Avg Nodes:16790212   Branching = 3.71

Here is the number of new solutions by depth of Komodo:


WAC_depth.jpg


Leela (42620) has a big, irrecuperable trouble with this very easy for Komodo tactical suite:

1s/position
score=98/145 [averages on correct positions: depth=4.1 time=0.07 nodes=498]

10s/poisition
score=105/145 [averages on correct positions: depth=4.4 time=0.14 nodes=1620]


In 10 seconds per position (on strong GPU), Leela fares worse than Komodo in milliseconds to depth 9. Leela misses tactical 2-3-4 mover shots quite easily. Can we say that "Leela is weak tactically"? And all in all, beats the crap out of Komodo on my PC due to "something else"? If I call this "something else" as roughly the "positional play", I am outside the usual terminology? Yes, as a patzer human player, I can hardly grasp what exactly "positional play" means, so I use engines, and for example these extreme positional/tactical test suites. If 2 years ago with usual engines, the separation tactical/positional was unclear indeed, with Leela this separation is quite extreme. So, I am unprepared now to blur the separation tactical/positional (was more prepared 2 years ago).

Tactical-positional is a falser than false dichotomy. They are both words associated with weak human players learning chess. Firstly, weak player knows about material 95331. Then he blunders around, leaves pieces en prise and learns to guard his material. Then he learns there are things called tactics, maybe a knight fork or something. From then on he plays tactics, always looking for a trick. Some few then learn there’s a bit more, called positional, like double pawn. These few then play tactics to try and get some positional advantage. That’s where it stops, also with most chess programmers. Everything is a combination of positional things and some lookahead tactics. The belief being that chess is entirely tactics with a bit of position knowledge. Eg, chess is won by tactics.
Fast forward to 2018. Whoops. Everything everybody knew was wrong. But they are not really very sure why, so they carry on babbling using language that doesn’t actually fit, tactics this, positional that, bla bla. Like Sisyphus, they want to progress up the hill, but they have this giant tactics-positional dichotomy stone that keeps rolling backwards. Worst, probably, are these 75 move deep SF lines people, who want to prove everything, but can’t.
Throw away the words, they were useful in learning, but are a handicap to understanding deeper. There is no tactics. Everything is positional. Except for beginners.

I am not sure what to make of this. Take an engine as following: only material evaluation (no PST, no bonuses or penalties), pure AB search. Even I can write such an engine. If not buggy, it should go to 8-9 depth in blitz on an i5 core, and depth of 10-12 in LTC. It will beat you in both blitz and LTC. Should I say that it beats you because of your positional flaws? Or you have an excuse being a beginner and losing to tactics?

[Laskos]

snip...

If we know what "tactics" is, then we know that WAC suite is a very tactical test-suite. I trimmed it from 300 to 145 positions which have a unique, game-changing solution. Komodo solves all of them in under 5 seconds/position, and the vast majority of them at depths 1-12 (125/145 solved), literally in 1-30 milliseconds.

Can we have your trimmed 145 positions ?
That would be nice.

Sure, but maybe you will re-check them, I checked them only twice. Attached.

WAC145.zip

[Laskos]

I just want to say that I think the distinction between "tactical" and "positional" problems is rather arbitrary and not so useful, because in real chess games good positional moves are found by tactical details. For example, let's say that rook on the 7th rank is usually good (with whatever conditions you want to specify). One engine may find some odd-looking move that after a deep search results in getting a rook to the 7th rank because preventing it loses material. Is this tactical or positional? I wonder if there is some set of problems taken from high level human games where the right move is difficult but 90% or so agreed upon as best, without distinguishing between tactical and positional problems? That might be a test with some predictive power for elo ratings.

I am not sure I would agree, more so seeing the playing of this weirdo called Leela. We do know what a "tactical" problem or puzzle is, don't we? In fact so much emphasis in this forum is about some tactical puzzles, that to me it became a clear, albeit often a bit obnoxious topic. We can find positions which per se don't pose any tactical complications, and aren't they, brushing aside elementary tactics, the majority of in-game chess positions? I do not know a strong human's perspective on that, maybe there are few "quiet" moves for humans, and even a strong human is wary of some hidden tactics move upon move. But this concept that "tactical" and "positional" are hard to separate came to me with the top regular AB engines, where I can clearly see that what I call "positional" strength is due to deeper search and deeper tactics. In case of regular engines, "positional" strength came mostly as a side effect of deeper, tactically accurate search. But Leela doesn't play this game. Leela can easily miss a three-mover shot, but in real games that is not what usually happens.

If we know what "tactics" is, then we know that WAC suite is a very tactical test-suite. I trimmed it from 300 to 145 positions which have a unique, game-changing solution. Komodo solves all of them in under 5 seconds/position, and the vast majority of them at depths 1-12 (125/145 solved), literally in 1-30 milliseconds.

Code: Select all

Engine: Komodo 13.02 64-bit (192 MB)
by Don Dailey, Larry Kaufman, Mark Lefler

1      sec    ->       142/145 
2      sec    ->       143/145 
3      sec    ->       143/145 
4      sec    ->       144/145 
5      sec    ->       145/145 

  n/s: 7.202.528  
  TotTime: 2:33m    SolTime: 11s
  Ply: 0   Positions:145   Avg Nodes:       0   Branching = 0.00
  Ply: 1   Positions:113   Avg Nodes:    1956   Branching = 0.00
  Ply: 2   Positions: 97   Avg Nodes:    4834   Branching = 2.47
  Ply: 3   Positions: 82   Avg Nodes:    6860   Branching = 1.42
  Ply: 4   Positions: 74   Avg Nodes:    9254   Branching = 1.35
  Ply: 5   Positions: 66   Avg Nodes:   14131   Branching = 1.53
  Ply: 6   Positions: 57   Avg Nodes:   19534   Branching = 1.38
  Ply: 7   Positions: 50   Avg Nodes:   26732   Branching = 1.37
  Ply: 8   Positions: 43   Avg Nodes:   36925   Branching = 1.38
  Ply: 9   Positions: 39   Avg Nodes:   50848   Branching = 1.38
  Ply:10   Positions: 32   Avg Nodes:   91416   Branching = 1.80
  Ply:11   Positions: 26   Avg Nodes:  134044   Branching = 1.47
  Ply:12   Positions: 20   Avg Nodes:  235666   Branching = 1.76
  Ply:13   Positions: 16   Avg Nodes:  372846   Branching = 1.58
  Ply:14   Positions: 12   Avg Nodes:  529047   Branching = 1.42
  Ply:15   Positions: 10   Avg Nodes:  794787   Branching = 1.50
  Ply:16   Positions:  7   Avg Nodes: 1289637   Branching = 1.62
  Ply:17   Positions:  6   Avg Nodes: 1792729   Branching = 1.39
  Ply:18   Positions:  5   Avg Nodes: 2959558   Branching = 1.65
  Ply:19   Positions:  4   Avg Nodes: 4611414   Branching = 1.56
  Ply:20   Positions:  4   Avg Nodes: 5795014   Branching = 1.26
  Ply:21   Positions:  3   Avg Nodes: 9113716   Branching = 1.57
  Ply:22   Positions:  2   Avg Nodes: 4529585   Branching = 0.50
  Ply:23   Positions:  1   Avg Nodes:16790212   Branching = 3.71

Here is the number of new solutions by depth of Komodo:


WAC_depth.jpg


Leela (42620) has a big, irrecuperable trouble with this very easy for Komodo tactical suite:

1s/position
score=98/145 [averages on correct positions: depth=4.1 time=0.07 nodes=498]

10s/poisition
score=105/145 [averages on correct positions: depth=4.4 time=0.14 nodes=1620]


In 10 seconds per position (on strong GPU), Leela fares worse than Komodo in milliseconds to depth 9. Leela misses tactical 2-3-4 mover shots quite easily. Can we say that "Leela is weak tactically"? And all in all, beats the crap out of Komodo on my PC due to "something else"? If I call this "something else" as roughly the "positional play", I am outside the usual terminology? Yes, as a patzer human player, I can hardly grasp what exactly "positional play" means, so I use engines, and for example these extreme positional/tactical test suites. If 2 years ago with usual engines, the separation tactical/positional was unclear indeed, with Leela this separation is quite extreme. So, I am unprepared now to blur the separation tactical/positional (was more prepared 2 years ago).

Tactical-positional is a falser than false dichotomy. They are both words associated with weak human players learning chess. Firstly, weak player knows about material 95331. Then he blunders around, leaves pieces en prise and learns to guard his material. Then he learns there are things called tactics, maybe a knight fork or something. From then on he plays tactics, always looking for a trick. Some few then learn there’s a bit more, called positional, like double pawn. These few then play tactics to try and get some positional advantage. That’s where it stops, also with most chess programmers. Everything is a combination of positional things and some lookahead tactics. The belief being that chess is entirely tactics with a bit of position knowledge. Eg, chess is won by tactics.
Fast forward to 2018. Whoops. Everything everybody knew was wrong. But they are not really very sure why, so they carry on babbling using language that doesn’t actually fit, tactics this, positional that, bla bla. Like Sisyphus, they want to progress up the hill, but they have this giant tactics-positional dichotomy stone that keeps rolling backwards. Worst, probably, are these 75 move deep SF lines people, who want to prove everything, but can’t.
Throw away the words, they were useful in learning, but are a handicap to understanding deeper. There is no tactics. Everything is positional. Except for beginners.

I am not sure what to make of this. Take an engine as following: only material evaluation (no PST, no bonuses or penalties), pure AB search. Even I can write such an engine. If not buggy, it should go to 8-9 depth in blitz on an i5 core, and depth of 10-12 in LTC. It will beat you in both blitz and LTC. Should I say that it beats you because of your positional flaws? Or you have an excuse being a beginner and losing to tactics?

Even worse: take SF search, but modify the eval to elementary bare material (no anything else). I bet on 4 i5 cores, it will beat Carlsen. It will also often show some "deep positional understanding of chess" or something nice on these lines.

[chrisw]

I just want to say that I think the distinction between "tactical" and "positional" problems is rather arbitrary and not so useful, because in real chess games good positional moves are found by tactical details. For example, let's say that rook on the 7th rank is usually good (with whatever conditions you want to specify). One engine may find some odd-looking move that after a deep search results in getting a rook to the 7th rank because preventing it loses material. Is this tactical or positional? I wonder if there is some set of problems taken from high level human games where the right move is difficult but 90% or so agreed upon as best, without distinguishing between tactical and positional problems? That might be a test with some predictive power for elo ratings.

I am not sure I would agree, more so seeing the playing of this weirdo called Leela. We do know what a "tactical" problem or puzzle is, don't we? In fact so much emphasis in this forum is about some tactical puzzles, that to me it became a clear, albeit often a bit obnoxious topic. We can find positions which per se don't pose any tactical complications, and aren't they, brushing aside elementary tactics, the majority of in-game chess positions? I do not know a strong human's perspective on that, maybe there are few "quiet" moves for humans, and even a strong human is wary of some hidden tactics move upon move. But this concept that "tactical" and "positional" are hard to separate came to me with the top regular AB engines, where I can clearly see that what I call "positional" strength is due to deeper search and deeper tactics. In case of regular engines, "positional" strength came mostly as a side effect of deeper, tactically accurate search. But Leela doesn't play this game. Leela can easily miss a three-mover shot, but in real games that is not what usually happens.

If we know what "tactics" is, then we know that WAC suite is a very tactical test-suite. I trimmed it from 300 to 145 positions which have a unique, game-changing solution. Komodo solves all of them in under 5 seconds/position, and the vast majority of them at depths 1-12 (125/145 solved), literally in 1-30 milliseconds.

Code: Select all

Engine: Komodo 13.02 64-bit (192 MB)
by Don Dailey, Larry Kaufman, Mark Lefler

1      sec    ->       142/145 
2      sec    ->       143/145 
3      sec    ->       143/145 
4      sec    ->       144/145 
5      sec    ->       145/145 

  n/s: 7.202.528  
  TotTime: 2:33m    SolTime: 11s
  Ply: 0   Positions:145   Avg Nodes:       0   Branching = 0.00
  Ply: 1   Positions:113   Avg Nodes:    1956   Branching = 0.00
  Ply: 2   Positions: 97   Avg Nodes:    4834   Branching = 2.47
  Ply: 3   Positions: 82   Avg Nodes:    6860   Branching = 1.42
  Ply: 4   Positions: 74   Avg Nodes:    9254   Branching = 1.35
  Ply: 5   Positions: 66   Avg Nodes:   14131   Branching = 1.53
  Ply: 6   Positions: 57   Avg Nodes:   19534   Branching = 1.38
  Ply: 7   Positions: 50   Avg Nodes:   26732   Branching = 1.37
  Ply: 8   Positions: 43   Avg Nodes:   36925   Branching = 1.38
  Ply: 9   Positions: 39   Avg Nodes:   50848   Branching = 1.38
  Ply:10   Positions: 32   Avg Nodes:   91416   Branching = 1.80
  Ply:11   Positions: 26   Avg Nodes:  134044   Branching = 1.47
  Ply:12   Positions: 20   Avg Nodes:  235666   Branching = 1.76
  Ply:13   Positions: 16   Avg Nodes:  372846   Branching = 1.58
  Ply:14   Positions: 12   Avg Nodes:  529047   Branching = 1.42
  Ply:15   Positions: 10   Avg Nodes:  794787   Branching = 1.50
  Ply:16   Positions:  7   Avg Nodes: 1289637   Branching = 1.62
  Ply:17   Positions:  6   Avg Nodes: 1792729   Branching = 1.39
  Ply:18   Positions:  5   Avg Nodes: 2959558   Branching = 1.65
  Ply:19   Positions:  4   Avg Nodes: 4611414   Branching = 1.56
  Ply:20   Positions:  4   Avg Nodes: 5795014   Branching = 1.26
  Ply:21   Positions:  3   Avg Nodes: 9113716   Branching = 1.57
  Ply:22   Positions:  2   Avg Nodes: 4529585   Branching = 0.50
  Ply:23   Positions:  1   Avg Nodes:16790212   Branching = 3.71

Here is the number of new solutions by depth of Komodo:


WAC_depth.jpg


Leela (42620) has a big, irrecuperable trouble with this very easy for Komodo tactical suite:

1s/position
score=98/145 [averages on correct positions: depth=4.1 time=0.07 nodes=498]

10s/poisition
score=105/145 [averages on correct positions: depth=4.4 time=0.14 nodes=1620]


In 10 seconds per position (on strong GPU), Leela fares worse than Komodo in milliseconds to depth 9. Leela misses tactical 2-3-4 mover shots quite easily. Can we say that "Leela is weak tactically"? And all in all, beats the crap out of Komodo on my PC due to "something else"? If I call this "something else" as roughly the "positional play", I am outside the usual terminology? Yes, as a patzer human player, I can hardly grasp what exactly "positional play" means, so I use engines, and for example these extreme positional/tactical test suites. If 2 years ago with usual engines, the separation tactical/positional was unclear indeed, with Leela this separation is quite extreme. So, I am unprepared now to blur the separation tactical/positional (was more prepared 2 years ago).

Tactical-positional is a falser than false dichotomy. They are both words associated with weak human players learning chess. Firstly, weak player knows about material 95331. Then he blunders around, leaves pieces en prise and learns to guard his material. Then he learns there are things called tactics, maybe a knight fork or something. From then on he plays tactics, always looking for a trick. Some few then learn there’s a bit more, called positional, like double pawn. These few then play tactics to try and get some positional advantage. That’s where it stops, also with most chess programmers. Everything is a combination of positional things and some lookahead tactics. The belief being that chess is entirely tactics with a bit of position knowledge. Eg, chess is won by tactics.
Fast forward to 2018. Whoops. Everything everybody knew was wrong. But they are not really very sure why, so they carry on babbling using language that doesn’t actually fit, tactics this, positional that, bla bla. Like Sisyphus, they want to progress up the hill, but they have this giant tactics-positional dichotomy stone that keeps rolling backwards. Worst, probably, are these 75 move deep SF lines people, who want to prove everything, but can’t.
Throw away the words, they were useful in learning, but are a handicap to understanding deeper. There is no tactics. Everything is positional. Except for beginners.

I am not sure what to make of this. Take an engine as following: only material evaluation (no PST, no bonuses or penalties), pure AB search. Even I can write such an engine. If not buggy, it should go to 8-9 depth in blitz on an i5 core, and depth of 10-12 in LTC. It will beat you in both blitz and LTC. Should I say that it beats you because of your positional flaws? Or you have an excuse being a beginner and losing to tactics?

Initial premise false. This material engine would stand no chance at all. But that’s besides the point. It’s not a useful question. At thinkable time controls why one side loses, draws or wins (other than by blunder, eg some stupid oversight) is a very complex question, not reducible to tactical-positional concept. Wins are always because positional. Tactics arise at the end if a game because positional. Technique arises at the end of a game because positional.
Where does technique fit into this beginners tactical-positional worldview, btw?

[chrisw]

I just want to say that I think the distinction between "tactical" and "positional" problems is rather arbitrary and not so useful, because in real chess games good positional moves are found by tactical details. For example, let's say that rook on the 7th rank is usually good (with whatever conditions you want to specify). One engine may find some odd-looking move that after a deep search results in getting a rook to the 7th rank because preventing it loses material. Is this tactical or positional? I wonder if there is some set of problems taken from high level human games where the right move is difficult but 90% or so agreed upon as best, without distinguishing between tactical and positional problems? That might be a test with some predictive power for elo ratings.

I am not sure I would agree, more so seeing the playing of this weirdo called Leela. We do know what a "tactical" problem or puzzle is, don't we? In fact so much emphasis in this forum is about some tactical puzzles, that to me it became a clear, albeit often a bit obnoxious topic. We can find positions which per se don't pose any tactical complications, and aren't they, brushing aside elementary tactics, the majority of in-game chess positions? I do not know a strong human's perspective on that, maybe there are few "quiet" moves for humans, and even a strong human is wary of some hidden tactics move upon move. But this concept that "tactical" and "positional" are hard to separate came to me with the top regular AB engines, where I can clearly see that what I call "positional" strength is due to deeper search and deeper tactics. In case of regular engines, "positional" strength came mostly as a side effect of deeper, tactically accurate search. But Leela doesn't play this game. Leela can easily miss a three-mover shot, but in real games that is not what usually happens.

If we know what "tactics" is, then we know that WAC suite is a very tactical test-suite. I trimmed it from 300 to 145 positions which have a unique, game-changing solution. Komodo solves all of them in under 5 seconds/position, and the vast majority of them at depths 1-12 (125/145 solved), literally in 1-30 milliseconds.

Code: Select all

Engine: Komodo 13.02 64-bit (192 MB)
by Don Dailey, Larry Kaufman, Mark Lefler

1      sec    ->       142/145 
2      sec    ->       143/145 
3      sec    ->       143/145 
4      sec    ->       144/145 
5      sec    ->       145/145 

  n/s: 7.202.528  
  TotTime: 2:33m    SolTime: 11s
  Ply: 0   Positions:145   Avg Nodes:       0   Branching = 0.00
  Ply: 1   Positions:113   Avg Nodes:    1956   Branching = 0.00
  Ply: 2   Positions: 97   Avg Nodes:    4834   Branching = 2.47
  Ply: 3   Positions: 82   Avg Nodes:    6860   Branching = 1.42
  Ply: 4   Positions: 74   Avg Nodes:    9254   Branching = 1.35
  Ply: 5   Positions: 66   Avg Nodes:   14131   Branching = 1.53
  Ply: 6   Positions: 57   Avg Nodes:   19534   Branching = 1.38
  Ply: 7   Positions: 50   Avg Nodes:   26732   Branching = 1.37
  Ply: 8   Positions: 43   Avg Nodes:   36925   Branching = 1.38
  Ply: 9   Positions: 39   Avg Nodes:   50848   Branching = 1.38
  Ply:10   Positions: 32   Avg Nodes:   91416   Branching = 1.80
  Ply:11   Positions: 26   Avg Nodes:  134044   Branching = 1.47
  Ply:12   Positions: 20   Avg Nodes:  235666   Branching = 1.76
  Ply:13   Positions: 16   Avg Nodes:  372846   Branching = 1.58
  Ply:14   Positions: 12   Avg Nodes:  529047   Branching = 1.42
  Ply:15   Positions: 10   Avg Nodes:  794787   Branching = 1.50
  Ply:16   Positions:  7   Avg Nodes: 1289637   Branching = 1.62
  Ply:17   Positions:  6   Avg Nodes: 1792729   Branching = 1.39
  Ply:18   Positions:  5   Avg Nodes: 2959558   Branching = 1.65
  Ply:19   Positions:  4   Avg Nodes: 4611414   Branching = 1.56
  Ply:20   Positions:  4   Avg Nodes: 5795014   Branching = 1.26
  Ply:21   Positions:  3   Avg Nodes: 9113716   Branching = 1.57
  Ply:22   Positions:  2   Avg Nodes: 4529585   Branching = 0.50
  Ply:23   Positions:  1   Avg Nodes:16790212   Branching = 3.71

Here is the number of new solutions by depth of Komodo:


WAC_depth.jpg


Leela (42620) has a big, irrecuperable trouble with this very easy for Komodo tactical suite:

1s/position
score=98/145 [averages on correct positions: depth=4.1 time=0.07 nodes=498]

10s/poisition
score=105/145 [averages on correct positions: depth=4.4 time=0.14 nodes=1620]


In 10 seconds per position (on strong GPU), Leela fares worse than Komodo in milliseconds to depth 9. Leela misses tactical 2-3-4 mover shots quite easily. Can we say that "Leela is weak tactically"? And all in all, beats the crap out of Komodo on my PC due to "something else"? If I call this "something else" as roughly the "positional play", I am outside the usual terminology? Yes, as a patzer human player, I can hardly grasp what exactly "positional play" means, so I use engines, and for example these extreme positional/tactical test suites. If 2 years ago with usual engines, the separation tactical/positional was unclear indeed, with Leela this separation is quite extreme. So, I am unprepared now to blur the separation tactical/positional (was more prepared 2 years ago).

Tactical-positional is a falser than false dichotomy. They are both words associated with weak human players learning chess. Firstly, weak player knows about material 95331. Then he blunders around, leaves pieces en prise and learns to guard his material. Then he learns there are things called tactics, maybe a knight fork or something. From then on he plays tactics, always looking for a trick. Some few then learn there’s a bit more, called positional, like double pawn. These few then play tactics to try and get some positional advantage. That’s where it stops, also with most chess programmers. Everything is a combination of positional things and some lookahead tactics. The belief being that chess is entirely tactics with a bit of position knowledge. Eg, chess is won by tactics.
Fast forward to 2018. Whoops. Everything everybody knew was wrong. But they are not really very sure why, so they carry on babbling using language that doesn’t actually fit, tactics this, positional that, bla bla. Like Sisyphus, they want to progress up the hill, but they have this giant tactics-positional dichotomy stone that keeps rolling backwards. Worst, probably, are these 75 move deep SF lines people, who want to prove everything, but can’t.
Throw away the words, they were useful in learning, but are a handicap to understanding deeper. There is no tactics. Everything is positional. Except for beginners.

I am not sure what to make of this. Take an engine as following: only material evaluation (no PST, no bonuses or penalties), pure AB search. Even I can write such an engine. If not buggy, it should go to 8-9 depth in blitz on an i5 core, and depth of 10-12 in LTC. It will beat you in both blitz and LTC. Should I say that it beats you because of your positional flaws? Or you have an excuse being a beginner and losing to tactics?

Even worse: take SF search, but modify the eval to elementary bare material (no anything else). I bet on 4 i5 cores, it will beat Carlsen. It will also often show some "deep positional understanding of chess" or something nice on these lines.

Absolute fantasy.

[Rebel]

Even my crap suite is better.

Ah, now I finally yet did get your point.
Yes of course, you're right. Your crap suite is the very best for crap results, but then again, that I never doubted.

By the way, what are you talking about geniuses like Tsvetkov?
I thought we were talking about people like Swaminathan and Corbit, and who were the authors of "your suite" quickly again?
Some special geniuses of their own for sure too.

Well, Kai is right. STS was developed with the help of the top engines of its time, Rybka and friends. When you run STS nowadays Rybka and friends will top the list and not the engines that are 200-300 elo stronger. That should make you think. It's outdated and served its purpose at the time. It's still a good test for starters.

Well, Ed, of course I could imagine better test suites than STS also still, but do you really think, Kai's 200 positions are anything like that?
Did you ever have a look at the positions?

Do you remember the disussions many years ago, when Swaminathan and Corbit came along with the early versions of STS?

At least about this kind of discussion, not so much has changed till then, don't you think so too?

And did you also notice once again, that almost never in such cases, there's a discussion about certain single positions of the suites, just always only about the "results", most of the times even without mentioning better or worse conditions of hardware- time and pool of engines to be compared to each other by different conditions.

There are other discussions about single positions of interest often enough, but it's other people disussing such most of the times, did you notice that too?

I'm outa here again, high time for me for that since quite a while, shouldn't have even started to try one more time to touch one of the holy cows, but now and then, I just can't resist.

There never was an ultimate test-suite. Best thing that can happen to STS is to revisit it again with the current tops and then it will be outdated again in about 4-5 years.

[peter]

There never was an ultimate test-suite. Best thing that can happen to STS is to revisit it again with the current tops and then it will be outdated again in about 4-5 years.

There never will be an ultimate test-suite.
But to me the only reason for a test- suite to get outdated was, if it was solved too easily and too completely by new arising engines. No reason to update because a single one new engine doesn't perform good enough for some fans of the new engine.
As we see once again (and we saw that often enough, didn't we?), one can easily find new positions and form new test- suites of them to fit best for a single one engine, that's the test- suites that get outdated with the engines they fit for only.

Look at 200 positions of STS of your choice and have a look at some of Kai's 200 positions, then you know the difference between a good and a bad suite.

[Rebel]

There never was an ultimate test-suite. Best thing that can happen to STS is to revisit it again with the current tops and then it will be outdated again in about 4-5 years.

There never will be an ultimate test-suite.
But to me the only reason for a test- suite to get outdated was, if it was solved too easily and too completely by new arising engines. No reason to update because a single one new engine doesn't perform good enough for some fans of the new engine.

That's pretty unreasonable to Dann & Swami considering the energy and massive computer time they spend. It was good at the time, now it needs an update.

As we see once again (and we saw that often enough, didn't we?), one can easily find new positions and form new test- suites of them to fit best for a single one engine, that's the test- suites that get outdated with the engines they fit for only.

I don't believe that happened with the creation of STS. In the 90's people cooperated online to create a tactical suite called ECM. Many programmers profited despite its errors.

Look at 200 positions of STS of your choice and have a look at some of Kai's 200 positions, then you know the difference between a good and a bad suite.

I started talking to you because I disagreed about things you said about STS and he was right. I don't think 200 positions can ever be proof of overall superiority. I did Kai's test and 2 program in the range of 2400-2500 elo are in the middle between the tops.

Code: Select all

Engine: Lc0 v0.21.2-rc1         125/200
Engine: Stockfish 10             93/200
Engine: Ethereal 11.25           86/200
Engine: Senpai 1.0               78/200
Engine: Mephisto Gideon          76/200
Engine: Xiphos 0.5               72/200
Engine: Laser 1.6                71/200
Engine: Rebel Century            66/200
Engine: Sting SF 9.6             66/200
Engine: Texel 1.06a45            65/200
Engine: Rodent III               64/200
Engine: Rybka 4.1                61/200

[peter]

But to me the only reason for a test- suite to get outdated was, if it was solved too easily and too completely by new arising engines. No reason to update because a single one new engine doesn't perform good enough for some fans of the new engine.

That's pretty unreasonable to Dann & Swami considering the energy and massive computer time they spend. It was good at the time, now it needs an update.
...
I don't believe that happened with the creation of STS. In the 90's people cooperated online to create a tactical suite called ECM. Many programmers profited despite its errors.

I didn't mean STS being outdated neither bulit for a single engine (neither Rybka nor any other of that time) I meant Kai's one.

As we see once again (and we saw that often enough, didn't we?), one can easily find new positions and form new test- suites of them to fit best for a single one engine, that's the test- suites that get outdated with the engines they fit for only.

I started talking to you because I disagreed about things you said about STS and he was right. I don't think 200 positions can ever be proof of overall superiority. I did Kai's test and 2 program in the range of 2400-2500 elo are in the middle between the tops.

Code: Select all

Engine: Lc0 v0.21.2-rc1         125/200
Engine: Stockfish 10             93/200
Engine: Ethereal 11.25           86/200
Engine: Senpai 1.0               78/200
Engine: Mephisto Gideon          76/200
Engine: Xiphos 0.5               72/200
Engine: Laser 1.6                71/200
Engine: Rebel Century            66/200
Engine: Sting SF 9.6             66/200
Engine: Texel 1.06a45            65/200
Engine: Rodent III               64/200
Engine: Rybka 4.1                61/200

I know the result is better for LC0, but did you have a look at some of the positions?
Try to find provable single best moves without at least almost as well alternatives and show them, if you want to go on discussing this "suite".

It's not a matter of to a certain favorite engine fitting results only, It's a matter of correctness of the positions too, don't you think so?
We all know the "solution for the false reasons", one of these reasons is often enought wrong or not unique "solution" to be "found" according to the definition made by the author of the suite.

In STS, if there are near to each other candidate moves to be found, more than a single move is counted as solved but with different number of points given.

So if your interest in comparing these two suites to each other is big enough, Ed, try to find incorrect positions in STS, and then try to find for sure correct ones in Kai's 200. Regard, I mean correct as for the result to be counted in points, so at Kai's you have to have all positions doubted, in which a single one alternative move comes near enough to the one, you think to be best, but doesn't get any points in result- measurement, because only "solved" or "not solved" is counted.

This "suite" of Kai's is as crappy as he by himself says it to be. Once again, that's what Kai's right about, not more and not less.
As long as there isn't any discussion about single positions here still, I'm outa it again.

[jp]

There is no tactics. Everything is positional. Except for beginners.

For all engines, including NN engines, it's the opposite.

There is no "positional". Everything is brute calculation.