87% of the poll voted that SF will win Tcec Sufi 19.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mwyoung
Posts: 2725
Joined: Wed May 12, 2010 8:00 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by mwyoung » Tue Oct 13, 2020 3:42 pm

Tord wrote:
Tue Oct 13, 2020 3:19 pm
nabildanial wrote:
Tue Oct 13, 2020 2:41 pm
It has been the case since forever, it's not only for these past 2 seasons. I've heard the same exact argument from the losing side, whether it's SF, Komodo, Leela or Houdini fans. Same ol' story.
It is true, though. It's a competition that tries to determine which engine is best at defending nearly lost positions on hardware you can't afford. Evidently, it's a form of entertainment many people enjoy, but it's not chess as it's usually played, and it's doubtful that any interesting conclusions can be made from the match results.
Bingo!
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Tue Oct 13, 2020 4:05 pm

Some seem to want to see in 95+ games out of 100 that Chess is a bloody draw.

mwyoung
Posts: 2725
Joined: Wed May 12, 2010 8:00 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by mwyoung » Tue Oct 13, 2020 4:13 pm

Laskos wrote:
Tue Oct 13, 2020 4:05 pm
Some seem to want to see in 95+ games out of 100 that Chess is a bloody draw.
Yes! True chess engine testing is hard, and like watching paint draw. And correct, when 2 4000+ Elo equally matched players play. The result should be very very close. :lol:
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

Guenther
Posts: 3676
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Guenther » Tue Oct 13, 2020 4:45 pm

Daniel Shawul wrote:
Tue Oct 13, 2020 2:51 pm
jorose wrote:
Tue Oct 13, 2020 2:23 pm
Daniel Shawul wrote:
Tue Oct 13, 2020 1:59 pm
It has been clear for sometime now that the unbalanced book is very unfair to Lc0.
TCEC does it for the sake of entertainment, but drawing any conclusions from it doesn't make any sense.
This SuFi and last one have been all about who can convert better given a position with a +1 advantage.
If every season is going to be like this, I guess there is no point watching TCEC SuFi.
Could you clarify what you mean by this?

I just took a look at the first 10 openings (20 games) and I don't think they are particularly unbalanced. In fact, I play several of those openings myself. The only ones I find somewhat odd are the French with Ng8 and KGA with Nc3.

Most of the lines are theoretically very well established and see a decent amount of play.I'm not sure where this notion comes from that TCEC has unbalanced openings this season in particular.

I also don't know what such fans want to see, would they prefer even less decisive games?
I am surprized you need clarification for this but here it goes.
These are scores of stockfish right out of book for the second half of the match starting from game 52

Code: Select all

52 - +1.31
54 - +1.06 => 10. 1.06 17. 0.69
56 - +1.19
58 - +0.80 => 1/2-1/2
60 - +1.14 => 7. 1.14 139. 0.62
62 - +1.17 => 1/2-1/2
64 - +1.17 => 1/2-1/2
66 - +1.12 => 1/2-1/2
68 - +1.20
70 - +0.80 => 1/2-1/2
72 - -0.93 => 1/2-1/2
74 - +0.70 ?? here I disagree ;-) and would give >1.00
76 - +0.66 ?? 25. 0.24
78 - +1.38
80 - +1.05
82 - +0.83 => 1/2-1/2
SF may well be superior to Lc0 but this unbalanced openings prove nothing.
Even SF without NNUE was able to get +7 over leela last season entirely due to the book.
The score right after book end is no verdict. Usually you need to look also at the eval several plies later to get a real clue
about an opening.
Even at this tc and with the hardware of TCEC the first move out of book/start position might be evaluated relatively wrong.

I added the result in case it still was a draw. For the others I added much lower later evals if existed
(move number out of book + score, move number with later lower score,
but first move number not given for score <= 0.70 don't consider this as unbalanced)

Here are just two counter examples which prove this, game 60 and 76 from your list above.
Also look at all the drawn ones listed above, despite the 'virtual' advantage dreamed by SF.

Image


Image


All games ran through a macro to get clean readable pgn, last number is TL (time left).
Graphs copied out of Tom McBurneys wonderful game analyzer.
Last edited by Guenther on Tue Oct 13, 2020 4:47 pm, edited 1 time in total.

JohnWoe
Posts: 280
Joined: Sat Mar 02, 2013 10:31 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by JohnWoe » Tue Oct 13, 2020 4:47 pm

When engines get better and better then draw rate should increase every season. As chess is draw.

Lc0 is put into tactical positions it wouldn't go itself. A little bit unfair. I think it would be better to give both own books and let them play itself. I haven't watched any (I'm too busy) TCEC so I don't how the games have been like. 100 games is a bit too little. But 100 draws isn't good entertainment. :D

mwyoung
Posts: 2725
Joined: Wed May 12, 2010 8:00 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by mwyoung » Tue Oct 13, 2020 5:03 pm

Guenther wrote:
Tue Oct 13, 2020 4:45 pm
Daniel Shawul wrote:
Tue Oct 13, 2020 2:51 pm
jorose wrote:
Tue Oct 13, 2020 2:23 pm
Daniel Shawul wrote:
Tue Oct 13, 2020 1:59 pm
It has been clear for sometime now that the unbalanced book is very unfair to Lc0.
TCEC does it for the sake of entertainment, but drawing any conclusions from it doesn't make any sense.
This SuFi and last one have been all about who can convert better given a position with a +1 advantage.
If every season is going to be like this, I guess there is no point watching TCEC SuFi.
Could you clarify what you mean by this?

I just took a look at the first 10 openings (20 games) and I don't think they are particularly unbalanced. In fact, I play several of those openings myself. The only ones I find somewhat odd are the French with Ng8 and KGA with Nc3.

Most of the lines are theoretically very well established and see a decent amount of play.I'm not sure where this notion comes from that TCEC has unbalanced openings this season in particular.


I also don't know what such fans want to see, would they prefer even less decisive games?
I am surprized you need clarification for this but here it goes.
These are scores of stockfish right out of book for the second half of the match starting from game 52

Code: Select all

52 - +1.31
54 - +1.06 => 10. 1.06 17. 0.69
56 - +1.19
58 - +0.80 => 1/2-1/2
60 - +1.14 => 7. 1.14 139. 0.62
62 - +1.17 => 1/2-1/2
64 - +1.17 => 1/2-1/2
66 - +1.12 => 1/2-1/2
68 - +1.20
70 - +0.80 => 1/2-1/2
72 - -0.93 => 1/2-1/2
74 - +0.70 ?? here I disagree ;-) and would give >1.00
76 - +0.66 ?? 25. 0.24
78 - +1.38
80 - +1.05
82 - +0.83 => 1/2-1/2
SF may well be superior to Lc0 but this unbalanced openings prove nothing.
Even SF without NNUE was able to get +7 over leela last season entirely due to the book.
The score right after book end is no verdict. Usually you need to look also at the eval several plies later to get a real clue
about an opening.
Even at this tc and with the hardware of TCEC the first move out of book/start position might be evaluated relatively wrong.

I added the result in case it still was a draw. For the others I added much lower later evals if existed
(move number out of book + score, move number with later lower score,
but first move number not given for score <= 0.70 don't consider this as unbalanced)

Here are just two counter examples which prove this, game 60 and 76 from your list above.
Also look at all the drawn ones listed above, despite the 'virtual' advantage dreamed by SF.

Image


Image


All games ran through a macro to get clean readable pgn, last number is TL (time left).
Graphs copied out of Tom McBurneys wonderful game analyzer.
You are correct that some bad position can be held. But that is not the point! Why are bad positions being played at all.

I will take TCEC words....from TCEC

Q: What do you mean when you say you “bias” openings, and why do you do it?


A: Bias is the degree to which a particular opening, or a set of openings, seem to favor one side or the other according to empirical data (historical outcomes) and evaluative data (deep analysis from top engines).

The opening position typically offers white a slightly favorable evaluation of +0.15 to +0.20 on traditional chess programs where the “comtempt” setting is neutralized. Low or zero bias in an opening book makes little difference in a tournament where time controls are shorter and contestants have a higher range of Elos. In that case, there will be ample decisive games.

However, if very strong, closely matched programs are playing at long time controls with no bias that invites a draw-rate approaching 90%. To combat this, we introduce bias in League 1 and to an even greater degree in Premier League and the Superfinal. In the latter two, you will see some openings where one color has no realistic winning chances and the game is all about successfully holding a draw from an almost desperate position.

"Note that both Nelson and Jeroen freely acknowledge that few traditional and neural net engines would find themselves in most TCEC opening positions of their own accord. This is deliberate. The whole point is to force engines to play unexpected positions and demonstrate their strength in all kinds of game situations. This is especially true for neural nets, which in training tend to emphasize favorite lines and neglect what they deem to be inferior ones."
Last edited by mwyoung on Tue Oct 13, 2020 5:09 pm, edited 2 times in total.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Tue Oct 13, 2020 5:04 pm

Moreover, Leela can be trained to play unbalanced or weird openings. And then the issue with always playing sound balanced openings is that Lc0 acts similarly to an opening book of sound openings, this question being raised since the appearance of DeepMind paper. SF should probably be allowed to use an opening book in this case.

Dann Corbit
Posts: 11923
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Dann Corbit » Tue Oct 13, 2020 5:14 pm

I do think Daniel Shawul has a point.

One might argue (and, indeed, I have) that since both sides must play the exact same lopsided opening from the disadvantaged and advantaged point of view it should be a wash. If I have a dominant position and win, next time, the opponent has the same dominant position and I should lose.

However, the clear strength of LC0 is in very quiet positions. If we start from a lopsided position, this does seem like a penalty on LC0 because of how it plays.

As further evidence of this disparity, SF abandons NN technology when the score becomes lopsided.

Food for thought.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

mwyoung
Posts: 2725
Joined: Wed May 12, 2010 8:00 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by mwyoung » Tue Oct 13, 2020 5:15 pm

Laskos wrote:
Tue Oct 13, 2020 5:04 pm
Moreover, Leela can be trained to play unbalanced or weird openings. And then the issue with always playing sound balanced openings is that Lc0 acts similarly to an opening book of sound openings, this question being raised since the appearance of DeepMind paper. SF should probably be allowed to use an opening book in this case.
Why would you do this, and if done. It would not be true chess. Chess starts with a unbias starting position. All moves matter in chess. And most importantly in the opening.

This is why true engine testing is not done with garbage openings. :lol:
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

Guenther
Posts: 3676
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Guenther » Tue Oct 13, 2020 5:16 pm

mwyoung wrote:
Tue Oct 13, 2020 5:03 pm

You are correct that some bad position can be held. But that is not the point! Why are bad positions being played at all.

...
It seems you did not read my post, or did not understand it, or both (there was no time to read all of it anyway before your reply).
Still you had to do a full quote...

Not all positions given by Daniel above were 'bad', but just falsely evaluated as 'bad' (some were even <= 0.70).

Post Reply