SF was more seriously handicapped than I thought

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Leo
Posts: 1000
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: SF was more seriously handicapped than I thought

Post by Leo » Thu Jan 04, 2018 5:26 pm

Rodolfo Leoni wrote:
Ras wrote:
Rodolfo Leoni wrote:As the real "match" conditions cannot be reproduced, I think Kai Laskos performed the best scientific approach possible to the question of how much SF got handicapped.
Right, but the best approach possible is so far off the match that conclusions aren't valid anymore.
As I said, it cannot be reproduced. So, we have no answers. Only questions. And the main question is:

Why did they need to give SF such handicaps?

My personal opinion is that they didn't really want a hard fight.
I want to throw out a previous comment I made.

If the Cerebellum Brainfish people would release their program publicly we could train it around the clock on a server and build our own chess engine book. I have played with cerebellum and it goes very deep. Give SF Dev 128 GB or more instead of 1 GB ?! of DDR4 4000 Mhz Ram and 30 overclocked cores plus 7 man EGTB. The 1 minute time control would have to be scotched. Maybe use a liquid nitrogen setup if that would help however that works. The mere 3 losses with black would be eliminated and maybe turned to wins. The 25 losses against white would be chipped away at. Much more drawn and some would be wins. AZ has accomplished something amazing and shaken up the chess computer world but they have not conquered the chess engine world until further notice. Why in the world didn't they let Deep Zero run for a week and learn. I think Milos might be right. After 4 hours it maxed out. It was saturated.
Advanced Micro Devices fan.

Rodolfo Leoni
Posts: 545
Joined: Tue Jun 06, 2017 2:49 pm
Location: Italy

Re: SF was more seriously handicapped than I thought

Post by Rodolfo Leoni » Thu Jan 04, 2018 9:31 pm

Leo wrote:.........................................................
I think Milos might be right. After 4 hours it maxed out. It was saturated.
It's my impression too. :wink:
F.S.I. Chess Teacher

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: SF was more seriously handicapped than I thought

Post by Laskos » Fri Jan 05, 2018 8:05 am

Ras wrote:
Laskos wrote:out of curiosity, I measured the effect
Yeah, of Stockfish against Stockfish, and that doesn't say that Stockfish boosted with better hardware would also have taken the same amount of Elo en plus against A0. This simply doesn't wash.

Of course you'll get effects because if the same engine can calculate a bit deeper, it will win more often than not. But Stockfish's problem in the matches was NOT that Stockfish didn't go deep enough. Even after long analysis and when making A0's moves, Stockfish still doesn't understand what was going on. More time and more speed would just have made Stockfish calculating the losing moves a bit deeper.
It's sure a toy model, but a toy model is the best we can do as of now. I too expect that the effect in real conditions is smaller. But not negligible. SF8 was pretty seriously handicapped, even if that is maybe not extremely relevant against A0. What you say its true, but sometimes SF misevaluates positions even against Komodo, losing games, being nevertheless at least the same level of strength, if not stronger.

I repeated the test with Komodo at 20s/move as A0. Against SF dev at 3s/move and 1MB hash with no book and TBs, and against full panoply BrainFish + 128MB hash + Cerebellum + Time Control + 6-men Syzygy on SSD.

First match:

+16 -2 =22

Second match:

+6 -4 =30

Again, the handicap seems pretty severe, this time against Komodo. In real A0 case, we would probably see higher draw rate and maybe some losses of A0.

Post Reply