SF was more seriously handicapped than I thought

[Leo]

As the real "match" conditions cannot be reproduced, I think Kai Laskos performed the best scientific approach possible to the question of how much SF got handicapped.

Right, but the best approach possible is so far off the match that conclusions aren't valid anymore.

As I said, it cannot be reproduced. So, we have no answers. Only questions. And the main question is:

Why did they need to give SF such handicaps?

My personal opinion is that they didn't really want a hard fight.

I want to throw out a previous comment I made.

If the Cerebellum Brainfish people would release their program publicly we could train it around the clock on a server and build our own chess engine book. I have played with cerebellum and it goes very deep. Give SF Dev 128 GB or more instead of 1 GB ?! of DDR4 4000 Mhz Ram and 30 overclocked cores plus 7 man EGTB. The 1 minute time control would have to be scotched. Maybe use a liquid nitrogen setup if that would help however that works. The mere 3 losses with black would be eliminated and maybe turned to wins. The 25 losses against white would be chipped away at. Much more drawn and some would be wins. AZ has accomplished something amazing and shaken up the chess computer world but they have not conquered the chess engine world until further notice. Why in the world didn't they let Deep Zero run for a week and learn. I think Milos might be right. After 4 hours it maxed out. It was saturated.

[Rodolfo Leoni]

.........................................................
I think Milos might be right. After 4 hours it maxed out. It was saturated.

It's my impression too.

[Laskos]

out of curiosity, I measured the effect

Yeah, of Stockfish against Stockfish, and that doesn't say that Stockfish boosted with better hardware would also have taken the same amount of Elo en plus against A0. This simply doesn't wash.

Of course you'll get effects because if the same engine can calculate a bit deeper, it will win more often than not. But Stockfish's problem in the matches was NOT that Stockfish didn't go deep enough. Even after long analysis and when making A0's moves, Stockfish still doesn't understand what was going on. More time and more speed would just have made Stockfish calculating the losing moves a bit deeper.

It's sure a toy model, but a toy model is the best we can do as of now. I too expect that the effect in real conditions is smaller. But not negligible. SF8 was pretty seriously handicapped, even if that is maybe not extremely relevant against A0. What you say its true, but sometimes SF misevaluates positions even against Komodo, losing games, being nevertheless at least the same level of strength, if not stronger.

I repeated the test with Komodo at 20s/move as A0. Against SF dev at 3s/move and 1MB hash with no book and TBs, and against full panoply BrainFish + 128MB hash + Cerebellum + Time Control + 6-men Syzygy on SSD.

First match:

+16 -2 =22

Second match:

+6 -4 =30

Again, the handicap seems pretty severe, this time against Komodo. In real A0 case, we would probably see higher draw rate and maybe some losses of A0.