The Last Survivor of AB engine, Stockfish! What position will Stockfish stand?

[Raphexon]

During the match they use for AB engines a server with 90 threads (HT = on) CPUs clock frequency 2.7 GHz and for NN engines 4xRTX 2080 Ti and a server with 36 physical cores (HT = off).
The main issues for AB engines the short TC (5 min + 2sec) and they no use opening book.
It would be a wonder if Stockfish can win the race.
Obviously it is not the aim of the organizing.

This is a testament to how really good Stockfish plays. With such obvious handicaps.

And when you split AB engine's work load 90 times to 90 threads. You are causing major overhead to the AB engine's real speed on this hardware. But it will look good to the public. When they see 90 CPU's, and huge NPS counts. And again this hardware is no match for 4x RTX 2080ti.

Was this done due to ignorance, or something else.

It looks to me the AB engines were setup to fail by the organizers.

You make an interesting point about AB engines being split to much. How many cores do you recommend for optimal performance for a AB engine?

Hard to tell really.
But it's plenty evident in some benchmarks.

https://sites.google.com/site/computers ... benchmarks**

An AMD Ryzen 7 1800X scores 60 points with the Komodo benchmark. (8 cores, 16 threads)
2x Intel Xeon E5-2660 v3* (20 cores, 40 threads) scores 59 points.
But the same Xeons with hyperthreading off (so 20 cores = 20 threads) score 62 points in the Komodo benchmark.**

The 2x Intel Xeon E5-2660 v3* are benched at 49-50 mnps on ipmanchess with HT on.
But only 38.1 mnps with hyperthreading off.
http://www.ipmanchess.yolasite.com/amd- ... -bench.php

While an overclocked Ryzen R7 1800X does "just" 28.5 mnps.

So 50 mnps on 40 threads is equal to at most 28.5 mnps on 16 threads.
And 38.1 mnps of 20 real cores outperform 40 threads (49-50 mnps) on the same machine!

*They're benched by the same person too.

**The Komodo benchmark tests a machine's ability to quickly find the solution to multiple tactical puzzles.

I honestly would not be surprised if an overclocked i9 7980XE easily beats the 96 core machine chess.com uses.

nps is only a measure of strength between 2 systems using an equal amount of physical cores.
Time to depth is a much better benchmark.

[Raphexon]

Anyone remember Zeta, the AB engine on a GPU.
It has the same problem where more NPS does not equal superior performance if NPS is the result of more cores. (workers)

As you can see beyond 128 workers its NPS keeps scaling, but its real strength (time to depth) stops scaling beyond 128 workers.
It has 35 mnps with 1024 workers, but it's TTD speedup is the highest with 128 workers. Despite nps only being a third of 1024 workers.

https://github.com/smatovic/Zeta/blob/m ... esults.txt

# Zeta 099k, startposition, depth 12, Nvidia Titan RTX:
# run with up to 1024 workers, parallel search scales to 128 workers

### workers #nps #nps speedup #time in s #ttd speedup #relative ttd speedup
### 1 104663 1.000000 108.546000 1.000000 1.000000
### 2 206745 1.975340 48.657000 2.230840 2.230840
### 4 409302 3.910666 28.482000 3.811039 1.708342
### 8 825953 7.891547 18.614000 5.831417 1.530139
### 16 1701699 16.258840 11.590000 9.365487 1.606040
### 32 3409805 32.578896 8.438000 12.863949 1.373548
### 64 6721376 64.219218 7.406000 14.656495 1.139346
### 128 11680025 111.596505 4.871000 22.284131 1.520427

### 256 16852466 161.016462 5.987000 18.130282 0.813596
### 512 24075509 230.028845 6.343000 17.112723 0.943875
### 1024 35080623 335.176930 8.272000 13.122099 0.766804

[mwyoung]

It is even worst then this. This is such a lopsided tournament in favor of the NN engines. I looked up the benchmark for the $6000 Intel Xeon Platinum 8168. And what a piece of dog crap for 6 grand. And they bought 2 pieces of dog crap. Below is my benchmark for my $800 2950x. For a extra $5000 you get a whopping 1.9% speed increase.

This tournament is a total hardware mismatch for the NN engines.

What do you think is the best CPU performance they could have got for $6000 or $12000?

Back when they got their server. The fastest chess server would be a AMD epyc 7601 32 core. This setup would give you more real cores 24 vs 32. And at a cheaper price $4600 vs $6000.

But the main selling point of the 7601. Is that you can overclock the 7601 in a dual socket configuration. This is what puts it over the top in the real world for a chess playing server.

As you can see from my bench test. My 16 core is about the same speed as $6000 Intel Xeon Platinum 8168 locked cpu when overclocked.

The Epyc with 32 cores would be epic x2.

[Raphexon]

My guess it that an overclocked i9-9960X or an overclocked i9-9980X are the two strongest CPUs for chess engines.
Their nps rivals systems with many more cores while only having 32 threads.

Another contender would be an overclocked Threadripper 2 2990WX.

[mwyoung]

My guess it that an overclocked i9-9960X or an overclocked i9-9980X are the two strongest CPUs for chess engines.
Their nps rivals much with many more cores while only have 32 threads.

Another contender would be an overclocked Threadripper 2 2990WX.

I agree.

It is better to have faster and fewer cores for AB engines. The more you split the workload. The more you hurt the real world performance of the AB engine.

[Raphexon]

It's easy to test really.

From a random position SF10 reached depth 25 in 45 million nodes on 1 core.
On 4 cores it reached depth 25 in 76.7 million nodes.
While nps increased close to 4 fold (3.8 times as many). TTD only quicked 2.5 times. From 37 seconds to 15.

A node is not a node when comparing 2 systems with differing amount of threads.

[Raphexon]

It is even worst then this. This is such a lopsided tournament in favor of the NN engines. I looked up the benchmark for the $6000 Intel Xeon Platinum 8168. And what a piece of dog crap for 6 grand. And they bought 2 pieces of dog crap. Below is my benchmark for my $800 2950x. For a extra $5000 you get a whopping 1.9% speed increase.

This tournament is a total hardware mismatch for the NN engines.

What do you think is the best CPU performance they could have got for $6000 or $12000?

An i9-9960x overclocked to its utmost limit and cooled with a phase change cooler.

[mwyoung]

Update, after 36 games! My prediction is correct!! ( with more and more game, the gap between LCO and SF become larger and larger)!

Yep, This will be ugly.

Folks, just remember Lc0 will win huge only because Lc0 has all the advantages. A huge hardware advantage, and how Stockfish was configured, and was forced to play on this system.

[Nay Lin Tun]

Update!
Leela performance is better than expected! 50 games without a single loss!

[Leo]

During the match they use for AB engines a server with 90 threads (HT = on) CPUs clock frequency 2.7 GHz and for NN engines 4xRTX 2080 Ti and a server with 36 physical cores (HT = off).
The main issues for AB engines the short TC (5 min + 2sec) and they no use opening book.
It would be a wonder if Stockfish can win the race.
Obviously it is not the aim of the organizing.

This is a testament to how really good Stockfish plays. With such obvious handicaps.

And when you split AB engine's work load 90 times to 90 threads. You are causing major overhead to the AB engine's real speed on this hardware. But it will look good to the public. When they see 90 CPU's, and huge NPS counts. And again this hardware is no match for 4x RTX 2080ti.

Was this done due to ignorance, or something else.

It looks to me the AB engines were setup to fail by the organizers.

You make an interesting point about AB engines being split to much. How many cores do you recommend for optimal performance for a AB engine?

Hard to tell really.
But it's plenty evident in some benchmarks.

https://sites.google.com/site/computers ... benchmarks**

An AMD Ryzen 7 1800X scores 60 points with the Komodo benchmark. (8 cores, 16 threads)
2x Intel Xeon E5-2660 v3* (20 cores, 40 threads) scores 59 points.
But the same Xeons with hyperthreading off (so 20 cores = 20 threads) score 62 points in the Komodo benchmark.**

The 2x Intel Xeon E5-2660 v3* are benched at 49-50 mnps on ipmanchess with HT on.
But only 38.1 mnps with hyperthreading off.
http://www.ipmanchess.yolasite.com/amd- ... -bench.php

While an overclocked Ryzen R7 1800X does "just" 28.5 mnps.

So 50 mnps on 40 threads is equal to at most 28.5 mnps on 16 threads.
And 38.1 mnps of 20 real cores outperform 40 threads (49-50 mnps) on the same machine!

*They're benched by the same person too.

**The Komodo benchmark tests a machine's ability to quickly find the solution to multiple tactical puzzles.

I honestly would not be surprised if an overclocked i9 7980XE easily beats the 96 core machine chess.com uses.

nps is only a measure of strength between 2 systems using an equal amount of physical cores.
Time to depth is a much better benchmark.

Thanks for the research. Some of us on here have been skeptical over the years about chess engines benefitting much over 16 cores. I wonder what system has the best time to depth. I think my dream machine will be the Ryzen 3000 series Threadripper with 16 cores and a lot of RAM. (Unless LCO dominates it at some point. Then I would be forced to foray into the world of GPS.)