Dann Corbit wrote: ↑Fri Jul 19, 2019 8:18 pm
Both hardware and software improve exponentially.
Now the "Moore's Law is dead" crowd have a point.
But I suspect that Kurzweil's insistence that a new technology will be found is very likely indeed.
After all, it happened with relays, then with vacuum tubes then transistors, then integrated circuits.
Why not a new thing to take the place?
And software has a long way to go to reach perfection also.
It seems that the evaluation of the NN stuff is absurdly superior (considering the very low nodes examined yet superior move choices).
So what should happen if the evaluation of the alpha-beta searchers reaches the current level of the NN engines?
The NN thing seemed to come out of the blue (and it took real vision to see it, not everyone was awestruck by Matthew Lai's initial project before the massive compute power of TPUs came into play).
Who is to say that we cannot have further revolution after further revolution after further revolution.
Look at Pohl's nice graphs. You will see that they look like nice, steady, linear advancement. Of course, with Elo, we are looking at an
exponential y-scale. So that means we have seen very steady exponential progress from the Stockfish team.
So, how much more Elo?
How much from hardware?
How much from software?
I don't think we know the answer to any of the three questions, but I guess it is "a lot".
From my perspective, GPUs are a superior architecture for chess analysis. Neural Networks "cheat" by using a GPU, but the programmer doesn't really understand how the GPU works. Most programmers these days don't know how to use a GPU yet, and are unfamiliar with basic (but important) patterns like "
Stream Compaction".
I'm working on allowing Alpha-Beta to be explored on a GPU. I still don't know if I'd be successful, but I would expect that any program that properly uses a 655,360 thread machine with 500GBps RAM access (ie: Vega64 GPU + HBM2) would be superior to programs that are only designed to use 64-threads on 100GBps RAM (ie: Threadripper 2990wx with Quad-channel DDR4).
The 480GBps HBM2 RAM has more bandwidth available to it than most CPU's L3 cache, and you get 8GB of that in a commodity GPU. NVidia uses GDDR6 to achieve similar speeds on its GPUs, all GPUs have VRAM that utterly crush a typical CPU's specs.
Faster RAM + more cores
should be faster. Now granted, individually, each of those GPU cores are slower, and there are significant issues like "Thread Divergence" which can destroy a GPU's efficiency. CPUs on the other hand, are out-of-order and speculatively branch predict. CPUs are "easy" to program, the CPU will automatically optimize your code (even rearrange it automatically) so that it will execute faster.
So no matter how I look at it: programming a GPU to be a chess engine will be harder, far harder, than writing an efficient CPU algorithm. The question is how to write software that properly exploits those advantages.
Neural Nets have been optimized to use the GPU architecture extremely efficiently. Neural Nets are transformed into a matrix-multiplication problem (which has already been solved by decades of research in the graphics community). CPUs these days are ~200 GFlops, maybe ~400GFlops. But an NVidia GPU will hit over 100 16-bit TFlops, almost 500x more operations per second than any commodity CPU. With such a crushing advantage in power, of course a GPU will be superior at chess playing.
For now, the Neural Network will stay as a crutch: used by those who don't really understand a GPU (or want to learn how to use it). But I think the next steps forward is to actually learn the GPU architecture and really think about new algorithms that will work on GPUs.