about the term of equal hardware

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

jp
Posts: 1470
Joined: Mon Apr 23, 2018 7:54 am

Re: about the term of equal hardware

Post by jp »

Robert Pope wrote: Mon Mar 04, 2019 7:54 pm
jp wrote: Mon Mar 04, 2019 7:08 pm
Robert Pope wrote: Mon Mar 04, 2019 6:10 pm If a tournament is run on 64-bit CPUs instead of 32-bit CPUs, are you giving bitboard engines an "advantage"?
Yes: Bitboard engines can provide a 64-bit compile that will run much more quickly on 64-bit CPUs, while mailbox engines hardly gain any speed at all.
No: Mailbox engine authors know full well that the tourney is run on a 64-bit CPU. If they decide not to invest the resources to take advantage of the architecture, that's on them.
And the same "advantage" comparison crops up when you go to 1 vs 4 vs 64 CPUs, or 1 GB hash limit to 128 GB hash limit, or no GPU to yes GPU, or own book access vs not. The same argument about giving advantage exists.
It's worse in the GPU/not, book/not scenario, I think, because the engines are so different it's not just deciding to or not to invest the resources.
Sure, you can argue it's worse, and maybe an alpha-beta style engine will never be able to leverage a GPU as well as a NN, but then it's still a conscious decision to stay on that path. Just as it is a conscious decision that I keep my engine single-threaded. But I'm not going to argue that because I have decided to keep my engine single-core, that other programmers are being given an advantage.
But e.g. it's not a conscious decision that they keep NN engines bad at endgames. They don't know how to make them good at endgames. Giving all engines TBs is the same as giving NN engines an advantage, because it is partly covering over one of their weaknesses.
Last edited by jp on Mon Mar 04, 2019 8:30 pm, edited 1 time in total.
smatovic
Posts: 2641
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: about the term of equal hardware

Post by smatovic »

j.korhonen wrote: Mon Mar 04, 2019 6:34 pm
smatovic wrote: Mon Mar 04, 2019 6:54 am
j.korhonen wrote: Sun Mar 03, 2019 11:59 pm
smatovic wrote: Sun Mar 03, 2019 7:06 pm
j.korhonen wrote: Sun Mar 03, 2019 6:46 pm Tell me more about Zeta GPU?
In short, it is a port of Zeta Dva engine to OpenCL to run on a GPU.

https://www.chessprogramming.org/Zeta

https://zeta-chess.app26.de/

--
Srdja
classic parallel Alpha-Beta on GPU and open source. WOW. Why you not in TCEC?
Well, to port a relative simple engine to run on a GPU and to get a
competitive engine running on GPU are two different tasks...

https://zeta-chess.app26.de/post/zeta-v099/

--
Srdja
How hard it will be to port SF on openCL, and how strong it will be if compare with Lc0?
Hard, otherwise team fish would already have done it.

I think a 3000+ Elo engine is possible on gpu,
but unless you get the same nps per worker like a cpu,
you can not outperform SF running on current many-core cpus.

--
Srdja
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: about the term of equal hardware

Post by Robert Pope »

jp wrote: Mon Mar 04, 2019 8:24 pm
Robert Pope wrote: Mon Mar 04, 2019 7:54 pm
jp wrote: Mon Mar 04, 2019 7:08 pm
Robert Pope wrote: Mon Mar 04, 2019 6:10 pm If a tournament is run on 64-bit CPUs instead of 32-bit CPUs, are you giving bitboard engines an "advantage"?
Yes: Bitboard engines can provide a 64-bit compile that will run much more quickly on 64-bit CPUs, while mailbox engines hardly gain any speed at all.
No: Mailbox engine authors know full well that the tourney is run on a 64-bit CPU. If they decide not to invest the resources to take advantage of the architecture, that's on them.
And the same "advantage" comparison crops up when you go to 1 vs 4 vs 64 CPUs, or 1 GB hash limit to 128 GB hash limit, or no GPU to yes GPU, or own book access vs not. The same argument about giving advantage exists.
It's worse in the GPU/not, book/not scenario, I think, because the engines are so different it's not just deciding to or not to invest the resources.
Sure, you can argue it's worse, and maybe an alpha-beta style engine will never be able to leverage a GPU as well as a NN, but then it's still a conscious decision to stay on that path. Just as it is a conscious decision that I keep my engine single-threaded. But I'm not going to argue that because I have decided to keep my engine single-core, that other programmers are being given an advantage.
But e.g. it's not a conscious decision that they keep NN engines bad at endgames. They don't know how to make them good at endgames. Giving all engines TBs is the same as giving NN engines an advantage, because it is partly covering over one of their weaknesses.
It's a conscious decision to create a NN engine. If pure NN is always bad at endgames, it's their choice to go down that route. They certainly have the ability to make design decisions to mitigate that weakness, e.g. switch to a more traditional alpha-beta search under certain conditions, or have an endgame NN. Before TB, many engines had special endgame code to play better than those who didn't. After TB, the engines who didn't have the special code suddenly made gains by adding TB support, and the others made smaller gains. Was adding TB availability "an advantage"? In the short term, you can argue yes. In the long-term, the landscape that engines compete in just changed, and most engines now reflect the new reality and don't bother coding special cases that TB can handle by itself.

If the tournament you care about says they will make 5 or 6 or 7 (or 8) piece tablebases available, it's a conscious decision to implement support for them or not, and to select algorithms that make best use of them, or not.

No program can perform equally well under every type of tournament. If I create a tournament where the engines must run on Apple IIe computers, Stockfish probably won't even enter. It's up to developers to decide what they care most about, and act accordingly. That's all I'm saying. Arguing whether conditions are an advantage or not, is in many ways a false discussion if the conditions are equal. GPUs in chess are here to stay, and every developer gets to decide how they react to that.
jp
Posts: 1470
Joined: Mon Apr 23, 2018 7:54 am

Re: about the term of equal hardware

Post by jp »

Robert Pope wrote: Mon Mar 04, 2019 9:49 pm Was adding TB availability "an advantage"? In the short term, you can argue yes. In the long-term, the landscape that engines compete in just changed, and most engines now reflect the new reality and don't bother coding special cases that TB can handle by itself.

If the tournament you care about says they will make 5 or 6 or 7 (or 8) piece tablebases available, it's a conscious decision to implement support for them or not, and to select algorithms that make best use of them, or not.
The AB engines play endgames well without TBs. The NN engines don't. We don't know what conscious decision the NN engine makers can make to improve their EGs if the tournament does not make TBs available.
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: about the term of equal hardware

Post by Robert Pope »

jp wrote: Tue Mar 05, 2019 10:29 am
Robert Pope wrote: Mon Mar 04, 2019 9:49 pm Was adding TB availability "an advantage"? In the short term, you can argue yes. In the long-term, the landscape that engines compete in just changed, and most engines now reflect the new reality and don't bother coding special cases that TB can handle by itself.

If the tournament you care about says they will make 5 or 6 or 7 (or 8) piece tablebases available, it's a conscious decision to implement support for them or not, and to select algorithms that make best use of them, or not.
The AB engines play endgames well without TBs. The NN engines don't. We don't know what conscious decision the NN engine makers can make to improve their EGs if the tournament does not make TBs available.
They can switch to alphabeta methods, if they feel that will improve their engine. Nothing says they have to be pure NN. And there are already clear ways that they can improve in a NN framework as well -- dkappe's ender networks have shown that.
j.korhonen
Posts: 19
Joined: Tue Feb 26, 2019 12:34 am
Full name: Juhani Korhonen

Re: about the term of equal hardware

Post by j.korhonen »

smatovic wrote: Mon Mar 04, 2019 8:27 pm
j.korhonen wrote: Mon Mar 04, 2019 6:34 pm
smatovic wrote: Mon Mar 04, 2019 6:54 am
j.korhonen wrote: Sun Mar 03, 2019 11:59 pm
smatovic wrote: Sun Mar 03, 2019 7:06 pm
j.korhonen wrote: Sun Mar 03, 2019 6:46 pm Tell me more about Zeta GPU?
In short, it is a port of Zeta Dva engine to OpenCL to run on a GPU.

https://www.chessprogramming.org/Zeta

https://zeta-chess.app26.de/

--
Srdja
classic parallel Alpha-Beta on GPU and open source. WOW. Why you not in TCEC?
Well, to port a relative simple engine to run on a GPU and to get a
competitive engine running on GPU are two different tasks...

https://zeta-chess.app26.de/post/zeta-v099/

--
Srdja
How hard it will be to port SF on openCL, and how strong it will be if compare with Lc0?
Hard, otherwise team fish would already have done it.

I think a 3000+ Elo engine is possible on gpu,
but unless you get the same nps per worker like a cpu,
you can not outperform SF running on current many-core cpus.

--
Srdja
Why OpenCL? Why not CUDA? Isn't CUDA faster?
smatovic
Posts: 2641
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: about the term of equal hardware

Post by smatovic »

j.korhonen wrote: Wed Mar 06, 2019 2:52 am
smatovic wrote: Mon Mar 04, 2019 8:27 pm
j.korhonen wrote: Mon Mar 04, 2019 6:34 pm
smatovic wrote: Mon Mar 04, 2019 6:54 am
j.korhonen wrote: Sun Mar 03, 2019 11:59 pm
smatovic wrote: Sun Mar 03, 2019 7:06 pm
j.korhonen wrote: Sun Mar 03, 2019 6:46 pm Tell me more about Zeta GPU?
In short, it is a port of Zeta Dva engine to OpenCL to run on a GPU.

https://www.chessprogramming.org/Zeta

https://zeta-chess.app26.de/

--
Srdja
classic parallel Alpha-Beta on GPU and open source. WOW. Why you not in TCEC?
Well, to port a relative simple engine to run on a GPU and to get a
competitive engine running on GPU are two different tasks...

https://zeta-chess.app26.de/post/zeta-v099/

--
Srdja
How hard it will be to port SF on openCL, and how strong it will be if compare with Lc0?
Hard, otherwise team fish would already have done it.

I think a 3000+ Elo engine is possible on gpu,
but unless you get the same nps per worker like a cpu,
you can not outperform SF running on current many-core cpus.

--
Srdja
Why OpenCL? Why not CUDA? Isn't CUDA faster?
Not on AMD and Intel gpus. There are papers that claim with the right
optimization, OpenCL runs as fast as CUDA. On Nvidia devices CUDA offers some
advantages over OpenCL. But one aim of the Zeta project was to support all kind
of gpus, so i am still on OpenCL 1.x, and do not use the features newer
gpu architectures offer. I want Zeta to run on my first gpgpu device,
the Nvidia 8800 GT. Zeta started as a bet with a fellow student, i claimed that
i am able to write a chess engine that beats MSCP within two weaks. I thought
with a gpu this would be an easy tasks, cos the 8800 GT has 112 cores, compared
to only core MSCP is able to run on, silly me. After two weaks i had an chess
engine running on the CPU, but i got the move ordering wrong and was not able
to outperform MSCP.

--
Srdja
jp
Posts: 1470
Joined: Mon Apr 23, 2018 7:54 am

Re: about the term of equal hardware

Post by jp »

Robert Pope wrote: Tue Mar 05, 2019 3:19 pm
jp wrote: Tue Mar 05, 2019 10:29 am The AB engines play endgames well without TBs. The NN engines don't.
They can switch to alphabeta methods, if they feel that will improve their engine. Nothing says they have to be pure NN. And there are already clear ways that they can improve in a NN framework as well -- dkappe's ender networks have shown that.
I'd prefer to find out what's the best a "zero" NN can do. If not enough people care about "zero", then that won't happen, but at least a non-zero NN (ender is one). This would be helped long term if tournaments don't just cover up NN weaknesses through their conditions.
j.korhonen
Posts: 19
Joined: Tue Feb 26, 2019 12:34 am
Full name: Juhani Korhonen

Re: about the term of equal hardware

Post by j.korhonen »

CMCanavessi wrote: Mon Mar 04, 2019 1:54 am
j.korhonen wrote: Sun Mar 03, 2019 11:59 pm
smatovic wrote: Sun Mar 03, 2019 7:06 pm
j.korhonen wrote: Sun Mar 03, 2019 6:46 pm Tell me more about Zeta GPU?
In short, it is a port of Zeta Dva engine to OpenCL to run on a GPU.

https://www.chessprogramming.org/Zeta

https://zeta-chess.app26.de/

--
Srdja
classic parallel Alpha-Beta on GPU and open source. WOW. Why you not in TCEC?
~2200 elo for now
And what? TCEC Season 15. Division 4a. Igel 1.2 2386 Elo
Such a unique engine as Zeta should be play in the TCEC
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: about the term of equal hardware

Post by CMCanavessi »

j.korhonen wrote: Wed Mar 06, 2019 8:31 pm
CMCanavessi wrote: Mon Mar 04, 2019 1:54 am
j.korhonen wrote: Sun Mar 03, 2019 11:59 pm
smatovic wrote: Sun Mar 03, 2019 7:06 pm
j.korhonen wrote: Sun Mar 03, 2019 6:46 pm Tell me more about Zeta GPU?
In short, it is a port of Zeta Dva engine to OpenCL to run on a GPU.

https://www.chessprogramming.org/Zeta

https://zeta-chess.app26.de/

--
Srdja
classic parallel Alpha-Beta on GPU and open source. WOW. Why you not in TCEC?
~2200 elo for now
And what? TCEC Season 15. Division 4a. Igel 1.2 2386 Elo
Such a unique engine as Zeta should be play in the TCEC
Igel is "at least" 2800-2900 on TCEC hardware.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls