I was looking at the CEGT 40/20 rating list and was surprised to see that Houdini 6 doesn't seem to scale well with extra CPU's on that list:
Houdini 6.0 x64 8 CPU rating 3442
Houdini 6.0 x64 4 CPU rating 3437
A 5 pt. increase.
By comparison:
Stockfish 9.0 x64 8 CPU rating 3506
Stockfish 9.0 x64 4 CPU rating 3471
A 35 pt. increase.
Has anyone else done testing to confirm Houdini's poor scaling, or is this a margin-of-error issue?
Houdini Scaling on CEGT Rating List
Moderators: hgm, Rebel, chrisw
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Houdini Scaling on CEGT Rating List
Sure error margin issue and maybe pool of opponents issue. We saw Houdini 6 in TCEC and WCCC on 40+ cores, it scales in the same league with the best, SF, Komodo, etc.Milton wrote: ↑Mon Jan 28, 2019 8:54 pm I was looking at the CEGT 40/20 rating list and was surprised to see that Houdini 6 doesn't seem to scale well with extra CPU's on that list:
Houdini 6.0 x64 8 CPU rating 3442
Houdini 6.0 x64 4 CPU rating 3437
A 5 pt. increase.
By comparison:
Stockfish 9.0 x64 8 CPU rating 3506
Stockfish 9.0 x64 4 CPU rating 3471
A 35 pt. increase.
Has anyone else done testing to confirm Houdini's poor scaling, or is this a margin-of-error issue?
-
- Posts: 4606
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Houdini Scaling on CEGT Rating List
If you would post the error margin we could tell more w/o extra workMilton wrote: ↑Mon Jan 28, 2019 8:54 pm I was looking at the CEGT 40/20 rating list and was surprised to see that Houdini 6 doesn't seem to scale well with extra CPU's on that list:
Houdini 6.0 x64 8 CPU rating 3442
Houdini 6.0 x64 4 CPU rating 3437
A 5 pt. increase.
By comparison:
Stockfish 9.0 x64 8 CPU rating 3506
Stockfish 9.0 x64 4 CPU rating 3471
A 35 pt. increase.
Has anyone else done testing to confirm Houdini's poor scaling, or is this a margin-of-error issue?
Have you compared to CCRL too?
-
- Posts: 2871
- Joined: Wed Mar 08, 2006 10:09 pm
- Location: Germany
- Full name: Werner Schüle
Re: Houdini Scaling on CEGT Rating List
Hi,
of course, 370 games do not show too much. But I have had some troubles with Houdini after a Windows update: it was not longer registered.
We will check this on the new pc of our tester. Thanks for the posting.
best wishes
Werner
of course, 370 games do not show too much. But I have had some troubles with Houdini after a Windows update: it was not longer registered.
We will check this on the new pc of our tester. Thanks for the posting.
best wishes
Werner
Werner
-
- Posts: 139
- Joined: Thu Mar 09, 2006 12:58 am
Re: Houdini Scaling on CEGT Rating List
I have looked at the CCRL 40/40 rating list.Guenther wrote: ↑Mon Jan 28, 2019 9:07 pmIf you would post the error margin we could tell more w/o extra workMilton wrote: ↑Mon Jan 28, 2019 8:54 pm I was looking at the CEGT 40/20 rating list and was surprised to see that Houdini 6 doesn't seem to scale well with extra CPU's on that list:
Houdini 6.0 x64 8 CPU rating 3442
Houdini 6.0 x64 4 CPU rating 3437
A 5 pt. increase.
By comparison:
Stockfish 9.0 x64 8 CPU rating 3506
Stockfish 9.0 x64 4 CPU rating 3471
A 35 pt. increase.
Has anyone else done testing to confirm Houdini's poor scaling, or is this a margin-of-error issue?
Have you compared to CCRL too?
As near as I can tell, all Houdini testing was done with 4 CPUs, so this list does not have comparative data about how well Houdini scales with additional CPUs.
-
- Posts: 2871
- Joined: Wed Mar 08, 2006 10:09 pm
- Location: Germany
- Full name: Werner Schüle
Re: Houdini Scaling on CEGT Rating List
I can help with our 40/4 list:
Houdini 6.0 x64 12CPU 3592
Houdini 6.0 x64 4CPU 3470
Houdini 6.0 x64 2CPU 3435
Houdini 6.0 x64 1CPU 3372
Houdini 6.0 x64 12CPU 3592
Houdini 6.0 x64 4CPU 3470
Houdini 6.0 x64 2CPU 3435
Houdini 6.0 x64 1CPU 3372
Werner
-
- Posts: 1080
- Joined: Fri Sep 16, 2016 6:55 pm
- Location: USA/Minnesota
- Full name: Leo Anger
Re: Houdini Scaling on CEGT Rating List
Houdini might go the way of Rybka and become a dead engine.
Advanced Micro Devices fan.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: Houdini Scaling on CEGT Rating List
At least rybka has a son, Fritz 16
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 41423
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
-
- Posts: 139
- Joined: Thu Mar 09, 2006 12:58 am
Re: Houdini Scaling on CEGT Rating List
Thanks for pointing me to the single cpu list Graham:Graham Banks wrote: ↑Tue Jan 29, 2019 5:40 amhttp://www.computerchess.org.uk/ccrl/40 ... ons_only=1
http://www.computerchess.org.uk/ccrl/40 ... ons_only=1
So comparing the same programs on the CCRL lists as I did on the CEGT list:
Stockfish 9 64-bit 4CPU Rating: 3432
Stockfish 9 64-bit 1cpu Rating: 3367
A difference of 65 points.
Houdini 6 64-bit 4CPU Rating: 3396
Houdini 6 64-bit 1CPU Rating: 3336
A difference of 60 points.
So the scaling of the two programs from 1 CPU to 4 CPUs in CCRL are very close.