lkaufman wrote:Since it is clear that Komodo scales better with more time than Houdini
I think it is a Komodo problem, not an advantage - Komodo scales worse than Houdini at STC.
lkaufman wrote:I think it is fair to say that we have finally surpassed Houdini in strength at most reasonable time controls, at least on Intel machines
I'm really glad to hear this, but Houdini 4 released six months ago - who knows how strong Houdini 5 will be?
Regarding scaling, since Stockfish also scales better than Houdini it looks more like a Houdini problem. Anyway, that doesn't matter, the point is that whatever Komodo's results at short time control are vs. Houdini they should be better at long time control. This was borne out by the current TCEC semifinal, where Komodo finished a full four points ahead of Houdini.
Regarding Houdini, who knows if there will even be a Houdini 5?
..This was borne out (?!).. in 16 games
with Komodo dev at drawscore=0, while Houdini with default contempt
While you don't react on the formentioned 50 LTC games in Clemens Keck match 27,5 - 22,5 for Houdini 4
The TCEC semifinal was 48 rounds, not 16. Komodo scored 4 points more than Houdini after 48 round robin games each.
Regarding Clemens match, first of all Komodo 7 is version 1227, four improvements later than the one he ran, and furthermore, he doesn't use color reversal of openings, which greatly increases the luck factor.
Only 16 games played against Komodo dev at TCEC and drawscore Komodo at zero
I very much doubt that Komodo 7 will surpass Houdini 4 at CCRL and CEGT whether short or long time control 1 or 4cpu
But if so you will fully deserve it
beram wrote:
I very much doubt that Komodo 7 will surpass Houdini 4 at CCRL and CEGT whether short or long time control 1 or 4cpu
But if so you will fully deserve it
OK, first of all, even Komodo tcec outrates Houdini 4 on the CEGT 40/2 hours list. I do not expect Komodo 7 to catch Houdini 4 at the blitz levels. At the intermediate levels (40/20 on CEGT, 40/40 on CCRL) the average gap between Komodo tcec and Houdini 4, averaging both 1 core and 4 core differences, is 30 elo points. If we have gained 40 elo at blitz levels, this probably means at least 30 at intermediate levels. So my best guess is that we will top Houdini 4 on two out of those 4 lists, as well as the 40/2hour list. Anyway, it should be extremely close. Houdini 4 does have the advantage on these lists of using high contempt to rack up big scores against weak opponents; the real test is how the top engines do against each other. I would bet money on Komodo 7 against Houdini 4 at any time limit of 30 minutes or longer plus one percent increment, on one or 4 cores, run on an Intel machine by a neutral party.
beram wrote:
I very much doubt that Komodo 7 will surpass Houdini 4 at CCRL and CEGT whether short or long time control 1 or 4cpu
But if so you will fully deserve it
OK, first of all, even Komodo tcec outrates Houdini 4 on the CEGT 40/2 hours list. I do not expect Komodo 7 to catch Houdini 4 at the blitz levels. At the intermediate levels (40/20 on CEGT, 40/40 on CCRL) the average gap between Komodo tcec and Houdini 4, averaging both 1 core and 4 core differences, is 30 elo points. If we have gained 40 elo at blitz levels, this probably means at least 30 at intermediate levels. So my best guess is that we will top Houdini 4 on two out of those 4 lists, as well as the 40/2hour list. Anyway, it should be extremely close. Houdini 4 does have the advantage on these lists of using high contempt to rack up big scores against weak opponents; the real test is how the top engines do against each other. I would bet money on Komodo 7 against Houdini 4 at any time limit of 30 minutes or longer plus one percent increment, on one or 4 cores, run on an Intel machine by a neutral party.
SF is really strong, but judging by performance of Komodo 7 in superfinal it's really hard to get away from feeling that Komodo is in superfinal by sheer luck.
Almost as if RH didn't want to do anything in Stage 4 (not even change contempt) in order to avoid exactly this type of humiliation in superfinal that Komodo 7 is experiencing atm.
lkaufman wrote:I'm running matches on three Windows 7/8 Intel computers with a total of 40 cores between Komodo 7a (released this morning) and Houdini 4, single core games. Most of the games are at 5' + 3" except the games on the quad are at 2' + 1". Five man Syzygy for both, ponder off, book derived from TWIC high-level games, variable length.
Results after 1552 games are a virtual tie; Komodo is just 7 games ahead. Since it is clear that Komodo scales better with more time than Houdini, I think it is fair to say that we have finally surpassed Houdini in strength at most reasonable time controls, at least on Intel machines. Now there remains only Stockfish to surpass, a tall order to be sure.
Since in IPON, which is due to ponder almost 2x 5'+3'', Komodo 7 is not more than 15Elo better than Komodo TCEC and at least 45Elo weaker than H4, what you publish here is nothing but flagrant commercial exhortation i.e. pure advertising (which is btw. against the charter).
lkaufman wrote:I'm running matches on three Windows 7/8 Intel computers with a total of 40 cores between Komodo 7a (released this morning) and Houdini 4, single core games. Most of the games are at 5' + 3" except the games on the quad are at 2' + 1". Five man Syzygy for both, ponder off, book derived from TWIC high-level games, variable length.
Results after 1552 games are a virtual tie; Komodo is just 7 games ahead. Since it is clear that Komodo scales better with more time than Houdini, I think it is fair to say that we have finally surpassed Houdini in strength at most reasonable time controls, at least on Intel machines. Now there remains only Stockfish to surpass, a tall order to be sure.
Since in IPON, which is due to ponder almost 2x 5'+3'', Komodo 7 is not more than 15Elo better than Komodo TCEC and at least 45Elo weaker than H4, what you publish here is nothing but flagrant commercial exhortation i.e. pure advertising (which is btw. against the charter).
That figure (15 elo) must have been based on quite few games; it's now 22 elo and still test is only about 25% done. Moreover the elo duiring the test often underestimates the final elo which is calculated differently. Our estimate of 40 was partly based on a private test for us by Ingo of the version just before Komodo 7, with the only change being minor and safe. Perhaps that version was a bit lucky in Ingo's test, but I'll be disappointed if the IPON final figure is below thirty elo gain. We'll still be below Houdini 4 on IPON due to Houdini's contempt helping against weaker opponents and to IPON using AMD since Komodo does noticeably better compared to Houdini on Intel than on AMD overall.
lkaufman wrote: Houdini 4 does have the advantage on these lists of using high contempt to rack up big scores against weak opponents; the real test is how the top engines do against each other. I would bet money on Komodo 7 against Houdini 4 at any time limit of 30 minutes or longer plus one percent increment, on one or 4 cores, run on an Intel machine by a neutral party.
Hello Larry,
Today the new Komodo 7a will replace Komodo TCECr in my Endless-RoundRobin tournament.
There are only 3 strong opponents and the timecontrol is 15'+3'' (and all engines running on 4 cores, not singlecore (=around 45-55 minutes + 10'' on singlecore)) - so this is high quality chess and Komodo 7a will play this tournament until a new Komodo is released. Around 40 games per day are played. So in a few weeks, we will see, how strong Komodo 7a really is with enough thinking time, 4 cores and against only strong opponents on an intel-machine. And the results are getting more and more valid, the longer the tournament runs.
Will Komodo 7a really surpass Houdini 4? We will see. But Stockfish is out of reach at the moment, I believe...
lkaufman wrote: We'll still be below Houdini 4 on IPON due to Houdini's contempt helping against weaker opponents and to IPON using AMD since Komodo does noticeably better compared to Houdini on Intel than on AMD overall.
Apologies for my bad memory Larry... what tests have been run to show that on a comparable basis ?
lkaufman wrote: We'll still be below Houdini 4 on IPON due to Houdini's contempt helping against weaker opponents and to IPON using AMD since Komodo does noticeably better compared to Houdini on Intel than on AMD overall.
Apologies for my bad memory Larry... what tests have been run to show that on a comparable basis ?
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
