TCEC stage 3 , New Houdini starts with a bang

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
APassionForCriminalJustic
Posts: 415
Joined: Sat May 24, 2014 7:16 am

Re: TCEC stage 3 , New Houdini starts with a bang

Post by APassionForCriminalJustic » Sun Jul 24, 2016 5:02 am

mjlef wrote:
IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
Contempt = 7 is pretty important. We lowered the Contempt for this stage because the average opponent rating is higher than the last stage. Using a higher Contempt would have encouraged Komodo to try to avoid draws harder, and to keep more pieces on the board at some sacrifice of positional play. Dynamism also was wrong and our testing showed in Komodo 10.1, the higher setting was better.

TCEC has been very accurate at settings in the past, but we are all only human and some errors can occur. When they are found it should be corrected and replayed, which is what they did, and swiftly too.

There will be lots of games in this Stage.
People are just desperate for a houdini miracle... Houdini will fail like it did before with houdini 4... while Houdart was on vacation both Komodo and Stockfish walked away with the crown.

User avatar
Laskos
Posts: 9441
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Laskos » Sun Jul 24, 2016 5:05 am

IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
The values "Before" were close to to optimal. The ELO loss was no larger than 5-10 ELO points and hardly the reason for the lost game. Seems unfair to Houdini to replay the game.

carldaman
Posts: 1717
Joined: Sat Jun 02, 2012 12:13 am

Re: TCEC stage 3 , New Houdini starts with a bang

Post by carldaman » Sun Jul 24, 2016 5:13 am

Laskos wrote:
IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
The values "Before" were close to to optimal. The ELO loss was no larger than 5-10 ELO points and hardly the reason for the lost game. Seems unfair to Houdini to replay the game.
That was my reasoning as well.

User avatar
Laskos
Posts: 9441
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Laskos » Sun Jul 24, 2016 5:33 am

carldaman wrote:
Laskos wrote:
IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
The values "Before" were close to to optimal. The ELO loss was no larger than 5-10 ELO points and hardly the reason for the lost game. Seems unfair to Houdini to replay the game.
That was my reasoning as well.
To see how disproportionate the action to replay the game is, for at most 5-10 ELO points loss in ONE game, they changed the result in the whole Stage 3 by 1 point between Komodo and Houdini, and that in 56 games each amounts to 13 ELO points in the WHOLE stage.

Uri Blass
Posts: 8586
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Uri Blass » Sun Jul 24, 2016 7:46 am

Laskos wrote:
IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
The values "Before" were close to to optimal. The ELO loss was no larger than 5-10 ELO points and hardly the reason for the lost game. Seems unfair to Houdini to replay the game.
I do not think that we know the elo loss at long time control and there is no basis to claim that the wrong setting is not the reason for the lost game.

I see nothing unfair for Houdini.

Uri Blass
Posts: 8586
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Uri Blass » Sun Jul 24, 2016 7:51 am

APassionForCriminalJustic wrote:
mjlef wrote:
IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
Contempt = 7 is pretty important. We lowered the Contempt for this stage because the average opponent rating is higher than the last stage. Using a higher Contempt would have encouraged Komodo to try to avoid draws harder, and to keep more pieces on the board at some sacrifice of positional play. Dynamism also was wrong and our testing showed in Komodo 10.1, the higher setting was better.

TCEC has been very accurate at settings in the past, but we are all only human and some errors can occur. When they are found it should be corrected and replayed, which is what they did, and swiftly too.

There will be lots of games in this Stage.
People are just desperate for a houdini miracle... Houdini will fail like it did before with houdini 4... while Houdart was on vacation both Komodo and Stockfish walked away with the crown.
It is not clear that Houdini will fail and I do not think that you know what Robert Houdart did in the time since Houdini4.

The fact that he did not send a new version earlier does not mean that he had not a new version.

For example it is possible that he did not send a new version because he was not sure that his version is stable with 20 cores or because he knew it was not stable and there are bugs that he need to fix after changing the code in order to earn more from many cores.

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: TCEC stage 3 , New Houdini starts with a bang

Post by tpoppins » Sun Jul 24, 2016 10:34 am

APassionForCriminalJustic wrote:we all know that Komodo is stronger... get some brains...
Why run TCEC at all then? Let's just have Adam list the final rankings and save a lot of time, effort and electricity.

Branko Radovanovic
Posts: 59
Joined: Sat Sep 13, 2014 2:12 pm

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Branko Radovanovic » Sun Jul 24, 2016 10:38 am

Laskos wrote:To see how disproportionate the action to replay the game is, for at most 5-10 ELO points loss in ONE game, they changed the result in the whole Stage 3 by 1 point between Komodo and Houdini, and that in 56 games each amounts to 13 ELO points in the WHOLE stage.
If you're saying that changing the result in the whole stage by 13 Elo points is a big thing, you're contravening your own argument.

User avatar
Laskos
Posts: 9441
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Laskos » Sun Jul 24, 2016 10:42 am

Uri Blass wrote:
Laskos wrote:
IanO wrote:
carldaman wrote:What was wrong with Komodo's settings? People should at least be entitled to a proper explanation by the organizers.
The differences can be found in the initial game comment.

Before:

Code: Select all

BlackEngineOptions: Use Syzygy=true; Syzygy Probe Depth=2; Syzygy Probe Limit=6; Dynamism=110;
After:

Code: Select all

BlackEngineOptions: Contempt=7;
To summarize, removing custom Syzygy probe options and "Dynamism" (whatever that is), and adding Contempt 7.

It would be nice to hear from Mark what the effects of these changes are likely to be.
The values "Before" were close to to optimal. The ELO loss was no larger than 5-10 ELO points and hardly the reason for the lost game. Seems unfair to Houdini to replay the game.
I do not think that we know the elo loss at long time control and there is no basis to claim that the wrong setting is not the reason for the lost game.

I see nothing unfair for Houdini.
With your usual "we do not know anything" are you sure you usually see something at all? One can try to see things by a rough estimate. The drawelo here is about 250 ELO points, and if the differnce from best settings is 5 ELO points, then we have 98% that the wrong settings did not affect the result. Only if the wrong settings are affecting by an order of 250 ELO points Komodo, we can say that probably this was the cause for the loss. Is it plausible?

User avatar
Laskos
Posts: 9441
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: TCEC stage 3 , New Houdini starts with a bang

Post by Laskos » Sun Jul 24, 2016 10:46 am

Branko Radovanovic wrote:
Laskos wrote:To see how disproportionate the action to replay the game is, for at most 5-10 ELO points loss in ONE game, they changed the result in the whole Stage 3 by 1 point between Komodo and Houdini, and that in 56 games each amounts to 13 ELO points in the WHOLE stage.
If you're saying that changing the result in the whole stage by 13 Elo points is a big thing, you're contravening your own argument.
The point you apparently missed was that a 5 ELO points mistake in a SINGLE game resulted in a 13 ELO points mistake for the whole stage.

Post Reply