nTCEC simulation

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: nTCEC simulation

Post by Don »

Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%

To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   2%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
Hi Kai,

I'm doing this for fun, so it will be fun to compare my results to yours too.

I'm going to extend my simulation to go to the stage 4 and then superfinal, but it's all subject to the error inherent in actually predicting how strong each of the program are - that is subject to a lot of guesswork - but it's fun anyway.

The superfinal winner will have a lot to do if whether Komodo really is stronger than Houdini under these exact conditions and to me that is a huge question mark. Both of us have upgraded our programs and Stockfish too is an unknown.

I set Komodo to Houdini + 10 just to be conservative - I do believe it is probably a little stronger under these conditions but even that is a guess.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: nTCEC simulation

Post by Laskos »

Don wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%

To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   2%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
Hi Kai,

I'm doing this for fun, so it will be fun to compare my results to yours too.

I'm going to extend my simulation to go to the stage 4 and then superfinal, but it's all subject to the error inherent in actually predicting how strong each of the program are - that is subject to a lot of guesswork - but it's fun anyway.

The superfinal winner will have a lot to do if whether Komodo really is stronger than Houdini under these exact conditions and to me that is a huge question mark. Both of us have upgraded our programs and Stockfish too is an unknown.

I set Komodo to Houdini + 10 just to be conservative - I do believe it is probably a little stronger under these conditions but even that is a guess.

Don
I too take a lot of informed guesses, by the way, the same 10 ELO points at this TC and hardware for Komodo above Houdini. In some aspects I don't think that we are wrong by more than 10 ELO points 2SD, in other maybe 20 points or so. But it's fun, and let's see, SF is downgraded for Stage 4 and Superfinal predictions because it yet has 40-50% to not qualify to Stage 4. But if it qualifies, it will be a worthy adversary in higher stages.
Last edited by Laskos on Mon Oct 28, 2013 6:27 pm, edited 1 time in total.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: nTCEC simulation

Post by Milos »

Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%
To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   3%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
From your results if Komodo gets into the final it has 76% chance of winning it. That means it is on average 180Elo stronger than other engines. What a load of crap...
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: nTCEC simulation

Post by Laskos »

Milos wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%
To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   3%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
From your results if Komodo gets into the final it has 76% chance of winning it. That means it is on average 180Elo stronger than other engines. What a load of crap...
Look silly, in 48 games even 20 points difference will reflect in high percentage of win. 53% for 20 points difference is for ONE game, silly.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: nTCEC simulation

Post by Don »

Laskos wrote:
Don wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%

To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   2%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
Hi Kai,

I'm doing this for fun, so it will be fun to compare my results to yours too.

I'm going to extend my simulation to go to the stage 4 and then superfinal, but it's all subject to the error inherent in actually predicting how strong each of the program are - that is subject to a lot of guesswork - but it's fun anyway.

The superfinal winner will have a lot to do if whether Komodo really is stronger than Houdini under these exact conditions and to me that is a huge question mark. Both of us have upgraded our programs and Stockfish too is an unknown.

I set Komodo to Houdini + 10 just to be conservative - I do believe it is probably a little stronger under these conditions but even that is a guess.

Don
I too take a lot of informed guesses, by the way the same 10 ELO points at this TC and hardware for Komodo above Houdini. In some aspects I don't think that we are wrong by more than 10 ELO points 2SD, in other maybe 20 points or so. But it's fun, and let's see, SF is downgraded for Stage 4 and Superfinal predictions because it yet has 40-50% to not qualify to Stage 4. But if it qualifies, it will be a worthy adversary in higher stages.
I think SF has just had a unfortunate bad result, I don't think there is anything wrong with the version nor do I doubt it's strength. That's just my opinion, but many of these games hang on a simple mistakes made by one side or the other. It could just as easily have been Komodo that had some bad fortune. Even Houdini seemed to be struggling but has roared back - such is how these competitions go. Komodo could easily suffer a loss, perhaps to Stockfish as it will be facing Stockfish with the black pieces soon.

Another potential issue with SF is that it's strength has been over-hyped so the expectations have been set ridiculously high, making it seem like something is horribly wrong if it doesn't completely dominate Houdini and Komodo.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: nTCEC simulation

Post by Milos »

Laskos wrote:
Milos wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%
To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   3%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
From your results if Komodo gets into the final it has 76% chance of winning it. That means it is on average 180Elo stronger than other engines. What a load of crap...
Look silly, in 48 games even 20 points difference will reflect in high percentage of win. 53% for 20 points difference is for ONE game, silly.
There is zero evidence Komodo is stronger even than H3, so assuming 20Elo between Komodo and Hdev is pure nonsense.
I understand that ppl have a lot of pity for Don's illness, but this is just becoming ridiculous.

P.S. Just quote me please 2SD error bars for 48 games with 70% draw probability :lol:
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: nTCEC simulation

Post by Laskos »

Milos wrote:
Laskos wrote:
Milos wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%
To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   3%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
From your results if Komodo gets into the final it has 76% chance of winning it. That means it is on average 180Elo stronger than other engines. What a load of crap...
Look silly, in 48 games even 20 points difference will reflect in high percentage of win. 53% for 20 points difference is for ONE game, silly.
There is zero evidence Komodo is stronger even than H3, so assuming 20Elo between Komodo and Hdev is pure nonsense.
I understand that ppl have a lot of pity for Don's illness, but this is just becoming ridiculous.
1) I assumed on reasonable grounds that Komodo is 10 points above Houdini at this TC and hardware (before Don did it).
2) If Komodo reaches Superfinal, it will have only 53% or so to meet Houdini, therefore, it could well meet weaker opposition, and Komodo chances will improve in the Superfinal.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: nTCEC simulation

Post by Don »

Laskos wrote:
Milos wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%
To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   3%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
From your results if Komodo gets into the final it has 76% chance of winning it. That means it is on average 180Elo stronger than other engines. What a load of crap...
Look silly, in 48 games even 20 points difference will reflect in high percentage of win. 53% for 20 points difference is for ONE game, silly.
Milos is criticizing things he doesn't even understand. Your odds of winning a single game if you are 180 ELO stronger is about 74% - but winning a 48 game match if you are 180 stronger would be almost certain. This guy needs to go back to school or at least leave the statistics to people who know what they are talking about.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: nTCEC simulation

Post by Milos »

Laskos wrote:1) I assumed on reasonable grounds that Komodo is 10 points above Houdini at this TC and hardware (before Don did it).
2) If Komodo reaches Superfinal, it will have only 53% or so to meet Houdini, therefore, it could well meet weaker opposition, and Komodo chances will improve in the Superfinal.
1)You reasonable ground is just horse manure. Just changing default contempt of H3 in matches against Komodo and SF improves H3 score for 30+ Elo.
You assume that for more than a year Houdini didn't improve at all???
And that myth about Komodo and good scaling and H3 and bad scaling is just ridiculous. There is not a single shred of evidence about it except Don's shameful advertizing of his product on this forum.

2)If Komodo reaches final Houdini chance to reach final is certainly not 53% but higher. You should maybe go to basics and try to repeat stuff like a priori and a posteriori probabilities (or just go and try to figure out basic stuff like Monty Hall problem) ;).
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: nTCEC simulation

Post by Milos »

Don wrote:
Laskos wrote:
Milos wrote:
Laskos wrote:I can give my simulations for Stage 4 and the Superfinal, after 45 games in Stage 3:

To qualify for the Superfinal:

Code: Select all

Komodo:   62%
Houdini:  53%
SF:       27%
Rybka:    15%
Critter:  15%
Bouquet:  14%
Gull:     11%
Hiarcs:    1%
Naum:      1%
Junior:    0%
To win nTCEC:

Code: Select all

Komodo:   47%
Houdini:  30%
SF:       12%
Rybka:     3%
Critter:   3%
Bouquet:   3%
Gull:      2%
Hiarcs:    0%
Naum:      0%
Junior:    0%
From your results if Komodo gets into the final it has 76% chance of winning it. That means it is on average 180Elo stronger than other engines. What a load of crap...
Look silly, in 48 games even 20 points difference will reflect in high percentage of win. 53% for 20 points difference is for ONE game, silly.
Milos is criticizing things he doesn't even understand. Your odds of winning a single game if you are 180 ELO stronger is about 74% - but winning a 48 game match if you are 180 stronger would be almost certain. This guy needs to go back to school or at least leave the statistics to people who know what they are talking about.
Quote me first 2SD error bars for 48 games with 70% draw rate.
Don if anyone is here ignorant about statistics it is you. It is well known thing in this forum.