Houdini 5 in TCEC

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

mjlef
Posts: 1494
Joined: Thu Mar 30, 2006 2:08 pm

Re: Houdini 5 in TCEC

Post by mjlef »

gotogo wrote:It's possible Robert doesn't want anyone trying to tune against Houdini 5!
That might make things more interesting.
Komodo in the late stages has always tuned values against Stockfish.

Should be interesting even if Houdini 5 doesn't win!
The idea alone will make competitors nervous.
if by "tuned" you mean we just change Contempt, that is true. But that is a single value. We have lower Contempt as the average elo of our opponents gets closer to Komodo's. But say "always tuned values against Stockfish" is not true. I do not even know how to tune an engine against a specific opponent. I would say at least 98% of our testing is small changes in Komodo against itself. It takes more games to test against another engine. I am not saying we would not try to come up with values that would be better against a specific opponent, but I frankly know of no way to do this in any practical amount of time. If you do, please let us know how.

Mark
Top
Uri Blass
Posts: 10282
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Houdini 5 in TCEC

Post by Uri Blass »

gotogo wrote:It's possible Robert doesn't want anyone trying to tune against Houdini 5!
That might make things more interesting.
Komodo in the late stages has always tuned values against Stockfish.

Should be interesting even if Houdini 5 doesn't win!
The idea alone will make competitors nervous.
I would like to know if you know about a single case when A beat B
B beat C and C beat A in computer chess.

If not then probably there is no successful tuning in computer chess.

improving A of course help but it is not tuning.

tuning is something that help A to beat B without improvement so it still lose against C that is weaker than B.
Top
shrapnel
Posts: 1339
Joined: Fri Nov 02, 2012 9:43 am
Location: New Delhi, India

Re: Houdini 5 in TCEC

Post by shrapnel »

gotogo wrote:It's possible Robert doesn't want anyone trying to tune against Houdini 5!
That might make things more interesting.
Komodo in the late stages has always tuned values against Stockfish.

Should be interesting even if Houdini 5 doesn't win!
The idea alone will make competitors nervous.
Hmm... you may have something there, no matter the protestations....
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
Top
Henk
Posts: 7216
Joined: Mon May 27, 2013 10:31 am

Re: Houdini 5 in TCEC

Post by Henk »

And what about the posts a year ago that there would never be a Houdini 5.

Maybe create another department GossipChessClub.com.
Top
User avatar
reflectionofpower
Posts: 1610
Joined: Fri Mar 01, 2013 5:28 pm
Location: USA

Re: Houdini 5 in TCEC

Post by reflectionofpower »

Henk wrote:And what about the posts a year ago that there would never be a Houdini 5.

Maybe create another department GossipChessClub.com.
It also MY opinion that Bruce will come out with a version of ferret that will crush all comers beyond description.
"Without change, something sleeps inside us, and seldom awakens. The sleeper must awaken." (Dune - 1984)

Lonnie
Top
acase
Posts: 983
Joined: Wed Mar 15, 2006 1:14 am
Location: Columbus, Ohio USA
Full name: Andrew R Case

Re: Houdini 5 in TCEC

Post by acase »

Dan Cooper,


Did you get away with the loot after jumping out of the plane? :wink:
Top
supersharp77
Posts: 1242
Joined: Sat Jul 05, 2014 7:54 am
Location: Southwest USA

Re: Houdini 5 in TCEC

Post by supersharp77 »

JohnW wrote:Glad to see Houdini 5 is real..
http://www.chessdom.com/houdini-5-to-pa ... -season-9/
DustyMonkey wrote:People seem to have forgotten that this is operating procedure for Houdini.
TCEC Season 5 began with H3.
H4 officially released prior to the season 5 superfinal (which was SF vs K.)
The real issue people should be wondering about.. is also in evidence with season 5, as H4 turned out to only be a tiny bit stronger than H3.
Yes friends a very important development indeed! Houdini 5...!!
Thanks to Fritz 15 (Rybka) and the rumours of a soon to be released
Hiarcs 15 and Shredder 13 for leading to the Houdini announcement..
my tests show houdini 4 as slightly better than the strong Houdini 3 so Houdini 5 should be slightly better than the stong Houdini 4..should be a interesting summer!! AR :) :wink:
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Houdini 5 in TCEC

Post by bob »

Uri Blass wrote:
gotogo wrote:It's possible Robert doesn't want anyone trying to tune against Houdini 5!
That might make things more interesting.
Komodo in the late stages has always tuned values against Stockfish.

Should be interesting even if Houdini 5 doesn't win!
The idea alone will make competitors nervous.
I would like to know if you know about a single case when A beat B
B beat C and C beat A in computer chess.

If not then probably there is no successful tuning in computer chess.

improving A of course help but it is not tuning.

tuning is something that help A to beat B without improvement so it still lose against C that is weaker than B.
Are you talking about consistently (i.e. long matches where a beats b, b beats c and c beats a? Or a single tournament? The latter has happened multiple times. I can't say much about matches but it wouldn't surprise me a bit if it did happen.
Top
Uri Blass
Posts: 10282
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Houdini 5 in TCEC

Post by Uri Blass »

bob wrote:
Uri Blass wrote:
gotogo wrote:It's possible Robert doesn't want anyone trying to tune against Houdini 5!
That might make things more interesting.
Komodo in the late stages has always tuned values against Stockfish.

Should be interesting even if Houdini 5 doesn't win!
The idea alone will make competitors nervous.
I would like to know if you know about a single case when A beat B
B beat C and C beat A in computer chess.

If not then probably there is no successful tuning in computer chess.

improving A of course help but it is not tuning.

tuning is something that help A to beat B without improvement so it still lose against C that is weaker than B.
Are you talking about consistently (i.e. long matches where a beats b, b beats c and c beats a? Or a single tournament? The latter has happened multiple times. I can't say much about matches but it wouldn't surprise me a bit if it did happen.
I talk about consistently that mean a result that is significant statistically and not because of specific opening books but in the way that people test in tournaments like TCEC(fixed opening that the opponents do not know in advance).
Top
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: Houdini 5 in TCEC

Post by IWB »

Uri Blass wrote:
I talk about consistently that mean a result that is significant statistically and not because of specific opening books but in the way that people test in tournaments like TCEC(fixed opening that the opponents do not know in advance).
The IPON has some examples (with fixed unknown openings, changing colors), the statistical significance might be the problem ... however, 220 games with your given conditions will be hard to find in another rating list.

Code: Select all

12) iCE 3.0          2908 :   3300 (+555,=1398,-1347),  38.0 %

    vs.                    :  games (   +,    =,    -),   (%) :   Diff,   SD, CFS (%)
    Komodo 9.42            :    220 (   3,   51,  166),  13.0 :   -330,    7,    0.0
    Stockfish 7            :    220 (   3,   63,  154),  15.7 :   -326,    7,    0.0
    Houdini 4              :    220 (  24,   64,  132),  25.5 :   -215,    6,    0.0
    Gull 3                 :    220 (  19,   91,  110),  29.3 :   -165,    6,    0.0
    Ginkgo 1.7             :    220 (  26,   98,   96),  34.1 :   -119,    6,    0.0
    Equinox 3.30           :    220 (  28,  109,   83),  37.5 :    -98,    6,    0.0
    Fritz 15               :    220 (  28,  112,   80),  38.2 :    -97,    6,    0.0
    Critter 1.6a           :    220 (  37,   89,   94),  37.0 :    -85,    6,    0.0
    Andscacs 0.86          :    220 (  44,  107,   69),  44.3 :    -58,    6,    0.0
    Protector 1.9.0        :    220 (  44,   98,   78),  42.3 :    -33,    6,    0.0
    Nirvanachess 2.2       :    220 (  46,  110,   64),  45.9 :    -19,    6,    0.1
    Jonny 7.01             :    220 (  59,  105,   56),  50.7 :     +6,    6,   81.3
    Texel 1.05             :    220 (  58,   97,   65),  48.4 :    +12,    6,   97.5
    Chiron 2               :    220 (  68,   99,   53),  53.4 :    +22,    6,  100.0
    Naum 4.6               :    220 (  68,  105,   47),  54.8 :    +43,    6,  100.0

13) Jonny 7.01       2902 :   3300 (+559,=1342,-1399),  37.3 %

    vs.                    :  games (   +,    =,    -),   (%) :   Diff,   SD, CFS (%)
    Komodo 9.42            :    220 (   5,   49,  166),  13.4 :   -336,    7,    0.0
    Stockfish 7            :    220 (   0,   46,  174),  10.5 :   -332,    7,    0.0
    Houdini 4              :    220 (  17,   66,  137),  22.7 :   -221,    6,    0.0
    Gull 3                 :    220 (  18,   87,  115),  28.0 :   -170,    6,    0.0
    Ginkgo 1.7             :    220 (  25,  103,   92),  34.8 :   -125,    6,    0.0
    Equinox 3.30           :    220 (  29,   92,   99),  34.1 :   -104,    6,    0.0
    Fritz 15               :    220 (  31,   94,   95),  35.5 :   -103,    6,    0.0
    Critter 1.6a           :    220 (  34,   96,   90),  37.3 :    -91,    6,    0.0
    Andscacs 0.86          :    220 (  47,   97,   76),  43.4 :    -64,    6,    0.0
    Protector 1.9.0        :    220 (  43,  115,   62),  45.7 :    -39,    6,    0.0
    Nirvanachess 2.2       :    220 (  45,  101,   74),  43.4 :    -24,    6,    0.0
    iCE 3.0                :    220 (  56,  105,   59),  49.3 :     -6,    6,   18.7
    Texel 1.05             :    220 (  62,  104,   54),  51.8 :     +6,    6,   85.0
    Chiron 2               :    220 (  72,   92,   56),  53.6 :    +17,    6,   99.6
    Naum 4.6               :    220 (  75,   95,   50),  55.7 :    +37,    6,  100.0

14) Texel 1.05       2896 :   3300 (+530,=1345,-1425),  36.4 %

    vs.                    :  games (   +,    =,    -),   (%) :   Diff,   SD, CFS (%)
    Komodo 9.42            :    220 (   3,   44,  173),  11.4 :   -342,    7,    0.0
    Stockfish 7            :    220 (   1,   54,  165),  12.7 :   -338,    7,    0.0
    Houdini 4              :    220 (  11,   63,  146),  19.3 :   -227,    6,    0.0
    Gull 3                 :    220 (  16,   83,  121),  26.1 :   -177,    6,    0.0
    Ginkgo 1.7             :    220 (  22,   94,  104),  31.4 :   -131,    6,    0.0
    Equinox 3.30           :    220 (  28,   96,   96),  34.5 :   -110,    6,    0.0
    Fritz 15               :    220 (  32,   93,   95),  35.7 :   -109,    6,    0.0
    Critter 1.6a           :    220 (  35,   96,   89),  37.7 :    -97,    6,    0.0
    Andscacs 0.86          :    220 (  52,   88,   80),  43.6 :    -70,    6,    0.0
    Protector 1.9.0        :    220 (  44,  104,   72),  43.6 :    -45,    6,    0.0
    Nirvanachess 2.2       :    220 (  38,  111,   71),  42.5 :    -31,    6,    0.0
    iCE 3.0                :    220 (  65,   97,   58),  51.6 :    -12,    6,    2.5
    Jonny 7.01             :    220 (  54,  104,   62),  48.2 :     -6,    6,   15.0
    Chiron 2               :    220 (  64,  119,   37),  56.1 :    +10,    6,   95.6
    Naum 4.6               :    220 (  65,   99,   56),  52.0 :    +31,    6,  100.0
iCE won against Jonny, lost against Texel
Jonny lost against iCE won against Texel
Texel won against iCE, lost against Jonny


or

Code: Select all

 6) Equinox 3.30     3006 :   3300 (+868,=1654,-778),  51.4 %

    vs.                    :  games (   +,    =,   -),   (%) :   Diff,   SD, CFS (%)
    Komodo 9.42            :    220 (  12,   85, 123),  24.8 :   -232,    7,    0.0
    Stockfish 7            :    220 (   5,   93, 122),  23.4 :   -228,    7,    0.0
    Houdini 4              :    220 (  24,   92, 104),  31.8 :   -117,    6,    0.0
    Gull 3                 :    220 (  40,  110,  70),  43.2 :    -66,    6,    0.0
    Ginkgo 1.7             :    220 (  41,  128,  51),  47.7 :    -21,    6,    0.0
    Fritz 15               :    220 (  29,  144,  47),  45.9 :     +1,    6,   57.7
    Critter 1.6a           :    220 (  49,  139,  32),  53.9 :    +13,    6,   98.3
    Andscacs 0.86          :    220 (  66,  121,  33),  57.5 :    +40,    6,  100.0
    Protector 1.9.0        :    220 (  71,  114,  35),  58.2 :    +65,    6,  100.0
    Nirvanachess 2.2       :    220 (  69,  114,  37),  57.3 :    +79,    6,  100.0
    iCE 3.0                :    220 (  83,  109,  28),  62.5 :    +98,    6,  100.0
    Jonny 7.01             :    220 (  99,   92,  29),  65.9 :   +104,    6,  100.0
    Texel 1.05             :    220 (  96,   96,  28),  65.5 :   +110,    6,  100.0
    Chiron 2               :    220 (  83,  115,  22),  63.9 :   +120,    6,  100.0
    Naum 4.6               :    220 ( 101,  102,  17),  69.1 :   +141,    6,  100.0

 7) Fritz 15         3005 :   3300 (+886,=1607,-807),  51.2 %

    vs.                    :  games (   +,    =,   -),   (%) :   Diff,   SD, CFS (%)
    Komodo 9.42            :    220 (   5,   86, 129),  21.8 :   -233,    7,    0.0
    Stockfish 7            :    220 (   3,   83, 134),  20.2 :   -229,    7,    0.0
    Houdini 4              :    220 (  14,   90, 116),  26.8 :   -118,    6,    0.0
    Gull 3                 :    220 (  32,  120,  68),  41.8 :    -68,    6,    0.0
    Ginkgo 1.7             :    220 (  44,  111,  65),  45.2 :    -22,    6,    0.0
    Equinox 3.30           :    220 (  47,  144,  29),  54.1 :     -1,    6,   42.3
    Critter 1.6a           :    220 (  38,  138,  44),  48.6 :    +12,    6,   97.8
    Andscacs 0.86          :    220 (  67,  116,  37),  56.8 :    +39,    6,  100.0
    Protector 1.9.0        :    220 (  76,  115,  29),  60.7 :    +64,    6,  100.0
    Nirvanachess 2.2       :    220 (  82,  107,  31),  61.6 :    +78,    6,  100.0
    iCE 3.0                :    220 (  80,  112,  28),  61.8 :    +97,    6,  100.0
    Jonny 7.01             :    220 (  95,   94,  31),  64.5 :   +103,    6,  100.0
    Texel 1.05             :    220 (  95,   93,  32),  64.3 :   +109,    6,  100.0
    Chiron 2               :    220 ( 103,   92,  25),  67.7 :   +119,    6,  100.0
    Naum 4.6               :    220 ( 105,  106,   9),  71.8 :   +140,    7,  100.0

 8) Critter 1.6a     2993 :   3300 (+820,=1632,-848),  49.6 %

    vs.                    :  games (   +,    =,   -),   (%) :   Diff,   SD, CFS (%)
    Komodo 9.42            :    220 (   6,   75, 139),  19.8 :   -245,    7,    0.0
    Stockfish 7            :    220 (   4,   95, 121),  23.4 :   -241,    7,    0.0
    Houdini 4              :    220 (  18,  110,  92),  33.2 :   -130,    6,    0.0
    Gull 3                 :    220 (  28,  112,  80),  38.2 :    -79,    6,    0.0
    Ginkgo 1.7             :    220 (  39,  128,  53),  46.8 :    -34,    6,    0.0
    Equinox 3.30           :    220 (  32,  139,  49),  46.1 :    -13,    6,    1.7
    Fritz 15               :    220 (  44,  138,  38),  51.4 :    -12,    6,    2.2
    Andscacs 0.86          :    220 (  61,  105,  54),  51.6 :    +27,    6,  100.0
    Protector 1.9.0        :    220 (  71,  118,  31),  59.1 :    +52,    6,  100.0
    Nirvanachess 2.2       :    220 (  79,  102,  39),  59.1 :    +66,    6,  100.0
    iCE 3.0                :    220 (  94,   89,  37),  63.0 :    +85,    6,  100.0
    Jonny 7.01             :    220 (  90,   96,  34),  62.7 :    +91,    6,  100.0
    Texel 1.05             :    220 (  89,   96,  35),  62.3 :    +97,    6,  100.0
    Chiron 2               :    220 (  72,  123,  25),  60.7 :   +107,    6,  100.0
    Naum 4.6               :    220 (  93,  106,  21),  66.4 :   +128,    6,  100.0
Equinox lost against Fritz, won gainst Critter
Fritz won against Equinox, lost against Critter
Critter lost against Equinox, won against Fritz
(Not quite the same as the first example)

2 examples within only 16 engines ...

I agree that "statisticaly relvant" is the problem in my examples but seeing for years that engines perform differently against other engines I have no problem "believing" that such a uneven 3 way combination is possible even with enough games ...

And the winning percentage is not allways gaing straigt from low % to high % from top down, thare are jumps in which are quite big (winning % > vs a stronger opponent than vs a weaker opponent)

Bye
Ingo

PS: The full lists can be downloaded at IPON
Last edited by IWB on Sun Apr 17, 2016 9:35 am, edited 2 times in total.
Top

Return to “Computer Chess Club: General Topics”