Houdini 2.0 running for the IPON

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: Houdini 2.0 running for the IPON

Post by IWB »

Hi Don,
Don wrote: Are these tables used in the tree search, or just to find the best move from the root position?

Or does it depend on the program?
If the Tables are used at root or in the search is completly up to the program! Right now I dont know of any prog. which uses the Tables only at root by default - except Critter (but I do not pay much attention as I strongly believe that it doesent matter much for gameplay and elo performance!). Of course some use it earlier, some later ...

But: I do not use Tablebases in the GUI. The Enigne has to support it or it has to run without TBs.

BYe
Ingo
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Houdini 2.0 running for the IPON

Post by lkaufman »

IWB wrote:
lkaufman wrote:
IWB wrote:Hi,

Nonetheless, I already started the run with the GTB (on one comp) for Houdini, but on request I use the standard version and this will be the Version I include in the IPON.

Most likely I am courious enough to repeat it a third time with GTB AND the PRO version to see if there is a difference.

Everything on start in a few minutes!

Bye
Ingo
Is there any measurable difference in speed (NPS or time to complete N ply) between the two versions on your hardware?
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Houdini 2.0 running for the IPON

Post by Laskos »

IWB wrote:Hi,

This morning I discovered that I forgot to include the Gaviota Tablebases. Even if I only use the 4pc version the current run is not according to my normal standard. As it is interesting to see if there is any difference I will finish this run and start it imediately again WITH 4pc GTB. The latter will be included in my list.

As a side effect we have a nice test about the influence of Tbs! :-)

Sorry for the inconvinience
Ingo
In 2,100 games there will be no side effect. In all my experience, usually TBases give from +3 to -3 Elo points, which are measurable in 30,000+ games with 2 standard deviations (95% confidence, I mean the sign).

Kai
ernest
Posts: 2041
Joined: Wed Mar 08, 2006 8:30 pm

Re: Houdini 2.0 running for the IPON

Post by ernest »

Hi Kai,

I figure maybe you have an answer to this question, pertaining to Elo calculation:

If you look at Ingo's result
http://forum.computerschach.de/cgi-bin/ ... 1#pid41321
can you explain why Houdini's calculated Elo (3016, resulting from Elostat or Bayeselo) differs so much from the average of the individual matches Elo (the so called Perfs, at right), which I calculated to be 3045 ?
Uri Blass
Posts: 10299
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Houdini 2.0 running for the IPON

Post by Uri Blass »

Note that I expect the performance average to be too high because performance is not linear.

An extreme example is that 100-0 and 50-50 is going to give an infinite performance for the 100-0 part so it is certainly higher average performance than
75-25 and 75-25

I still suspect that something is wrong with the result because the difference from 3016 to 3045 seems to me too high and houdini got higher performance than 3016 in 18 out of 21 matches based on the data.
Karol Majewski
Posts: 106
Joined: Sat Jul 09, 2011 4:18 pm

Re: Houdini 2.0 running for the IPON

Post by Karol Majewski »

Poor Strelka. It loses 0-40 so far :lol:
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: Houdini 2.0 running for the IPON

Post by IWB »

Hello Larry,
lkaufman wrote: Is there any measurable difference in speed (NPS or time to complete N ply) between the two versions on your hardware?
Here is the full 5 min output on one core on one of my computer for the IPON:

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Engine: Houdini 2.0 Pro (1024 MB)
by Robert Houdart

15/37  0:01   +0.17    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 Bf5 
                       5.Bd3 Bxd3 6.Qxd3 e6 7.Qb5 Bd6 
                       8.Qxb7 Nxd4 9.exd4 Bxf4 10.Qc6+ Nd7 
                       11.Nf3 (1.905.048) 1562 

16/37  0:01   +0.14    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 Bf5 
                       5.Bd3 e6 6.Bxf5 exf5 7.Nf3 Bb4 8.Qd3 Bxc3+ 
                       9.bxc3 Ne4 10.O-O h6 (2.913.631) 1563 

17/37  0:03   +0.13    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 Bf5 
                       5.Bd3 e6 6.Nf3 Bd6 7.Bxf5 exf5 8.Bg5 Bb4 
                       9.O-O Bxc3 10.Bxf6 Qxf6 11.bxc3 (5.813.902) 1581 

18/37  0:08   +0.13    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 Nc6 
                       5.Bd3 e6 6.Nf3 Bd6 7.Bxf5 exf5 8.Bg5 Bb4 
                       9.O-O Bxc3 10.Bxf6 Qxf6 11.bxc3 (12.899.751) 1570 

19/40  0:19   +0.20++  1.d4 (31.120.019) 1566 

19/40  0:20   +0.19    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 e6 
                       5.Bd3 Bb4 6.Bxf5 Bxc3+ 7.bxc3 exf5 
                       8.Qd3 O-O 9.Rb1 Qc8 10.Nf3 Nc6 
                       11.O-O Ne4 (32.621.938) 1567 

20/40  0:27   +0.13--  1.d4 Nf6 (43.466.031) 1573 

20/42  0:43   +0.13    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 e6 
                       5.Nf3 Bd6 6.Bd3 Bxf4 7.exf4 O-O 
                       8.O-O a6 9.Qd2 Qd6 10.a3 Bd7 11.h3 h6 (68.403.432) 1575 

21/44  1:12   +0.08    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 e6 
                       5.Bd3 Bxd3 6.Qxd3 Nc6 7.Nf3 Bd6 
                       8.O-O O-O 9.a3 a6 10.h3 h6 11.Ne5 Bxe5 
                       12.Bxe5 Nxe5 13.dxe5 (114.108.564) 1584 

22/46  2:03   +0.08    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 e6 
                       5.Bd3 Bxd3 6.Qxd3 Nc6 7.Nf3 Bd6 
                       8.O-O O-O 9.a3 a6 10.h3 h6 11.Ne5 Bxe5 
                       12.Bxe5 Nxe5 13.dxe5 (196.021.521) 1587 

22/47  2:48   +0.15++  1.e4 (268.090.578) 1592 

22/47  3:13   +0.17    1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bc5 
                       5.Nxe5 Nxe5 6.d4 a6 7.dxe5 axb5 
                       8.exf6 Qxf6 9.Nc3 O-O 10.Be3 Bxe3 
                       11.fxe3 Qg5 12.Qd3 d6 13.Nxb5 Qc5 
                       14.Nc3 (308.710.339) 1592 

23/47  4:22   +0.17    1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bc5 
                       5.Nxe5 Nxe5 6.d4 a6 7.dxe5 axb5 
                       8.exf6 Qxf6 9.Nc3 O-O 10.Be3 Bxe3 
                       11.fxe3 Qg5 12.Qd3 d6 13.Nxb5 Qc5 
                       14.Nc3 (418.427.093) 1595 

23/47  5:01   +0.17    1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bc5 
                       5.Nxe5 Nxe5 6.d4 a6 7.dxe5 axb5 
                       8.exf6 Qxf6 9.Nc3 O-O 10.Be3 Bxe3 
                       11.fxe3 Qg5 12.Qd3 d6 13.Nxb5 Qc5 
                       14.Nc3 (481.010.046) 1596 

best move: e2-e4 time: 5:01.500 min  n/s: 1.596.000  CPU 99.7%   n/s(1CPU): 1.600.802  nodes: 481.010.046 



rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Engine: Houdini 2.0 STD (1024 MB)
by Robert Houdart

15/37  0:01   +0.17    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 Bf5 
                       5.Bd3 Bxd3 6.Qxd3 e6 7.Qb5 Bd6 
                       8.Qxb7 Nxd4 9.exd4 Bxf4 10.Qc6+ Nd7 
                       11.Nf3 (1.905.048) 1588 

16/37  0:01   +0.14    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 Bf5 
                       5.Bd3 e6 6.Bxf5 exf5 7.Nf3 Bb4 8.Qd3 Bxc3+ 
                       9.bxc3 Ne4 10.O-O h6 (2.913.631) 1589 

17/37  0:03   +0.13    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 Bf5 
                       5.Bd3 e6 6.Nf3 Bd6 7.Bxf5 exf5 8.Bg5 Bb4 
                       9.O-O Bxc3 10.Bxf6 Qxf6 11.bxc3 (5.813.902) 1596 

18/37  0:07   +0.13    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 Nc6 
                       5.Bd3 e6 6.Nf3 Bd6 7.Bxf5 exf5 8.Bg5 Bb4 
                       9.O-O Bxc3 10.Bxf6 Qxf6 11.bxc3 (12.899.751) 1613 

19/40  0:19   +0.20++  1.d4 (31.120.019) 1610 

19/40  0:20   +0.19    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 e6 
                       5.Bd3 Bb4 6.Bxf5 Bxc3+ 7.bxc3 exf5 
                       8.Qd3 O-O 9.Rb1 Qc8 10.Nf3 Nc6 
                       11.O-O Ne4 (32.621.938) 1615 

20/40  0:26   +0.13--  1.d4 Nf6 (43.466.031) 1622 

20/42  0:42   +0.13    1.d4 Nf6 2.Nc3 d5 3.Bf4 Nc6 4.e3 e6 
                       5.Nf3 Bd6 6.Bd3 Bxf4 7.exf4 O-O 
                       8.O-O a6 9.Qd2 Qd6 10.a3 Bd7 11.h3 h6 (68.403.432) 1625 

21/44  1:10   +0.08    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 e6 
                       5.Bd3 Bxd3 6.Qxd3 Nc6 7.Nf3 Bd6 
                       8.O-O O-O 9.a3 a6 10.h3 h6 11.Ne5 Bxe5 
                       12.Bxe5 Nxe5 13.dxe5 (114.108.564) 1627 

22/46  2:00   +0.08    1.d4 Nf6 2.Nc3 d5 3.Bf4 Bf5 4.e3 e6 
                       5.Bd3 Bxd3 6.Qxd3 Nc6 7.Nf3 Bd6 
                       8.O-O O-O 9.a3 a6 10.h3 h6 11.Ne5 Bxe5 
                       12.Bxe5 Nxe5 13.dxe5 (196.021.521) 1627 

22/47  2:44   +0.15++  1.e4 (268.090.578) 1630 

22/47  3:09   +0.17    1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bc5 
                       5.Nxe5 Nxe5 6.d4 a6 7.dxe5 axb5 
                       8.exf6 Qxf6 9.Nc3 O-O 10.Be3 Bxe3 
                       11.fxe3 Qg5 12.Qd3 d6 13.Nxb5 Qc5 
                       14.Nc3 (308.710.339) 1630 

23/47  4:16   +0.17    1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bc5 
                       5.Nxe5 Nxe5 6.d4 a6 7.dxe5 axb5 
                       8.exf6 Qxf6 9.Nc3 O-O 10.Be3 Bxe3 
                       11.fxe3 Qg5 12.Qd3 d6 13.Nxb5 Qc5 
                       14.Nc3 (418.427.093) 1633 

23/50  5:01   +0.17    1.e4 e5 2.Nf3 Nc6 3.Bb5 Nf6 4.O-O Bc5 
                       5.Nxe5 Nxe5 6.d4 a6 7.dxe5 axb5 
                       8.exf6 Qxf6 9.Nc3 O-O 10.Be3 Bxe3 
                       11.fxe3 Qg5 12.Qd3 d6 13.Nxb5 Qc5 
                       14.Nc3 (493.019.042) 1636 

best move: e2-e4 time: 5:01.500 min  n/s: 1.636.000  CPU 99.8%   n/s(1CPU): 1.639.278  nodes: 493.019.042
The versions produce the same PV, the node count is slightly faster on the STD version (2.5%), but for whatever reason the last depth on the Pro version is doubled, while the standard goes a little deeper.

Overall I would say there are slight differences. Whats the impact of this is unknown.

Bye
Ingo
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: Houdini 2.0 running for the IPON

Post by IWB »

Hi

The calculation seems to be correct. The individual perfomances are calculated vs the former rating. E.G. 2800 of Shredder. while the overall performance is calculated vs the Elo average of all 21 opponents. This average is 2769 (2768.857 - in the given example)
80.67% vs a 2768Elo opponent is 3016 Elo.

What I want to say is, that the average of the individual perfomance is something completly different than the performance vs the average Elo.

Bye
Ingo
Uri Blass
Posts: 10299
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Houdini 2.0 running for the IPON

Post by Uri Blass »

I think that using the average is not correct calculation for rating.

An extreme example
A player with rating 2000 play against 2 opponents:

case 1:
both of the opponent have rating of 3000

The 2000 player can expect to get a score that is close to 0(less than 0.01 out of 2)

case 2:one of the opponent has a rating 2000 and one of the opponents has a rating of 4000

The player can expect to get a score that is close to 0.5 points out of 2.
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: Houdini 2.0 running for the IPON

Post by IWB »

Hello Uri,

I know that your mathematical skills are far beyond my capabilities but I knoww as well that the calculation done by the Shredder-Classic-GUI is allways right for Elostat and differs just slightly for BayesElo. In the current discussion it is even right for both. When I take the games and throw them into Bayes or Elostat I get 3016.

If this is "right" or not is an intersting discussion, but it is the way Elos are calculated. Nonetheless I still hope for something better which will be accepted ... ! :-)

Bye
Ingo