Houdini 3 : +40 after 20.000 games

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

User avatar
Peter Skinner
Posts: 1763
Joined: Sun Feb 26, 2006 1:49 pm
Location: Edmonton, Alberta, Canada
Full name: Peter Skinner

Re: Houdini 3 : +40 after 20.000 games

Post by Peter Skinner »

MM wrote:I apologize to all if i am wrong but we are talking about 10 million games. If i had the time and the tools to run 10 millions game i could try all the settings that i want and after 13 months i cannot ''avoid'' to find a formula to gain at least 50 elo.

If he worked on the search, i hardly can believe that he needed so many games to achieve and to be sure of the improvement.
Best Regards
We run millions of games on the Crafty team to test out new ideas in the search and eval. It literally takes that many games to figure out if something is good or not in testing.

I've only played about with Houdini 3 for a few hours, but I get the increase is from more than just tuning. I would go so far to say the MP search has been improved as well.. significantly. There is definitely some added knowledge, as well as other speed improvements. Gaining 50+ elo on an already tuned 3000+ engine does _not_ come from simple tuning.

So I would have to say you are wrong.

Peter
I was kicked out of Chapters because I moved all the Bibles to the fiction section.
MM
Posts: 766
Joined: Sun Oct 16, 2011 11:25 am

Re: Houdini 3 : +40 after 20.000 games

Post by MM »

Peter Skinner wrote:
MM wrote:I apologize to all if i am wrong but we are talking about 10 million games. If i had the time and the tools to run 10 millions game i could try all the settings that i want and after 13 months i cannot ''avoid'' to find a formula to gain at least 50 elo.

If he worked on the search, i hardly can believe that he needed so many games to achieve and to be sure of the improvement.
Best Regards
We run millions of games on the Crafty team to test out new ideas in the search and eval. It literally takes that many games to figure out if something is good or not in testing.

I've only played about with Houdini 3 for a few hours, but I get the increase is from more than just tuning. I would go so far to say the MP search has been improved as well.. significantly. There is definitely some added knowledge, as well as other speed improvements. Gaining 50+ elo on an already tuned 3000+ engine does _not_ come from simple tuning.

So I would have to say you are wrong.

Peter
Hi Peter,

you can say it, i regard you very much, thanks for having added examples of your work, i don't know much about the way of testing, i thought some thousand games were enough but i trust you.

Thank you

Best Regards
MM
User avatar
Rolf
Posts: 6081
Joined: Fri Mar 10, 2006 11:14 pm
Location: Munster, Nuremberg, Princeton

Re: Houdini 3 : +40 after 20.000 games

Post by Rolf »

Graham Banks wrote:
Carlos Ylich wrote:
Graham Banks wrote:
Carlos Ylich wrote:This is what others think. Graham And you, what do you think? :twisted:
I am not a programmer.
The quote I provided is from a well respected programmer within the community. He is the author of Sloppy and the programmer of Cutechess-cli.
Ok Graham, Great
But, what is your opinion? You need to be a programmer to have an opinion about a program that is for sale?
What is your opinion?
I don't have an opinion. I just enjoy testing engines.
I'm not interested in computer chess politics anymore.
Exactly because you are just a tester, Graham, would you explain to me please, what such 40 Elo should mean in chess? Know what I mean? Where is the pointwith such a result with that TC? Does it mean something for the chess players? This isnt a political question.

I forgot another point and want to ask, for what Hardware these results are relevant? Could it come out that for 32 bit and only 2 cores on a laptop there might only be 4 points of Elo? Because why should I buy it in that case if only the stronger HW would profit? Logically then a real cluster would be even better but I also wouldnt install it on my little lap. Do you see my point?
Last edited by Rolf on Wed Oct 17, 2012 1:32 am, edited 1 time in total.
-Popper and Lakatos are good but I'm stuck on Leibowitz
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Houdini 3 : +40 after 20.000 games

Post by mwyoung »

mwyoung wrote:
Dr.Wael Deeb wrote:
Tennison wrote:Hash = 32MB
Time / Move = 10ms (+/- 1.70" per game)
Book = openings.epd
Threads = 1

1. Houdini 2.0c x64 8750.5/20000 (d=8.9 nps=1493097)
2. Houdini 3.0 x64 11249.5/20000 (d=9.2 nps=1564498)

Houdini 3.0 scores 56% and have a gain equal to 40 elo (+/- 4).

This time control is very short but I'll try later with longer time controls.
Again,my prediction of +40 Elo at best is becoming reality.....
Dr.D
You can stop breaking your arm patting yourself on your back, Ipon has Houdini 3 +50 Elo better. :)
Final Ipon update. Houdini 3. +65 :)
User avatar
Graham Banks
Posts: 41415
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: Houdini 3 : +40 after 20.000 games

Post by Graham Banks »

Rolf wrote:
Graham Banks wrote:
Carlos Ylich wrote:
Graham Banks wrote:
Carlos Ylich wrote:This is what others think. Graham And you, what do you think? :twisted:
I am not a programmer.
The quote I provided is from a well respected programmer within the community. He is the author of Sloppy and the programmer of Cutechess-cli.
Ok Graham, Great
But, what is your opinion? You need to be a programmer to have an opinion about a program that is for sale?
What is your opinion?
I don't have an opinion. I just enjoy testing engines.
I'm not interested in computer chess politics anymore.
Exactly because you are just a tester, Graham, would you explain to me please, what such 40 Elo should mean in chess? Know what I mean? Where is the pointwith such a result with that TC? Does it mean something for the chess players? This isnt a political question.

I forgot another point and want to ask, for what Hardware these results are relevant? Could it come out that for 32 bit and only 2 cores on a laptop there might only be 4 points of Elo? Because why should I buy it in that case if only the stronger HW would profit? Logically then a real cluster would be even better but I also wouldnt install it on my little lap. Do you see my point?
I'm not bothered about such things Rolf. I simply enjoy watching engines playing chess against each other. Very nerdy I know, but.......... :lol:
gbanksnz at gmail.com
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: Houdini 3 : +40 after 20.000 games

Post by Dr.Wael Deeb »

mwyoung wrote:
Dr.Wael Deeb wrote:
Tennison wrote:Hash = 32MB
Time / Move = 10ms (+/- 1.70" per game)
Book = openings.epd
Threads = 1

1. Houdini 2.0c x64 8750.5/20000 (d=8.9 nps=1493097)
2. Houdini 3.0 x64 11249.5/20000 (d=9.2 nps=1564498)

Houdini 3.0 scores 56% and have a gain equal to 40 elo (+/- 4).

This time control is very short but I'll try later with longer time controls.
Again,my prediction of +40 Elo at best is becoming reality.....
Dr.D
You can stop breaking your arm patting yourself on your back, Ipon has Houdini 3 +50 Elo better. :)
Indeed an impressive result.....
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….