Ingo was kind enough to run a development version of Komodo against his standard list.
To be sure, his test is not very favorable to Komodo which excells at longer time controls and this is a blitz time control list.
Komodo beat every single program on the list including Houdini, but falls just short of Houdini due to the fact that Houdini does slightly better against weak program at this time control.
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
Don wrote:
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
I don't know if you ppl just pretend or like to stress only the facts the go into your benefit.
All the tests so far including short, medium and moderetly long TCs suggest both Komodo 6 and SFdev have 4% over default H3 in direct matches.
However H3 is with contempt 1. Depending on the average rating of the whole field this contempt brings more overall rating than what H3 looses to Komodo and SF.
If you ran the same matches with contempt 0 you'd see that H3 is 2-3% stronger than both latest SFdef and K6.
However, you prefer to pretend that your program is the strongest in direct matches with Houdini and than suggest some flaw or whatever in rating lists methods since H3 still has better rating.
Don wrote:
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
I don't know if you ppl just pretend or like to stress only the facts the go into your benefit.
All the tests so far including short, medium and moderetly long TCs suggest both Komodo 6 and SFdev have 4% over default H3 in direct matches.
However H3 is with contempt 1. Depending on the average rating of the whole field this contempt brings more overall rating than what H3 looses to Komodo and SF.
If you ran the same matches with contempt 0 you'd see that H3 is 2-3% stronger than both latest SFdef and K6.
However, you prefer to pretend that your program is the strongest in direct matches with Houdini and than suggest some flaw or whatever in rating lists methods since H3 still has better rating.
Thank you.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Don wrote:Ingo was kind enough to run a development version of Komodo against his standard list.
To be sure, his test is not very favorable to Komodo which excells at longer time controls and this is a blitz time control list.
Komodo beat every single program on the list including Houdini, but falls just short of Houdini due to the fact that Houdini does slightly better against weak program at this time control.
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
It looks like Jonny 6.00 and Hannibal 1.4a are now included if I am not wrong. Disregarding this issue, this development version of Komodo has earned around 24 Elo plus/minus uncertainties (around ± 14 Elo taking into account a difference between two normal distributions of 3036 ± 10 and 3060 ± 10, writing from memory) since version 5.1r2 or similar. Am I right?
Anyway, well done Komodo team! I wish Don a speedy recovery.
Don wrote:
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
I don't know if you ppl just pretend or like to stress only the facts the go into your benefit.
All the tests so far including short, medium and moderetly long TCs suggest both Komodo 6 and SFdev have 4% over default H3 in direct matches.
However H3 is with contempt 1. Depending on the average rating of the whole field this contempt brings more overall rating than what H3 looses to Komodo and SF.
If you ran the same matches with contempt 0 you'd see that H3 is 2-3% stronger than both latest SFdef and K6.
However, you prefer to pretend that your program is the strongest in direct matches with Houdini and than suggest some flaw or whatever in rating lists methods since H3 still has better rating.
I think you explained why Houdini does better against weak programs, it's probably the aggressive contempt factor. Komodo probably respects other programs way too much.
And I agree with you that Komodo is way too strong for Houdini to have contempt for it.
I doubt Ingo would run this test again as it costs him precious electricity which is expensive where he lives, but if I could convince him to do so do you believe setting Houdini to contempt zero will increase it's overall rating on this list?
Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Don wrote:
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
I don't know if you ppl just pretend or like to stress only the facts the go into your benefit.
All the tests so far including short, medium and moderetly long TCs suggest both Komodo 6 and SFdev have 4% over default H3 in direct matches.
However H3 is with contempt 1. Depending on the average rating of the whole field this contempt brings more overall rating than what H3 looses to Komodo and SF.
If you ran the same matches with contempt 0 you'd see that H3 is 2-3% stronger than both latest SFdef and K6.
However, you prefer to pretend that your program is the strongest in direct matches with Houdini and than suggest some flaw or whatever in rating lists methods since H3 still has better rating.
I think you explained why Houdini does better against weak programs, it's probably the aggressive contempt factor. Komodo probably respects other programs way too much.
And I agree with you that Komodo is way too strong for Houdini to have contempt for it.
I doubt Ingo would run this test again as it costs him precious electricity which is expensive where he lives, but if I could convince him to do so do you believe setting Houdini to contempt zero will increase it's overall rating on this list?
Don
I don't believe it would help H3 on Ingo's list, on the contrary. There are too many weak opponents (300Elo weaker) so high contempt there brings more points overall (more wins instead of draws) than what H3 looses against SF and Komodo (there it has rougly 6% of the games as losses that would be draws with contempt 0).
Don wrote:
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
I don't know if you ppl just pretend or like to stress only the facts the go into your benefit.
All the tests so far including short, medium and moderetly long TCs suggest both Komodo 6 and SFdev have 4% over default H3 in direct matches.
However H3 is with contempt 1. Depending on the average rating of the whole field this contempt brings more overall rating than what H3 looses to Komodo and SF.
If you ran the same matches with contempt 0 you'd see that H3 is 2-3% stronger than both latest SFdef and K6.
However, you prefer to pretend that your program is the strongest in direct matches with Houdini and than suggest some flaw or whatever in rating lists methods since H3 still has better rating.
I think you explained why Houdini does better against weak programs, it's probably the aggressive contempt factor. Komodo probably respects other programs way too much.
And I agree with you that Komodo is way too strong for Houdini to have contempt for it.
I doubt Ingo would run this test again as it costs him precious electricity which is expensive where he lives, but if I could convince him to do so do you believe setting Houdini to contempt zero will increase it's overall rating on this list?
Don
I don't believe it would help H3 on Ingo's list, on the contrary. There are too many weak opponents (300Elo weaker) so high contempt there brings more points overall (more wins instead of draws) than what H3 looses against SF and Komodo (there it has rougly 6% of the games as losses that would be draws with contempt 0).
So it's probably the case that Komodo would actually top this list if Houdini's contempt was zero. Houdini is optimized to do well on lists.
I'll see if Ingo is willing to run another test with contempt = 0 for Houdini.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Don wrote:
Here are the invidual guantlet results against the top few programs, note that Komodo wins almost 55% against Houdini 3 even at this blitze time control. We are now quite curious about how Houdini beats weaker programs more decisively than us.
I don't know if you ppl just pretend or like to stress only the facts the go into your benefit.
All the tests so far including short, medium and moderetly long TCs suggest both Komodo 6 and SFdev have 4% over default H3 in direct matches.
However H3 is with contempt 1. Depending on the average rating of the whole field this contempt brings more overall rating than what H3 looses to Komodo and SF.
If you ran the same matches with contempt 0 you'd see that H3 is 2-3% stronger than both latest SFdef and K6.
However, you prefer to pretend that your program is the strongest in direct matches with Houdini and than suggest some flaw or whatever in rating lists methods since H3 still has better rating.
I think you explained why Houdini does better against weak programs, it's probably the aggressive contempt factor. Komodo probably respects other programs way too much.
And I agree with you that Komodo is way too strong for Houdini to have contempt for it.
I doubt Ingo would run this test again as it costs him precious electricity which is expensive where he lives, but if I could convince him to do so do you believe setting Houdini to contempt zero will increase it's overall rating on this list?
Don
I don't believe it would help H3 on Ingo's list, on the contrary. There are too many weak opponents (300Elo weaker) so high contempt there brings more points overall (more wins instead of draws) than what H3 looses against SF and Komodo (there it has rougly 6% of the games as losses that would be draws with contempt 0).
So it's probably the case that Komodo would actually top this list if Houdini's contempt was zero. Houdini is optimized to do well on lists.
I'll see if Ingo is willing to run another test with contempt = 0 for Houdini.
I pretty sure RH has the same kind of setup with avarage opponent rating as in Ingo list or CCRL, and after he's satisfied with contempt 0 strength of the engine, he then optimizes contempt to provide highest rating and than normalizes that one to 1 .