CEGT 40/20 update (May 20th 2007)

[Shaun]

OK, understood - Hiarcs has caught the 32-bit version of Rybka 1.0 beta

Steady progress - good to see!!!

Shaun

[Harvey Williamson]

Hi Werner,

I'll be following your quad testing with interest. Great to see. For dual engines you use twice the hash size than for single CPU engines. What policy have you settled on for your quad engines ?

sadly another group that tests engines+hash rather than engines.

Hi Harvey,

I thought we cleared this up?

As long as enought hash is allocated for a single cpu version giving more will not help (and for some engines hinder) - where as with say 4 cpus the hash will be used quicker so more is required.

Yes one could run 1 CPU 1GB hash v 4 CPUs 1GB hash but the extra hash for the single CPU version is not going to help - at least we extrapolated that for the 40/40 ccrl time control from the hundreds of games I ran at 40/4?

Shaun.

P.S. Are we going to see a H11.2 soon looking forward to testing it.

Hi Shaun,

Regardless of any results engines should be tested not engines + hash.

We will probably release an 11.2 soon but it will not be a major release. It will have MP support for > 4 CPUs and a few other minor changes.

Harvey

[Uri Blass]

Hi Werner,

I'll be following your quad testing with interest. Great to see. For dual engines you use twice the hash size than for single CPU engines. What policy have you settled on for your quad engines ?

sadly another group that tests engines+hash rather than engines.

Hi Harvey,

I thought we cleared this up?

As long as enought hash is allocated for a single cpu version giving more will not help (and for some engines hinder) - where as with say 4 cpus the hash will be used quicker so more is required.

Yes one could run 1 CPU 1GB hash v 4 CPUs 1GB hash but the extra hash for the single CPU version is not going to help - at least we extrapolated that for the 40/40 ccrl time control from the hundreds of games I ran at 40/4?

Shaun.

P.S. Are we going to see a H11.2 soon looking forward to testing it.

I think that every engine should have the same rights.
You may be right for most of the engines but I think that you cannot generalize about all of them.

There is no problem if you give more hash only to part of the single cpu engine based on request of the author.

Note that I do not ask to give movei more hash (movei does not use hash efficiently and it is possible that more hash is counter productive for it)

Note also that my guess is that at 40/40 it is logical to use hash that is 10 times bigger and I tend to believe that it is not the case

In 40/40 you use 256 mbyte hash so 40/4 should use 25.6 mbyte hash and I guess that you use more than it.

Uri

[Spock]

Well there are many ratings lists out there and they all do something slightly different. You decide which ones you like and that's it. Ignore the ones that you don't. All I can say is that you can't please all the people all the time.... I think you'd say the same thing of your Hiarcs customers as well Harvey..

For example, yes we and CEGT do test engines+hash essentially (although the hash effect is likely to be minuscule), but SSDF for example test engine+book. That is different from our philosophy but that's fine, it's their list and it's up to them

Anyway, I hope Werner answers my original question

[Shaun]

Regarding Hiarcs - Mark though that about 32mb was optimal for our 40/4 time control based on his testing at similar time controls although more hash should not hurt - or at least that is my recollection of his email to me - I can't check now as the PSU is bust on the PC with all my emails and I don't want to restore image backups on another box just to check email

For 40/4 our hash policy is 64-128mb per core and this is perhaps a little generous.

My (personal) ideal would be a huge amount of ram on all my machines and that I could allocate a large amount of hash to each engine and it would only allocate a sensible amount based on processsor speed and time control - although I doubt that happening on both counts.

Anyway from my own testing I am confident that Hiarcs performance is not impacted by our hash policy.

Shaun

[Harvey Williamson]

Regarding Hiarcs - Mark though that about 32mb was optimal for our 40/4 time control based on his testing at similar time controls although more hash should not hurt - or at least that is my recollection of his email to me - I can't check now as the PSU is bust on the PC with all my emails and I don't want to restore image backups on another box just to check email

For 40/4 our hash policy is 64-128mb per core and this is perhaps a little generous.

My (personal) ideal would be a huge amount of ram on all my machines and that I could allocate a large amount of hash to each engine and it would only allocate a sensible amount based on processsor speed and time control - although I doubt that happening on both counts.

Anyway from my own testing I am confident that Hiarcs performance is not impacted by our hash policy.

Shaun

But I think he would say give it 32 or = on 1 2 or 4 cores - not the way that you do it.

Anyway i guess you wont change so we will have to live with it.

[Spock]

Werner,

My apologies to you and the CEGT team that my innocent question resulted in all that has followed.

When we introduced quad engines we faced a dilemma as to what to do with hash sizes, and had a bit of discussion about it. I was just curious as to what you had decided to do

[Uri Blass]

Note also that my guess is that at 40/40 it is logical to use hash that is 10 times bigger and I tend to believe that it is not the case

In 40/40 you use 256 mbyte hash so 40/4 should use 25.6 mbyte hash and I guess that you use more than it.

Uri

I use 64-96 MB hash for 40/4 testing. I also make engines solve test suites during which I regularly check (when using UCI engines) to how much extent the hash is filled. I have found that in almost all positions it takes about 5 minutes to reach 100 %. With the 40/4 time control, the thinking time for a single move has never reached 2 minutes for me, meaning that the hash full indicator will usually remain well under the limit. Can the hit rate be higher with bigger hash if even the smaller hash is not full? Even if yes, I don't think it would matter much (up to now I have seen only one engine, Bringer, that displayed not only hash full but also hit rate values).

Now at 40/40 the average time for a move is 1 minute. Even if a move takes 10 minutes, probably it will not fill a 256 MB hash table. My conclusion is that no bigger hash is needed.

1)Note that test suites prove nothing because in games there are programs that can learn from previous searches.

2)For your question
programs can earn speed from more hash even when the hash is not full.

The point is that the fact that the size of the hash tables is X positions and you tried to store Y position in the hash when Y<X does not mean that no positions was rejected.

If you take 23 random people and you put everyone in a room based on his birthday you can expect at least one room to have more than one person with probability of more than 50% so at least one room will reject a person.

http://en.wikipedia.org/wiki/Birthday_paradox

You can do better by allowing 5 people to be in one room based on their birthday divided by 5 and have 73 rooms but even in this case you do not need 365 people(hash full) to have people rejected because the room that they should go already has 5 people.

The same is correct also for chess positions and even if hash entry is a room that can include more than one position then it still cannot include infinite number of positions so there are simply positions that are rejected
clearly before the hash is full.

Uri

[Werner]

Ok Ray,
here is a short answer:
When I bougt the Quad - only 1GB fast Ram modules have been availabe to a payable price. So the PC has 2 GB RAM. The result is: also quad engines get 512 MB each because not more Ram is available. And e.g. see what happens if DeepShredder 10 x64 has add. 400 MB for its shredderbases .
And inside Shredder GUI I could see thís is enough RAM for 40/20. There are no rules in CEGT for Quad engines but one rule says its possible to use less hash if the PC does not have enough RAM. Here is the original text from February last year:

"Hash given is usually 256 MB for each engine. Very few testers who have less RAM available are allowed to give 128 MB.
Deep versions: Deep Shredder 9. Deep Fritz 8, Deep Junior 9 and others are tested on dual machines using 2 CPU´s and 512 MB hash. There is an exception for Junior 9.003 using only 256 MB, because there seem to occur bugs when giving 512 MB to this one."