OliThink 5.9.5 is very small

[Ras]

This is a simple performance issue, it's somewhat slower even though it's hardy measurable.

If it's hardly measurable, the Elo loss might rather be the usual tournament noise because +/- sqrt(N) is what you should expect as error margin. If you start from the initial position and let both versions calculate to a certain depth so that the time needed is around 30 seconds, and you have no other applications running, what is the speed difference?

One thing that you can try in the allocation routine after the (successful) calloc() call:

• use memset() to fill the whole hashtable with anything non-zero, such as 42
• call __sync_synchronize()
• use memset() again to fill the hashtable with 0
• call sync_synchronize() again.

This forces the OS to actually blend in the pages which, right after calloc(), rather point to the zero page and are only blended in upon usage, which will be slower. The __sync_synchronise() calls commit that stuff to main memory, and they keep the compiler from optimising the redundant memset() calls away. This will not count against your thinking time because the GUI will be using CECP's ping and wait for a pong, which you will not answer until this initialisation is done.

[Dann Corbit]

Is there a C++11 atomic equivalent to: __sync_synchronize() because that one is GCC specific.

[Ras]

Is there a C++11 atomic equivalent to: __sync_synchronize() because that one is GCC specific.

Olithink is in C, and __sync_synchronize() isn't just GCC, but also Clang. For C++11, I'd look into std::atomic_thread_fence() without function argument, which issues a full memory fence.

[OliverBr]

This is a simple performance issue, it's somewhat slower even though it's hardy measurable.

If it's hardly measurable, the Elo loss might rather be the usual tournament noise because +/- sqrt(N) is what you should expect as error margin. If you start from the initial position...

I never start from the initial position. I am using pre-defined opening books with thousand of openings and let play ten-thousands of games.
I got a couple of tourneys with small negative ELOs with the dynamic allocation, then I changed to to malloc instead of calloc. It got better, this may be just accident. Of course those numbers are always within the error margin, the last one (c has dynamic allocation, I decided to keep it for 5.9.6) looks ok:

Code: Select all

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)     W     D     L  D(%)  CFS(%)
   1 OliThink 5.9.5c    :       3      6  4326.5    8581  50.4  2430  3793  2358  44.2      86
   2 OliThink 5.9.5b    :       0   ----  4254.5    8581  49.6  2358  3793  2430  44.2     ---

White advantage = 71.22 +/- 2.81
Draw rate (equal opponents) = 45.92 % +/- 0.56

On the other site, combining "generateNoisy" and "generateQuiet" on Version d didn't do any good for the perfomance. In each tourney I get an 7-8 points loss of ELO:

Code: Select all

   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)     W     D     L  D(%)  CFS(%)
   1 OliThink 5.9.5c    :       0   ----  5102.0   10000  51.0  3028  4148  2824  41.5     100
   2 OliThink 5.9.5d    :      -7      5  4898.0   10000  49.0  2824  4148  3028  41.5     ---

White advantage = 70.21 +/- 2.65
Draw rate (equal opponents) = 42.97 % +/- 0.51

OliThink 5.9.5d is over 50 lines smaller and it is more elegant, but is this worth -7 ELO?
What do you think?

If you are interested, here is the commit, that I haven't included in 5.9.6:

https://github.com/olithink/OliThink/co ... c39904c6bc

[Ras]

I never start from the initial position. I am using pre-defined opening books with thousand of openings and let play ten-thousands of games.

I was asking about a speed benchmark, not full games. If you start from the initial position and let both versions calculate to a certain depth so that the time needed (for this mini-benchmark!) is around 30 seconds, and you have no other applications running, what is the speed difference? Is the speed difference still there if you add the double init with memset() and __sync_synchronize() as I suggested?

then I changed to to malloc instead of calloc. It got better, this may be just accident.

Probably just noise, because once you have the memory, the main difference is that with malloc(), you have to initialise it manually - and that won't play a role once the game has started.

OliThink 5.9.5d is over 50 lines smaller and it is more elegant, but is this worth -7 ELO?
What do you think?

That's a question of values. I would take the 7 Elo over small and clean code any day, but small source code size isn't an engine objective for me.

[OliverBr]

I was asking about a speed benchmark, not full games. If you start from the initial position and let both versions calculate to a certain depth so that the time needed (for this mini-benchmark!) is around 30 seconds, and you have no other applications running, what is the speed difference?

All speed benchmarks do vary each time a couple of per cent, even there is no other big process running.

Is the speed difference still there if you add the double init with memset() and __sync_synchronize() as I suggested?

I initialize (clean) the hash anyway before each match.
There shouldn't be any double init, because the "memory" command in CECP is supposed to be only sent once at all.
I am using cutechess-cli for testing and it looks as it doesn't send it at all in the standard configuration.

That's a question of values. I would take the 7 Elo over small and clean code any day, but small source code size isn't an engine objective for me.

This is really a difficult question.
It's not only about size, it's also about elegance and effectiveness. There is a lot of duplicate code vanishing in the commit. Unfortunately the ELO difference is reproducible :/

[Ras]

All speed benchmarks do vary each time a couple of per cent, even there is no other big process running.

If the speed difference is not even measurable, it will not impact playing strength. That has to be statistical error noise. With equal hash size, the node counts etc. will be the same, or else there would be a bug somewhere.

There shouldn't be any double init, because the "memory" command in CECP is supposed to be only sent once at all.

Nothing of what I wrote had anything to do with double CECP commands.

It's not only about size, it's also about elegance and effectiveness.

I'd take "ugly but stronger" any day. If line count is an objective, and if the two routines are like 95% the same, you could of course write a parametrised macro and "instantiate" that two times. That depends on much much you'd hate preprocessor hacks.

[Dann Corbit]

All speed benchmarks do vary each time a couple of per cent, even there is no other big process running.

If the speed difference is not even measurable, it will not impact playing strength. That has to be statistical error noise. With equal hash size, the node counts etc. will be the same, or else there would be a bug somewhere.

There shouldn't be any double init, because the "memory" command in CECP is supposed to be only sent once at all.

Nothing of what I wrote had anything to do with double CECP commands.

It's not only about size, it's also about elegance and effectiveness.

I'd take "ugly but stronger" any day. If line count is an objective, and if the two routines are like 95% the same, you could of course write a parametrised macro and "instantiate" that two times. That depends on much much you'd hate preprocessor hacks.

I don't think either side of the coin is a slam dunk. There is a very old measure that is still true. 80% of the cost of software is maintenance. That is why small and elegant is good, unless it is so terse that the shrink is just a pissing contest (e.g. one letter variable and method names).

On the other hand, what is the goal of chess engines? To win games more often and to analyze better.

So pick your poison. Both of them have a bite, and both of them have a payoff.

[Damir]

OliThink 5.9.6

Thanks to the Author Oliver Brausch smile16

DOWNLOAD : http://brausch.org/home/chess/index.html

[Ras]

Hm besides the missing null pointer check in the memory allocation, 5.9.6 also has a memory leak because the old pointer is neither freed nor reallocated.