Daydreamer 1.7

[Graham Banks]

I'm not sure. I run most of my testing games at game in 10s or 1.0s+0.1s and haven't seen any time-outs since the 1.61 release, so I'm guessing it's a bad interaction between the engine and GUI. What interface are you using?

edit: somehow I'm sure the answer will be Arena. Nothing ever works right for me in Arena.

No problems with the 32-bit version running under ChessGUI.

[Aaron Becker]

After some experimentation, I'm relatively sure that this problem is caused by a bad interaction with the Windows pthreads library that my Gaviota tablebase support relies on. When you saw the losses on time, was gtb support turned on? If so, please try turning it off for now and see if that fixes the problem. If not, please let me know so that I'll know to keep looking for the problem.

Assuming I'm correct about the cause, turning off gtb support should eliminate any time losses for now. A proper fix will take a while for me to figure out how to achieve the results I want on Windows. I should also be clear that this is a problem with my own code, and doesn't indicate any problem with the gtb library.

[michiguel]

After some experimentation, I'm relatively sure that this problem is caused by a bad interaction with the Windows pthreads library that my Gaviota tablebase support relies on. When you saw the losses on time, was gtb support turned on? If so, please try turning it off for now and see if that fixes the problem. If not, please let me know so that I'll know to keep looking for the problem.

Assuming I'm correct about the cause, turning off gtb support should eliminate any time losses for now. A proper fix will take a while for me to figure out how to achieve the results I want on Windows. I should also be clear that this is a problem with my own code, and doesn't indicate any problem with the gtb library.

Hi Aaron,

Let me know if there is anything I should be paying attention to in the gtb library. Probably you already looked, but everything related to threads is contained in sysport/sysport.c. However, GTB is using in Windows only a mutex despite several things are defined. Sysport.c is a general file that I use to wrap everything that is not portable between Windows and Linux. Would that mutex have a bad interaction with a pthread in windows?

These are the wrappers GTB is using

Code: Select all

extern void mythread_mutex_init		(mythread_mutex_t *m) { *m = CreateMutex(0, FALSE, 0)      ;}
extern void mythread_mutex_destroy	(mythread_mutex_t *m) { CloseHandle(*m)                    ;}
extern void mythread_mutex_lock     (mythread_mutex_t *m) { WaitForSingleObject(*m, INFINITE)  ;}
extern void mythread_mutex_unlock   (mythread_mutex_t *m) { ReleaseMutex(*m)                   ;}

but if you define POSIX_THREADS, this is what it will use

Code: Select all

extern void mythread_mutex_init		(mythread_mutex_t *m) { pthread_mutex_init   (m,NULL);}
extern void mythread_mutex_destroy	(mythread_mutex_t *m) { pthread_mutex_destroy(m)     ;}
extern void mythread_mutex_lock     (mythread_mutex_t *m) { pthread_mutex_lock   (m)     ;}
extern void mythread_mutex_unlock   (mythread_mutex_t *m) { pthread_mutex_unlock (m)     ;}

Miguel

[Aaron Becker]

The gtb libraries definitely aren't at fault. You're doing things the safe way, which is to just use Windows API functions for synchronization. Regardless of whether gtb uses pthreads or Windows mutexes, nothing in your library should cause any problems.

I'm relying on some less common pthreads functionality in my thread pool implementation, like pthread condition variables, that don't have a direct windows analogue. I suspect that's where my problem is coming from, although I haven't confirmed it yet. Probably what I should do is come up with a Windows implementation that just directly uses the Win32 API to do all my threading-related tasks, but that will require some learning on my end to figure out what all the relevant functions are.

[Martin Thoresen]

I'm not sure. I run most of my testing games at game in 10s or 1.0s+0.1s and haven't seen any time-outs since the 1.61 release, so I'm guessing it's a bad interaction between the engine and GUI. What interface are you using?

edit: somehow I'm sure the answer will be Arena. Nothing ever works right for me in Arena.

Hello Aaron,

I am not using Arena but Chessbase 12. Your Daydreamer 1.61 works just fine, but when testing 1.7 JA x64 for the upcoming Swiss Open it had
several losses on time.

I will try to disable Gaviota tablebases to see if that is somehow making the engine non-responsive.

Best Regards,
Martin

[Aaron Becker]

Thanks for the update, Martin. Another thing worth trying is keeping tablebase support turned on, but decreasing the "Endgame database thread pool size" variable from 2 to 1. This eliminated all the time losses in my tests, even with very fast time controls, while still allowing tablebase access. This might be very system-dependent, though. Sorry for the trouble with this; I was really hoping this would be a bug-free release.

[Martin Thoresen]

Thanks for the update, Martin. Another thing worth trying is keeping tablebase support turned on, but decreasing the "Endgame database thread pool size" variable from 2 to 1. This eliminated all the time losses in my tests, even with very fast time controls, while still allowing tablebase access. This might be very system-dependent, though. Sorry for the trouble with this; I was really hoping this would be a bug-free release.

No worries Aaron, I'll play around a bit with the settings as soon as I'm home from work.

I'm sure it's only a minor detail that needs adjusting, or perhaps I've set some wrong parameters somehow.

Best Regards,
Martin

[michiguel]

Thanks for the update, Martin. Another thing worth trying is keeping tablebase support turned on, but decreasing the "Endgame database thread pool size" variable from 2 to 1. This eliminated all the time losses in my tests, even with very fast time controls, while still allowing tablebase access. This might be very system-dependent, though. Sorry for the trouble with this; I was really hoping this would be a bug-free release.

With only one HD having a pool size of 1 may be almost the same as 2, speed wise. Is that true? The bottleneck will the HD access.

Miguel

[michiguel]

The gtb libraries definitely aren't at fault. You're doing things the safe way, which is to just use Windows API functions for synchronization. Regardless of whether gtb uses pthreads or Windows mutexes, nothing in your library should cause any problems.

I'm relying on some less common pthreads functionality in my thread pool implementation, like pthread condition variables, that don't have a direct windows analogue. I suspect that's where my problem is coming from, although I haven't confirmed it yet. Probably what I should do is come up with a Windows implementation that just directly uses the Win32 API to do all my threading-related tasks, but that will require some learning on my end to figure out what all the relevant functions are.

That is the reason I avoided condition variables when I implemented the SMP version of Gaviota (Darn it! it would have been good to have them!). I could not find an analog in Windows XP (I think there is one in Vista, IIRC). I ended up using semaphores, which I found they are easy to port (at least there is a good correlation between the functionality of both). That is how I come up with the sysport.c file. It was a good learning experience for me (no comp. scientist is bothered by the messy design of the windows threads? or is it just me?)

Miguel

[Aaron Becker]

The gtb libraries definitely aren't at fault. You're doing things the safe way, which is to just use Windows API functions for synchronization. Regardless of whether gtb uses pthreads or Windows mutexes, nothing in your library should cause any problems.

I'm relying on some less common pthreads functionality in my thread pool implementation, like pthread condition variables, that don't have a direct windows analogue. I suspect that's where my problem is coming from, although I haven't confirmed it yet. Probably what I should do is come up with a Windows implementation that just directly uses the Win32 API to do all my threading-related tasks, but that will require some learning on my end to figure out what all the relevant functions are.

That is the reason I avoided condition variables when I implemented the SMP version of Gaviota (Darn it! it would have been good to have them!). I could not find an analog in Windows XP (I think there is one in Vista, IIRC). I ended up using semaphores, which I found they are easy to port (at least there is a good correlation between the functionality of both). That is how I come up with the sysport.c file. It was a good learning experience for me (no comp. scientist is bothered by the messy design of the windows threads? or is it just me?)

Miguel

There's a gain to be had by using multiple threads to decompress tb entries, but it's not a big gain in my testing.
You're definitely not alone in being bothered by the design of the windows threading/concurrency api. The design of the concurrency libraries for Vista and later is a lot better, but I'm not interested in giving up XP support, so that's not much help.