TinyThread++

[Rein Halbersma]

But the Do It Yourself attitude will lose in the long run.

But quite the contrary! It's actually a HUGE win.
Those libraries you talk about were written by someone and that someone did it himself (themselves).

In the case of threading libraries, Boost.Thread was written by Anthony Williams, who literally wrote the book (and many parts of the Standard) on C++11 threading. I don't think many people can write parallel code themselves better than he can

How many times did you get a buggy new library version here and there (being afraid and reverting to old version), tons of warnings, how long did it take you to seamlessly integrate some third-party library?
Wait it won't compile on that platform, I have to wait for a new version...
Or they just completely revamped the whole interface, nice
So in the end you spend more time twiddling with third party code than writing your own.

If you ever ran into problems with third-party libraries (like zlib reading uninitialized memory etc. - but hey it's ok, they say, it's for performance reasons , you know what I'm talking about.

Well about the entire world of GitHub / SourceForge etc., you are absolutely right. But there are some high-quality libraries (Boost, POCO, Folly) that will be much better than I could ever write myself. And furthermore, because these libraries are so popular, they get tested extremely intensively, so that any remaining bugs are found and fixed much quicker than in your own code.

I'm not talking about writing your own operating system or compiler (I'm not that crazy - but there are people out there who could do that).
I'm talking about thinking carefully about how much dependencies you drag into your (very small if it's a chess engine) project.

Re: depencies. I decided very early on in my draughts project to treat Boost as a kind of pre-Standard Library. So I feel free to use almost everything from Boost (apart from the really scary stuff that I don't understand 100%, such as Proto, Fusion, Xpressive, Spirt etc.).

The actual Boost dependencies are quite small, however. I just did a check and it's mostly Iterator, Range and Operators to get nice classes and algorithms quickly, a little bit of MPL / TTI (used to much more, with C++14 constexpr there are better alternatives) to do some compile-time magic, and a lot of Test for my unit testing.

At the application level, I use Boost.Accumulators which is a real nice way of collecting search statistics without having to write your own boilerplate. I'm experimenting with Log, FileSystem and Asio (which are both likely to be standardized in C++17) to add logging, file I/O for tablebases and networking.

Last thing, what do you enjoy more? Programming or gluing third-party/middleware stuff together? I prefer the former.

This is a matter of taste of course, and I respect yours I actually appreciate high-quality low-level stuff as well, but I prefer to code at the domain-level, to put in AI knowledge and think about the general structure of algorithms, and to let libraries give me the high-quality implementation of those ideas.

E.g., I really hate the current std::thread low-levelness, and I would much prefer to just express the potential parallelism of an algorithm such as with the Cilk-style spawn keyword. Such a declarative programming style is only possible with a taks-based concurrency library that will take care of load-balancing and work-stealing behind my back. They are coming in C++17 hopefully. It's my private theory that algorithms like DTS are just a very low-level and intricate way of reimplementing optimal load-balancing schedulers that can be much easier created with work-stealing library frameworks like Cilk.

[syzygy]

Full control is an illusion at worst, and a very expensive dream at best.

It's reality and works very well for me My code does exactly what I want and exactly the way I want it to (read it does nothing unexpected behind the curtains).

So you have implemented a multithreaded search using only syscalls?

But I wonder why you equate "do it yourself" with "wrap OS functionality"... you also wrote your own smp-enabled OS?

[mar]

So you have implemented a multithreaded search using only syscalls?

But I wonder why you equate "do it yourself" with "wrap OS functionality"... you also wrote your own smp-enabled OS?

Of course, you don't have do be a genius to wrap os functionality,
That was the point. I don't need 90 megs of boost for that.
How many OSes have you written btw?

[bob]

Full control is an illusion at worst, and a very expensive dream at best.

It's reality and works very well for me My code does exactly what I want and exactly the way I want it to (read it does nothing unexpected behind the curtains).

So you have implemented a multithreaded search using only syscalls?

But I wonder why you equate "do it yourself" with "wrap OS functionality"... you also wrote your own smp-enabled OS?

I don't see any particular reason for the sarcasm. But to answer your question myself, YES I have versions of Crafty that do not use posix threads at all. You might look at the clone() system call which is just as easy to use as posix threads, and since I use my own spin-locks, those programs use no thread library at all.

Does it REALLY matter? Most C++ stuff is bloated. Most C11++ stuff is even more bloated. I don't see any negative regarding rolling your own thread stuff, it is not like using pthreads saves you hundreds (or even 10's) of lines of code-writing...

The clone stuff is in the 18.x versions on, until posix threads was finally debugged and stable enough to use... It was a giant mess at first, changing too frequently to be usable.

[mar]

Fair enough Rein. You know I have a lot of respect for you
even if we disagree in subtle details;)
Unfortunately some people mistake the teasing
with their own holy war, I'm not sure I want to be part of that:)

[Rein Halbersma]

Ha Martin, yes you are one of the good guys in my book too. I like having to think about your arguments. Indeed, language wars are not fun, and haters gonna hate etc. It's like mud wrestling a pig: you both get dirty but only the pig actually enjoys it.

[syzygy]

Full control is an illusion at worst, and a very expensive dream at best.

It's reality and works very well for me My code does exactly what I want and exactly the way I want it to (read it does nothing unexpected behind the curtains).

So you have implemented a multithreaded search using only syscalls?

But I wonder why you equate "do it yourself" with "wrap OS functionality"... you also wrote your own smp-enabled OS?

I don't see any particular reason for the sarcasm.

Interesting that you interpret it as sarcasm!!
Apparently you do not seriously believe either that Martin implemented a multithreaded search using only syscalls.

Please don't bother replying, thanks.

[syzygy]

So you have implemented a multithreaded search using only syscalls?

But I wonder why you equate "do it yourself" with "wrap OS functionality"... you also wrote your own smp-enabled OS?

Of course, you don't have do be a genius to wrap os functionality,
That was the point. I don't need 90 megs of boost for that.
How many OSes have you written btw?

You're not exactly answering my question.

I assume you are using pthreads (on Linux). I don't see why that should count as "do it yourself" if, say, using C++11 threads does not.

Maybe my assumption is wrong, but then you might want to clarify.

[bob]

Full control is an illusion at worst, and a very expensive dream at best.

It's reality and works very well for me My code does exactly what I want and exactly the way I want it to (read it does nothing unexpected behind the curtains).

So you have implemented a multithreaded search using only syscalls?

But I wonder why you equate "do it yourself" with "wrap OS functionality"... you also wrote your own smp-enabled OS?

I don't see any particular reason for the sarcasm.

Interesting that you interpret it as sarcasm!!
Apparently you do not seriously believe either that Martin implemented a multithreaded search using only syscalls.

Please don't bother replying, thanks.

Unfortunately for you, I DID believe him. It is trivial.

All one needs under linux is clone() and spin locks. As I said, look at later 18.x and into the 19.x versions of Crafty which did exactly that...

[mar]

You're not exactly answering my question.

I assume you are using pthreads (on Linux). I don't see why that should count as "do it yourself" if, say, using C++11 threads does not.

Maybe my assumption is wrong, but then you might want to clarify.

Of course I'm using pthreads everywhere except on Windows and I excpect C++11 threads to do exactly that so I don't see a difference here.
I don't like the way it's designed in C++11 but that just my opinion and I expect others to disagree. I don't have problem with that.
If I wrap the functionality then I don't really care and I can run on 98 or 11 compiler and it will work just fine.
Broken C++11 library implementations won't however (IIRC someone here had problems with that on Android) but it will sort itself out given some time.

However you seem to be misinterpreting what I call DIY.
If I write my own heap allocator, I call that DIY (and I don't need any 3rd party leak checker), sure it's not as good as jemalloc and others plus I don't have local per-thread pools and use global heap lock but it gets the job done and is pretty fast if there is no contention while trying to keep the fragmentation low.
If I write my own alignment-aware templated containers, I don't need to use stl or write custom allocators and I can even use realloc when growing a dynamic array. If I write my own inflate/deflate, I call it DIY. and so on and so on.
I never intended to write my own OS, even though I bet GDT/IDT/selectors still mean what they meant in old 386 times when we used to write dos extenders.
Entering protected mode was as simple as setting up and setting lsbit of CR0 (or MSW). And I still expect that on x86/x64, IRQ 0 is still timer and 1 still keyboard and that they still use out 20h/0a0h,20h to acknowledge hw interrupts. I still assume that CR3 is used as pointer to paging directory. I'm not interested in super low level like that anymore (even though having all physical memory for yourself was fun), modern OSes are way more complicated than that and I believe that people who write kernels are vastly more competent than I'll ever be.
When it comes to dependencies on 3rd party libraries however, I prefer zero if possible. This gives you more freedom but also more responsibility, naturally. That was the whole point.