Building the Ultimate Chess Machine!

[M ANSARI]

This has also been posted in the General section ... I hope it is OK to post it here as well.

I have for some time now been thinking of a better way to utilize hardware to play chess. I am sure many of you have heard of clustering, where many computers are networked and the power of each of the individual clusters or computers work together to search deeper and thus play stronger than a standalone engine (recently with Toga cluster doing well). I like to call this active clustering. I have another idea which I think would be much easier to implement and would still be able to dramatically increase the strength of a chess engine. I like to call it passive clustering. Only I am not a programmer and do not have the knowledge necessary to do the software side. Therefore I would appreciate any help from the many gifted programmers here. I will be happy to financially compensate anyone who is qualified and interested. In the end if this goes through and a stable good software is developed I plan to have it available as a free download for everyone where they can add or improve on it. I am sure it would be a beneficial contribution to chess analysis. So if you are interested or can recommend someone please PM me for more details. The following is a basic idea of what I am thinking.

Basically the idea is that we have a master computer running a certain type of engine that would have the ability to share with several other slave computers a chess game. The other clusters or slave computers would be specialized in certain aspects of the game. These slave computers would be passive and have absolutely no interaction with the master computer unless a pre-determined threshold is reached. In many cases a change in chess parameters or chess knowledge will allow certain positions to be played perfectly, yet it would decrease the overall strength of the engine. A good example would be Rybka Winfinder … where it can find tactical shots maybe 10X faster than normal Rybka … yet it is weaker overall. Also many versions of Crafty can play certain problematic positions perfectly but somehow play much weaker overall.

First Generation: This would be basically a Master computer with a dedicated slave. This slave computer would act as the endgame module and have all the important EGTB’s. It would also have an engine tuned with all sort of knowledge for endgames. In this case merely offloading the task of EGTB’s should make the master computer play stronger … without risking a draw or loss if an EGTB hit is found.

Second Generation: This would introduce a very tactical engine that would look for tactical shots. Also different contempt factors could be experimented with. This would give the master engine the added ability to be tactically aware without sacrificing overall playing ability.

Third Generation: This would add a multitude of specific engines that can play certain problematic positions perfectly. This could be tested with different positional suites and adjust the number of computers accordingly.

Final Generation: This would add additional computers that would act similar to a cluster today. It would take the first 5 best moves that the master computer likes and each computer would take a line and investigate it thoroughly. This can obviously include an infinite number of computers as each individual slave could also have their own slaves. But I see that only as a theoretical idea and not really practical. Five computers working on the best 5 lines should be already quite good.

An obvious problem here would be the latencies involved in such a system since it would most likely be running on a wired LAN. Also the algorithm to decide what the threshold should be to override the main engine would have to be experimented with and most likely would have to be a dynamic algorithm which can be changed according to where the game has progressed. The most important thing would be to first of all make sure that this system does not lower the overall playing strength of the original engine and thus thresholds have to be conservative initially. But I think it should improve engine performance quite a bit.

The good thing about such a system would be the ability to have a trickle down effect to general engines as stand alone computers have access to more and more cores. Obviously if such a system can have reduced latencies by having all cores on one motherboard there will be a huge advantage. Also if hash in memory can be shared that would be a big boost … but I would hate to think of all the different bugs that would come out and thus I see this as a solution that will come gradually. Passive clustering is something that can be done today relatively easily with probably only the requirement of a driver and some communications code.

I have already approached Vas to do this but he is swamped with work prior to release of Rybka 3.0. I am sure we can talk Vas into providing the different experimental engines that might be useful in such a project.

So if you have the talent or know how or know someone who might be please PM me.