Distributed auto test

[Don]

There has been good response to my invitation to help test Komodo on the general topics forum.

I just wanted to say a bit about how the distributed tester works. It's basically database driven and works much like a job queuing system. You run a standalone client on your computer and it manages the games.

Whenever a binary is needed, the client pulls it from the server. When the client first starts up a test is run to estimate the speed of the computer under the expected load. The client allows you to specify how many CPU cores to keep occupied, and the test is based on this.

The tester is somewhat cross platform - it works on windows 64 bit and Linux 64 bit. For any tests I must provide a binary for each platform and the client knows which to get.

After the test is complete, the client asks for a game to run by basically popping it off a queue from the server. The server hosts a redis database which supports more data structures than just key/value pairs.

The game record specifies which binaries, options to use, opening to play and a Fischer time control. However the time control is modified by the client to adjust for the power of the computer system. This of course takes into consider how many CPU cores the user set it up to run. So on a quad a user might specify to run 4 cores. (I have found that it's actually slightly more efficient to over-specify, so on my quad I specify 6 cpu.)

When the game is complete, it is scored and with the gameID, game score and other information sent back to the server.

It's up to the server to keep the game queue full and manage all the other record keeping. Once a client grabs a game job, the server assumes it will get played, but on occasion it could get "orphaned" if a client disconnects. However is it possible to recover these lost games after enough time has passed. The server knows what it has scheduled and knows what games have been completed.

The redis database supports many kinds of transactional operations, such as pushing and popping lists, sets, hash tables and so on. This makes it relatively easy to build the tester.

The client is actually the most complex part. It is single threaded tcl, but it makes good use of tcl's event loop and does not use any blocking operations. The redis client library has a non-blocking mode, so that if a program responds to a move while the client is fetching a binary, there is no significant delay. The client must multiplex several games simultaneously and tcl is a good fit for this. However I plan to eventually build a java and/or C client but it's good to figure this out in a very high level language first. The tcl client is packaged up in executable form and comes as a single executable so that no runtime or installation hassle is involved.

This is still alpha quality but so far it's been relatively robust. It still needs some checking to deal with disonnects - I would like this to continue to run even with temporary or flaky network outages and that should be easily possible. At the moment it can falter due to this.

If anyone want to help me test Komodo, send me an email at drd@mit.edu

[BrandonSi]

Can you comment real quick on the communication used in this test? Is this all via standard HTTP/80? Any possibility for changing tcp port usage, etc..? I have one computer I have in mind that could be dedicated almost 24x7, but due to network configuration, it would need to go out port 8080, assuming you're using HTTP.

[Don]

Can you comment real quick on the communication used in this test? Is this all via standard HTTP/80? Any possibility for changing tcp port usage, etc..? I have one computer I have in mind that could be dedicated almost 24x7, but due to network configuration, it would need to go out port 8080, assuming you're using HTTP.

The only communication between client and server is via a redis database and port 8080 or 80 is not used. And it's initiated by the client anyway, so even if the redis port is blocked I think it will still work.

My internet provider blocks 80 and 8080 I think, but I can still use my brower, I just cannot set up a web server on those ports.

Actually, I changed providers recently so it's possible I have those ports, but it's my understanding most providers block the web server ports.

[Don]

The distributed testing is working great! And I want to thank the many volunteers who are letting me use their spare CPU cycles for testing!