Sven Schüle wrote:This incremental approach drastically reduces the number of games you need to run. It requires, though, to keep your testing conditions (TC, hardware etc.) unchanged. Each time you change the conditions this means you open a new game pool, simply a new directory to store the PGN.
How strict is this?
E.g. I have 4 computers that I can use. If I now run the same collection of engines on all computers, would that work? I think it would because if a computer is faster than the others, then all programs would benefit from that. Agreed?
Sven Schüle wrote:This incremental approach drastically reduces the number of games you need to run. It requires, though, to keep your testing conditions (TC, hardware etc.) unchanged. Each time you change the conditions this means you open a new game pool, simply a new directory to store the PGN.
How strict is this?
E.g. I have 4 computers that I can use. If I now run the same collection of engines on all computers, would that work? I think it would because if a computer is faster than the others, then all programs would benefit from that. Agreed?
I also let games run on more computers (they are comparable). But always keep the TC constanvt (1'+1"). When I release a new version, I cleanup games of the very old branches and only the newer (and better) remain in the pool.
Sven Schüle wrote:This incremental approach drastically reduces the number of games you need to run. It requires, though, to keep your testing conditions (TC, hardware etc.) unchanged. Each time you change the conditions this means you open a new game pool, simply a new directory to store the PGN.
How strict is this?
E.g. I have 4 computers that I can use. If I now run the same collection of engines on all computers, would that work? I think it would because if a computer is faster than the others, then all programs would benefit from that. Agreed?
I also let games run on more computers (they are comparable). But always keep the TC constanvt (1'+1"). When I release a new version, I cleanup games of the very old branches and only the newer (and better) remain in the pool.
Conditions should be comparable to use the same pool. When using computers A and B where B is "faster" on average by a factor K, and everything else can be considered equal, then I think TC(A) = K * TC(B) would roughly mean comparable conditions on A and B. You give more time per move on the slower computer so that the available number of CPU cycles per move on average remains about the same on A and B. I think CCRL uses some tool to determine the right TC to use for a given tester's hardware.