Jouni wrote:Yes it's stunning, that SF has got only +23 after version 8, but Komodo and Houdini one/two man team +50
. Obviously SF team should look at Komodo/Houdini code now
. And I don't understand endless tuning of time management. Isn't it so easy thing once programmed no need to ever change?
1)Time management is not something that no need to ever change.
There are many possible ideas that may work or not work and people did not try all ideas.
2)It is not surprising that SF has got only +23 after version 8 and houdini got more than it.
The fact that is surprising for me is that so many people give computer time to stockfish when stockfish is optimized only for bullet and also does not test the real value of every patch they accept
SPRT does not give unbiased estimate and I think that it is better at least to test every patch that pass in a fixed number of games against previous version(let say 40000 games at STC and LTC and VLTC) let say 15+0.15 60+0.6 and 240+2.4
I think that it is better for computer chess if the stockfish team stop testing new patches in the near future and simply test patches that they accepted in the past against previous version with 40000 games in these time control so we can know better if there are patches that scale well(earn more elo at LTC).
If we find few patches that scale well out of many patches that do not scale well then I think that we should test them again in 100,000 games at all time controls to verify that we did not get a lucky run.
I believe that the knowledge that we can get by this process can help to improve stockfish and other programs in the future because people will have a better idea if a patch scale well or does not scale well before testing it.
Today people have almost no idea about the value of patches at different time control(number of games to pass SPRT means almost nothing).
I believe that it is bad for Komodo or Houdini and also bad for Stockfish because knowing the value of the patches of the past can help to suggest better patches in the future.