lucasart wrote:Uri Blass wrote:I do not agree with marco (the leader of stockfish) about a lot of things and one of the problems is that he seems to be too careful not to lose elo and even reject simplifications that are not proved to be bad because of unproved fear that they are 1 elo loss.
I wonder if it is possible to have a different leader with testers who support him to compete against stockfish of marco and accept changes relative to marco's version based on different rules.
1)Simplifications are accepted if they pass twice sprt(-4,0) in first try
and removing one line of if condition is certainly a simplification.
In order to accept other changes they need to pass SPRT(-1.5,4.5) at some time control and SPRT(0,6) at slower time control when the slower time control is not faster than 1 minute per game.
It may be possible also to use lower margin than 6 but the bottom line in the second test needs to be 0.
It does not mean that changes are going to be accepted if they pass and the leader has the freedom not to accept changes(for example if something similar was already tried many times) but tests that pass without being accepted may get a second chance(for example if some patch failed SPRT(0.6) and passed SPRT(0,4) at the same time control then it may be accepted if it pass again SPRT(0.4) without failures with SPRT(0,4))
2)People who commit patches have the freedom to push every patch that they want at every time control that they want and with the number of games that they want or with the SPRT parameters that they want.
3)People who give their machine for testing have the option to decide
which patch to test(they can choose default that is the option that the leader suggest but also can choose to increase or to reduce priority of every candidate to test when priority below -5 means not testing it even if there are no other tests) and I guess that if somebody give obviously bad patches that interest nobody then people including the leader may put them with priority below -5 so no waste of computer time on them.
You can create your own SF derivative, and be the team leader, and use whatever commit rules you want. But you're forgetting two very important things:
1/ how will all the testing be performed ? are you going to create your own fishtest equivalent ?
2/ how will you attract intelligent people to work with you ? generally smart people want to work with smart people, and that's why many of them like to work with Marco.
Note that I did not suggest myself as the leader for a reason.
The main problems is programming knowledge in order to create my own fishtest equivalent and a server running 24 hours a day (like Gary's one for the fishtest) that is mentioned as a problem by other people.
I also agree that people want to work with marco only because stockfish is the top freeware program and not because marco is smart.
marco with no doubt know more than me about programming but I doubt if he is more intelligent than me and I can see that he makes obviously wrong claims more than once.
Here is one example from the stockfish page:
http://abrok.eu/stockfish/
Author: Marco Costalba
Date: Wed Feb 12 14:16:21 2014 +0100
Timestamp: 1392210981
"Revert "Retire null search verification"
Although does not change ELO level, it seems
verification is useful in many zugzwang positions
as reported by many sources."
This is obviously wrong.
There is no proof that removing null search verification does not change elo level.
passing SPRT(-4,0) twice only shows no significant regression and does not show no improvement.
so the claim of marco is obviously wrong because no regression is not equivalent to no change.
Note that it is possible to justify the revert by other words(for example that playing strength is not the only important thing) but marco did not do it.
I do not think that a person who justifies his decisions by wrong claims in more than one case is smarter than me.