Why are the Ippo derivative stronger than Stockfish?

[lkaufman]

It seems to me that both the Ippo family programs and SF use many similar ideas, and both are extremely well written and fast programs. Moreover SF had access to the Ippo open-source code for a long time now, and yet despite huge progress it still lacks behind all of these programs, though the gap is getting small. Our experience with Komodo is that SF ideas or something at least vaguely similar to them seem to work better for us than Ippo ideas, despite the fact that our eval is more like the Ippo family than like the SF eval, since I was responsible for the eval weights in both Rybka and Komodo. So I just cannot see any reason for the superiority of the Ippo programs over SF, and yet it's hard to argue with results. Any opinions out there?

[silentshark]

It seems to me that both the Ippo family programs and SF use many similar ideas, and both are extremely well written and fast programs. Moreover SF had access to the Ippo open-source code for a long time now, and yet despite huge progress it still lacks behind all of these programs, though the gap is getting small. Our experience with Komodo is that SF ideas or something at least vaguely similar to them seem to work better for us than Ippo ideas, despite the fact that our eval is more like the Ippo family than like the SF eval, since I was responsible for the eval weights in both Rybka and Komodo. So I just cannot see any reason for the superiority of the Ippo programs over SF, and yet it's hard to argue with results. Any opinions out there?

is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?

[mcostalba]

is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?

Beeeep !!! Sorry, Wrong Answer. Thanks for having tried !

Because this question has been, for some reason, posted on the programmers subforum I assume the author was interested to give some fact and receive some fact back. Amazingly currently I don't see neither

Anyhow I'll keep looking at this post in the remote case something semi-serious emerges....

[Joost Buijs]

is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?

Beeeep !!! Sorry, Wrong Answer. Thanks for having tried !

Why is this the wrong answer? I really think speed can be an issue here.

For instance lets compare Stockfish to Houdini and assume both engines do approximately the same work at each node. Probably most of the nodes counted will be quiescense nodes heavely influenced by the speed of the evaluation function.
On my core i7 980x Stockfish 2.01 JA runs at 7500 to 8000 knps while Houdini 1.5a runs at 15000 to 16000 knps. That means that Houdini does twice the amount of work in the same time. This can easely account for a difference of 70 ELO points.

[zamar]

It seems to me that both the Ippo family programs and SF use many similar ideas, and both are extremely well written and fast programs. Moreover SF had access to the Ippo open-source code for a long time now, and yet despite huge progress it still lacks behind all of these programs, though the gap is getting small. Our experience with Komodo is that SF ideas or something at least vaguely similar to them seem to work better for us than Ippo ideas, despite the fact that our eval is more like the Ippo family than like the SF eval, since I was responsible for the eval weights in both Rybka and Komodo. So I just cannot see any reason for the superiority of the Ippo programs over SF, and yet it's hard to argue with results. Any opinions out there?

Their search algorithms still differ a lot. IPP* has different pruning rules close to the leaves compared to Stockfish. You can't just blindly copy Rule X from IPP* to SF. The search is one picture. And we don't want to copy the whole search, it would just be pointless.

Anyway we got a lot of good ideas from IPP*:

- Singular extensions
- Aggressive LMR
- Aggressive neg. SEE moves pruning

[mcostalba]

Why is this the wrong answer? I really think speed can be an issue here.

For instance lets compare Stockfish to Houdini and assume both engines do approximately the same work at each node. Probably most of the nodes counted will be quiescense nodes heavely influenced by the speed of the evaluation function.
On my core i7 980x Stockfish 2.01 JA runs at 7500 to 8000 knps while Houdini 1.5a runs at 15000 to 16000 knps. That means that Houdini does twice the amount of work in the same time. This can easely account for a difference of 70 ELO points.

I don't know how houdini counts the nodes. You should compare with Ippolit.

Bare speed is at least equal: do a perft test to check speed of move generation functions. Above that we have the actual search that is at the same level and the evaluation that perhaps is slower (because we don't use lazy evaluation), but I don't expect by so much. Even a 10% faster evaluation per node on average (that is a lot !) it means a 3-4% faster engine (because evaluation accounts for no more then 35-40% of total used time) and this translates in less then 5 ELO due to only speed difference, but probably even less.

[bob]

is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?

Beeeep !!! Sorry, Wrong Answer. Thanks for having tried !

Why is this the wrong answer? I really think speed can be an issue here.

For instance lets compare Stockfish to Houdini and assume both engines do approximately the same work at each node. Probably most of the nodes counted will be quiescense nodes heavely influenced by the speed of the evaluation function.
On my core i7 980x Stockfish 2.01 JA runs at 7500 to 8000 knps while Houdini 1.5a runs at 15000 to 16000 knps. That means that Houdini does twice the amount of work in the same time. This can easely account for a difference of 70 ELO points.

It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.

[mcostalba]

It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.

Yes, even worse than that. If node count is the double and would be all due to different evaluation, because evaluation does not account for more then 40% of total time, it would mean that SF evaluation is 4X slower then Houdini !!

This is almost nonsense. What I think is:

1) You should compare single core nodes because according to different SMP schemes node count could seem different on multi thread.

2) You should compare apples with apples: I don't know how Houdini counts, I know SF has recently changed node accounting and this showed in a -20% reported speed, but it is only an artifact because actual engine speed of 2.0.1 is always the same of 1.9, actually even a bit faster.

3) We already know hardware POPCOUNT gives a 3-4% boost and the JA versions don't have it.

At the end I think speed difference, if any, is really a misleading factor in trying to understand the differences. I think post of Joona makes a lot more sense.

[Joost Buijs]

It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.

I don't know the innards of both engines very well, but I assumed the evaluation function of Stockfish and Houdini to be roughly of the same complexity.

[bob]

It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.

I don't know the innards of both engines very well, but I assumed the evaluation function of Stockfish and Houdini to be roughly of the same complexity.

Evaluation is not the only thing. What about the code used to deal with pruning, reductions and such. Some use simple rules. Some toss in an eval call, + possibly a SEE call. Making a single node more expensive, but possibly more accurate as well...