asmFish

[syzygy]

It would not be a superfluous test because Asmfish is not an asm-copy of Stockfish and the difference between them more then the speed. There was a real copy named pedantFish, but this version ended in the past.

The old asmFIsh was replaced with pedantFish when it was found that pedantFish was about 2 Elo better than asmFish (despite being very slightly slower in nps). asmFish has been functionally identical to Stockfish ever since. So the difference between asmFish and the corresponding version of Stockfish is speed and only speed.

The speed of PedantFish was about 10 % higher than Stockfish but this 10 % was important in very short games only.

Being 10% faster in nps is equivalent to having 10% more thinking time. This makes a difference at both short and long time controls.

Of course the Elo improvements made to Stockfish since September 2018 (or to whatever point asmFish has been updated) may well exceed the Elo gained by 10-25% more speed.

[syzygy]

I'm auditing the games now and something interesting is apparent. It was my perception while watching the games that while asmFish was 1,000-2,000 kN/s faster, SF seemed to search deeper. I just shrugged this notion off, thinking "That's impossible!" After all, asmFish IS Stockfish, just an outdated version that's much faster. So if one SF version is faster than the other, surely it's searching to greater overall depths too. But now after auditing the first three games (OK, I've got 97 more to go, so this data is preliminary), I have hard evidence supporting my perception.

So you have learned that the logic behind your "surely" is flawed.

So nps is not everything. The next thing you may learn is that a comparison of search depths reached by different engines is not everything either. (Yes, SF of today is a different engine than SF of last year.)

[corres]

...
asmFish has been functionally identical to Stockfish ever since. So the difference between asmFish and the corresponding version of Stockfish is speed and only speed.

If you are right what was the sense to make asmFish AND pedantFish simultaneously?

Being 10% faster in nps is equivalent to having 10% more thinking time. This makes a difference at both short and long time controls.

I agree.
But in an STC match the Elo difference becomes obvious after some tens games meanwhile in a LTC match you need many thousand games to see the difference. An average engine user surely does not play such a lot of games.

Of course the Elo improvements made to Stockfish since September 2018 (or to whatever point asmFish has been updated) may well exceed the Elo gained by 10-25% more speed.

I agree.

[Guenther]

...
asmFish has been functionally identical to Stockfish ever since. So the difference between asmFish and the corresponding version of Stockfish is speed and only speed.

If you are right what was the sense to make asmFish AND pedantFish simultaneously?

Being 10% faster in nps is equivalent to having 10% more thinking time. This makes a difference at both short and long time controls.

I agree.
But in an STC match the Elo difference becomes obvious after some tens games...

Huh? Still on the strange side...

[syzygy]

...
asmFish has been functionally identical to Stockfish ever since. So the difference between asmFish and the corresponding version of Stockfish is speed and only speed.

If you are right what was the sense to make asmFish AND pedantFish simultaneously?

asmFish included a few speed optimizations that caused node counts to be different from those of Stockfish because they caused some changes in move ordering.
I think the main differences were:
- the implementation in do_move() of castling, and
- the elimination of piece lists.

Because different node counts make it really difficult to find bugs in asmFish, asmFish's source also implemented SF's way of doing castling and piece lists. asmFish and pedantFish were "compiled" (assembled) from the same code, with either the asmFIsh or the pedantFish implementations for castling and piece lists enabled.

So asmFish development would have been impossible without pedantFish. But since the higher speed of asmFish did not translate into more Elo, there was no reason to keep it. Therefore asmFish was removed and pedantFish was renamed into asmFish.

Being 10% faster in nps is equivalent to having 10% more thinking time. This makes a difference at both short and long time controls.

I agree.
But in an STC match the Elo difference becomes obvious after some tens games meanwhile in a LTC match you need many thousand games to see the difference. An average engine user surely does not play such a lot of games.

I don't think so. STC matches are more noisy (= lower draw rate) than LTC matches, so you need more games rather than less. (Of course playing thousands of games is much easier at STC than at LTC.)

[corres]

...
asmFish has been functionally identical to Stockfish ever since. So the difference between asmFish and the corresponding version of Stockfish is speed and only speed.

If you are right what was the sense to make asmFish AND pedantFish simultaneously?

asmFish included a few speed optimizations that caused node counts to be different from those of Stockfish because they caused some changes in move ordering.
I think the main differences were:
- the implementation in do_move() of castling, and
- the elimination of piece lists.
Because different node counts make it really difficult to find bugs in asmFish, asmFish's source also implemented SF's way of doing castling and piece lists. asmFish and pedantFish were "compiled" (assembled) from the same code, with either the asmFIsh or the pedantFish implementations for castling and piece lists enabled.
So asmFish development would have been impossible without pedantFish. But since the higher speed of asmFish did not translate into more Elo, there was no reason to keep it. Therefore asmFish was removed and pedantFish was renamed into asmFish.

Being 10% faster in nps is equivalent to having 10% more thinking time. This makes a difference at both short and long time controls.

I agree.
But in an STC match the Elo difference becomes obvious after some tens games meanwhile in a LTC match you need many thousand games to see the difference. An average engine user surely does not play such a lot of games.

I don't think so. STC matches are more noisy (= lower draw rate) than LTC matches, so you need more games rather than less. (Of course playing thousands of games is much easier at STC than at LTC.)

Maybe it would be better to keep the name "pedantFish" - for the shake of unanimity, I think.

[Leo]

I'm auditing the games now and something interesting is apparent. It was my perception while watching the games that while asmFish was 1,000-2,000 kN/s faster, SF seemed to search deeper. I just shrugged this notion off, thinking "That's impossible!" After all, asmFish IS Stockfish, just an outdated version that's much faster. So if one SF version is faster than the other, surely it's searching to greater overall depths too. But now after auditing the first three games (OK, I've got 97 more to go, so this data is preliminary), I have hard evidence supporting my perception.

So you have learned that the logic behind your "surely" is flawed.

So nps is not everything. The next thing you may learn is that a comparison of search depths reached by different engines is not everything either. (Yes, SF of today is a different engine than SF of last year.)

What is everything?

[syzygy]

I'm auditing the games now and something interesting is apparent. It was my perception while watching the games that while asmFish was 1,000-2,000 kN/s faster, SF seemed to search deeper. I just shrugged this notion off, thinking "That's impossible!" After all, asmFish IS Stockfish, just an outdated version that's much faster. So if one SF version is faster than the other, surely it's searching to greater overall depths too. But now after auditing the first three games (OK, I've got 97 more to go, so this data is preliminary), I have hard evidence supporting my perception.

So you have learned that the logic behind your "surely" is flawed.

So nps is not everything. The next thing you may learn is that a comparison of search depths reached by different engines is not everything either. (Yes, SF of today is a different engine than SF of last year.)

What is everything?

Playing strength is everything. The only way to measure it is many many games.

(But if you are comparing functionally identical engines, such as asmFish and the corresponding version of Stockfish having identical node counts, then it is enough to look at speed.)

[Stephen Ham]

Thanks for your valued input, Syzygy.

Rather than overall playing strength, I was instead testing tactical acuity. In that 100 game match with long TC, asmFish/PedantFish failed. That was a surprise as the test was skewed in its favor, as previously explained, given the latter's roughly 25% speed advantage. Tactics is generally all about searching faster/more than the opponent. Also, that's the equivalent of 25% more time.

Instead, in a 100 game match with a shared "normal" opening book leading to "standard" chess positions, I expect that SF would have won by a far larger number of games than the 3-to-1 victory rate it scored. In normal chess, there would have been greater opportunities for SF to manifest superior technical ability and evaluation. So again, full credit to SF's programmers.

I'm still auditing the match games. However, I want to report that I did find a couple of games where asmFish/PedantFish's search average was deeper than SF's. But when it happened, it was only by roughly one ply.

My next and last planned match is between the two engines that defeated SF: Raubfisch X40 and Cfish.

All the best,
-Steve-

[MikeB]

Hi Mike,

Thanks for your interest. Like you, I wish the asmFish effort was sustainable since conceptually, its superior speed would make it the strongest Stockfish (SF).

I'm auditing the games now and something interesting is apparent. It was my perception while watching the games that while asmFish was 1,000-2,000 kN/s faster, SF seemed to search deeper. I just shrugged this notion off, thinking "That's impossible!" After all, asmFish IS Stockfish, just an outdated version that's much faster. So if one SF version is faster than the other, surely it's searching to greater overall depths too. But now after auditing the first three games (OK, I've got 97 more to go, so this data is preliminary), I have hard evidence supporting my perception.

In game #1, SF's speed average was 6,309 kN/s versus 7,887 for asmFish. But SF searched to an average depth of 40.5 plies, versus 37.0 for asmFish.

In game #2, asmFish searched at an average of 7,419 kN/s to an average depth of 42.5 plies. But SF's numbers are: 6,018 kN/s and 49.6 plies.

In game #3, asmFish searched at an average of 6,344 kN/s to average depth of 36.9 plies. But SF's numbers are: 5,131 kN/s and 39.5 plies. I'll bet the remaining 97 games confirm my perception from watching some latter games.

Mike, does this suggest that recent SF coding results in more radical pruning to reach greater search depth?

All the very best,
-Steve-

Generally speaking , meaning not always true, but when you see the node count for the bench going down, it points to more pruning going on ( and resulting in deeper depth) , conversely when the bench nodes increases, it would would mean that less pruning ( or perhaps more extensions ) are happening. So your conjecture that "that recent SF coding results in more radical pruning to reach greater search depth" is correct. Quite frankly , that has been magic over the last ten years - such so that even when SF is searching at just 65K nodes per second - it is playing at GM strength today. The smartness in SF's pruning in amazing - but also leads to some well known blindspots that much weaker engines find instantly.