Stockfish_dev is probably stronger than Sargon 1978 v1.00

[Ovyron]

Yeah, if you want to see an amazing Stockfish at depth 1, just make it not report any iteration change until depth 10, then subtract 9 from the displayed depth. It'll be untouchable by other fixed depth engines, and it'll be as meaningful as node or depth comparisons that are being done (apples to oranges and all that.)

[Bill Forster]

I chuckled at this. What is the win expectation with a 2000 Elo delta? According to https://wismuth.com/elo/calculator.html#elo_diff=2000 it's actually 1.000000000. Which surprises me actually, I would have thought someone who had just learned the moves would get a draw against Magnus before playing a billion games, although maybe it would only be because they were learning

When I read a bit more and saw the conditions of the match, I realised you were inviting Stockfish to play in Sargon's garden. A fixed depth of 5 or 6 is ideal for Sargon, because it does full width, shallow search. It uses SOMA in the leaf nodes, so in effect it is peering further (I was talking about this yesterday in the programming forum on this site). Reading some of the other replies it seems that Stockfish when limited to fixed depth might imagine taking a protected pawn with its queen at the leaf node was a good idea to win a pawn.

[Of course in other conditions (as I mention in the project page), Sargon takes years to get to depth 13, Stockfish seconds ...]

Given all of this, it would come down to whether Sargon's SOMA to avoid gross blunders in the leaves was sufficient compensation for a total lack of chess knowledge compared to Stockfish's presumably elaborate evaluator. Possibly not surprising it's something of a wash.

Incidentally, I don't think Sargon is anywhere near 1900. That's about my strength, and playing dozens of games during development it was clear that it could never beat me unless I really asked for it. Sometimes not even then. I also tried a simple engine match (not my wheelhouse though really) and my estimate, as I mentioned in my project page, was 1200.

[Uri Blass]

I don't think the weak 6 ply searches matter at all.
You can get to 15 plies in an eyeblink.
If you like ultra-hyper-bullet game in one tenth of a second, then use Komodo.
I am not sure how the NN programs would do at ludicrously short time control.
It would be interesting to see an LC0/Komodo match at very high speed.

For reasonable time control on both play and analysis, Stockfish does very well indeed.

The weak 6 ply searches matter because they mean stockfish is useless as a tool to teach humans how to search in chess.

Humans cannot look at stockfish's tree at small depth and learn from it a better strategy to search.
We need a better engine for this target.

[Joerg Oster]

I do not see the point of comparing stockfish with sargon at small fixed depth.

Stockfish is not optimized to play well at small fixed depth and does some strange pruning that no human does.
I do not see the point of using it at fixed depth of 5 plies when it believes that after 1.e4 d5 2.exd5 Qxd5 white wins the queen by 3.Nc3



FEN: rnb1kbnr/ppp1pppp/8/3q4/8/8/PPPP1PPP/RNBQKBNR w KQkq - 0 3

Stockfish_20052607_x64_modern:
1/1 00:00 395 198k +0.92 3.Qe2
2/2 00:00 564 282k +9.08 3.Nc3 c6
3/3 00:00 708 354k +9.10 3.Nc3 e6 4.Nxd5
4/4 00:00 865 433k +9.16 3.Nc3 e6 4.Nxd5 exd5 5.Bb5+ Nc6
5/6 00:00 1k 520k +9.38 3.Nc3 e6 4.Nxd5 exd5
6/7 00:00 2k 842k +9.04 3.Nc3 Be6 4.Nxd5 Bxd5 5.h4

I would like to see a different strong engine that try to search like a human.
Unfortunately the stockfish team care only about elo and not about trying to use some smart algorithm that humans can learn from it useful things.

If humans try to look at stockfish's tree to learn how to search and see lines like 3,Nc3 e6 they can get the conclusion that stockfish is a very weak and useless engine.

In this case, I tend to agree.
Especially, since this odd behavior is so easy to fix.

[Joerg Oster]

I don't think the weak 6 ply searches matter at all.
You can get to 15 plies in an eyeblink.
If you like ultra-hyper-bullet game in one tenth of a second, then use Komodo.
I am not sure how the NN programs would do at ludicrously short time control.
It would be interesting to see an LC0/Komodo match at very high speed.

For reasonable time control on both play and analysis, Stockfish does very well indeed.

Where do you draw the line?
At 12 plies, 15 plies, or even 24 plies?

Don't we want the best possible move(s) at any given time or depth?

[Laskos]

I don't think the weak 6 ply searches matter at all.
You can get to 15 plies in an eyeblink.
If you like ultra-hyper-bullet game in one tenth of a second, then use Komodo.
I am not sure how the NN programs would do at ludicrously short time control.
It would be interesting to see an LC0/Komodo match at very high speed.

For reasonable time control on both play and analysis, Stockfish does very well indeed.

The weak 6 ply searches matter because they mean stockfish is useless as a tool to teach humans how to search in chess.

Humans cannot look at stockfish's tree at small depth and learn from it a better strategy to search.
We need a better engine for this target.

Lc0 at 200 nodes with a large net is roughly depth 5-6 in openings-middlegame, and in strength comparable to SF_dev depth=15, serious strength comparable to that of a GM at LTC. Lc0 tree search can be visualized and it IS useful to teach humans. Depth of 5-6 for Lc0 is meaningless, but the shape of the MCTS search tree is meaningful and useful. Only endgames suffer with Lc0 trolling and even blundering. I think the better part of 200 nodes can almost be remembered as viable possibilities chosen by Lc0 according its policy and value. But tactical combinations and shots are better with SF depth=15 or 20.

[Laskos]

I don't think the weak 6 ply searches matter at all.
You can get to 15 plies in an eyeblink.
If you like ultra-hyper-bullet game in one tenth of a second, then use Komodo.
I am not sure how the NN programs would do at ludicrously short time control.
It would be interesting to see an LC0/Komodo match at very high speed.

For reasonable time control on both play and analysis, Stockfish does very well indeed.

You are wrong that Komodo or Lc0 are better in ultra-fast games. At fixed time, SF_dev is by far the strongest engine on a regular PC in hyper-ultra-fast games (say 1s+0.01s), it is Lc0 surpassing it at slower games and Komodo approaching SF level at slower games. Simply, depth 6 is completed by SF in maybe a millisecond, practically unplayable times using time controls. But I was curious how the notion of "ply" has changed over time comparing a full width Sargon A/B search with current vague "ply" (iteration) of top A/B engines and even more vague "ply" of Lc0. Top A/B engines today seem to use their strong eval to guide the search so that it compensates for sparsely searched vague "plies", and after all EBF reductions to come up with vague "plies" as strong as old real plies. Lc0 MCTS (so called MCTS) is even more extreme in this respect, its eval guides the search so well, that it needs a very sparse search to play well.

[Raphexon]

I tried Sargon depth 8 vs SF.
But I got bored from waiting, those are some humongous time odds in favour of the ancient engine.

But thanks for linking it, it's an interesting engine.

[Raphexon]

I don't think the weak 6 ply searches matter at all.
You can get to 15 plies in an eyeblink.
If you like ultra-hyper-bullet game in one tenth of a second, then use Komodo.
I am not sure how the NN programs would do at ludicrously short time control.
It would be interesting to see an LC0/Komodo match at very high speed.

For reasonable time control on both play and analysis, Stockfish does very well indeed.

You are wrong that Komodo or Lc0 are better in ultra-fast games. At fixed time, SF_dev is by far the strongest engine on a regular PC in hyper-ultra-fast games (say 1s+0.01s), it is Lc0 surpassing it at slower games and Komodo approaching SF level at slower games. Simply, depth 6 is completed by SF in maybe a millisecond, practically unplayable times using time controls. But I was curious how the notion of "ply" has changed over time comparing a full width Sargon A/B search with current vague "ply" (iteration) of top A/B engines and even more vague "ply" of Lc0. Top A/B engines today seem to use their strong eval to guide the search so that it compensates for sparsely searched vague "plies", and after all EBF reductions to come up with vague "plies" as strong as old real plies. Lc0 MCTS (so called MCTS) is even more extreme in this respect, its eval guides the search so well, that it needs a very sparse search to play well.

Its policy (move ordering) guides the search so well.*

During the TCEC Stockfish-Allie SuFi, Leela seemed equally blind to certain Allie blunders. But Leela would have never played those moves.
Her eval had (still has?) the same blindspots as Allie, but her move ordering vastly superior.

I wonder if SF's move ordering being turned into an 1 ply multiPV Leela search would be a big ELO gainer.
It should theoretically vastly improve SF's move ordering making its search more efficient.

[Laskos]

I chuckled at this. What is the win expectation with a 2000 Elo delta? According to https://wismuth.com/elo/calculator.html#elo_diff=2000 it's actually 1.000000000. Which surprises me actually, I would have thought someone who had just learned the moves would get a draw against Magnus before playing a billion games, although maybe it would only be because they were learning

When I read a bit more and saw the conditions of the match, I realised you were inviting Stockfish to play in Sargon's garden. A fixed depth of 5 or 6 is ideal for Sargon, because it does full width, shallow search. It uses SOMA in the leaf nodes, so in effect it is peering further (I was talking about this yesterday in the programming forum on this site). Reading some of the other replies it seems that Stockfish when limited to fixed depth might imagine taking a protected pawn with its queen at the leaf node was a good idea to win a pawn.

[Of course in other conditions (as I mention in the project page), Sargon takes years to get to depth 13, Stockfish seconds ...]

Given all of this, it would come down to whether Sargon's SOMA to avoid gross blunders in the leaves was sufficient compensation for a total lack of chess knowledge compared to Stockfish's presumably elaborate evaluator. Possibly not surprising it's something of a wash.

Incidentally, I don't think Sargon is anywhere near 1900. That's about my strength, and playing dozens of games during development it was clear that it could never beat me unless I really asked for it. Sometimes not even then. I also tried a simple engine match (not my wheelhouse though really) and my estimate, as I mentioned in my project page, was 1200.

Thank you very much for porting this engine to modern hardware and allowing UCI standard, it's very helpful in concocting all sorts of experiments with Sargon 1.00, which is amazing. Also, on modern hardware what experiments I am doing in ultra-blitz was long time control back then (say depth=6) on Z80. Sargon 1 and 2 set the standard of microcomputer engines we have up to now, right?

I think the first Mephistos of Richard Lang used too SOMA in early 1980s, and this was probably inspired by "the mighty Sargon" .

I now googled for a rating list, and the follow-up engine, "Chafitz Sargon 2.5 6502 2 MHz" is mentioned on this FIDE list:
https://www.schach-computer.info/wiki/i ... -Elo-Liste
as having about 1420 FIDE level at LTC on 2 MHz 6502. It was created only 1 year after Sargon 1978 v1.00 you ported, so my estimate for Sargon v1.00 on Z80 of 1350-1400 FIDE Elo level at LTC shouldn't be that far off, maybe 1300-1350 would be closer. I got this estimate by pitting Sargon to another ported UCI engine, Mephisto Amsterdam, for which I knew the LTC FIDE rating. However, the rating of Sargon 1 in Blitz was higher even back then in 1978, as it always happens with engines, maybe 1500-1600. And on modern hardware maybe 1900 FIDE Blitz Elo level. I played one fast blitz game against it, blundered and lost . It would be fun if more people play this engine on modern hardware.