The link above reminded me that I need to finish off my Sargon V1.01 project. I devoted the day to it and got it done. The engine itself is unchanged from the beta, except that it's name is now "Sargon 1978 V1.01" not "Sargon 1978 V1.01 beta". Sorry not very overwhelming I know. The latest version of the engine can be downloaded from Github here:
https://github.com/billforsternz/retro- ... engine.exe.
It will be interesting to see if the various engine testers pick up the new release and measure a significant Elo change compared to V1.00.
I have updated the documentation to reflect V1.01. If you've been following along you will know that V1.01 is basically an attempt to work-around Sargon 1978's tendency to repeat positions and concede repetition draws far too easily, and also to finish off opponents and deliver mate once it has calculated a mating PV. (Also, less importantly, if the GUI provides a depth parameter to the go command it now goes straight to that target depth rather than iterating its way there, saving significant time).
Getting this release out was a whole day's work mainly because I updated sargon-tests.exe available here:
https://github.com/billforsternz/retro- ... -tests.exe to include some timing tests. On the principle that 'the code should be the documentation' the program explains itself;
Code: Select all
C:\Users\Bill\Documents\Github\sargon\Release>sargon-tests t -1 -v
On the Talkchess computer chess forum, user mwyoung revealed some Sargon
timing information from the (paper) manual that came with an early Z80
version of Sargon:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=40#p866311
Hardware needed: one TRS-80 level II with 32k of memory minimum.
Level 1: 20 secs (per move)
Level 2: 60 secs, 3 x slower vs level 1
Level 3: 120 secs, 2 x slower vs level 2
Level 4: 360 secs, 3 x slower vs level 3
Level 5: 2400 secs, 5 x slower vs level 4
Level 6: 14400 secs, 6 x slower vs level 5
The following measurements on a variety of positions (the positions used for
position tests are reused) serve to provide a comparison to these move times.
Level 1: ............ 1200 tests
Avg=0.002 secs, 11445 x faster vs TRS-80
Level 2: ............ 120 tests
Avg=0.009 secs, 4.9 x slower vs level 1, 6943 x faster vs TRS-80
Level 3: ............ 12 tests
Avg=0.117 secs, 13.5 x slower vs level 2, 1027 x faster vs TRS-80
Level 4: ............ 12 tests
Avg=0.352 secs, 3.0 x slower vs level 3, 1022 x faster vs TRS-80
Level 5: ............ 12 tests
Avg=5.005 secs, 14.2 x slower vs level 4, 479 x faster vs TRS-80
Level 6: ............ 12 tests
Avg=13.131 secs, 2.6 x slower vs level 5, 1097 x faster vs TRS-80
The Sargon benchmark (=level 6, 12 tests TRS-80 speedup factor) is 1097.
An important caveat to the TRS-80 comparison above is that this is definitely
not an apples to apples comparison. How do we know this? The TRS-80 manual
entry above also provides an 'instantaneous' time for level 0. But the Sargon
1978 source code, and our translation of it to Intel x86 constrains the level
to the range 1-6 inclusive. Level 0 is definitely not allowed! Refer to the
paragraph of assembly language starting with label IN08:
Since we are not simply comparing the same software running on different
platforms, the comparisons are perhaps moot. With tongue in cheek I have used
the measurements to define a 'Sargon' benchmark above which is a measurement
of a computer's single core performance. The modest circa 2010 Acer Aspire
5741G laptop this software was developed has a benchmark score of ~1140.
In light of these considerations, the best estimate of how much faster Sargon
1978 runs on a modern PC versus the original Z80 code on Z80 hardware was
probably provided by user Tibono on the talkchess forums:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=10#p845624
Tibono went above and beyond the call of duty, comparing the x86 UCI engine
to original Z80 code running on a Z80 emulator with known timing. He not
only verified identical moves were calculated, he hand timed and tabulated
the moves. He provided two averages, a speedup ratio of 6917 at level 2 and
6125 at level 3. He also used a modest old laptop, he reports it has a CPU &
RAM win perf index of 5.8. Sadly a comparable score for my laptop under
Windows 10 doesn't seem to be available. My laptop has Windows System
Assessment Tool (WinSAT) scores of 8.2 (both CPU and RAM).
In summary we have considered two independent speedup ratios, on two different
modest laptops. One is approx 1000, one is approx 6000. We consider the latter
to be a more reliable and better indicator.
All tests passed. Elapsed time = 226.410 seconds
C:\Users\Bill\Documents\Github\sargon\Release>
To summarise, it basically attempts to answer the question; "How much faster is Sargon 1978 x86 than Sargon 1978 Z80 running on a Z80?" Larry Kaufman was asking this question (way) up thread, so it seemed worthwhile to gather, re-measure and summarise all available information. The best answer seems to be about 6000 times faster (thanks to user Tibono who performed the measurement). I have defined a "Sargon" benchmark to measure how fast your computer runs Sargon. My circa 2010 Acer 5741G (a very modest machine indeed) has a Sargon benchmark of about 1100 (run the sargon-tests.exe on your machine if you are interested in your Sargon benchmark). The Sargon benchmark number is also a speedup ratio (like Tibono's 6000) measured slightly differently. Perhaps actual speedup is Sargon benchmark multiplied by 6 approximately.
