Sargon 1978 UCI Available

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

It was NOT that easy to beat Sargon at depth 7 but slowly I got the best of Sargon Depth7. I wonder in how many moves can other players beat Sargon at depth7 :?: :mrgreen:


Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

ThatsIt wrote: Fri Jun 05, 2020 2:07 pm Hi !

I've started a test for our 40/4 + ... ratinglist.

https://cegt.forumieren.com/t1308-testi ... 8-1-00-uci

Best wishes,
G.S.
(CEGT team)
Please run the same match with Sargon at depth 6 and include Cicada 0.1 64 bit :roll:
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

Chessqueen wrote: Wed Feb 03, 2021 6:45 am
ThatsIt wrote: Fri Jun 05, 2020 2:07 pm Hi !

I've started a test for our 40/4 + ... ratinglist.

https://cegt.forumieren.com/t1308-testi ... 8-1-00-uci

Best wishes,
G.S.
(CEGT team)
Please run the same match with Sargon at depth 6 and include Cicada 0.1 64 bit :roll:
I played a match of 10 games Sargon V.01Beta at Depth6 Versus a newer version of Mizar_21 and after 10 games the final score was 5 Wins 3 Losts and 2 Draws in favor of Sargon Depth at Depth 6

Bill Forster
Posts: 76
Joined: Mon Sep 21, 2015 7:47 am
Location: New Zealand

Re: Sargon 1978 UCI Available

Post by Bill Forster »

Chessqueen wrote: Wed Feb 03, 2021 9:38 pm
Chessqueen wrote: Wed Feb 03, 2021 6:45 am
ThatsIt wrote: Fri Jun 05, 2020 2:07 pm Hi !

I've started a test for our 40/4 + ... ratinglist.

https://cegt.forumieren.com/t1308-testi ... 8-1-00-uci

Best wishes,
G.S.
(CEGT team)
The link above reminded me that I need to finish off my Sargon V1.01 project. I devoted the day to it and got it done. The engine itself is unchanged from the beta, except that it's name is now "Sargon 1978 V1.01" not "Sargon 1978 V1.01 beta". Sorry not very overwhelming I know. The latest version of the engine can be downloaded from Github here: https://github.com/billforsternz/retro- ... engine.exe.

It will be interesting to see if the various engine testers pick up the new release and measure a significant Elo change compared to V1.00.

I have updated the documentation to reflect V1.01. If you've been following along you will know that V1.01 is basically an attempt to work-around Sargon 1978's tendency to repeat positions and concede repetition draws far too easily, and also to finish off opponents and deliver mate once it has calculated a mating PV. (Also, less importantly, if the GUI provides a depth parameter to the go command it now goes straight to that target depth rather than iterating its way there, saving significant time).

Getting this release out was a whole day's work mainly because I updated sargon-tests.exe available here: https://github.com/billforsternz/retro- ... -tests.exe to include some timing tests. On the principle that 'the code should be the documentation' the program explains itself;

Code: Select all

C:\Users\Bill\Documents\Github\sargon\Release>sargon-tests t -1 -v
On the Talkchess computer chess forum, user mwyoung revealed some Sargon
timing information from the (paper) manual that came with an early Z80
version of Sargon:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=40#p866311

Hardware needed: one TRS-80 level II with 32k of memory minimum.
Level 1: 20 secs (per move)
Level 2: 60 secs, 3 x slower vs level 1
Level 3: 120 secs, 2 x slower vs level 2
Level 4: 360 secs, 3 x slower vs level 3
Level 5: 2400 secs, 5 x slower vs level 4
Level 6: 14400 secs, 6 x slower vs level 5

The following measurements on a variety of positions (the positions used for
position tests are reused) serve to provide a comparison to these move times.

Level 1: ............ 1200 tests
         Avg=0.002 secs, 11445 x faster vs TRS-80
Level 2: ............  120 tests
         Avg=0.009 secs, 4.9 x slower vs level 1, 6943 x faster vs TRS-80
Level 3: ............   12 tests
         Avg=0.117 secs, 13.5 x slower vs level 2, 1027 x faster vs TRS-80
Level 4: ............   12 tests
         Avg=0.352 secs, 3.0 x slower vs level 3, 1022 x faster vs TRS-80
Level 5: ............   12 tests
         Avg=5.005 secs, 14.2 x slower vs level 4, 479 x faster vs TRS-80
Level 6: ............   12 tests
         Avg=13.131 secs, 2.6 x slower vs level 5, 1097 x faster vs TRS-80

The Sargon benchmark (=level 6, 12 tests TRS-80 speedup factor) is 1097.

An important caveat to the TRS-80 comparison above is that this is definitely
not an apples to apples comparison. How do we know this? The TRS-80 manual
entry above also provides an 'instantaneous' time for level 0. But the Sargon
1978 source code, and our translation of it to Intel x86 constrains the level
to the range 1-6 inclusive. Level 0 is definitely not allowed! Refer to the
paragraph of assembly language starting with label IN08:

Since we are not simply comparing the same software running on different
platforms, the comparisons are perhaps moot. With tongue in cheek I have used
the measurements to define a 'Sargon' benchmark above which is a measurement
of a computer's single core performance. The modest circa 2010 Acer Aspire
5741G laptop this software was developed has a benchmark score of ~1140.
In light of these considerations, the best estimate of how much faster Sargon
1978 runs on a modern PC versus the original Z80 code on Z80 hardware was
probably provided by user Tibono on the talkchess forums:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=10#p845624

Tibono went above and beyond the call of duty, comparing the x86 UCI engine
to original Z80 code running on a Z80 emulator with known timing. He not
only verified identical moves were calculated, he hand timed and tabulated
the moves. He provided two averages, a speedup ratio of 6917 at level 2 and
6125 at level 3. He also used a modest old laptop, he reports it has a CPU &
RAM win perf index of 5.8. Sadly a comparable score for my laptop under
Windows 10 doesn't seem to be available. My laptop has Windows System
Assessment Tool (WinSAT) scores of 8.2 (both CPU and RAM).

In summary we have considered two independent speedup ratios, on two different
modest laptops. One is approx 1000, one is approx 6000. We consider the latter
to be a more reliable and better indicator.

All tests passed. Elapsed time = 226.410 seconds

C:\Users\Bill\Documents\Github\sargon\Release>
To summarise, it basically attempts to answer the question; "How much faster is Sargon 1978 x86 than Sargon 1978 Z80 running on a Z80?" Larry Kaufman was asking this question (way) up thread, so it seemed worthwhile to gather, re-measure and summarise all available information. The best answer seems to be about 6000 times faster (thanks to user Tibono who performed the measurement). I have defined a "Sargon" benchmark to measure how fast your computer runs Sargon. My circa 2010 Acer 5741G (a very modest machine indeed) has a Sargon benchmark of about 1100 (run the sargon-tests.exe on your machine if you are interested in your Sargon benchmark). The Sargon benchmark number is also a speedup ratio (like Tibono's 6000) measured slightly differently. Perhaps actual speedup is Sargon benchmark multiplied by 6 approximately.
User avatar
AdminX
Posts: 6363
Joined: Mon Mar 13, 2006 2:34 pm
Location: Acworth, GA

Re: Sargon 1978 UCI Available

Post by AdminX »

Thanks Bill,

Here is what is reported on a old i7-4790 w/ 32GB Ram

Code: Select all

D:\Users\tedsu\Downloads>sargon-tests t -1 -v
On the Talkchess computer chess forum, user mwyoung revealed some Sargon
timing information from the (paper) manual that came with an early Z80
version of Sargon:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=40#p866311

Hardware needed: one TRS-80 level II with 32k of memory minimum.
Level 1: 20 secs (per move)
Level 2: 60 secs, 3 x slower vs level 1
Level 3: 120 secs, 2 x slower vs level 2
Level 4: 360 secs, 3 x slower vs level 3
Level 5: 2400 secs, 5 x slower vs level 4
Level 6: 14400 secs, 6 x slower vs level 5

The following measurements on a variety of positions (the positions used for
position tests are reused) serve to provide a comparison to these move times.

Level 1: ............ 1200 tests
         Avg=0.001 secs, 22749 x faster vs TRS-80
Level 2: ............  120 tests
         Avg=0.005 secs, 5.2 x slower vs level 1, 13115 x faster vs TRS-80
Level 3: ............   12 tests
         Avg=0.057 secs, 12.6 x slower vs level 2, 2090 x faster vs TRS-80
Level 4: ............   12 tests
         Avg=0.175 secs, 3.0 x slower vs level 3, 2058 x faster vs TRS-80
Level 5: ............   12 tests
         Avg=2.260 secs, 12.9 x slower vs level 4, 1062 x faster vs TRS-80
Level 6: ............   12 tests
         Avg=6.231 secs, 2.8 x slower vs level 5, 2311 x faster vs TRS-80

The Sargon benchmark (=level 6, 12 tests TRS-80 speedup factor) is 2311.

An important caveat to the TRS-80 comparison above is that this is definitely
not an apples to apples comparison. How do we know this? The TRS-80 manual
entry above also provides an 'instantaneous' time for level 0. But the Sargon
1978 source code, and our translation of it to Intel x86 constrains the level
to the range 1-6 inclusive. Level 0 is definitely not allowed! Refer to the
paragraph of assembly language starting with label IN08:

Since we are not simply comparing the same software running on different
platforms, the comparisons are perhaps moot. With tongue in cheek I have used
the measurements to define a 'Sargon' benchmark above which is a measurement
of a computer's single core performance. The modest circa 2010 Acer Aspire
5741G laptop this software was developed has a benchmark score of ~1140.
In light of these considerations, the best estimate of how much faster Sargon
1978 runs on a modern PC versus the original Z80 code on Z80 hardware was
probably provided by user Tibono on the talkchess forums:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=10#p845624

Tibono went above and beyond the call of duty, comparing the x86 UCI engine
to original Z80 code running on a Z80 emulator with known timing. He not
only verified identical moves were calculated, he hand timed and tabulated
the moves. He provided two averages, a speedup ratio of 6917 at level 2 and
6125 at level 3. He also used a modest old laptop, he reports it has a CPU &
RAM win perf index of 5.8. Sadly a comparable score for my laptop under
Windows 10 doesn't seem to be available. My laptop has Windows System
Assessment Tool (WinSAT) scores of 8.2 (both CPU and RAM).

In summary we have considered two independent speedup ratios, on two different
modest laptops. One is approx 1000, one is approx 6000. We consider the latter
to be a more reliable and better indicator.

All tests passed. Elapsed time = 106.297 seconds
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers
Essay
Posts: 6
Joined: Fri Apr 10, 2020 10:27 pm
Full name: Sophie Abbot

Re: Sargon 1978 UCI Available

Post by Essay »

Hi Bill,

Many thanks, great work and excellent documentation!

On my Intel Core i7-10700 CPU @2.90GHz - 16GB ram I got these results:

Sargon 1978 move calculation time measurements.

Level 1: ............ 1200 tests
Avg=0.001 secs, 28812 x faster vs TRS-80
Level 2: ............ 120 tests
Avg=0.004 secs, 5.1 x slower vs level 1, 17102 x faster vs TRS-80
Level 3: ............ 12 tests
Avg=0.047 secs, 13.3 x slower vs level 2, 2562 x faster vs TRS-80
Level 4: ............ 12 tests
Avg=0.143 secs, 3.1 x slower vs level 3, 2509 x faster vs TRS-80
Level 5: ............ 12 tests
Avg=1.850 secs, 12.9 x slower vs level 4, 1297 x faster vs TRS-80
Level 6: ............ 12 tests
Avg=5.077 secs, 2.7 x slower vs level 5, 2836 x faster vs TRS-80

The Sargon benchmark (=level 6, 12 tests TRS-80 speedup factor) is 2836.

All tests passed. Elapsed time = 86.664 seconds
Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

Bill Forster wrote: Fri Feb 05, 2021 7:30 am
Chessqueen wrote: Wed Feb 03, 2021 9:38 pm
Chessqueen wrote: Wed Feb 03, 2021 6:45 am
ThatsIt wrote: Fri Jun 05, 2020 2:07 pm Hi !

I've started a test for our 40/4 + ... ratinglist.

https://cegt.forumieren.com/t1308-testi ... 8-1-00-uci

Best wishes,
G.S.
(CEGT team)
The link above reminded me that I need to finish off my Sargon V1.01 project. I devoted the day to it and got it done. The engine itself is unchanged from the beta, except that it's name is now "Sargon 1978 V1.01" not "Sargon 1978 V1.01 beta". Sorry not very overwhelming I know. The latest version of the engine can be downloaded from Github here: https://github.com/billforsternz/retro- ... engine.exe.

It will be interesting to see if the various engine testers pick up the new release and measure a significant Elo change compared to V1.00.

I have updated the documentation to reflect V1.01. If you've been following along you will know that V1.01 is basically an attempt to work-around Sargon 1978's tendency to repeat positions and concede repetition draws far too easily, and also to finish off opponents and deliver mate once it has calculated a mating PV. (Also, less importantly, if the GUI provides a depth parameter to the go command it now goes straight to that target depth rather than iterating its way there, saving significant time).

Getting this release out was a whole day's work mainly because I updated sargon-tests.exe available here: https://github.com/billforsternz/retro- ... -tests.exe to include some timing tests. On the principle that 'the code should be the documentation' the program explains itself;

Code: Select all

C:\Users\Bill\Documents\Github\sargon\Release>sargon-tests t -1 -v
On the Talkchess computer chess forum, user mwyoung revealed some Sargon
timing information from the (paper) manual that came with an early Z80
version of Sargon:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=40#p866311

Hardware needed: one TRS-80 level II with 32k of memory minimum.
Level 1: 20 secs (per move)
Level 2: 60 secs, 3 x slower vs level 1
Level 3: 120 secs, 2 x slower vs level 2
Level 4: 360 secs, 3 x slower vs level 3
Level 5: 2400 secs, 5 x slower vs level 4
Level 6: 14400 secs, 6 x slower vs level 5

The following measurements on a variety of positions (the positions used for
position tests are reused) serve to provide a comparison to these move times.

Level 1: ............ 1200 tests
         Avg=0.002 secs, 11445 x faster vs TRS-80
Level 2: ............  120 tests
         Avg=0.009 secs, 4.9 x slower vs level 1, 6943 x faster vs TRS-80
Level 3: ............   12 tests
         Avg=0.117 secs, 13.5 x slower vs level 2, 1027 x faster vs TRS-80
Level 4: ............   12 tests
         Avg=0.352 secs, 3.0 x slower vs level 3, 1022 x faster vs TRS-80
Level 5: ............   12 tests
         Avg=5.005 secs, 14.2 x slower vs level 4, 479 x faster vs TRS-80
Level 6: ............   12 tests
         Avg=13.131 secs, 2.6 x slower vs level 5, 1097 x faster vs TRS-80

The Sargon benchmark (=level 6, 12 tests TRS-80 speedup factor) is 1097.

An important caveat to the TRS-80 comparison above is that this is definitely
not an apples to apples comparison. How do we know this? The TRS-80 manual
entry above also provides an 'instantaneous' time for level 0. But the Sargon
1978 source code, and our translation of it to Intel x86 constrains the level
to the range 1-6 inclusive. Level 0 is definitely not allowed! Refer to the
paragraph of assembly language starting with label IN08:

Since we are not simply comparing the same software running on different
platforms, the comparisons are perhaps moot. With tongue in cheek I have used
the measurements to define a 'Sargon' benchmark above which is a measurement
of a computer's single core performance. The modest circa 2010 Acer Aspire
5741G laptop this software was developed has a benchmark score of ~1140.
In light of these considerations, the best estimate of how much faster Sargon
1978 runs on a modern PC versus the original Z80 code on Z80 hardware was
probably provided by user Tibono on the talkchess forums:
Ref: http://talkchess.com/forum3/viewtopic.php?f=2&t=74027&start=10#p845624

Tibono went above and beyond the call of duty, comparing the x86 UCI engine
to original Z80 code running on a Z80 emulator with known timing. He not
only verified identical moves were calculated, he hand timed and tabulated
the moves. He provided two averages, a speedup ratio of 6917 at level 2 and
6125 at level 3. He also used a modest old laptop, he reports it has a CPU &
RAM win perf index of 5.8. Sadly a comparable score for my laptop under
Windows 10 doesn't seem to be available. My laptop has Windows System
Assessment Tool (WinSAT) scores of 8.2 (both CPU and RAM).

In summary we have considered two independent speedup ratios, on two different
modest laptops. One is approx 1000, one is approx 6000. We consider the latter
to be a more reliable and better indicator.

All tests passed. Elapsed time = 226.410 seconds

C:\Users\Bill\Documents\Github\sargon\Release>
To summarise, it basically attempts to answer the question; "How much faster is Sargon 1978 x86 than Sargon 1978 Z80 running on a Z80?" Larry Kaufman was asking this question (way) up thread, so it seemed worthwhile to gather, re-measure and summarise all available information. The best answer seems to be about 6000 times faster (thanks to user Tibono who performed the measurement). I have defined a "Sargon" benchmark to measure how fast your computer runs Sargon. My circa 2010 Acer 5741G (a very modest machine indeed) has a Sargon benchmark of about 1100 (run the sargon-tests.exe on your machine if you are interested in your Sargon benchmark). The Sargon benchmark number is also a speedup ratio (like Tibono's 6000) measured slightly differently. Perhaps actual speedup is Sargon benchmark multiplied by 6 approximately.

Bill you created an expert Engine, Isn't Mizar 3.0 rated 1683
Mizar 3.0 1683 +10 −10 47.8% +21.0 16.1% 4426
86.8%
555‑557 BACE 0.45 1675 +11 −11 49.1% +4.2 17.9% 3517
48.4%
555‑557 PolarChess 1.3 1675 +13 −13 52.3% −19.3 16.8% 2719
49.5%
555‑557 SharpChess2 2.6.9 1675 +31 −31 55.8% −77.5 7.0% 511
5

Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

Here is the 3dr game, the 2nd game was a boring Draw, but based on what I see if Mizar 3.0 is rated 1683 I am starting to assume that Sargon at depth 6 must be at least 1750 :roll:

Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

Sorry ,but I only mentioned that the 2nd game was a boring draw, you should all see it :roll:

Chessqueen
Posts: 5685
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Sargon 1978 UCI Available

Post by Chessqueen »

CCRL Needs to do another testing of the New Sargon with Depth 6, since this test result was NOT done with Depth 6 ==>

644 Sargon 1978 1.00 1251 +19 −19 44.4% +36.8 39.3% 1029
69.1%
645 Tikov 0.6.3 1244 +17 −17 45.1% +61.9 17.3% 1647
75.2%
646 Pierre 1.7 1236 +19 −19 54.3% −38.4 12.1% 1306
50.3%
647 Dragontooth 0.2 Bahamut 64-bit 1235 +23 −23 54.3% −34.6 13.3% 804
61.2%
648 Gringo 1.4.9b 1231 +17 −17 48.8% +30.5 16.1% 1570
53.3%
649 Raffaela 0.1.4 1230 +18 −17 42.2% +94.1 26.8% 1429
71.0%