Arasan 20.0 is available from http://www.arasanchess.org.
Changes in Arasan 20.0:
1) Fix to handling of UCI_LimitStrength and UCI_Elo options. These can now be processed in either order and will set the search strength correctly.
2) Revised tuning program. Only the "Texel" method is supported now. Tuner uses a "mean-squared error" objective and takes a labeled EPD file
as input. Various optimization methods can be selected. Optimization steps are scaled appropriately based on parameter ranges.
3) Changes to king cover and king safety scoring.
4) Bug fix: Ensure hash move is always checked for validity (was not being done for evasions).
5) Fix possible race condition updating root PV.
6) Add some utility programs and Python scripts to source package.
7) Considerable code cleanup and fixing warnings and possible bugs, notably in Syzygy tb code.
This version scores quite a bit better than 19.2 against a gauntlet of opponents in my fast time-control testing, but actually did not win a 2400 game blitz match directly against version 19.2. So I don't know quite what to make of that. Anyway I am releasing it, partly because of the bug fixes, and hopefully more testing will make clearer what the relative strength is.
--Jon
Arasan 20.0
Moderators: hgm, Rebel, chrisw
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
-
- Posts: 216
- Joined: Sun Jan 22, 2017 8:30 pm
- Location: Russia
Re: Arasan 20.0
Congrats on the release!
Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.
Considering that any chess algorithm is necessarily highly heuristic (the search tree is pruned so the search doesn't always return the true minimax), the self-play failure means only that the tuning of v. 20.0 is unfit against one particular opponent - Arasan 19.2 - which tells little about its extent of optimality against a random computer, let alone human, adversary.
Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.
Considering that any chess algorithm is necessarily highly heuristic (the search tree is pruned so the search doesn't always return the true minimax), the self-play failure means only that the tuning of v. 20.0 is unfit against one particular opponent - Arasan 19.2 - which tells little about its extent of optimality against a random computer, let alone human, adversary.
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Arasan 20.0
Well that is true to an extent and I do not usually rely on self-play results.
But there are two variables there: the time control is different and so is the opponent. So it is unknown to me at present if 20.0 just does worse against 19.2, or if the time control is a factor.
--Jon
But there are two variables there: the time control is different and so is the opponent. So it is unknown to me at present if 20.0 just does worse against 19.2, or if the time control is a factor.
--Jon
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Arasan 20.0
In my experience (with Rybka and Komodo) the time control is more likely to be a major factor than the choice of opponent. It's true that some engines do better against specific opponents, but when comparing versions of the same engine, unless really drastic changes have been made, the one that does better in self-play will also do better against a range of opponents. Clearly that principle has worked for the Stockfish team as well.jdart wrote:Well that is true to an extent and I do not usually rely on self-play results.
But there are two variables there: the time control is different and so is the opponent. So it is unknown to me at present if 20.0 just does worse against 19.2, or if the time control is a factor.
--Jon
Komodo rules!
-
- Posts: 41455
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Re: Arasan 20.0
Thanks Jon.jdart wrote:Arasan 20.0 is available from http://www.arasanchess.org.
gbanksnz at gmail.com
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Arasan 20.0
Thanks for this new version!
As the computer power goes slowly cheaper, I think that this will be an area of improvement of the engines at some point. I mean that of course an engine can be retuned to play more optimal against other engines.
As always, while you have a system of improvement that works, is difficult at least psychologically to go to another system.
Initially I used a gauntlet also for Andscacs, but maybe two years ago I went to selfplay, mostly due to the increased sensitivity that reduces the necessary number of games. Anyway from time to time I do a verification test against a gauntlet.Tony P. wrote: Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.
As the computer power goes slowly cheaper, I think that this will be an area of improvement of the engines at some point. I mean that of course an engine can be retuned to play more optimal against other engines.
As always, while you have a system of improvement that works, is difficult at least psychologically to go to another system.
Daniel José - http://www.andscacs.com
-
- Posts: 216
- Joined: Sun Jan 22, 2017 8:30 pm
- Location: Russia
Re: Arasan 20.0
That's actually a deep topic on which I might post a separate thread soon. The thing is that, nowadays when any top engine is more than enough for postmortem blunder checks already, humans no longer require an engine to play even better vs other engines. They need it to show and explain devastating strategies (primarily opening novelties) against fellow humans whom the users are about to face over the board.cdani wrote:As the computer power goes slowly cheaper, I think that this will be an area of improvement of the engines at some point. I mean that of course an engine can be retuned to play more optimal against other engines.
As always, while you have a system of improvement that works, is difficult at least psychologically to go to another system.
Thus, ideally, an engine would need to include separate opponent models whose thinking processes it would be emulating and adjusting against.
But this of course requires a ton of computational power or some kind of Monte Carlo search because the alpha-beta technique is obviously inapplicable if the opponent is assumed to evaluate positions differently or even have a different search routine from the engine's.
-
- Posts: 12541
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Arasan 20.0
Big thanks,
There seems to be a real avalanche of fun the last few days.
There seems to be a real avalanche of fun the last few days.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 4889
- Joined: Thu Mar 09, 2006 6:34 am
- Location: Pen Argyl, Pennsylvania
Re: Arasan 20.0
"...due to the increased sensitivity that reduces the necessary number of games..." agree 100% - I think the SF team has pretty much proven self-play works...cdani wrote:Thanks for this new version!
Initially I used a gauntlet also for Andscacs, but maybe two years ago I went to selfplay, mostly due to the increased sensitivity that reduces the necessary number of games. Anyway from time to time I do a verification test against a gauntlet.Tony P. wrote: Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.
....
-
- Posts: 4889
- Joined: Thu Mar 09, 2006 6:34 am
- Location: Pen Argyl, Pennsylvania
Re: Arasan 20.0
"...but actually did not win a 2400 game blitz match directly against version 19.2...." don't you hate that?jdart wrote:Arasan 20.0 is available from http://www.arasanchess.org.
Changes in Arasan 20.0:
1) Fix to handling of UCI_LimitStrength and UCI_Elo options. These can now be processed in either order and will set the search strength correctly.
2) Revised tuning program. Only the "Texel" method is supported now. Tuner uses a "mean-squared error" objective and takes a labeled EPD file
as input. Various optimization methods can be selected. Optimization steps are scaled appropriately based on parameter ranges.
3) Changes to king cover and king safety scoring.
4) Bug fix: Ensure hash move is always checked for validity (was not being done for evasions).
5) Fix possible race condition updating root PV.
6) Add some utility programs and Python scripts to source package.
7) Considerable code cleanup and fixing warnings and possible bugs, notably in Syzygy tb code.
This version scores quite a bit better than 19.2 against a gauntlet of opponents in my fast time-control testing, but actually did not win a 2400 game blitz match directly against version 19.2. So I don't know quite what to make of that. Anyway I am releasing it, partly because of the bug fixes, and hopefully more testing will make clearer what the relative strength is.
--Jon
It's amazing you been doing this since 1994, I can still remember playing Arasan 1.0 when you first published it and it still runs today on my Mac Pro using wine ...amazing, just as much fun playing now as it was back then... if anybody else it looking for v1.0 , Jon has it on website , no need to go to a germ infested site to download it...