Arasan 20.0

[jdart]

Arasan 20.0 is available from http://www.arasanchess.org.

Changes in Arasan 20.0:
1) Fix to handling of UCI_LimitStrength and UCI_Elo options. These can now be processed in either order and will set the search strength correctly.
2) Revised tuning program. Only the "Texel" method is supported now. Tuner uses a "mean-squared error" objective and takes a labeled EPD file
as input. Various optimization methods can be selected. Optimization steps are scaled appropriately based on parameter ranges.
3) Changes to king cover and king safety scoring.
4) Bug fix: Ensure hash move is always checked for validity (was not being done for evasions).
5) Fix possible race condition updating root PV.
6) Add some utility programs and Python scripts to source package.
7) Considerable code cleanup and fixing warnings and possible bugs, notably in Syzygy tb code.

This version scores quite a bit better than 19.2 against a gauntlet of opponents in my fast time-control testing, but actually did not win a 2400 game blitz match directly against version 19.2. So I don't know quite what to make of that. Anyway I am releasing it, partly because of the bug fixes, and hopefully more testing will make clearer what the relative strength is.

--Jon

[Tony P.]

Congrats on the release!

Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.

Considering that any chess algorithm is necessarily highly heuristic (the search tree is pruned so the search doesn't always return the true minimax), the self-play failure means only that the tuning of v. 20.0 is unfit against one particular opponent - Arasan 19.2 - which tells little about its extent of optimality against a random computer, let alone human, adversary.

[jdart]

Well that is true to an extent and I do not usually rely on self-play results.

But there are two variables there: the time control is different and so is the opponent. So it is unknown to me at present if 20.0 just does worse against 19.2, or if the time control is a factor.

--Jon

[lkaufman]

Well that is true to an extent and I do not usually rely on self-play results.

But there are two variables there: the time control is different and so is the opponent. So it is unknown to me at present if 20.0 just does worse against 19.2, or if the time control is a factor.

--Jon

In my experience (with Rybka and Komodo) the time control is more likely to be a major factor than the choice of opponent. It's true that some engines do better against specific opponents, but when comparing versions of the same engine, unless really drastic changes have been made, the one that does better in self-play will also do better against a range of opponents. Clearly that principle has worked for the Stockfish team as well.

[Graham Banks]

Arasan 20.0 is available from http://www.arasanchess.org.

Thanks Jon.

[cdani]

Thanks for this new version!

Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.

Initially I used a gauntlet also for Andscacs, but maybe two years ago I went to selfplay, mostly due to the increased sensitivity that reduces the necessary number of games. Anyway from time to time I do a verification test against a gauntlet.

As the computer power goes slowly cheaper, I think that this will be an area of improvement of the engines at some point. I mean that of course an engine can be retuned to play more optimal against other engines.

As always, while you have a system of improvement that works, is difficult at least psychologically to go to another system.

[Tony P.]

As the computer power goes slowly cheaper, I think that this will be an area of improvement of the engines at some point. I mean that of course an engine can be retuned to play more optimal against other engines.

As always, while you have a system of improvement that works, is difficult at least psychologically to go to another system.

That's actually a deep topic on which I might post a separate thread soon. The thing is that, nowadays when any top engine is more than enough for postmortem blunder checks already, humans no longer require an engine to play even better vs other engines. They need it to show and explain devastating strategies (primarily opening novelties) against fellow humans whom the users are about to face over the board.

Thus, ideally, an engine would need to include separate opponent models whose thinking processes it would be emulating and adjusting against.

But this of course requires a ton of computational power or some kind of Monte Carlo search because the alpha-beta technique is obviously inapplicable if the opponent is assumed to evaluate positions differently or even have a different search routine from the engine's.

[Dann Corbit]

Big thanks,
There seems to be a real avalanche of fun the last few days.

[MikeB]

Thanks for this new version!

Btw, with the immense hardware power that allows to run blitz gauntlets fast enough nowadays, why do chess programmers still rely on the self-play result as an accurate measure of improvement? A gauntlet vs a mixture of opponents with different styles is so much closer to the conditions of official computer tournaments and matches.

Initially I used a gauntlet also for Andscacs, but maybe two years ago I went to selfplay, mostly due to the increased sensitivity that reduces the necessary number of games. Anyway from time to time I do a verification test against a gauntlet.
....

"...due to the increased sensitivity that reduces the necessary number of games..." agree 100% - I think the SF team has pretty much proven self-play works...

[MikeB]

Arasan 20.0 is available from http://www.arasanchess.org.

Changes in Arasan 20.0:
1) Fix to handling of UCI_LimitStrength and UCI_Elo options. These can now be processed in either order and will set the search strength correctly.
2) Revised tuning program. Only the "Texel" method is supported now. Tuner uses a "mean-squared error" objective and takes a labeled EPD file
as input. Various optimization methods can be selected. Optimization steps are scaled appropriately based on parameter ranges.
3) Changes to king cover and king safety scoring.
4) Bug fix: Ensure hash move is always checked for validity (was not being done for evasions).
5) Fix possible race condition updating root PV.
6) Add some utility programs and Python scripts to source package.
7) Considerable code cleanup and fixing warnings and possible bugs, notably in Syzygy tb code.

This version scores quite a bit better than 19.2 against a gauntlet of opponents in my fast time-control testing, but actually did not win a 2400 game blitz match directly against version 19.2. So I don't know quite what to make of that. Anyway I am releasing it, partly because of the bug fixes, and hopefully more testing will make clearer what the relative strength is.

--Jon

"...but actually did not win a 2400 game blitz match directly against version 19.2...." don't you hate that?

It's amazing you been doing this since 1994, I can still remember playing Arasan 1.0 when you first published it and it still runs today on my Mac Pro using wine ...amazing, just as much fun playing now as it was back then... if anybody else it looking for v1.0 , Jon has it on website , no need to go to a germ infested site to download it...