skill levels of engines Stockfish, Igel, Phalanx, Crafty

[chesskobra]

I am trying to test skill levels of Stockfish, Igel, Phalanx and Crafty. I am running a round robin with these engines (SF 16, with skill levels 20, 19, 18, 17, and Igel 3.5.5 skill levels 20, 19, 18, 17) with a few other engines in the mix. I must admit that my test is on a very small scale on a laptop with Intel Core 5, with 4 GB ram. But what I observe is Stockfish strength drops drastically from skill level 20 (default) to 19, but 19,18,17 are pretty close. On the other hand Igel's drop in strength is more gradual. Also, for Phalanx XXV, from easy level 0 (default) to easy level 1, the drop is very big, where the easy levels are from 1 to 100.

Could someone please share data about this if they have done somewhat larger scale testing of skill levels.

[JoAnnP38]

I am trying to test skill levels of Stockfish, Igel, Phalanx and Crafty. I am running a round robin with these engines (SF 16, with skill levels 20, 19, 18, 17, and Igel 3.5.5 skill levels 20, 19, 18, 17) with a few other engines in the mix. I must admit that my test is on a very small scale on a laptop with Intel Core 5, with 4 GB ram. But what I observe is Stockfish strength drops drastically from skill level 20 (default) to 19, but 19,18,17 are pretty close. On the other hand Igel's drop in strength is more gradual. Also, for Phalanx XXV, from easy level 0 (default) to easy level 1, the drop is very big, where the easy levels are from 1 to 100.

Could someone please share data about this if they have done somewhat larger scale testing of skill levels.

I've been staying away from the SkillLevel settings and instead concentrating on the UCI "standard" UCI_LimitStrength and UCI_Elo. In theory, when UCI_LimitStrength is set to true then the engine should try to play at the Elo specified in UCI_Elo. However, in the two engines I have tested they seem to be really off from the Elo established by CCRL. For instance, SF 15.1 needs to be set to about 2150 and Komodo Dragon 3.2 needs to be set to 2650 to play around a 2700 Elo established by CCRL. I don't think SF really cares about these settings as of course they don't add Elo. But I wish they would come up with a way to scale them better.

[chesskobra]

The original motivation for such tests was to find engines or settings that can play in the Elo range 1800-2200, but that don't just play strong game most of the time and then randomly hang pieces. Also, the engines should play a reasonable endgame without tablebases. If anybody knows such engines, I would like to try them out.

What are some standard parameters that one should experiment with? I am also willing to mess with the code of an existing engine (if it is in C, works on linux, and is readable). I am considering phalanx, but would appreciate other suggestions.

[Alexander Schmidt]

I once put some effort in implementing the UCI_Limit_Stregth into Slowchess, you can find v2.960 here: https://drive.google.com/drive/folders/ ... bukS3wrgGg

The ELO is calibrated to the SSDF List and should at least be somehow linear.

Here are some of the results of engines with ELO set to 2000: https://open-chess.org/viewtopic.php?t=2167

I only used engines that limit the nps because I want them to play the same level on different systems. Also I dislike all kind of implemented stupidity that make engines play silly moves.

[chesskobra]

Is there a linux build of SlowChess 2.96? I found 2.9; does it have the limit Elo strength feature?

[Alexander Schmidt]

Is there a linux build of SlowChess 2.96? I found 2.9; does it have the limit Elo strength feature?

Jim Ablett made a linux build, but afaik it doesn't work correctly. I don't know about newer versions of Slowchess, but if it has the feature it is made by the original author.

http://kirr.homeunix.org/chess/engines/ ... ENGINES%20++/

Since I use linux since a while i can have a look on it next week.

[jdart]

Engines implement strength reduction differently, and I wouldn't expect the ratings when UCI_Elo is used to be comparable across engines.

To get to low rating levels, 2000 or below, you have to drastically limit search depth and also do other things. Arasan even allows some occasional blunders.