OKE - one year after

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Rebel
Posts: 5685
Joined: Thu Aug 18, 2011 10:04 am

OKE - one year after

Post by Rebel » Sun Sep 06, 2020 4:08 pm

June 2019
OKE is a research project how well nowadays (mostly top) chess engines play the early opening, the first 6 moves in particular. Since the arrival of the first serious rating list in 1979 -- the SSDF -- engines competed with opening books allowed, later in 2006 when CEGT and CCRL took over the role as leading rating lists that changed. Books were not allowed and engines were tested on pure engine strength. Yet engines for the most part were given the first 6-8-10-12 moves in advance to avoid doubles.

On this page we will show the early opening hasn't received the attention it deserved likely caused by the fact engines weren't challenged in this area. We will see engines rated hundreds elo points lower surpassing the stronger ones, in some extreme cases more than 500 elo points.

The message is that there is a lot room for improvement and we will offer a simple platform for engine programmers to improve in this area. The base is the program MEA from engine author Ferninand Mosca which allows engines to analyze all positions in an EPD file which contains a bonus for moves. Examples:

http://rebel13.nl/rebel13/oke.html

September 2020 - Update
1. We can see that the NNUE aproach did Stockfish 12 etremely good.
2. Lc0 running on a RTX 1060 6Gb remains best.
3. Rodent keeps doing well between the Giants

Next engines to test
Amoeba 3.1, Asymptote 0.8, Combusken 1.3, Counter 3.5, Defenchess 2.2, Demolito_2020_05_14, FabChess 1.15, Ice 4.0, Igel 2.60, Komodo 11, rofChade 2.3, RubiChess 1.8, Schooner 2.2, Vajolet 2.8, Wasp 4.00, Weiss 1.1, Winter 0.8, Xiphos 0.6
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Rebel
Posts: 5685
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - one year after

Post by Rebel » Sun Sep 06, 2020 9:12 pm

Ice, one of my favorite engines regarding playing style, 3th place after SF and Lc0.

http://rebel13.nl/html/3-moves.html
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Rebel
Posts: 5685
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - one year after

Post by Rebel » Mon Sep 07, 2020 4:03 am

Ice removed, I noticed it has an internal book.

Code: Select all

Deep iCE 4.0.853 x64/popcnt by Thomas Petzke, 2019.10.7
uci
id name Deep iCE 4.0.853 x64/popcnt
id author Thomas Petzke
option name Ponder type check default true
option name Threads type spin default 1 min 1 max 128
option name Hash type spin default 128 min 1 max 8192
option name Clear Hash type button
option name OwnBook type check default true
option name Book File type string default medium.ibx
option name Fallback to Internal Book type check default true
option name MultiPV type spin default 1 min 1 max 64
option name Output Search Depth Info type check default true
option name Output Current Move Info type check default false
option name Output Total Time and Nodes type check default true
option name Console Mode type check default false
option name CommandFile type string default <NONE>
info internal book positions 24278 moves 34622
uciok
90% of coding is debugging, the other 10% is writing bugs.

Dann Corbit
Posts: 11626
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: OKE - one year after

Post by Dann Corbit » Mon Sep 07, 2020 4:11 am

Looks like SF 12 has almost caught LC0 by 5 seconds.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Frank Quisinsky
Posts: 5160
Joined: Wed Nov 18, 2009 6:16 pm
Location: Trier, Germany
Contact:

Re: OKE - one year after

Post by Frank Quisinsky » Mon Sep 07, 2020 5:13 am

Hi Ed,

iCE have an own style and yes the style is interesting.
The eval is much higher as for other engines.

Same for Texel or Fizbo.

For the moment iCE have to many holes in king safty, easy to see with FCP Tourney stats.
For opening analyzes not one of the best engines (problems in closed positions).
But the engine is on the way and fact is the style is very own.

Nice program!
Not one of my favorits in question of "playing styles" but fact ist iCE have a very own and interesting style.

My TOPs in question of human-style!

1. Wasp
2. Fizbo
3. A lot of others I like here, Pedone or Andscacs for two examples more.

My TOPs in questions of most aggressive style:
What most humans like to see.

1. Stockfish
2. Booot
3. Andscacs, Wasp, Fritz 17, Fizbo, Komodo and different others like Schooner!

Strong allrounder for analyzes ... can find very deep "best moves".
1. Stockfish
2. Komodo
3. Ethereal
4. SlowChess, I saw here a lot of nice king attacks.

From Schooner, xiphos, Defenchess I must see more games, must have more stats.
But fact is, I like all of them.

iCE, Winter are on the way.
Arasan, Pedone are very interesting engines, with an own interesting style too. Arasan today, isn't only in endgames strong!!
Strongest fighter with small sources ... a little wonder in playing strength ... Demolito!!
A little wonder in strength is Senpai with the small sources too but Fabien should try to give Senpai an own face. Senpai is playing computer chess pure.

And Rodent is interesting with all the possibilites Rodent have.
Interesting for DGT-Pi !!

Best
Frank

Interesting is Chiron, Nemorino and Texel.
Never other engines can be sure that suddenly a danger combination ... very magic ... is on the board. Very interesting engines with an own style. I wish me that from Chiron will be newer versions available.
I like computer chess!

User avatar
Rebel
Posts: 5685
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - one year after

Post by Rebel » Tue Sep 08, 2020 7:06 am

Dann Corbit wrote:
Mon Sep 07, 2020 4:11 am
Looks like SF 12 has almost caught LC0 by 5 seconds.
How does SF NPS (1.2 million) vs Lc0 NPS (15.000) compare ?
90% of coding is debugging, the other 10% is writing bugs.

Dann Corbit
Posts: 11626
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: OKE - one year after

Post by Dann Corbit » Tue Sep 08, 2020 8:19 am

Well, considering that LC0 gets 1000 times as much computational power, I would say it fares very well.
As you know, the nodes and depths are not measuring the same thing anyway.
Both approaches are really interesting, and they are 1 and 2 in the TCEC contest right now
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
Rebel
Posts: 5685
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - one year after

Post by Rebel » Tue Sep 08, 2020 3:05 pm

90% of coding is debugging, the other 10% is writing bugs.

Terje
Posts: 257
Joined: Tue Nov 19, 2019 3:34 am
Location: https://github.com/TerjeKir/weiss
Full name: Terje Kirstihagen

Re: OKE - one year after

Post by Terje » Tue Sep 08, 2020 3:33 pm

As expected Weiss is no opening master :D

Would be interesting to see normal minic on there for comparison.

User avatar
xr_a_y
Posts: 1319
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: OKE - one year after

Post by xr_a_y » Tue Sep 08, 2020 3:50 pm

I guess most of that is due to the NNUE part of course and the strong Sergio net.

Post Reply