OKE - Opening Knowledge Engines

[Rebel]

Nice work Ferdy, runs smoothly. Some minor points.

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - c0 "c4=0";
rnbqkbnr/pppppppp/8/8/2P5/8/PP1PPPPP/RNBQKBNR b KQkq c3 c0 "Nf6=2";

1. Try this EPD, MEA --> ZeroDivisionError: float division by zero + crash.
Changing "c4=0"; to "c4=1"; and all is okay.

2. The EPD output looks great, 2 remarks/ wishes:
2a. Don't append, always overwrite. Time controls mixup else in one file.
2b. Store every EPD in an OUT folder, the main folder easily becomes messy and in a folder a delete all is more handy.

3. A wish regarding the time control, add "depth=" as an alternative for movetime. Movetime cancels the search in the middle of an iteration when time is up, depth always finishes the iteration. The easy way out, when movetime < 100 then that number is the iteration depth.

[Ferdy]

Nice work Ferdy, runs smoothly. Some minor points.

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - c0 "c4=0";
rnbqkbnr/pppppppp/8/8/2P5/8/PP1PPPPP/RNBQKBNR b KQkq c3 c0 "Nf6=2";

1. Try this EPD, MEA --> ZeroDivisionError: float division by zero + crash.
Changing "c4=0"; to "c4=1"; and all is okay.

Will fix this ASAP.

2. The EPD output looks great, 2 remarks/ wishes:
2a. Don't append, always overwrite. Time controls mixup else in one file.

I have a code deleting existing epd output filename, but there can be a bug somewhere, will look into it.
Sorry I don't understand by this "Time controls mixup else in one file."

2b. Store every EPD in an OUT folder, the main folder easily becomes messy and in a folder a delete all is more handy.

Right will save epd output to out folder.

3. A wish regarding the time control, add "depth=" as an alternative for movetime. Movetime cancels the search in the middle of an iteration when time is up, depth always finishes the iteration. The easy way out, when movetime < 100 then that number is the iteration depth.

A. For uci engines one can send

Code: Select all

go infinite

to stop the engine, one can send

Code: Select all

stop

Don't really like this as it interrupts the engine search.

B. Another case also for uci engines, one can send

Code: Select all

go movetime

with this I don't need to send the stop, uci engine will stop by itself.

In mea I use case B. So it is up to the engine how it would search based on allocated time.

There is one engine I observed that does not support movetime properly, and it will not stop, it continues to search, what I did with this engine is send the stop command.
In contrast there is one engine that does not support movetime properly if you send go movetime 60000, 60000 is just 60 sec or 1 minute, it will return its bestmove earlier than 10sec or so. mea does not have a work around on this type of engine. If I have to implement this, I can send go infinite to this engine and at appropriate time send the stop command, (assuming of course that such uci engine has support for go infinite) this would force the engine to send its bestmove.

I can implement your suggestion by adding a --depth option. Then I can send to uci engine,

Code: Select all

go movetime [time value in ms] depth [depth value]

It now depends on the engine how it would terminate its search based on movetime and depth values.

[Ferdy]

mea v0.3.3
https://drive.google.com/file/d/1SUte6B ... sp=sharing

Code: Select all

Update mea.py

* Fixed div by zero when move score is 0.
* Add --depth option, default is 128.
* Refactor epd output filenaming.
* Save epd output to epd_out folder.

Sample batch file, depth is added.

set MT=1000
set DEPTH=4
set HASH=256
set THREADS=1
set OUTPUT=2-moves.txt
set EPD=.\epd\2-moves.epd

mea.exe -h >help.txt

mea.exe --engine "./engines/Deuterium v14.3.34.130 32bit.exe" ^
--name "Deuterium v2014.3.34.130" --eoption "hash=256, multipv=8" --rating 2740 ^
--protocol uci --epd %EPD% --movetime %MT% --depth %DEPTH% --output %OUTPUT% --log

pause

[Ferdy]

mea v0.3.3
https://drive.google.com/file/d/1SUte6B ... sp=sharing

Code: Select all

Update mea.py

* Fixed div by zero when move score is 0.
* Add --depth option, default is 128.
* Refactor epd output filenaming.
* Save epd output to epd_out folder.

Sample batch file, depth is added.

set MT=1000
set DEPTH=4
set HASH=256
set THREADS=1
set OUTPUT=2-moves.txt
set EPD=.\epd\2-moves.epd

mea.exe -h >help.txt

mea.exe --engine "./engines/Deuterium v14.3.34.130 32bit.exe" ^
--name "Deuterium v2014.3.34.130" --eoption "hash=256, multipv=8" --rating 2740 ^
--protocol uci --epd %EPD% --movetime %MT% --depth %DEPTH% --output %OUTPUT% --log

pause

mea v0.3.4

Update mea.py

* Full epd line from input file is now shown in log file.
* Remove detection of different threads names such as cores, threads, cpus etc. User can now use --eoption "cores=1" or --eoption "Cpu=1". Only for uci engines.
* Check presence of actual log file itself before moving it to log directory.
* Check presence of epd output file itself before moving it to epd_out directory.
* Also print to console the overall performance result of winboard engines.

Updated mea.exe is same link to v0.3.3. Just redownload the mea.zip file.

[Rebel]

lc0 + Multi-pv is funny.

On Depth=1 every move has the same (ce) score.

Code: Select all

rnbqkb1r/pppp1ppp/4pn2/8/2P5/6P1/PP1PPP1P/RNBQKBNR w KQkq - id "2-moves.epd pos 74 MultiPV=1"; bm Nf3; ce 7; acd 1;
rnbqkb1r/pppp1ppp/4pn2/8/2P5/6P1/PP1PPP1P/RNBQKBNR w KQkq - id "2-moves.epd pos 74 MultiPV=2"; bm Bg2; ce 7; acd 1;
rnbqkb1r/pppp1ppp/4pn2/8/2P5/6P1/PP1PPP1P/RNBQKBNR w KQkq - id "2-moves.epd pos 74 MultiPV=3"; bm d4; ce 7; acd 1;

Depth=2 even more funny, cases of second and third best move having a higher score than the best move.

Code: Select all

rnbqkb1r/pp1ppppp/5n2/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - id "2-moves.epd pos 55 MultiPV=1"; bm e5; ce 2; acd 2;
rnbqkb1r/pp1ppppp/5n2/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - id "2-moves.epd pos 55 MultiPV=2"; bm Nc3; ce 5; acd 2;
rnbqkb1r/pp1ppppp/5n2/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - id "2-moves.epd pos 55 MultiPV=3"; bm d3; ce 5; acd 2;

[Ovyron]

Depth=2 even more funny, cases of second and third best move having a higher score than the best move.

Note this is more common than funny. There's even some positions where engines will not find the best move at all unless one uses MultiPV or excludes main moves from analysis.

(Example: a position where move B is best, but Stockfish after 5 minutes at depth 40 doesn't see the move and insists move A is best, but on MultiPV=4 it sees the move is best in 1 minute at depth 30, so when it does second will have a higher score than the best move, and if move C transposes it'll do it as well.)

It's so common programmers have made features that search like this on single PV (a single PV that uses MultiPV internally) like McCain's Tactical setting or Shash's "Deep Analysis Mode".

It's a feature that gives advantage to interactive analyzers of positions in correspondence chess (against opponents that just reach as much depth as possible, so they don't find this move with a score higher than the best), and happens to all engines that support MultiPV or Exclude Moves.

[Ferdy]

mea v0.3.5
https://drive.google.com/file/d/1SUte6B ... sp=sharing

Update mea.py

* Use python 3 subprocess module, tested on Python 3.6.
* Terminate engine process if it does not respond to quit command.
* Only send go movetime mt_value depth d_value if there is depth in --eoption, that is --eoption "depth=8, hash=128" otherwise just send go movetime mt_value.
* Remove --depth option.
* Fix bug, when there is no --eoption "threads=value", option --threads value is not followed and default engine threads is used. Now --threads value is followed. If --threads value is specified and --eoption "threads=value" is also specified in command line, threads=value found in --eoption will be the one to use.

Sample entry in a batch file.

set MT=1000
set HASH=256
set THREADS=1
set OUTPUT=2-moves.txt
set EPD=.\epd\2-moves.epd


mea.exe -h >help.txt


:: A. uci engine

:: (1) Example with depth limit and multipv
start "deuterium" mea.exe --engine ".\engines\Deuterium_v2019.1.36.50_x64_pop.exe" ^
--name "Deuterium v2019.1.36.50" --hash %HASH% --eoption "depth=8, multipv=3" ^
--rating 2800 --protocol uci --epd %EPD% --movetime %MT% ^
--output %OUTPUT% --log

[Rebel]

Lc0 and threads, something strange.

set THREADS=x works of course for engines except for Lc0. I checked the logfile, the uci command is ok.

Then I checked Arena, doesn't work also, always runs 1 thread whatever the setting.

Then I checked ChessPartner and Lc0 runs on the number of threads given.

Not sure if you can fix this.

[Ferdy]

Lc0 and threads, something strange.

set THREADS=x works of course for engines except for Lc0. I checked the logfile, the uci command is ok.

Then I checked Arena, doesn't work also, always runs 1 thread whatever the setting.

Then I checked ChessPartner and Lc0 runs on the number of threads given.

Not sure if you can fix this.

Did you use mea v0.3.5? It should work.

[Rebel]

Lc0 and threads, something strange.

set THREADS=x works of course for engines except for Lc0. I checked the logfile, the uci command is ok.

Then I checked Arena, doesn't work also, always runs 1 thread whatever the setting.

Then I checked ChessPartner and Lc0 runs on the number of threads given.

Not sure if you can fix this.

Did you use mea v0.3.5? It should work.

Works with 0.3.5 indeed. Running it directly from the download MEA gives an error message when the analysis is finished but all tasks are done properly as far as I can see.

Code: Select all

epd 403 / 404
epd 404 / 404
ExpectedTime     : 40.4s
ActualTime       : 41.2s
MarginTime/pos   : 0.2s
MarginTime       : 80.8s
usage: mea.exe [-h] -i EPD [-o OUTPUT] -e ENGINE [--eoption EOPTION] -n NAME
               [-t THREADS] [-m HASH] [-a MOVETIME] [-r RATING] [-p PROTOCOL]
               [-s {0,1}] [--stmode {0,1}] [--protover {1,2}] [--log]
mea.exe: error: the following arguments are required: -i/--epd, -e/--engine, -n/
--name

To see it happen run MEA directly without the START option.

mea.exe --engine ".\engines\Deuterium_v2019.1.36.50_x64_pop.exe" ^
--name "Deuterium v2019.1.36.50" --hash %HASH% --eoption "depth=8, multipv=3" ^
--rating 2800 --protocol uci --epd %EPD% --movetime %MT% ^
--output %OUTPUT% --log