OKE - Opening Knowledge Engines

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Rebel
Posts: 4663
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - Opening Knowledge Engines

Post by Rebel » Mon Jun 17, 2019 9:22 am

Nice work Ferdy, runs smoothly. Some minor points.

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - c0 "c4=0";
rnbqkbnr/pppppppp/8/8/2P5/8/PP1PPPPP/RNBQKBNR b KQkq c3 c0 "Nf6=2";
1. Try this EPD, MEA --> ZeroDivisionError: float division by zero + crash.
Changing "c4=0"; to "c4=1"; and all is okay.

2. The EPD output looks great, 2 remarks/ wishes:
2a. Don't append, always overwrite. Time controls mixup else in one file.
2b. Store every EPD in an OUT folder, the main folder easily becomes messy and in a folder a delete all is more handy.

3. A wish regarding the time control, add "depth=" as an alternative for movetime. Movetime cancels the search in the middle of an iteration when time is up, depth always finishes the iteration. The easy way out, when movetime < 100 then that number is the iteration depth.
Everybody is unique, except me.

Ferdy
Posts: 4079
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: OKE - Opening Knowledge Engines

Post by Ferdy » Mon Jun 17, 2019 10:06 am

Rebel wrote:
Mon Jun 17, 2019 9:22 am
Nice work Ferdy, runs smoothly. Some minor points.

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - c0 "c4=0";
rnbqkbnr/pppppppp/8/8/2P5/8/PP1PPPPP/RNBQKBNR b KQkq c3 c0 "Nf6=2";
1. Try this EPD, MEA --> ZeroDivisionError: float division by zero + crash.
Changing "c4=0"; to "c4=1"; and all is okay.
Will fix this ASAP.
Rebel wrote:
Mon Jun 17, 2019 9:22 am
2. The EPD output looks great, 2 remarks/ wishes:
2a. Don't append, always overwrite. Time controls mixup else in one file.
I have a code deleting existing epd output filename, but there can be a bug somewhere, will look into it.
Sorry I don't understand by this "Time controls mixup else in one file."
Rebel wrote:
Mon Jun 17, 2019 9:22 am
2b. Store every EPD in an OUT folder, the main folder easily becomes messy and in a folder a delete all is more handy.
Right will save epd output to out folder.
Rebel wrote:
Mon Jun 17, 2019 9:22 am
3. A wish regarding the time control, add "depth=" as an alternative for movetime. Movetime cancels the search in the middle of an iteration when time is up, depth always finishes the iteration. The easy way out, when movetime < 100 then that number is the iteration depth.
A. For uci engines one can send

Code: Select all

go infinite
to stop the engine, one can send

Code: Select all

stop
Don't really like this as it interrupts the engine search.

B. Another case also for uci engines, one can send

Code: Select all

go movetime
with this I don't need to send the stop, uci engine will stop by itself.

In mea I use case B. So it is up to the engine how it would search based on allocated time.

There is one engine I observed that does not support movetime properly, and it will not stop, it continues to search, what I did with this engine is send the stop command.
In contrast there is one engine that does not support movetime properly if you send go movetime 60000, 60000 is just 60 sec or 1 minute, it will return its bestmove earlier than 10sec or so. mea does not have a work around on this type of engine. If I have to implement this, I can send go infinite to this engine and at appropriate time send the stop command, (assuming of course that such uci engine has support for go infinite) this would force the engine to send its bestmove.

I can implement your suggestion by adding a --depth option. Then I can send to uci engine,

Code: Select all

go movetime [time value in ms] depth [depth value]
It now depends on the engine how it would terminate its search based on movetime and depth values.

Ferdy
Posts: 4079
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: OKE - Opening Knowledge Engines

Post by Ferdy » Mon Jun 17, 2019 12:29 pm

mea v0.3.3
https://drive.google.com/file/d/1SUte6B ... sp=sharing

Code: Select all

Update mea.py

* Fixed div by zero when move score is 0.
* Add --depth option, default is 128.
* Refactor epd output filenaming.
* Save epd output to epd_out folder.
Sample batch file, depth is added.
set MT=1000
set DEPTH=4
set HASH=256
set THREADS=1
set OUTPUT=2-moves.txt
set EPD=.\epd\2-moves.epd

mea.exe -h >help.txt

mea.exe --engine "./engines/Deuterium v14.3.34.130 32bit.exe" ^
--name "Deuterium v2014.3.34.130" --eoption "hash=256, multipv=8" --rating 2740 ^
--protocol uci --epd %EPD% --movetime %MT% --depth %DEPTH% --output %OUTPUT% --log

pause

Ferdy
Posts: 4079
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: OKE - Opening Knowledge Engines

Post by Ferdy » Mon Jun 17, 2019 3:59 pm

Ferdy wrote:
Mon Jun 17, 2019 12:29 pm
mea v0.3.3
https://drive.google.com/file/d/1SUte6B ... sp=sharing

Code: Select all

Update mea.py

* Fixed div by zero when move score is 0.
* Add --depth option, default is 128.
* Refactor epd output filenaming.
* Save epd output to epd_out folder.
Sample batch file, depth is added.
set MT=1000
set DEPTH=4
set HASH=256
set THREADS=1
set OUTPUT=2-moves.txt
set EPD=.\epd\2-moves.epd

mea.exe -h >help.txt

mea.exe --engine "./engines/Deuterium v14.3.34.130 32bit.exe" ^
--name "Deuterium v2014.3.34.130" --eoption "hash=256, multipv=8" --rating 2740 ^
--protocol uci --epd %EPD% --movetime %MT% --depth %DEPTH% --output %OUTPUT% --log

pause
mea v0.3.4

Update mea.py

* Full epd line from input file is now shown in log file.
* Remove detection of different threads names such as cores, threads, cpus etc. User can now use --eoption "cores=1" or --eoption "Cpu=1". Only for uci engines.
* Check presence of actual log file itself before moving it to log directory.
* Check presence of epd output file itself before moving it to epd_out directory.
* Also print to console the overall performance result of winboard engines.

Updated mea.exe is same link to v0.3.3. Just redownload the mea.zip file.

User avatar
Rebel
Posts: 4663
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - Opening Knowledge Engines

Post by Rebel » Mon Jun 17, 2019 10:47 pm

lc0 + Multi-pv is funny.

On Depth=1 every move has the same (ce) score.

Code: Select all

rnbqkb1r/pppp1ppp/4pn2/8/2P5/6P1/PP1PPP1P/RNBQKBNR w KQkq - id "2-moves.epd pos 74 MultiPV=1"; bm Nf3; ce 7; acd 1;
rnbqkb1r/pppp1ppp/4pn2/8/2P5/6P1/PP1PPP1P/RNBQKBNR w KQkq - id "2-moves.epd pos 74 MultiPV=2"; bm Bg2; ce 7; acd 1;
rnbqkb1r/pppp1ppp/4pn2/8/2P5/6P1/PP1PPP1P/RNBQKBNR w KQkq - id "2-moves.epd pos 74 MultiPV=3"; bm d4; ce 7; acd 1;
Depth=2 even more funny, cases of second and third best move having a higher score than the best move.

Code: Select all

rnbqkb1r/pp1ppppp/5n2/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - id "2-moves.epd pos 55 MultiPV=1"; bm e5; ce 2; acd 2;
rnbqkb1r/pp1ppppp/5n2/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - id "2-moves.epd pos 55 MultiPV=2"; bm Nc3; ce 5; acd 2;
rnbqkb1r/pp1ppppp/5n2/2p5/4P3/1P6/P1PP1PPP/RNBQKBNR w KQkq - id "2-moves.epd pos 55 MultiPV=3"; bm d3; ce 5; acd 2;
Everybody is unique, except me.

User avatar
Ovyron
Posts: 2519
Joined: Tue Jul 03, 2007 2:30 am

Re: OKE - Opening Knowledge Engines

Post by Ovyron » Tue Jun 18, 2019 1:24 am

Rebel wrote:
Mon Jun 17, 2019 10:47 pm
Depth=2 even more funny, cases of second and third best move having a higher score than the best move.
Note this is more common than funny. There's even some positions where engines will not find the best move at all unless one uses MultiPV or excludes main moves from analysis.

(Example: a position where move B is best, but Stockfish after 5 minutes at depth 40 doesn't see the move and insists move A is best, but on MultiPV=4 it sees the move is best in 1 minute at depth 30, so when it does second will have a higher score than the best move, and if move C transposes it'll do it as well.)

It's so common programmers have made features that search like this on single PV (a single PV that uses MultiPV internally) like McCain's Tactical setting or Shash's "Deep Analysis Mode".

It's a feature that gives advantage to interactive analyzers of positions in correspondence chess (against opponents that just reach as much depth as possible, so they don't find this move with a score higher than the best), and happens to all engines that support MultiPV or Exclude Moves.
Great spirits have always encountered violent opposition from mediocre minds.

Ferdy
Posts: 4079
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: OKE - Opening Knowledge Engines

Post by Ferdy » Tue Jun 18, 2019 10:39 am

mea v0.3.5
https://drive.google.com/file/d/1SUte6B ... sp=sharing
Update mea.py

* Use python 3 subprocess module, tested on Python 3.6.
* Terminate engine process if it does not respond to quit command.
* Only send go movetime mt_value depth d_value if there is depth in --eoption, that is --eoption "depth=8, hash=128" otherwise just send go movetime mt_value.
* Remove --depth option.
* Fix bug, when there is no --eoption "threads=value", option --threads value is not followed and default engine threads is used. Now --threads value is followed. If --threads value is specified and --eoption "threads=value" is also specified in command line, threads=value found in --eoption will be the one to use.
Sample entry in a batch file.
set MT=1000
set HASH=256
set THREADS=1
set OUTPUT=2-moves.txt
set EPD=.\epd\2-moves.epd


mea.exe -h >help.txt


:: A. uci engine

:: (1) Example with depth limit and multipv
start "deuterium" mea.exe --engine ".\engines\Deuterium_v2019.1.36.50_x64_pop.exe" ^
--name "Deuterium v2019.1.36.50" --hash %HASH% --eoption "depth=8, multipv=3" ^
--rating 2800 --protocol uci --epd %EPD% --movetime %MT% ^
--output %OUTPUT% --log

User avatar
Rebel
Posts: 4663
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - Opening Knowledge Engines

Post by Rebel » Sat Jun 22, 2019 8:01 am

Lc0 and threads, something strange.

set THREADS=x works of course for engines except for Lc0. I checked the logfile, the uci command is ok.

Then I checked Arena, doesn't work also, always runs 1 thread whatever the setting.

Then I checked ChessPartner and Lc0 runs on the number of threads given.

Not sure if you can fix this.
Everybody is unique, except me.

Ferdy
Posts: 4079
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: OKE - Opening Knowledge Engines

Post by Ferdy » Sat Jun 22, 2019 9:14 am

Rebel wrote:
Sat Jun 22, 2019 8:01 am
Lc0 and threads, something strange.

set THREADS=x works of course for engines except for Lc0. I checked the logfile, the uci command is ok.

Then I checked Arena, doesn't work also, always runs 1 thread whatever the setting.

Then I checked ChessPartner and Lc0 runs on the number of threads given.

Not sure if you can fix this.
Did you use mea v0.3.5? It should work.

User avatar
Rebel
Posts: 4663
Joined: Thu Aug 18, 2011 10:04 am

Re: OKE - Opening Knowledge Engines

Post by Rebel » Sun Jun 23, 2019 6:02 am

Ferdy wrote:
Sat Jun 22, 2019 9:14 am
Rebel wrote:
Sat Jun 22, 2019 8:01 am
Lc0 and threads, something strange.

set THREADS=x works of course for engines except for Lc0. I checked the logfile, the uci command is ok.

Then I checked Arena, doesn't work also, always runs 1 thread whatever the setting.

Then I checked ChessPartner and Lc0 runs on the number of threads given.

Not sure if you can fix this.
Did you use mea v0.3.5? It should work.
Works with 0.3.5 indeed. Running it directly from the download MEA gives an error message when the analysis is finished but all tasks are done properly as far as I can see.

Code: Select all

epd 403 / 404
epd 404 / 404
ExpectedTime     : 40.4s
ActualTime       : 41.2s
MarginTime/pos   : 0.2s
MarginTime       : 80.8s
usage: mea.exe [-h] -i EPD [-o OUTPUT] -e ENGINE [--eoption EOPTION] -n NAME
               [-t THREADS] [-m HASH] [-a MOVETIME] [-r RATING] [-p PROTOCOL]
               [-s {0,1}] [--stmode {0,1}] [--protover {1,2}] [--log]
mea.exe: error: the following arguments are required: -i/--epd, -e/--engine, -n/
--name
To see it happen run MEA directly without the START option.

mea.exe --engine ".\engines\Deuterium_v2019.1.36.50_x64_pop.exe" ^
--name "Deuterium v2019.1.36.50" --hash %HASH% --eoption "depth=8, multipv=3" ^
--rating 2800 --protocol uci --epd %EPD% --movetime %MT% ^
--output %OUTPUT% --log
Everybody is unique, except me.

Post Reply