UCI protocol and engine strength

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
Kempelen
Posts: 620
Joined: Fri Feb 08, 2008 10:44 am
Location: Madrid - Spain

UCI protocol and engine strength

Post by Kempelen »

Hi
While doing UCI implementation I have noted my engine has notable ELO loss. I have lost a lot of time trying to figure what could cause this situation, but the implementation seems good and I dont seen any performance bottleneck. Talking to Pedro Castro some time ago he noted similar thing while testing Danasah. Has anybody noted something similar? any ideas?
Regards
Fermín
Fermin Serrano
Author of 'Rodin' engine
http://sites.google.com/site/clonfsp/
brtzsnr
Posts: 433
Joined: Fri Jan 16, 2015 4:02 pm

Re: UCI protocol and engine strength

Post by brtzsnr »

Hi Fermin!

Given that IO code does very little work, I find this surprising so we need more information to track down the regression.

How did you test the regression? Did you play enough tests? What's the Elo drop? A typical no-regression tests needs about 30k-40k for my engine.

As for sources of Elo loss:
- are you clearing the hash every position?
- does cpu profile shows anything unusual in the new UCI code?


Edit: For Zurichess I remember that printing the PV lines were very time consuming showing something 3-4 Elo drop. Newer versions of the Go compilers fixed that, fortunately.
mjlef
Posts: 1494
Joined: Thu Mar 30, 2006 2:08 pm

Re: UCI protocol and engine strength

Post by mjlef »

Kempelen wrote:Hi
While doing UCI implementation I have noted my engine has notable ELO loss. I have lost a lot of time trying to figure what could cause this situation, but the implementation seems good and I dont seen any performance bottleneck. Talking to Pedro Castro some time ago he noted similar thing while testing Danasah. Has anybody noted something similar? any ideas?
Regards
Fermín
One possibility is time it takes to send information to and from the engine. Most programs do periodic polling or run the I/O in another thread. In Komodo, we have an option to suppress much of the engine output for the first X plies. It just puts out the best move and a few other things, and not each current move notification and long PVs.
matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: UCI protocol and engine strength

Post by matthewlai »

Kempelen wrote:Hi
While doing UCI implementation I have noted my engine has notable ELO loss. I have lost a lot of time trying to figure what could cause this situation, but the implementation seems good and I dont seen any performance bottleneck. Talking to Pedro Castro some time ago he noted similar thing while testing Danasah. Has anybody noted something similar? any ideas?
Regards
Fermín
Time management bug would be my prime suspect. Does it use reasonable amount of time?
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: UCI protocol and engine strength

Post by Ferdy »

Kempelen wrote:Hi
While doing UCI implementation I have noted my engine has notable ELO loss. I have lost a lot of time trying to figure what could cause this situation, but the implementation seems good and I dont seen any performance bottleneck. Talking to Pedro Castro some time ago he noted similar thing while testing Danasah. Has anybody noted something similar? any ideas?
Regards
Fermín
position startpos moves e2e4 e7e5 ... <last move> till move 80 say.

When playing games (even in analysis mode), don't verify the legality of those moves. Verify only the last move sent by the gui. Save those moves incrementally in history, the first time you receive e2e4 save e2e4, you receive e2e4 e5e7, save e7e5 only because e2e4 was already in your move history. When you receive ucinewgame, clear your history. If last move is illegal, setup everything from start.
whereagles
Posts: 565
Joined: Thu Nov 13, 2014 12:03 pm

Re: UCI protocol and engine strength

Post by whereagles »

Possibly related to this, I noticed a very considerable slow-down from running Stockfish 7 under the ChessX interface on Mac. Using the SF7 Mac app, I get to much higher depths for the same time span.

Perhaps ChessX invokes SF7 from a clear cache whereas the Mac app keeps the cache? (Cache size is the same for both.)