Test epd for Linux ?

[hgm]

Yes, this should be simple. I could have it print total time used for all solved positions, or average time, or both. Just tell me what you would prefer. The current version might calculate the time to solution wrong anyway, I will also fix that.

Currently it can only do bm-type EPDs; I still would like it to handle avoid-moves too. I don't know if EPDs can have both a best move and an avoid move. This seems a theoretical possiblity. So I could award 0.5 point for playing another move than the avoid move, (i.e. count it as draw), and 1 point for playing the best move. It is not clear how I should measure the solving time in that case, however.

[Dann Corbit]

Yes, this should be simple. I could have it print total time used for all solved positions, or average time, or both. Just tell me what you would prefer. The current version might calculate the time to solution wrong anyway, I will also fix that.

Currently it can only do bm-type EPDs; I still would like it to handle avoid-moves too. I don't know if EPDs can have both a best move and an avoid move. This seems a theoretical possiblity. So I could award 0.5 point for playing another move than the avoid move, (i.e. count it as draw), and 1 point for playing the best move. It is not clear how I should measure the solving time in that case, however.

I have many problems with both AM and BM.
Here is the value:
The AM is a move that looks like a solution. But in the end it loses out badly compared to some other choices. So if your engine finds the AM move and gives it a good score, it is a hint that you did not find a cook, but rather that your program blundered.

If a program has both AM and BM it must pick a move from the BM list to get the right answer. The AM is just a hint to say, "No, really, it simply is not this move that is the correct answer and if you think it is a new solution, I promise you it is not."

[hgm]

Do I understand correctly the bm does not need to be a single move, but can be a list? Currently I only check for the first move after " bm ".

If a move is an am, (or in the am list) the problem should in any case be considered unsolved. A move that is neither in the am or bm list can be considered half-solved, and can be counted as a draw. I am not sure what I should use as solving time when the engine first half-solved it, then fully solves it, to then switches back to a half-solution. Should I include the time for half-solved positions in the average at all? Or should there be a separate average for half-solved problems?

In any case, I now pushed a commit that correctly calculates the solving time, for bm problems (with a single bm), and prints the average time in the engnine-output pane at the end. There is one problem: XBoard does not allow you to copy-paste from that window after popping up the 'match' result, and exits when you close that popup. That is, when you started it with -mg on the command line. So the recommended way to use it would be to only put the -epd option on the command line, and start the match through the Tournament Options dialog (instating the EPD file as position file, and requesting the desired number of match games), and then press the Machine Match option in the mode menu. For such interactively started matches XBoard will not exit after the match finishes, so that you will have the opportunity to copy-paste the result.

[hgm]

I now pushed a patch that can handle a list of bm moves. (Supposedly. I had no way to test it.) No handling of am yet.

[hgm]

The latest patch now also checks for avoid moves. When no bm was given, avoiding all of the avoid moves counts as solving the problem. When a bm was given it just decides whether not having a bm is considered a draw or a loss. (Without am field not solving the bm will now also be counted as draw).

All of this untested, as I did not have the appropriate EPDs to test it. (It works on EPD files that specify single best moves, though.)

As far as I am concerned this completes XBoard/WinBoard's EPD capabilities. If it should do more, or should do things differently, please tell me what or how.

[Canoike]

Do I understand correctly the bm does not need to be a single move, but can be a list? Currently I only check for the first move after " bm ".

In any case, I now pushed a commit that correctly calculates the solving time, for bm problems (with a single bm), and prints the average time in the engnine-output pane at the end. That is, when you started it with -mg on the command line. So the recommended way to use it would be to only put the -epd option on the command line, and start the match through the Tournament Options dialog (instating the EPD file as position file, and requesting the desired number of match games), and then press the Machine Match option in the mode menu. For such interactively started matches XBoard will not exit after the match finishes, so that you will have the opportunity to copy-paste the result.

Thank you.
I need a global time of all the best moves found.
If I want to use a UCI engine instead of fairymax, I get an error message "Error writing to first chess program: broken pipe".
The clock should stop and go to the next position when the engine finds the bm for 3 consecutive plies. No need to use all the maximum allowed time.

[hgm]

Currently XBoard prints the average time for all the solved moves. This seems a more sensible measure than the total time, which would go up when the engine solves more problems. I now have it print the total time (in parentheses) as well, though.

As to the UCI engines: it doesn't do that for me. But it could depend on the engine. At the moment XBoard does start the second engine, even if it doesn't ever get to move in -epd mode. And in the latest XBoard version the default for the second engine is the same as the first engine. But if that engine is UCI, and you have not specified -sUCI, XBoard will send WB-protocol commands to that engine. It could be that the engine chokes on that, and exits.

So try to add -sUCI, and see if the problem persists.

[hgm]

BTW, I have my doubts on that "3 consecutive moves" rule. An engine could give the correct solution at 1, 2 and 3 ply for silly reasons, and then change to something completely different. Counting the problem as rapidly solved in that case seems just wrong...

[jdart]

I posted a Python script to do this some time back. Latest version below. Requires Python 3 and python-chess (https://pypi.python.org/pypi/python-chess).

Usage:
analyze.py -c <cores> -H <hash size in MB> -t <time in sec.> -m <multipv count> -e <engine path> <position file>

Code: Select all

#!/usr/bin/python3
# -*- coding: utf-8 -*-

import re, sys, subprocess, time, chess, chess.uci

engine_name = 'stockfish'

hash_size = 4000

multi_pv_value = 3

search_time = 60

cores = 1

solved = 0

results = {}

done = False

bestmove = None

class MyInfoHandler(chess.uci.InfoHandler):
    pat = re.compile("multipv [0-9]+")
    pat2 = re.compile(" pv ")

    def pre_info(self,line):
        super(MyInfoHandler, self).pre_info(line)
        global results
        #print(line)
        match = MyInfoHandler.pat.search(line)
        if match != None:
            parts = match.group().split()
            if len(parts)>1:
                results[int(parts[1])] = line
        elif multi_pv_value == 1:
            match = MyInfoHandler.pat2.search(line)
            if match != None:
                results[1] = line

    def on_bestmove(self,bestm,ponder):
        global bestmove
        global done
        uci_move = str(bestm)
        print("bestmove " + uci_move)
        bestmove = bestm
        done = True

def init():
    global hash_size
    global cores
    global multi_pv_value
    engine.uci()
    engine.setoption({'Hash': hash_size, 'Threads': cores, 'MultiPV': multi_pv_value})
    engine.isready()                       
    info_handler = MyInfoHandler()
    engine.info_handlers.append(info_handler)
    print("engine ready")            

def search(engine,board,ops,search_time):
    global results
    engine.position(board)
    engine.go(movetime=search_time)
    # print last group of results
    for i in range(1,multi_pv_value+1):
        print(results[i])

def main(argv = None):
    global search_time
    global engine_name
    global engine
    global cores
    global hash_size
    global multi_pv_value
    global bestmove
    global solved

    if argv is None:
        argv = sys.argv[1:]

    arg = 0
    while (&#40;arg < len&#40;argv&#41;) and &#40;argv&#91;arg&#93;&#91;0&#58;1&#93; == '-'))&#58;
        if &#40;argv&#91;arg&#93;&#91;1&#93; == 'c')&#58;
            arg = arg + 1
            if &#40;arg < len&#40;argv&#41;)&#58;
                try&#58;
                    cores = int&#40;argv&#91;arg&#93;)
                except exceptions.ValueError&#58;
                    print&#40;('Invalid value for parameter %s&#58; %s' % &#40;argv&#91;i&#93;, argv&#91;i + 1&#93;)),file=sys.stderr&#41;
                    return 2
            arg = arg + 1
        if &#40;argv&#91;arg&#93;&#91;1&#93; == 't')&#58;
            arg = arg + 1
            if &#40;arg < len&#40;argv&#41;)&#58;
                try&#58;
                    search_time = int&#40;argv&#91;arg&#93;)
                except exceptions.ValueError&#58;
                    print&#40;('Invalid value for parameter %s&#58; %s' % &#40;argv&#91;i&#93;, argv&#91;i + 1&#93;)),file=sys.stderr&#41;
                    return 2
            arg = arg + 1
        elif &#40;argv&#91;arg&#93;&#91;1&#93; == 'e')&#58;
            arg = arg + 1
            if &#40;arg < len&#40;argv&#41;)&#58;
                engine_name = argv&#91;arg&#93;
                arg = arg + 1
        elif &#40;argv&#91;arg&#93;&#91;1&#93; == 'H')&#58;
            arg = arg + 1
            if &#40;arg < len&#40;argv&#41;)&#58;
                try&#58;
                    hash_size = int&#40;argv&#91;arg&#93;)
                except exceptions.ValueError&#58;
                    print&#40;('Invalid value for parameter %s&#58; %s' % &#40;argv&#91;i&#93;, argv&#91;i + 1&#93;)),file=sys.stderr&#41;
                    return 2
            arg = arg+1
        elif &#40;argv&#91;arg&#93;&#91;1&#93; == 'm')&#58;
            arg = arg + 1
            if &#40;arg < len&#40;argv&#41;)&#58;
                try&#58;
                    multi_pv_value = int&#40;argv&#91;arg&#93;)
                except exceptions.ValueError&#58;
                    print&#40;('Invalid value for parameter %s&#58; %s' % &#40;argv&#91;i&#93;, argv&#91;i + 1&#93;)),file=sys.stderr&#41;
                    return 2
            arg = arg+1
        else&#58;
            print&#40;"Unrecognized switch&#58; " + argv&#91;arg&#93;, file=sys.stderr&#41;
            return
    time = search_time*1000

    if &#40;arg >= len&#40;argv&#41;)&#58;
        print&#40;"Expected a filename to analyze.", file=sys.stderr&#41;
        return

    try&#58;
        engine = chess.uci.popen_engine&#40;engine_name&#41;
    except&#58;
       print&#40;"failed to start child process " + engine_name, file=sys.stderr&#41;
       return

    init&#40;)

    pat = re.compile&#40;'^((&#91;pnbrqkPNBRQK1-8&#93;)+\/)+(&#91;pnbrqkPNBRQK1-8&#93;)+ &#91;wb&#93;+ &#91;\-kqKQ&#93;+ &#91;\-a-h1-8&#93;+')

    with open&#40;argv&#91;arg&#93;) as f&#58;
        for line in f&#58;
           # skip blank lines
           if len&#40;line.strip&#40;))==0&#58;
               continue
           m = pat.search&#40;line&#41;
           if m == None&#58;
               print&#40;"error&#58; invalid FEN in line&#58; %s" % line, file=sys.stderr&#41;
           else&#58;    
               print&#40;)
               print&#40;line&#41;
               board = chess.Board&#40;fen=m.group&#40;)+' 0 1')
               ops = board.set_epd&#40;line&#41;
               correct = 0
               bestmove = None
               search&#40;engine,board,ops,time&#41;
               if &#40;done and bestmove != None&#41;&#58;
                   if not ('bm' in ops&#41; and not ('am' in ops&#41;&#58;
                      print&#40;"error&#58; missing target move&#40;s&#41; in line&#58; " + line,file=sys.stderr&#41;
                   elif 'bm' in ops and bestmove in ops&#91;'bm'&#93;&#58;
                       correct = 1
                   elif 'am' in ops and not &#40;bestmove in ops&#91;'am'&#93;)&#58;
                       correct = 1
                   if correct != 0&#58;
                       print&#40;"++ solved")
                       solved = solved + 1
                   else&#58;
                       print&#40;"-- not solved")

    engine.quit&#40;)
    print&#40;)
    print&#40;"solved&#58; " + str&#40;solved&#41;)

if __name__ == "__main__"&#58;
    sys.exit&#40;main&#40;))

[Canoike]

I'm trying to make the UCI engine work without success.
./xboard -epd -sUCI
Is it right ? and after ?
Can you show an example please ?