A program that should be written

[sje]

A program that should be written: ucu, the Unified Chess Utility

There are a lot of common chess tasks that aren't specific to a particular engine, yet there isn't a common, portable open source utility to handle these. I propose the specification and coding of the command line utility program "ucu" that will handle these tasks.

The program would operate primarily in batch mode, although a simple interactive facility could be included. The utility would have a simple internal engine and could also interface to various other engines. Command line parameters would be sufficient for most actions, although alternate command and data file inputs could be specified. Output would appear on the standard output file unless an alternate output file were specified.

The Big Idea here is that such a utility could be called from a program instead of having the program's author write, test, and integrate his/her own code.

All command line parameters would be keyword based. The only required parameter would be "cmd" and its argument would identify the basic command request. An often used parameter would be "fen" and its argument would specify the primary position of interest.

Some examples:

(Enumerate movepaths to six ply for the given position)

ucu -cmd emp -ply 6 -fen k/8/8/8/8/8/8/4K2R w K - 0 1

(Output: 374670)

----

(Identify opening in the current locale language)

ucu -cmd opening -fen r1bqkbnr/1ppp1ppp/p1n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R w KQkq - 0 4

(Output: Ruy Lopez (Spanish opening))

[Aleks Peshkov]

It would be nice to have utility for:
1) running multi-engine tournaments with PGN output and ELO-rating;
2) running EPD batch tests collecting timing and performance;
3) connecting to available online chess servers and collecting PGN games.

IMHO it should be a run in host mode and be transparent for UCI-engines.

[sje]

Of course, the program would be open source and licensed under the GPL or the BSD agreement as appropriate. The source language would be ANSI C++ and the utility would be easily built and run on any sane Unix like platform. In the tradition of the most basic Unix utility programs, it would be easily configured as a data stream filter and would not be burdened with an unnecessary GUI.

The program would have both offset and bitboard generators and would use either or both as appropriate. One idea here is to deploy these as benchmark generators for measuring a chess MIPS metric for different machines.

The ucu program would be of great help for new authors and also established authors starting a new program. Pages and pages of opening book code, tablebase access code, game database code, and other non trivial grunt work could be delegated to a common utility.

--------

Other features (feel free to add your ideas):

1. ECO/NIC/SOC code determination

2. Dynamic opening book updating

3. HTML/PDF/PostScript/image generation (positions, games)

4. Verification and normalization of EPD and PGN datasets

5. Test suite record retrieval

6. Performance report generator

--------

Oh, and "ucu" is pronounced "you-chew".

[Uri Blass]

Of course, the program would be open source and licensed under the GPL or the BSD agreement as appropriate. The source language would be ANSI C++ and the utility would be easily built and run on any sane Unix like platform. In the tradition of the most basic Unix utility programs, it would be easily configured as a data stream filter and would not be burdened with an unnecessary GUI.

The program would have both offset and bitboard generators and would use either or both as appropriate. One idea here is to deploy these as benchmark generators for measuring a chess MIPS metric for different machines.

The ucu program would be of great help for new authors and also established authors starting a new program. Pages and pages of opening book code, tablebase access code, game database code, and other non trivial grunt work could be delegated to a common utility.

--------

Other features (feel free to add your ideas):

1. ECO/NIC/SOC code determination

2. Dynamic opening book updating

3. HTML/PDF/PostScript/image generation (positions, games)

4. Verification and normalization of EPD and PGN datasets

5. Test suite record retrieval

6. Performance report generator

--------

Oh, and "ucu" is pronounced "you-chew".

I would like the following option.

running 2 uci engines to fixed depth on some epd file and
give data to show the position when they have the biggest difference.

if engine A has evaluations
1)20.21 place 2
2)25.34 place 1
3)1.23 place 3
4)0.45 place 5
5)0.95 place 4
6)-1.45 place 6
7)-1.47 place 7

when engine B has evaluations
1)30.21 place 1(difference in places 2-1=1)
2)25.34 place 2(difference in places 2-1=1)
3)-0.19 place 6(difference in places 6-3=3)
4)0.57 place 3(difference in places 5-3=2)
5)0.37 place 4(difference in places 4-4=0)
6)-1.2 place 7(difference in places 7-6=1)
7)-0.18 place 5(difference in place 7-5=2)

It means that the biggest difference is in position 3 that changed 3 places in the list inspite of the fact that there is a difference of 10 pawns in position number 1 so the output should be
1)fen of position 3
2)fen of positions 4,7
3)fen of positions 1,2,6

Considering the fact that different programs may use different values for pieces difference in evaluation is not very important but difference in relative place of evaluation is important and finding positions when there is a big difference in relative place of evaluations may help to detect bugs.

I would like some tool that give both movei and rybka to analyze many positions to fixed depth in order to see the positions when movei and rybka has the biggest relative difference because it may give me hints what I need to fix in the evaluation.

Uri

[Marc Lacrosse]

It would be nice to have utility for ...

Most of these requested utilities already exist ...

1) running multi-engine tournaments with PGN output

There are quite a few good ones.
My present favorite is Pradu Kannan's PWBTM:
http://www.prism.gatech.edu/~gtg365v/PSWBTM/

and ELO-rating;

Bayeselo is perfect for this, better than EloStat.
http://remi.coulom.free.fr/Bayesian-Elo/

2) running EPD batch tests collecting timing and performance;

Lots of good ones too.
T Mayer's improved EPD2WB :
http://www.quarkchess.de/quark/epd2wb.zip
or undocumented Polyglot functions do this very well (see at end of this post)
PGSCID can also do this ( http://prolinux.free.fr/scid/fr/ )

3) connecting to available online chess servers and collecting PGN games.
IMHO it should be a run in host mode and be transparent for UCI-engines.

Formally forbidden on ICC but I published more than 75000 top-class games collected from ICC in 2006 here and on my site :
http://chessbazaar.mylivepage.com/file/ ... _ICC_games.
I promised not to continue publishing them and still hope ICC will offer their games for free.
Inbetween, automating game collection on ICC is in fact easy...
Besides they are quite a handful of sites with thousands of Playchess games...

Marc

EPD tests with Polyglot :

Its an undocumented feature

In the directory where I have Polyglot installed

I have the following command in a .bat file:

Code:

polyglot epd-test -epd GF1hour(2).epd -min-time 64 -max-time 3840 -depth-delta 5 > GF1hour(2)(64mLaptop-GF).txt


GF1hour(2).epd is my test suite
min-time is in seconds - 64 seconds
max-time is in seconds - 3840 secomds (64 minutes)
depth-delta assumed found when solution does not change over 5 iterations

.ini :
[PolyGlot]
EngineDir = .
EngineCommand = GF.exe
Log = false
LogFile = GFruit-4bx.log
[Engine]
Hash = 256
NalimovPath = C:\TB
NalimovCache = 128

other command line options are: -min-depth, -max-depth - I have not used these

Another useful option is -depth-delta, which is the minimum number of consecutive iterations the program must keep the correct best move before it is accepted as a solution. The default value is 3.

The output looks like this:

"ECM.1622" 0 162 182 - 4 0.01 2619 +0.57 Bd2 h6 Nf3 h5 Qh3 Ba4 Rab1 Qd7 Ng5 Rfe8 g4 Nc6 Qf3 hxg4 Qxg4
"ECM.1623" 0 162 183 - 8 0.49 303773 -0.14 Qd4 f6 Ng5 Bg7 Ne6 Qe7 Nxc7 Qxc7 Qe4 Qf7 Qf3 h5 Rh4
162/183 - 7.9 1.19 867464

The different columns in the output are: Fen ID string, solved or not (1 means solved, 0 means not solved), number of positions so far, number of correct solutions so far, search depth at which the solution was found, time to solution, nodes to solution, score, and PV.

The final line, which reads "162/183 - 7.9 1.19 867464", is a result summary. It means that 162 of 183 positions were solved, that the average depth to solution was 7.9 plies, the average time to solution was 1.19 seconds, and the average nodes to solution was 867464.

[sje]

I'm not very much interested in having the ucu program be capable of connecting to hordes of third party programs due in part to the relative brittleness of the interface specifications of said programs. Also, any connections that have tricky dependencies that can't be handled by basic Posix routines are out of luck.

[Jaap Weidemann]

PolyGlot EPD test options
--------------------------------

Usage: polyglot epd-test <options>

- "-epd" (default: wac.epd)

Name of the EPD file. PolyGlot should support any
standard-conforming file.

- "-min-depth" (default: 8)

Minimum depth in plies (half moves).

- "-max-depth" (default: 63)

Maximum depth in plies (half moves).

- "-min-time" (default: 1)

Minimum time allowed per position in seconds.

- "-max-time" (default: 5)

Maximum time allowed per position in seconds.

- "-depth-delta" (default: 3)

Minimum number of consecutive iterations the engine have to keep the
correct best move before it is accepted as a solution.

[jshriver]

It would be nice to have utility for:
1) running multi-engine tournaments with PGN output and ELO-rating;
2) running EPD batch tests collecting timing and performance;
3) connecting to available online chess servers and collecting PGN games.

IMHO it should be a run in host mode and be transparent for UCI-engines.

I already wrote a PGN harvest for FICS. It's been running off/on for a year. For about 4-5 months it was online 24/7 and I have built a huge dataset. Then it was offline for a month or two while I moved. But I have it in right now.

I'd love to put it on ICS. Though I'm not sure what their rules are on harvesting. I thought FICS would be a little iffy about it, but I'm in the process of getting my own TD account there just for this

It's all written in Perl btw.

[Oliver]

A program that should be written: ucu, the Unified Chess Utility

There are a lot of common chess tasks that aren't specific to a particular engine, yet there isn't a common, portable open source utility to handle these. I propose the specification and coding of the command line utility program "ucu" that will handle these tasks.

The program would operate primarily in batch mode, although a simple interactive facility could be included. The utility would have a simple internal engine and could also interface to various other engines. Command line parameters would be sufficient for most actions, although alternate command and data file inputs could be specified. Output would appear on the standard output file unless an alternate output file were specified.

The Big Idea here is that such a utility could be called from a program instead of having the program's author write, test, and integrate his/her own code.

All command line parameters would be keyword based. The only required parameter would be "cmd" and its argument would identify the basic command request. An often used parameter would be "fen" and its argument would specify the primary position of interest.

Some examples:

(Enumerate movepaths to six ply for the given position)

ucu -cmd emp -ply 6 -fen k/8/8/8/8/8/8/4K2R w K - 0 1

(Output: 374670)

----

(Identify opening in the current locale language)

ucu -cmd opening -fen r1bqkbnr/1ppp1ppp/p1n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R w KQkq - 0 4

(Output: Ruy Lopez (Spanish opening))

In principle i like this proposal very much. Nevertheless i dislike the design you propose, i think a general specification of an interface to a library would be much better. Very much better. Since if you have an implementation of such a library, it is only some tedious work to produce the program you wanted, just using this library. Furthermore:
A library is more efficient. No new process has to be started for every task.
An (invisible) library would have more acceptance than an external tool.
A library can give a much richer interface and much more functionality than an external utility, communicating over the commandline.

I know that you alreade have written a library (SAN), but as far as i know it is not widely used. This is sad, since it implements a lot of debugged functionality. There are other libraries, but they are not complete either.
They are to my knowledge no other libraries, implementing building blocks for chessengines systematically (such as hashtables, opening books, tablebaseaccess and so forth...)

My proposal would be:
Think big, such that many people are attracted.
Analyse, why current libraries are not in widespread use. In my opinion, a highclass external documentation in HTML would be very helpful.
Devise a useful libraryinterface, solving the identified problems and specify it functionally complete.
Specify the technical details. Using C++ and a deliberate use of templates could be beneficial.
Attract other coders, to do the stuff. I would be willing to participate.

Oliver

[Michael Sherwin]

I looked at SAN very recently and was very very impressed with the concept. But, the ifs about it are a barrier to actually trying to use it.

The ifs that I am concerned about might not be real, but I can not invest the time to find out.

The major if is, it looks slow to me. The code looks as though it might run well on a 486, but, not on a pentium with branch mispredictions penalties.

It is really just that simple. If Steven is still interested in promoting SAN and can give some favorable performance stats then I would give SAN a much closer look!