Perft speed optimization (renamed)

[mvanthoor]

Hi xr_a_y,

Yesterday I've been looking at Minic after you split it up into multiple files, because you said in the "raw speed" thread that your "apply()" function was ugly. I assume that this is your equivalent of my function make_move().

This is the most used function during perft. As far as I know:

Both our engines are using fancy magic bitboards
Both use no hash in perft (or it is/can be disabled)
Both do not do bulk counting (make/unmake including the leaf nodes)
Both use pseudo-legal move generation
Both don't use move optimizations such as special in-check or pinned-pieces routines

Because the engines look to be the same with regard to move generation, I would expect their speed to be comparable.

I wanted to take a look into it, as running Minic at 25 Mnps @ 9700K vs Rustic @ 28 Mnps @ 6700K, makes Minic slow in comparison. The 9700K is 13% faster in single-task workloads according to Passmark, so I would expect your engine to run at 31 Mnps at least, on your computer. (In the starting position.) As you're only achieving 25 Mnps, Minic is 20% slower in move generation than expected. I wanted to test Minic on my own machine to have the programs run on the same hardware to make a comparison that isn't a guess.

However, I can't get Minic to run perft. I downloaded the last 2.00 release bmi2-executable, but the only thing it does is give warnings that either a "key" or "ARGS" are not provided. I've also created the json-file from the readme, but I keep getting these warnings. The "-perft" command does nothing.

What am I missing?

Thanks for your assistance.

[Terje]

If you wouldn't mind throwing another engine into the mix, I'd love to see comparisons involving Weiss

To run perft compile with 'make dev', run it, and give commands like 'perft <depth> <fen>'.

Example: 'perft 6 rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1'

<depth> defaults to 5 (but must be supplied if you aren't using default fen)
<fen> defaults to "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1"

[xr_a_y]

For minic you will have to do this

Code: Select all

minic_exe -perft "fen here" 6 -quiet 0

And you'll get something like this

Code: Select all

minic_dev_linux_x64 -perft "8/2p3p1/k1p2b1p/P1P2K2/3P4/2B5/7P/8 w - - 1 46" 4 -quiet 0
# This is Minic version 2.04
# Info  2020-04-06 21:25:04-513: From ARG, quiet : 0
# Warn  2020-04-06 21:25:04-513: ARG key not given, debugMode
# Warn  2020-04-06 21:25:04-513: ARG key not given, debugFile
# Warn  2020-04-06 21:25:04-513: ARG key not given, book
# Warn  2020-04-06 21:25:04-513: ARG key not given, bookFile
# Warn  2020-04-06 21:25:04-513: ARG key not given, ttSizeMb
# Warn  2020-04-06 21:25:04-513: ARG key not given, threads
# Warn  2020-04-06 21:25:04-513: ARG key not given, mateFinder
# Warn  2020-04-06 21:25:04-513: ARG key not given, fullXboardOutput
# Warn  2020-04-06 21:25:04-513: ARG key not given, level
# Warn  2020-04-06 21:25:04-513: ARG key not given, syzygyPath
# Info  2020-04-06 21:25:04-513: Init hash
# Info  2020-04-06 21:25:04-513: Init TT
# Info  2020-04-06 21:25:04-513: Entry size 12
# Info  2020-04-06 21:25:04-563: Size of TT 96Mb
# Info  2020-04-06 21:25:04-563: Init lmr
# Info  2020-04-06 21:25:04-564: Init mvv-lva
# Info  2020-04-06 21:25:04-564: Init mask
# Info  2020-04-06 21:25:04-564: Init magic
# Info  2020-04-06 21:25:04-584: KPK init
# Info  2020-04-06 21:25:04-584: KPK table size : 24Kb
# Info  2020-04-06 21:25:04-593: Material hash total : 944784
# Info  2020-04-06 21:25:04-593: Material hash size : 10Mb
# Info  2020-04-06 21:25:04-593: Material hash init
# Info  2020-04-06 21:25:04-636: ...Done
# Info  2020-04-06 21:25:04-639: Starting worker 0
# Info  2020-04-06 21:25:04-639: Waiting for workers to join...
# Info  2020-04-06 21:25:04-640: Using 1 threads
# Info  2020-04-06 21:25:04-640: Init Pawn TT : 32768
# Info  2020-04-06 21:25:04-640: PawnEntry size 84
# Info  2020-04-06 21:25:04-641: Size of Pawn TT 2Mb
# Info  2020-04-06 21:25:04-641: Init tb from path 
# Info  2020-04-06 21:25:04-641: MAX_TB_MEN: 0
# Info  2020-04-06 21:25:04-641: You can use -uci command line option to enter uci mode
# Info  2020-04-06 21:25:04-641: You can use -xboard command line option to enter xboard mode
# Info  2020-04-06 21:25:04-641: Reading fen 8/2p3p1/k1p2b1p/P1P2K2/3P4/2B5/7P/8 w - - 1 46
# Info  2020-04-06 21:25:04-641: No en passant square given
# Info  2020-04-06 21:25:04-641: Position
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | |p| | | |p| |
#  +-+-+-+-+-+-+-+-+
#  |k| |p| | |b| |p|
#  +-+-+-+-+-+-+-+-+
#  |P| |P| | |K| | |
#  +-+-+-+-+-+-+-+-+
#  | | | |P| | | | |
#  +-+-+-+-+-+-+-+-+
#  | | |B| | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | |P|
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  Turn white
#  Phase 0.168004
#  Static score 30
#  Hash 9287054002257983662
#  FEN 8/2p3p1/k1p2b1p/P1P2K2/3P4/2B5/7P/8 w - - 0 1
# Info  2020-04-06 21:25:04-642: pseudoNodes   29080
# Info  2020-04-06 21:25:04-642: validNodes    25450
# Info  2020-04-06 21:25:04-642: captureNodes  0
# Info  2020-04-06 21:25:04-642: epNodes       0
# Info  2020-04-06 21:25:04-642: checkNode     0
# Info  2020-04-06 21:25:04-642: checkMateNode 0
# Info  2020-04-06 21:25:04-642: ... ok threadPool deleted
# Info  2020-04-06 21:25:04-642: Starting worker 0
# Info  2020-04-06 21:25:04-642: Waiting for workers to join...

[mvanthoor]

I'll try

I assume that both Weiss and Minic run perft the same way when providing the default starting position and depth 7?
- Pseudo-legal move generator
- No bulk counting at depth 1, but running up to and including make/unmake for depth 0
- No hash/transposition table I stripped mine from Rustic and put it aside for the moment, to keep version Alpha 1 as clean as possible with regard to chess playing code. Only the move generator, alpha/beta, time management, and the simplest evaluation that will work. Not even move ordering or anything, so I can see the improvement per added feature.

It seems I went a bit overboard regarding command-line control of the engine. I got tired of adding "perft::run()" or "perftsuite::all_tests()" in main.rs all the time to test a change, and I didn't want to type or copy/paste command-line arguments either. Also, I needed a way to put in moves for the search and for testing the UCI-commands (two last parts of the very first version of Rustic) so I added a complete command-line interpreter. When you type "wizardry", it even performs magic and generates 128 new numbers for rooks and bishops Completely useless for chess playing purposes, but Rustic is first and foremost an engine to show people how to write a chess engine, and do it using something OTHER than C. Therefore I left the function to finding magics in it, even though it very much looks like the function to initializing the rook/bishop tables.

At some point I'll obviously add UCI, and probably xboard as well.

The command-line interpreter...

Code: Select all

Engine: Rustic Alpha 1
Author: Marcel Vanthoor

8   r n b q k b n r
7   i i i i i i i i
6   . . . . . . . .
5   . . . . . . . .
4   . . . . . . . .
3   . . . . . . . .
2   I I I I I I I I
1   R N B Q K B N R

    A B C D E F G H

Zobrist key:        819aa694337673fb
Active Color:       White
Castling:           KQkq
En Passant:         -
Half-move clock:    0
Full-move number:   1

Rustic > _

[mvanthoor]

Some results. Minic doesn't output times when running perft (at least not when using the command you gave me above), so I used the "time" command in MSYS2 to time execution. This result obviously includes Minic's initialization routines. For comparison, I'll run Rustic both normally (it times its perft function), and using "time" as well.

Results on i7-6700K @ stock speeds, at default BIOS settings (no overclocks).
Minic: 2 minutes, 1 seconds 12 ms (including initialization, "time" command)
Rustic A: 1 minutes, 54 seconds, 53 ms (perft 7 only, self-timed)
Rustic B: 1 minutes, 54 seconds, 241 ms (including initialization, "time" command)

Initializing Rustic takes less than 0.2 seconds.
When ignoring the miliseconds, rustic takes 114 seconds, Minic takes 121.
With Minic being the 100% benchmark, Rustic is about 5-6% faster than Minic on the same system.
Tomorrow, I'll add Weiss as well.

Minic output:

Code: Select all

$ time ./minic_2.00_mingw_x64_avx2_bmi2.exe -perft "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" 7 -quiet 0
# This is Minic version 2.00
# Info  2020-04-06 22:13:34-800: From ARG, quiet : 0
# Warn  2020-04-06 22:13:34-800: ARG key not given, debugMode
# Warn  2020-04-06 22:13:34-800: ARG key not given, debugFile
# Warn  2020-04-06 22:13:34-800: ARG key not given, book
# Warn  2020-04-06 22:13:34-800: ARG key not given, bookFile
# Warn  2020-04-06 22:13:34-800: ARG key not given, ttSizeMb
# Warn  2020-04-06 22:13:34-800: ARG key not given, threads
# Warn  2020-04-06 22:13:34-800: ARG key not given, mateFinder
# Warn  2020-04-06 22:13:34-800: ARG key not given, fullXboardOutput
# Warn  2020-04-06 22:13:34-800: ARG key not given, level
# Warn  2020-04-06 22:13:34-800: ARG key not given, syzygyPath
# Info  2020-04-06 22:13:34-800: Init hash
# Info  2020-04-06 22:13:34-800: Init TT
# Info  2020-04-06 22:13:34-800: Entry size 12
# Info  2020-04-06 22:13:34-847: Size of TT 96Mb
# Info  2020-04-06 22:13:34-847: Init lmr
# Info  2020-04-06 22:13:34-847: Init mvv-lva
# Info  2020-04-06 22:13:34-847: Init mask
# Info  2020-04-06 22:13:34-847: Init magic
# Info  2020-04-06 22:13:34-863: KPK init
# Info  2020-04-06 22:13:34-863: KPK table size : 24Kb
# Info  2020-04-06 22:13:34-863: Material hash total : 944784
# Info  2020-04-06 22:13:34-863: Material hash size : 10Mb
# Info  2020-04-06 22:13:34-863: Material hash init
# Info  2020-04-06 22:13:34-910: ...Done
# Info  2020-04-06 22:13:34-910: Starting worker 0
# Info  2020-04-06 22:13:34-910: Waiting for workers to join...
# Info  2020-04-06 22:13:34-910: Using 1 threads
# Info  2020-04-06 22:13:34-910: Init Pawn TT : 32768
# Info  2020-04-06 22:13:34-910: PawnEntry size 84
# Info  2020-04-06 22:13:34-910: Size of Pawn TT 2Mb
# Info  2020-04-06 22:13:34-910: Init tb from path
# Info  2020-04-06 22:13:34-910: MAX_TB_MEN: 0
# Info  2020-04-06 22:13:34-910: You can use -uci command line option to enter uci mode
# Info  2020-04-06 22:13:34-910: You can use -xboard command line option to enter xboard mode
# Info  2020-04-06 22:13:34-910: Reading fen rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# Info  2020-04-06 22:13:34-910: No en passant square given
# Info  2020-04-06 22:13:34-910: Position
#  +-+-+-+-+-+-+-+-+
#  |r|n|b|q|k|b|n|r|
#  +-+-+-+-+-+-+-+-+
#  |p|p|p|p|p|p|p|p|
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  |P|P|P|P|P|P|P|P|
#  +-+-+-+-+-+-+-+-+
#  |R|N|B|Q|K|B|N|R|
#  +-+-+-+-+-+-+-+-+
#  Turn white
#  Phase 1
#  Static score 0
#  Hash 4715363712485770064
#  FEN rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# Info  2020-04-06 22:15:35-766: pseudoNodes   3222676138
# Info  2020-04-06 22:15:35-766: validNodes    3195901860
# Info  2020-04-06 22:15:35-766: captureNodes  0
# Info  2020-04-06 22:15:35-766: epNodes       0
# Info  2020-04-06 22:15:35-766: checkNode     0
# Info  2020-04-06 22:15:35-766: checkMateNode 0
# Info  2020-04-06 22:15:35-766: ... ok threadPool deleted
# Info  2020-04-06 22:15:35-766: Starting worker 0
# Info  2020-04-06 22:15:35-766: Waiting for workers to join...

real    2m1.012s
user    0m0.000s
sys     0m0.000s
[/quote]

Rustic output (command-line interpreter):
[code]
Marcel@WORKSTATION MINGW64 /c/code/rustic
$ ./target/release/rustic.exe

Engine: Rustic Alpha 1
Author: Marcel Vanthoor

8   r n b q k b n r
7   i i i i i i i i
6   . . . . . . . .
5   . . . . . . . .
4   . . . . . . . .
3   . . . . . . . .
2   I I I I I I I I
1   R N B Q K B N R

    A B C D E F G H

Zobrist key:        819aa694337673fb
Active Color:       White
Castling:           KQkq
En Passant:         -
Half-move clock:    0
Full-move number:   1

Rustic > perft
Benchmarking perft 1-7:

8   r n b q k b n r
7   i i i i i i i i
6   . . . . . . . .
5   . . . . . . . .
4   . . . . . . . .
3   . . . . . . . .
2   I I I I I I I I
1   R N B Q K B N R

    A B C D E F G H

Zobrist key:        819aa694337673fb
Active Color:       White
Castling:           KQkq
En Passant:         -
Half-move clock:    0
Full-move number:   1

Perft 1: 20 (0 ms, inf leaves/sec)
Perft 2: 400 (0 ms, inf leaves/sec)
Perft 3: 8902 (0 ms, inf leaves/sec)
Perft 4: 197281 (7 ms, 28183000 leaves/sec)
Perft 5: 4865609 (174 ms, 27963270 leaves/sec)
Perft 6: 119060324 (4322 ms, 27547506 leaves/sec)
Perft 7: 3195901860 (114053 ms, 28021199 leaves/sec)
Total time spent: 118556 ms
Execution speed: 28003933 leaves/second
Rustic > exit

Marcel@WORKSTATION MINGW64 /c/code/rustic
$

Rustic output with "time":
(I replaced the call to the command-line interpreter with a call directly to perft 7.)

Code: Select all

$ time ./target/release/rustic.exe

Engine: Rustic Alpha 1
Author: Marcel Vanthoor
Running perft 7 on starting position...
Done.

real    1m54.241s
user    0m0.000s
sys     0m0.000s

Marcel@WORKSTATION MINGW64 /c/code/rustic
$

[Terje]

Tomorrow, I'll add Weiss as well.

Cool, I've updated Weiss to allow depths above 6. I only had the correct counts for depths 1-6, and for the position I use as default, D6 already takes several minutes so I made an arbitrary cap. (It will still crash on multiple digits for depth, so don't try that )

To use magic bitboards instead of pext in dev build you will have to make this change in the makefile:

Code: Select all

dev:
	$(BENCH)-dev $(PEXT) $(DEV)

Code: Select all

dev:
	$(BENCH)-dev $(POPCNT) $(DEV)

[xr_a_y]

Ok I added a dedicated timer, without all init time (which is quite long for Minic as KPK table and material table are fully init ...).
And here it is 3.8sec

Code: Select all

# Info  2020-04-07 08:55:10-326: Reading fen rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# Info  2020-04-07 08:55:10-326: No en passant square given
# Info  2020-04-07 08:55:10-326: Position
#  +-+-+-+-+-+-+-+-+
#  |r|n|b|q|k|b|n|r|
#  +-+-+-+-+-+-+-+-+
#  |p|p|p|p|p|p|p|p|
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  |P|P|P|P|P|P|P|P|
#  +-+-+-+-+-+-+-+-+
#  |R|N|B|Q|K|B|N|R|
#  +-+-+-+-+-+-+-+-+
#  Turn white
#  Phase 1
#  Static score 0
#  Hash 4715363712485770064
#  FEN rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# Info  2020-04-07 08:55:14-212: Perft done in 3885ms
# Info  2020-04-07 08:55:14-212: pseudoNodes   119961738
# Info  2020-04-07 08:55:14-212: validNodes    119060324

Available in Minic 2.05.

[xr_a_y]

So for pertf 7 on start position 107sec

Code: Select all

# Info  2020-04-07 09:22:20-295: Reading fen rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# Info  2020-04-07 09:22:20-295: No en passant square given
# Info  2020-04-07 09:22:20-295: Position
#  +-+-+-+-+-+-+-+-+
#  |r|n|b|q|k|b|n|r|
#  +-+-+-+-+-+-+-+-+
#  |p|p|p|p|p|p|p|p|
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  | | | | | | | | |
#  +-+-+-+-+-+-+-+-+
#  |P|P|P|P|P|P|P|P|
#  +-+-+-+-+-+-+-+-+
#  |R|N|B|Q|K|B|N|R|
#  +-+-+-+-+-+-+-+-+
#  Turn white
#  Phase 1
#  Static score 0
#  Hash 4715363712485770064
#  FEN rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# Info  2020-04-07 09:24:07-943: Perft done in 107648ms
# Info  2020-04-07 09:24:07-943: pseudoNodes   3222676138
# Info  2020-04-07 09:24:07-943: validNodes    3195901860

[abulmo2]

IMHO, the reference program for perft without bulk counting nor hash table should be crafty. In my computer, it does perft 7 in 64.78s vs 3m30s for minic 2.00.

[xr_a_y]

IMHO, the reference program for perft without bulk counting nor hash table should be crafty. In my computer, it does perft 7 in 64.78s vs 3m30s for minic 2.00.

Quite a huge gap on your hardware !

On mine, Minic does 107 sec and Crafty 61 sec

What is your hardware and what Minic executable are you using ?