TalkChess.com

As many threads here discuss about weak and new engines, I take my minimalist chess engine Dumb 1.8 and strip all "sophisticated" heuristics, like history, killers, null move, lmr, lmp, check extension, razoring, SEE, etc. to just keep a plain alpha-beta, a simple quiescence search and MVVLVA move ordering to get Dumber 1.0. This Dumber engine is available on github here: https://github.com/abulmo/Dumb/releases/tag/v1.0
My purpose Is to compare Dumb to this new comers. For example, although simpler than Rustic Alpha 1, Dumber is about 100 Elo stronger, maybe because of a better evaluation function (material + pst +tempo, but with tuned weights), but also thanks to a faster search. On my computer Dumber usually analyzes around 10M nodes/s, but Rustic less than 5 M nodes/s, despite a 10% slower move generator.
Compared to dumb 1.8, Dumber is 1000 Elo weaker. All the removed codes just count for 200 lines, so they are worth 5 Elo per lines and are not that complicated.

Interesting experiment.

Rustic doesn't have tuned weights yet; that will be in the cards for Rustic 4 or 5 (the first version after the Alpha's). Rustic Alpha 1 basically has no evaluation: it just counts material + PST, and that's it. I often see that tuning PST's and material alone can add 60-100 Elo, so it could be the reason for Dumb being stronger.

It's almost impossible to compare node counts. When I compare Rustic Alpha 1 to other engines on my computer (older i7-6700K) it manages 3-5 million nps. What is your search doing that mine doesn't? If you have plain A/B, I doubt that it can be twice as fast. I'll have to take a look at it. On my computer, I can almost test against any 1600-1800 engine I want, and Rustic is often _much_ faster.

My NPS node count is actually low compared to what could be done. My NPS counts nodes that are actually searched. So if the engine reports 3.2 Mnps, it has generated moves for 3.2 M nodes in that second, and searched them. If the engine enters a node and then immediately exists because (for example) the TT cuts off the search, or is_draw() reports true, the node is NOT counted. (Same for QSearch, btw.)

If I would count a node immediately at the beginning of the search and qsearch functions, I'd not be counting nodes searched, but nodes visited (and left before actually doing anything). That would skyrocket the node count though.

Dumber has a transposition table?

hgm wrote: ↑Thu Mar 04, 2021 12:05 am Dumber has a transposition table?

No.

mvanthoor wrote: ↑Wed Mar 03, 2021 11:35 pm
My NPS node count is actually low compared to what could be done. My NPS counts nodes that are actually searched. So if the engine reports 3.2 Mnps, it has generated moves for 3.2 M nodes in that second, and searched them. If the engine enters a node and then immediately exists because (for example) the TT cuts off the search, or is_draw() reports true, the node is NOT counted. (Same for QSearch, btw.)

If I would count a node immediately at the beginning of the search and qsearch functions, I'd not be counting nodes searched, but nodes visited (and left before actually doing anything). That would skyrocket the node count though.

So you do not count terminal nodes?

If I understood Marcel correctly, then in any case he doesn't count nodes where a stand-pat cutoff occurs. I am not sure that this completely coincides with the concept of 'terminal node'. There must be QS nodes that do not have eval >= beta, so that you would start searching captures, but where you discover (after move generation) that there actually are none. (Or, when you prune bad captures, you discover all captures are bad.) I would also qualify these as 'terminal nodes', bu I am not sure whether Rustic counts these. Counting those would be tantamount to counting move generations.

I suspect that the speed difference between Rustic and Dumber is largely due to this.

The advantage of Rustic's way of counting is that the speedup you get by implementing futility pruning ('delta pruning') shows up in the node count. When you also count stand-pat nodes, futility pruning would reduce the node count, because it prevents that such nodes will be reached.

When comparing such things between engines, it would be a good idea to make a more differentiated 'node count': counts move generations and evaluations separately, and even distinguis capture-only move generation from that in the full-width search.

hgm wrote: ↑Thu Mar 04, 2021 9:17 am If I understood Marcel correctly, then in any case he doesn't count nodes where a stand-pat cutoff occurs. I am not sure that this completely coincides with the concept of 'terminal node'. There must be QS nodes that do not have eval >= beta, so that you would start searching captures, but where you discover (after move generation) that there actually are none. (Or, when you prune bad captures, you discover all captures are bad.) I would also qualify these as 'terminal nodes', bu I am not sure whether Rustic counts these. Counting those would be tantamount to counting move generations.

In Rustic it's simple: it counts a node, as soon as it generates moves in it. If it doesn't generate moves due to an early exit, it doesn't count the node. Depth == 0 is counted by qsearch of course, IF the node generates moves... if it doesn't, it's not counted. I could count both at the point where it generates moves (because it's doing work there), and at the point where it calls evaluation (also doing work there). I don't like just incrementing the node count at the beginning of search/qsearch, because it will massively inflate the speed when nodes are cut off with early exits.

Kiwipete position:
There's your 10M nodes/second, on an older i7-6700K

Counting directly after move generation (so it's doing/going to do work), and where a node is evaluated (not generating moves, but also work) This should count "work done nodes" (either move generation/searching or evaluation), but not early exits.

Strange. Hardly any difference with only counting nodes that do move generation.

What "early exits" do you still have when you count evaluation as "work done"? Is this the version with TT?

Much appreciated! Even though, if it's 100 ELO stronger than rustic, it's not quite in reach for my engine just yet!

abulmo2 wrote: ↑Thu Mar 04, 2021 1:17 am So you do not count terminal nodes?

It may be a stupid question but do you count every node evaluated in QSearch or stop counting as soon as you reach depth 0 and start QSearch?

Hmm... maybe you shouldn't have dumbed down the time control code, too!

Dumber-1.0 lost it's first game against MinimalChess on time. Played in CuteChess GUI. TC: 5seconds + 1sec increment. Still have it open if you need anything...

[pgn][Event "?"]
[Site "?"]
[Date "2021.03.04"]
[Round "?"]
[White "MinimalChess 0.2.6 SimpleMod"]
[Black "dumber-1.0"]
[Result "1-0"]
[ECO "B01"]
[GameDuration "00:01:26"]
[GameEndTime "2021-03-04T18:37:31.667 Mitteleuropäische Zeit"]
[GameStartTime "2021-03-04T18:36:05.343 Mitteleuropäische Zeit"]
[Opening "Scandinavian (center counter) defense"]
[PlyCount "77"]
[Termination "time forfeit"]
[TimeControl "40/5+1"]

1. e4 {+0.35/7 0.73s} d5 {+0.02/7 1.1s} 2. exd5 {+0.35/6 0.34s}
Qxd5 {0.00/7 1.1s} 3. Nc3 {+0.25/6 0.53s} Qe6+ {-0.35/6 1.1s}
4. Nge2 {+0.50/7 0.97s} Nh6 {0.00/7 1.1s} 5. d4 {+0.85/6 0.31s}
Nf5 {0.00/7 1.1s} 6. Bf4 {+0.75/6 1.00s} Qc6 {-0.43/6 1.1s} 7. d5 {+0.90/5 1.2s}
Qb6 {-0.46/6 0.86s} 8. Qd3 {+0.65/5 0.37s} Qxb2 {-0.42/6 1.1s}
9. Rb1 {+0.85/6 0.99s} Qa3 {-0.43/6 1.1s} 10. Bxc7 {+0.95/5 0.43s}
Na6 {0.00/6 1.1s} 11. Qb5+ {+1.55/5 1.3s} Bd7 {0.00/7 1.1s}
12. Qxb7 {+1.45/6 0.79s} Nxc7 {-0.61/5 0.96s} 13. Qxc7 {+1.45/5 0.28s}
Qd6 {-0.73/5 0.89s} 14. Qb7 {+1.90/6 0.60s} Rc8 {0.00/6 1.2s}
15. Qxa7 {+2.05/5 1.4s} Qe5 {0.00/6 1.2s} 16. f4 {+2.40/5 1.4s}
Qe3 {-0.84/6 1.2s} 17. Qxe3 {+2.15/6 0.73s} Nxe3 {0.00/7 1.2s}
18. Kd2 {+2.20/6 1.4s} Nc4+ {-0.69/6 1.2s} 19. Ke1 {+2.00/7 0.95s}
Na3 {-0.64/6 1.1s} 20. Rc1 {+2.00/6 1.4s} e6 {-0.35/7 1.2s}
21. dxe6 {+1.65/6 1.4s} Bxe6 {-0.05/7 0.83s} 22. Kf2 {+0.70/6 1.4s}
Rc4 {-0.15/6 0.84s} 23. Kf3 {+0.70/6 0.63s} Bb4 {0.00/7 1.2s}
24. Ne4 {+0.50/6 0.40s} f5 {+0.38/7 1.2s} 25. Ng5 {+0.65/7 1.0s}
Bd5+ {+0.20/7 1.2s} 26. Kf2 {+0.50/6 1.5s} Bc5+ {+0.23/7 1.2s}
27. Kg3 {+0.50/7 1.5s} Nxc2 {0.00/7 1.2s} 28. Nf3 {+0.30/6 0.81s}
Bxf3 {+0.17/6 1.2s} 29. gxf3 {+0.40/7 1.2s} Be3 {+0.22/7 1.2s}
30. Rb1 {+0.25/6 1.6s} O-O {+0.45/7 1.2s} 31. Rb3 {+0.25/6 0.69s}
Ra4 {+0.37/6 1.1s} 32. Rb7 {-0.35/6 0.36s} Bd2 {+0.41/6 1.2s}
33. Rb2 {-0.05/7 1.9s} Be1+ {+0.48/7 1.2s} 34. Kh3 {-0.55/7 0.48s}
Ne3 {+0.55/7 1.2s} 35. Bg2 {-0.65/6 2.2s} Bb4 {+0.50/6 0.99s}
36. Kg3 {-0.30/6 0.61s} Bd6 {+0.54/6 1.1s} 37. Rd2 {-0.35/6 2.6s}
Bb4 {+0.50/6 1.4s} 38. Rb2 {-0.60/7 1.2s} Bd6 {+0.54/6 1.1s}
39. Kf2 {-0.40/6 3.2s, Black loses on time} 1-0
[/pgn]