Thought experiment

[Rebel]

I wonder if there is a relationship between the percentage of failed nullmoves (those you have to research with full depth) vs succeeded null-moves and if that percentage has something to do with ELO and with how much. Anyone ever researched this? And secondly if it is interesting enough to make a comparison between engines posting the percentage of each engine running (say) an EPD of 10 positions at 1 minute.

I realize accuracy isn't perfect since nowadays R=3 is no longer the standard.

-----

And secondly the same procedure but now calculating the percentage of failed LMR (those you have to research with full depth) vs succeeded LMR.

Even less accurate as R never had a standard at all.

But somehow (I think, but entirely unsure, hence the subject line) there must(could) be some ideal percentage range for both which could be an indication you have some work to do.

[Patrice Duhamel]

I don't know if we can compare something with different engines / techniques used in the search, but I'm curious if there are other ways than ELO to find if null moves pruning, LMR or other ideas are working as expected.

[Ferdy]

I wonder if there is a relationship between the percentage of failed nullmoves (those you have to research with full depth) vs succeeded null-moves and if that percentage has something to do with ELO and with how much. Anyone ever researched this? And secondly if it is interesting enough to make a comparison between engines posting the percentage of each engine running (say) an EPD of 10 positions at 1 minute.

I realize accuracy isn't perfect since nowadays R=3 is no longer the standard.

-----

And secondly the same procedure but now calculating the percentage of failed LMR (those you have to research with full depth) vs succeeded LMR.

Even less accurate as R never had a standard at all.

But somehow (I think, but entirely unsure, hence the subject line) there must(could) be some ideal percentage range for both which could be an indication you have some work to do.

Sample 3 positions on LMR.

Code: Select all

CEGT 40/20 Ponder off
Deuterium v14.3.34.130 x64	2738

Search time: 1 minute
Hash: 128 MB
Eval: 32 MB
Pawn: 8 MB
Processor: i7-2600K

Startpos
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 
info string LMR attempts: 4118405, LMR failed: 1.3%, LMR succeeded: 98.7%
bestmove e2e4

Ruy Lopez
r1bqkbnr/pppp1ppp/2n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R b KQkq - 3 3
info string LMR attempts: 4382899, LMR failed: 1.6%, LMR succeeded: 98.4%
bestmove a7a6

Sicilian
rnbqkb1r/1p2pppp/p2p1n2/8/3NP3/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 6
info string LMR attempts: 4182720, LMR failed: 1.4%, LMR succeeded: 98.5%
bestmove d4b3

[cdani]

Same positions.

Code: Select all

Andscacs 0.86207
I7-5820K
60 seconds

Startpos
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 
lmr: 9411647 good: 9140639 97.12%
null: 5616750 good: 4082643 72.69%

Ruy Lopez
r1bqkbnr/pppp1ppp/2n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R b KQkq - 3 3
lmr: 18310723 good: 17777054 97.09%
null: 11368660 good: 8177980 71.93%
 
Sicilian
rnbqkb1r/1p2pppp/p2p1n2/8/3NP3/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 6
lmr: 26647815 good: 25903040 97.21%
null:16803117 good: 12104221 72.04%

[Rebel]

Same positions.

Code: Select all

Mine.
60 seconds

Startpos
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 
lmr: 9140639 | 14971 = 0.16% researched
null: 163977 | 32375 = 19.7% researched

Ruy Lopez
r1bqkbnr/pppp1ppp/2n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R b KQkq - 3 3
lmr: 2047831 | 19375 = 0.94% researched
null: 205325 | 33661 = 16,3% researched

Sicilian
rnbqkb1r/1p2pppp/p2p1n2/8/3NP3/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 6
lmr: 2069152 | 18392 = 0.88% researched
null: 133395 | 28580 = 21.4% researched

[Patrice Duhamel]

My null moves and LMR are very simple (null moves : R = 2 + 1 if depth > 5, and LMR : R = 1).

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 
Null moves success : 59.6 %
LMR success : 99.7 %

r1bqkbnr/pppp1ppp/2n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R b KQkq - 3 3 
Null moves success : 56.6 %
LMR success : 99.8 %

rnbqkb1r/1p2pppp/p2p1n2/8/3NP3/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 6 
Null moves success : 62.0 %
LMR success : 99.8 %

[AndrewGrant]

What are your conditions for your null move? Your success rates seem a bit low.

[Patrice Duhamel]

What are your conditions for your null move? Your success rates seem a bit low.

Null moves conditions :

- not a PV node
- not in check
- not in pawns ending
- not if last move was a null move
- depth > 1

[Ferdy]

Deuterium null move pruning.

Code: Select all

Search time: 1 minute

Startpos 
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 
info string NMP attempts: 2320986, NMP succeeded: 75.8%

Ruy Lopez 
r1bqkbnr/pppp1ppp/2n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R b KQkq - 3 3
info string NMP attempts: 2556155, NMP succeeded: 74.4%

Sicilian 
rnbqkb1r/1p2pppp/p2p1n2/8/3NP3/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 6 
info string NMP attempts: 2338703, NMP succeeded: 75.7%

[ZirconiumX]

Dorpsgek can't even beat TSCP yet, but for the sake of science:

Code: Select all

Startpos
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 
# Nulls: 9290 Succeeded (pruned): 7096 - 76.4%
# LMRs: 115238 Succeeded (no research): 112509 - 97.6%

Ruy Lopez
r1bqkbnr/pppp1ppp/2n5/1B2p3/4P3/5N2/PPPP1PPP/RNBQK2R b KQkq - 3 3
# Nulls: 14055 Succeeded (pruned): 10563 - 75.2%
# LMRs: 203646 Succeeded (no research): 201467 - 98.9%

Sicilian
rnbqkb1r/1p2pppp/p2p1n2/8/3NP3/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 6
# Nulls: 4590 Succeeded (pruned): 3600 - 78.4%
# LMRs: 152013 Succeeded (no research): 150220 - 98.8%

It looks like there is no real correlation, but we need more data.