How effective is move ordering from TT?

[bob]

The best way to compare move ordering is by turning off all your selectivity except for nullmove.

So just nullmove + hashtables.

No other forward pruning. And i disagree with Bob that hashtables are a form of forward pruning... ...as you remove a transposition you already searched so in the first place it belongs to a part of the search space you already searched to that depth...

Reason i wouldn't want to turn off hashtables nor nullmove is because otherwise we aren't gonna get some serious search depth and because when searching fullwidth 100% would have the disadvantage that any extension you do blows up the search tree too much - nullmove will avoid that.

If you forward prune last few plies and aren't doing nullmove there, keep it turned on.

That is a good comparision IMHO, though of course no replacement for the real thing.

You'll see then that LMR and such selectivity basically hide at many programs a very imperfect move ordering.

If authors like Don comment on that: "i don't need a good move ordering as LMR works magnificent for me so i save out time where you lost probably over a year building your move ordering at the time", there is no way to refute that argument from his viewpoint. Note this is a fictional example. Just an example.

Vincent,

I believe that LMR is useless UNLESS you have excellent move ordering.

If i make the distinction between different issues then you'll see we probably disagree less than you'd guess.

First of all LMR works magnificent for engines that have a worthless move ordering, as it will search them a ply or 5 deeper or so meaning their mainline gets checked quite deeper. This delivers them elo.

Engines with a worthless move ordering will reduce the wrong moves. That is a bad receipt for elo improvement. Don is right, the better your move ordering, the more secure the LMR reductions.

This does not have to be true. I don't presently reduce based on a moves position in the move list, I reduce on static criteria instead. The better your move ordering, the easier LMR becomes because you can reduce later moves more if you so choose. But it is not a necessary condition to reduce.

[diep]

The best way to compare move ordering is by turning off all your selectivity except for nullmove.

So just nullmove + hashtables.

No other forward pruning. And i disagree with Bob that hashtables are a form of forward pruning... ...as you remove a transposition you already searched so in the first place it belongs to a part of the search space you already searched to that depth...

Reason i wouldn't want to turn off hashtables nor nullmove is because otherwise we aren't gonna get some serious search depth and because when searching fullwidth 100% would have the disadvantage that any extension you do blows up the search tree too much - nullmove will avoid that.

If you forward prune last few plies and aren't doing nullmove there, keep it turned on.

That is a good comparision IMHO, though of course no replacement for the real thing.

You'll see then that LMR and such selectivity basically hide at many programs a very imperfect move ordering.

If authors like Don comment on that: "i don't need a good move ordering as LMR works magnificent for me so i save out time where you lost probably over a year building your move ordering at the time", there is no way to refute that argument from his viewpoint. Note this is a fictional example. Just an example.

Vincent,

I believe that LMR is useless UNLESS you have excellent move ordering.

If i make the distinction between different issues then you'll see we probably disagree less than you'd guess.

First of all LMR works magnificent for engines that have a worthless move ordering, as it will search them a ply or 5 deeper or so meaning their mainline gets checked quite deeper. This delivers them elo.

Engines with a worthless move ordering will reduce the wrong moves. That is a bad receipt for elo improvement. Don is right, the better your move ordering, the more secure the LMR reductions.

This does not have to be true. I don't presently reduce based on a moves position in the move list, I reduce on static criteria instead. The better your move ordering, the easier LMR becomes because you can reduce later moves more if you so choose. But it is not a necessary condition to reduce.

Bob, the big problem is this.

If you try the worst move first, you search DEEPER with LMR.

I win today 2 plies using reductions. The same search depth win i had in 1999 with reductions as well. I got 14 ply back in 1999 with reductions in experiments ( many minutes a move). Diep world champs 1999 used those reductions.

Today it's 13 years later and i'm still winning only 2 plies with LMR, Diep gets plies deeper obviously than back then at your Quad Xeon, which got diep around a 70k nps.

Getting 2 plies deeper is not much huh?

If you select a queen away giving move as first you can easily win 5 plies with LMR. That's my whole point.

The worse your move ordering, the better LMR seems to work for you.

If you do select queen away giving moves and win that 5 plies, your iteration depth is 5 plies deeper.

In superbullet 5 plies deeper is a lot, you won't find tactics quickly, yet you avoid losing for another 5 plies. That's a lot.

So if you have a crap move ordering, LMR works relative a lot better than if you have a magnificent move ordering.

Vincent

[Uri Blass]

The best way to compare move ordering is by turning off all your selectivity except for nullmove.

So just nullmove + hashtables.

No other forward pruning. And i disagree with Bob that hashtables are a form of forward pruning... ...as you remove a transposition you already searched so in the first place it belongs to a part of the search space you already searched to that depth...

Reason i wouldn't want to turn off hashtables nor nullmove is because otherwise we aren't gonna get some serious search depth and because when searching fullwidth 100% would have the disadvantage that any extension you do blows up the search tree too much - nullmove will avoid that.

If you forward prune last few plies and aren't doing nullmove there, keep it turned on.

That is a good comparision IMHO, though of course no replacement for the real thing.

You'll see then that LMR and such selectivity basically hide at many programs a very imperfect move ordering.

If authors like Don comment on that: "i don't need a good move ordering as LMR works magnificent for me so i save out time where you lost probably over a year building your move ordering at the time", there is no way to refute that argument from his viewpoint. Note this is a fictional example. Just an example.

Vincent,

I believe that LMR is useless UNLESS you have excellent move ordering.

If i make the distinction between different issues then you'll see we probably disagree less than you'd guess.

First of all LMR works magnificent for engines that have a worthless move ordering, as it will search them a ply or 5 deeper or so meaning their mainline gets checked quite deeper. This delivers them elo.

Engines with a worthless move ordering will reduce the wrong moves. That is a bad receipt for elo improvement. Don is right, the better your move ordering, the more secure the LMR reductions.

This does not have to be true. I don't presently reduce based on a moves position in the move list, I reduce on static criteria instead. The better your move ordering, the easier LMR becomes because you can reduce later moves more if you so choose. But it is not a necessary condition to reduce.

Bob, the big problem is this.

If you try the worst move first, you search DEEPER with LMR.

I win today 2 plies using reductions. The same search depth win i had in 1999 with reductions as well. I got 14 ply back in 1999 with reductions in experiments ( many minutes a move). Diep world champs 1999 used those reductions.

Today it's 13 years later and i'm still winning only 2 plies with LMR, Diep gets plies deeper obviously than back then at your Quad Xeon, which got diep around a 70k nps.

Getting 2 plies deeper is not much huh?

If you select a queen away giving move as first you can easily win 5 plies with LMR. That's my whole point.

The worse your move ordering, the better LMR seems to work for you.

If you do select queen away giving moves and win that 5 plies, your iteration depth is 5 plies deeper.

In superbullet 5 plies deeper is a lot, you won't find tactics quickly, yet you avoid losing for another 5 plies. That's a lot.

So if you have a crap move ordering, LMR works relative a lot better than if you have a magnificent move ordering.

Vincent

I do not buy it.

You can search deeper with LMR if you try the best move first.

If you try a bad move first then when you search later moves you
may fail high and need to do a research with bigger depth when it is not the case when you search the best move first.

[diep]

The best way to compare move ordering is by turning off all your selectivity except for nullmove.

So just nullmove + hashtables.

No other forward pruning. And i disagree with Bob that hashtables are a form of forward pruning... ...as you remove a transposition you already searched so in the first place it belongs to a part of the search space you already searched to that depth...

Reason i wouldn't want to turn off hashtables nor nullmove is because otherwise we aren't gonna get some serious search depth and because when searching fullwidth 100% would have the disadvantage that any extension you do blows up the search tree too much - nullmove will avoid that.

If you forward prune last few plies and aren't doing nullmove there, keep it turned on.

That is a good comparision IMHO, though of course no replacement for the real thing.

You'll see then that LMR and such selectivity basically hide at many programs a very imperfect move ordering.

If authors like Don comment on that: "i don't need a good move ordering as LMR works magnificent for me so i save out time where you lost probably over a year building your move ordering at the time", there is no way to refute that argument from his viewpoint. Note this is a fictional example. Just an example.

Vincent,

I believe that LMR is useless UNLESS you have excellent move ordering.

If i make the distinction between different issues then you'll see we probably disagree less than you'd guess.

First of all LMR works magnificent for engines that have a worthless move ordering, as it will search them a ply or 5 deeper or so meaning their mainline gets checked quite deeper. This delivers them elo.

Engines with a worthless move ordering will reduce the wrong moves. That is a bad receipt for elo improvement. Don is right, the better your move ordering, the more secure the LMR reductions.

This does not have to be true. I don't presently reduce based on a moves position in the move list, I reduce on static criteria instead. The better your move ordering, the easier LMR becomes because you can reduce later moves more if you so choose. But it is not a necessary condition to reduce.

Bob, the big problem is this.

If you try the worst move first, you search DEEPER with LMR.

I win today 2 plies using reductions. The same search depth win i had in 1999 with reductions as well. I got 14 ply back in 1999 with reductions in experiments ( many minutes a move). Diep world champs 1999 used those reductions.

Today it's 13 years later and i'm still winning only 2 plies with LMR, Diep gets plies deeper obviously than back then at your Quad Xeon, which got diep around a 70k nps.

Getting 2 plies deeper is not much huh?

If you select a queen away giving move as first you can easily win 5 plies with LMR. That's my whole point.

The worse your move ordering, the better LMR seems to work for you.

If you do select queen away giving moves and win that 5 plies, your iteration depth is 5 plies deeper.

In superbullet 5 plies deeper is a lot, you won't find tactics quickly, yet you avoid losing for another 5 plies. That's a lot.

So if you have a crap move ordering, LMR works relative a lot better than if you have a magnificent move ordering.

Vincent

I do not buy it.

You can search deeper with LMR if you try the best move first.

If you try a bad move first then when you search later moves you
may fail high and need to do a research with bigger depth when it is not the case when you search the best move first.

the proof is easy that bad moves search deeper with LMR.

I select in an upperbound node the queen giving away move.
Where i can capture a piece i select the move not taking it.

I'm gonna get 50 plies you know...

It won't be better elowise maybe, but sure it searches DEEPER.

Anyone not understanding that didn't program a chessprogram.
Oh wait - you didn't - you just improved TSCP a tad...

Didn't you do your PHD in statistics by the way, why aren't you commenting at any statistics thread here?

[plattyaj]

the proof is easy that bad moves search deeper with LMR.

I select in an upperbound node the queen giving away move.
Where i can capture a piece i select the move not taking it.

I'm gonna get 50 plies you know...

It won't be better elowise maybe, but sure it searches DEEPER.

Anyone not understanding that didn't program a chessprogram.
Oh wait - you didn't - you just improved TSCP a tad...

Didn't you do your PHD in statistics by the way, why aren't you commenting at any statistics thread here?

I don't understand. If I search bad moves first, I have to search all of them (until I get to the point where my LMR kicks in). So that means that my search tree is a lot wider. But it also means I won't be able to search so deep because I'll run out of time. What am I missing?

[diep]

the proof is easy that bad moves search deeper with LMR.

I select in an upperbound node the queen giving away move.
Where i can capture a piece i select the move not taking it.

I'm gonna get 50 plies you know...

It won't be better elowise maybe, but sure it searches DEEPER.

Anyone not understanding that didn't program a chessprogram.
Oh wait - you didn't - you just improved TSCP a tad...

Didn't you do your PHD in statistics by the way, why aren't you commenting at any statistics thread here?

I don't understand. If I search bad moves first, I have to search all of them (until I get to the point where my LMR kicks in). So that means that my search tree is a lot wider. But it also means I won't be able to search so deep because I'll run out of time. What am I missing?

LMR is NOT reducing the first moves, usually first 3. And it's reducing the remainder of the list.

So if we select a horrible move first in an upperbound node, then nullmove will reduce deeper in the tree that searchspace bigtime and the tactical moves that can create trouble that we can try in this position P we reduce an additional ply.

Most engines already reduce them nowadays 2 plies or so (on top of the normal reduction of 1 ply).

So mathematical it's very easy to prove selecting the first moves, say 3 moves we don't reduce, to select the most horrible moves there. Of c ourse that's not captures.

Just put your queen back on a square opponent can capture it.

Vincent

[diep]

Then shoot any programmer who can't get 2000 ELO out of raw evaluation only.

You don't need me to run this test nor do you need my permission.

Ah you were already shot by a derivative from the old boys school system...

Seems he has inversed the ideas of Pol Pot.

Yet he should start shooting all Anglosaxian judges they used at the olympics (non to confuse with yanks who are not involved in this as they don't know how to play games where you have to use a stick - they always want to grab the ball directly or throw it away).

They did do a bad job man, especially in the hockey.

How can you have an English judge judge the semi/quarter finals if in the first the British team just won, which normally spoken would play then the (reigning olympic champion) Dutch team that the English judge is busy judging in this quarter/semi finals?

I'd say start cleaning up the old boys school system way of doing things Chris!

[Don]

Micro-Max has a very poor move ordering (no killer, no history; it just does 1) null move, 2) hash move, 3) best MVV/LVA capture, 4) other captures in unspecified order, 5) non-captures in unspecified order). Yet LMR worked great. It reduces all non-captures that are not Pawn moves.

LMR might give no doubt, but the better the move ordering the better LMR will perform. Much of course will depend on the scheme in use for 1.0, 1.5, 2.0, 2.5, 3.0 reductions in relationship with the number of the move list. I am not using the latter myself but that's what I have understood what many programs do nowadays.

Ed, if i just look at rebel and compare it with the derivatives, the trick is the material evaluation of them.

It's more advanced than Rebel one. This where they indeed use a few simple concepts to get it done.

If none of your positional factors, other than passed pawn evaluation,
can ever overrule the material evaluation, then you can easily forward prune last 4 plies, not sure how this has been set in komodo.

Our evaluation is pretty aggressive and it can easily go over a pawn even without passed pawns.

A way to test your evaluation is to try a bunch of gambit positions. For example if you put Morra Gambit in and do a search the old programs with weak evaluation consider that you are down almost a pawn, but in reality you have a lot of compensation. I think current consensus is that you should not play most of these gambits, but they don't lose by force.

I just checked Komodo using the classic main line of the Smith Morra Gambit accepted:

1. e4 c5 2. d4 cxd4 3. c3 dxc3 4. Nxc3 Nc6

Now from the opening position Komodo thinks white has about 0.30 pawn advantage. If you play the above line Komodo thinks white is just slightly weaker - about -0.05 or so. So Komodo thinks by playing the gambit you are giving black a slight advantage that you didn't have to give. I'm sure people's opinions will vary on this but I think most strong players believe you are giving away the advantage of the white pieces by playing like this - so Komodo is pretty much correct.

Komodo is perfectly capable of positional pawn and exchange sacrifices too and they will be sound. A beancounter program cannot do that.

If someone want to build a good problem set I think a great one would present opportunites for SOUND positional sacrifices. These are difficult to construct because if a positional sacrifice is possible, it means you already have the advantage and for a good set the sacrifice needs to be the clearly best way to proceed. I have seen many positions where people show what a great move Komodo or some other program has played when it turns out that the program was winning and many moves worked, it's just more impressive if the program plays a "flashy" move. Komodo was not designed to play flashy just to impress people - but it will play them when they are needed.

That's what they do.

They win 3 plies there that i do not win.

Even if we are doing something special here, what is the point? With Komodo and I'm sure ALL the top programs we do what works. If we actually figure out how to win 3 ply it would be stupid not to do so unless it hurts the program. Nothing stops you from doing the same. If it hurts your program then don't do it! If there is something different about your program that makes it impossible to do so, then you must decide whether that is a good or bad thing.

What can be simpler than that?

Then they use nullmove with reduction factor that soon goes up 3,4,5,6,7 etc.

That they combine with agressive LMR.

If you'd retune Rebel, all you have to do is improve your material evaluation, have a 2 layer PSQ table and retune every parameter.

So layer 1 of the PSQ table is for opening, layer 2 is for endgame and then you find a formula to know how far in opening or endgame you are.

You cannot tune those parameters playing games.

You'd beat the crap out of Komodo in superbullet with Rebel then with your searchframe would you manage.

As for search, their strong point is Rebels strong point as well, namely see the mainline deep quickly. Additional to that you see more tactics with Rebel, so you win the superbullet bigtime from Komodo5 in such case.

I have huge respect for Rebel, if Ed really got serious about computer chess again I know that we would have our hands full trying to keep up with him.

But my only response to you is that I think our methods have been fully justified by the strength of Komodo. You can prove us wrong by beating us, otherwise all you are doing is making a fool out our yourself.

Just you lose it based upon the combination of material, and piece square tables. They have copied Fruit's idea, some of them in fact just cut'n pasted it, to have a piece square table for every piece in both opening as well as endgame. Then they average over that using a 32 intervals or so (with piece = 1, rook = 3, queen = 6 ; so in a full board position it gets 100% from the first PSQ with 2 * (6 + 3 + 3 + 1 + 1 + 1 + 1 ) = 2 * 16 = 32 points being the openingstable.

Fruit was using 1,2,4 by the way as values instead of 1,3,6...

This concept is surprisingly strong simply for endgame.

You don't manage to tune that at home that accurately though.

I've been toying at 80 cores and now at home i have 64 core cluster with 2 Tesla cards each having 448 cores. So i have in total for tuning nearly a 1000 cores.

In my experiments on how to improve Diep's tuning, i encountered many problems.

The PSQ tables play a total crucial role in the derivatives. Remove them and they're dead. The material evaluation none of them would invent it.

It has been tuned as it seems by neural net.

Now i'm not too impressed by that neural net, but tuning of your values highly is dependant upon how much knowledge you have got.

An engine with no knowledge except material, a bishop will be worth like 2.85 or so.

Your first goal is to figure out how to mathematically tune.

Playing games won't work.

If you have quite some knowledge you need a million games to figure out whether something needs to be 0.013 pawn bonus or 0.014

Forget it.

LMR is not the holy grail. It's the tuning and a simple yet effective setup of the evaluation. Material for the material. PSQ's for the positional evaluation and simple passed pawn knowledge for the passers.

Kingsafety?

Ah that's the big bummer. In many positions in chess it simply APPEARS that you can get away with relative simple kingsafety provided you get 20 ply. Most positional factors in kingsafety give a TEMPORARILY advantage.

That doesn't stop me from still working on Diep's kingsafety by the way.

Yet nearly all fixes is do right now have to do with simple things. Like i'm fixing now the willingness of diep to castle quick. Sure it wants to castle, but the derivatives simply give a big penalty for a king in the center.

Tuning fixes it for them everywhere....

In Diep just the material already is a 300+ parameters or so. Diep's material concept is totally different from the Fruit concept.

I really use knowledge rules to decide whether i give a bonus. I don't use a neural net tuned entity.

though it's all tuned by hand this material evaluation, i get impression diep is doing it better than the derivatives. But that feeling is very recent and i'm not sure it's already ok for most endgames.

As Diep has more parameters i can easier independantly decide to tune specific endgames. More possibilities also mean more problems for tuning though.

Tuning Diep is far more complex i'd say - even then the effort needed is similar in terms of expertise.

Tuning is in fact very important. No question about that, but you cannot tune something that is not there. If something important is missing from your program you are not going to "tune" it into the program.

[Uri Blass]

The best way to compare move ordering is by turning off all your selectivity except for nullmove.

So just nullmove + hashtables.

No other forward pruning. And i disagree with Bob that hashtables are a form of forward pruning... ...as you remove a transposition you already searched so in the first place it belongs to a part of the search space you already searched to that depth...

Reason i wouldn't want to turn off hashtables nor nullmove is because otherwise we aren't gonna get some serious search depth and because when searching fullwidth 100% would have the disadvantage that any extension you do blows up the search tree too much - nullmove will avoid that.

If you forward prune last few plies and aren't doing nullmove there, keep it turned on.

That is a good comparision IMHO, though of course no replacement for the real thing.

You'll see then that LMR and such selectivity basically hide at many programs a very imperfect move ordering.

If authors like Don comment on that: "i don't need a good move ordering as LMR works magnificent for me so i save out time where you lost probably over a year building your move ordering at the time", there is no way to refute that argument from his viewpoint. Note this is a fictional example. Just an example.

Vincent,

I believe that LMR is useless UNLESS you have excellent move ordering.

If i make the distinction between different issues then you'll see we probably disagree less than you'd guess.

First of all LMR works magnificent for engines that have a worthless move ordering, as it will search them a ply or 5 deeper or so meaning their mainline gets checked quite deeper. This delivers them elo.

Engines with a worthless move ordering will reduce the wrong moves. That is a bad receipt for elo improvement. Don is right, the better your move ordering, the more secure the LMR reductions.

This does not have to be true. I don't presently reduce based on a moves position in the move list, I reduce on static criteria instead. The better your move ordering, the easier LMR becomes because you can reduce later moves more if you so choose. But it is not a necessary condition to reduce.

Bob, the big problem is this.

If you try the worst move first, you search DEEPER with LMR.

I win today 2 plies using reductions. The same search depth win i had in 1999 with reductions as well. I got 14 ply back in 1999 with reductions in experiments ( many minutes a move). Diep world champs 1999 used those reductions.

Today it's 13 years later and i'm still winning only 2 plies with LMR, Diep gets plies deeper obviously than back then at your Quad Xeon, which got diep around a 70k nps.

Getting 2 plies deeper is not much huh?

If you select a queen away giving move as first you can easily win 5 plies with LMR. That's my whole point.

The worse your move ordering, the better LMR seems to work for you.

If you do select queen away giving moves and win that 5 plies, your iteration depth is 5 plies deeper.

In superbullet 5 plies deeper is a lot, you won't find tactics quickly, yet you avoid losing for another 5 plies. That's a lot.

So if you have a crap move ordering, LMR works relative a lot better than if you have a magnificent move ordering.

Vincent

I do not buy it.

You can search deeper with LMR if you try the best move first.

If you try a bad move first then when you search later moves you
may fail high and need to do a research with bigger depth when it is not the case when you search the best move first.

the proof is easy that bad moves search deeper with LMR.

I select in an upperbound node the queen giving away move.
Where i can capture a piece i select the move not taking it.

I'm gonna get 50 plies you know...

It won't be better elowise maybe, but sure it searches DEEPER.

Anyone not understanding that didn't program a chessprogram.
Oh wait - you didn't - you just improved TSCP a tad...

Didn't you do your PHD in statistics by the way, why aren't you commenting at any statistics thread here?

Your claims about me are not correct.
I started from some variables and some structures from tscp but I did not start from the full tscp program and movei has a legal move generator and I use array of 64 squares for the board unlike tscp.

For your claims
LMR is basically reducing only moves that cause fail low and researching fail high moves.

If you start with 1.Qa4 that is bad and
1.Qa4 is bad because of Bxa4 then

1...Bxa4 is going to fail high eventually with and without LMR.
If you search Bxa4 first it is going to fail high without LMR because you do not reduce first move and you are going to use LMR only for other moves.

If you search Bxa4 last then you are going to search Bxa4 twice (one time with reduced depth and one time with the full depth because of fail high)

Searching twice take more time so you are going to need more time to get the same depth.

Note that I ignore the fact that usually programs do not reduce captures with LMR and that they start to search good captures because I assume bad move ordering(and use the same idea as good capture last when the program does not know that it is a capture).

Note that I have M.sc in statistics and PHD formally in engineering but practically in combinatorial problems that are not about statistics and the subject of my ph.d is covering codes.

[lkaufman]

Ed, if i just look at rebel and compare it with the derivatives, the trick is the material evaluation of them.

It's more advanced than Rebel one. This where they indeed use a few simple concepts to get it done.

If none of your positional factors, other than passed pawn evaluation,
can ever overrule the material evaluation, then you can easily forward prune last 4 plies, not sure how this has been set in komodo.

King safety scores can be as much as the value of a rook in Komodo, but that doesn't stop us from forward pruning the last four plies.

Larry