Could EGTB make an Engine "weaker"?

[voyagerOne]

Let say we have a monster engine with 8-piece database.
Its playing against a weaker engine and towards end game the Eval is +.60 for the monster engine. The monster engine is surely going to win.

But wait...using TB the best outcome is a draw. The path to the draw is extremely difficult to find using search. So the monster plays a "sub-optimal" move, creating easier paths for the draw. The results then ends in a draw.

Without the TB the monster will win the game...since both engines are blind to the crazy path for a draw.

Thoughts?

[Daniel Shawul]

Hate to add to the current EGTB frenzy, but here it goes . The chinook guys had this as a serious problem when they started hitting the TBs from the initial position. Ofcourse this is only a problem if you are playing against weaker beings, say humans. Many engines switch to 'swindle mode' when they find out EGTBs say game is draw. Then the engine starts to play random moves because all moves report a score of 0, that is not adjusted by the ply where the final draw occurs, unlike the case of mate. This is a problem of MINIMAX, not TBs, where it is assumed the opponent plays the best move. In multi-player games, for instance, this is not the case. There are better strategies than minimax for multiplayer games. The direct equivalent of minimax for multiplayer games is the maxn strategy in which it is assumed all players collude against the player about to move. Another name for this is 'paranoid strategy' which also applies to mini-max. You can imagine this strategy is going to increasingly become less efficient with more number of players. For 5 or more players, maxn will think it is playing against 4 opponents merged into one, so you would start seeing mate on the first move, and start playing random move!
So to answer your question, no EGTBs are not the problem, but the less-than optimal mini-max search strategy when you are playing weak opponents.

[voyagerOne]

Right..we can forget about EGTB...

So would one expect a super engine to convert "wins to draws"...i.e. it is "pessimistic" as it will often see a crazy draw line, when it has an advantage...whereas a less superior engine will fight till the end as it is blinded by the draw and eventually win.

[syzygy]

Let say we have a monster engine with 8-piece database.
Its playing against a weaker engine and towards end game the Eval is +.60 for the monster engine. The monster engine is surely going to win.

But wait...using TB the best outcome is a draw. The path to the draw is extremely difficult to find using search. So the monster plays a "sub-optimal" move, creating easier paths for the draw. The results then ends in a draw.

Without the TB the monster will win the game...since both engines are blind to the crazy path for a draw.

Thoughts?

Certainly possible, but similar things happen as well without TBs. If a move seems winning at say 25 ply, it will probably win against weaker players. But at 26 ply the score may drop due to the discovery of a tactical variation that the opponent would never have seen and the engine may switch to a move that might lead to an easy draw for the weaker opponent.

I do not think that this means that an EGTB could make an engine "weaker". It is certainly possible that a particular game that would have been won without TBs ends in a draw with TBs, but it is quite likely that this is outweighed by the games that would have been drawn without TBs and are won with TBs. This is similar to other engine improvements including mere speedups: each improvement will change some of the moves that the engine plays, and some of these changes will be less than optimal.

[Daniel Shawul]

Yes, to nail the coffin this behaviour can happen in middle games too. The is a problem known as "opponent modeling". The best strategy against all kind of opponents is not to assume they play the best move, as mini-max does. If you know the opponent is weaker, then you assume it makes sub-optimal moves and adjust your strategy accordingly.

[hgm]

Thoughts?

This is a matter of faulty implementation (but alas, quite commonly occurring). Many engines set the score to zero if the EGT says the current position is a draw, and as a result they randomly pick a non-losing move, with only a very low probability of that move being the 'best' one to afford the opponent the possibility to make a mistake. Often quite the opposite: they allow themselves to be pushed to the very edge of losing, without offering any resistance at all, and only then they start too fight (successfully, of course) for the draw.

The proper implementation would be to just divide the normal evaluation of the engine by a factor (say 10) if the EGT says it is draw. Then the engine will continue to play its best, and put pressure on the opponent to which he might succumb if he is fallible.

Scores are not supposed to reflect the game-theoretical outcome of a position, but are a heuristic progress measure. If you would score all won KBBKN positions as +INF, the engine would also not be able to find the win. It is essential that it know that positions with the weak side's King in the center, although still theoretically won, are still worse then when he is in the corner.

[voyagerOne]

Lets extend this idea. Say its middle game...and our Super-Engine has a slight advantage and finds an impressive line to attack the king or even capture the queen. However, one depth deeper it finds a crazy line that the opponent can draw by king check repetition. So it plays a safe boring move...and the game eventually draws.

The draw would have been impossible to find for the other engine and if the super-engine made the attack it would no doubt win.

Testers are frustrated because there Super-Engine ELO seems to drop...when it actually really did improved.

[hgm]

Indeed, this is a problem. Engines can be too smart for their own sake. This already happened very long ago, when humans still had the advantage over engines. In a game against GMs, the engine sacrificed a Rook on ply 1, as the cheapest way to avoid an extremely deep mate, that the GM would have never seen. But if course he did see that he's better capture a Rook that was offered to him for free, and easily won the game.

I hard that in the last WCCC there was another example; Johnny, playing on 2000 cores, was searching so deep that in a better position it saw the opponent could force a draw, and stopped trying to win altogether. (Again, under the misconception that all draws are equal.)

An engine that would play 'perfectly' might win very little against a strong opponent, as the initial position is not won, and it might play so cowardly to ovoid positions that could be won by other perfect opponents, that its winning rate suffers.

So 'stronger' and 'weaker' are relative concepts. If you play against beginners opening with 1.e4 followed by 2. Qh5 will bring you more points quicker than any other line of play. Knowing that it is pointless and even harmful makes it just more difficult to win.

[Don]

Let say we have a monster engine with 8-piece database.
Its playing against a weaker engine and towards end game the Eval is +.60 for the monster engine. The monster engine is surely going to win.

But wait...using TB the best outcome is a draw. The path to the draw is extremely difficult to find using search. So the monster plays a "sub-optimal" move, creating easier paths for the draw. The results then ends in a draw.

Without the TB the monster will win the game...since both engines are blind to the crazy path for a draw.

Thoughts?

This is a classic problem even without databases. In a famous computer chess game from decades ago a superior deeper searching program found it was getting checkmated and played a move which delayed the mate (causing it to take longer.) But the delaying move was a sacrifice that the weaker program would easily refute. Had it "bluffed" by playing the move it would have played before discovering the mate, the inferior program would not have seen the mate and would have surely lost.

As a result of this game an algorithm was proposed to deal with it. If you suddenly find a major score drop which will cause a lost games in one iteration, play the move you would have played on the previous iteration.

I have no idea how helpful that idea is.

Don

[bob]

Let say we have a monster engine with 8-piece database.
Its playing against a weaker engine and towards end game the Eval is +.60 for the monster engine. The monster engine is surely going to win.

But wait...using TB the best outcome is a draw. The path to the draw is extremely difficult to find using search. So the monster plays a "sub-optimal" move, creating easier paths for the draw. The results then ends in a draw.

Without the TB the monster will win the game...since both engines are blind to the crazy path for a draw.

Thoughts?

This is a classic problem even without databases. In a famous computer chess game from decades ago a superior deeper searching program found it was getting checkmated and played a move which delayed the mate (causing it to take longer.) But the delaying move was a sacrifice that the weaker program would easily refute. Had it "bluffed" by playing the move it would have played before discovering the mate, the inferior program would not have seen the mate and would have surely lost.

As a result of this game an algorithm was proposed to deal with it. If you suddenly find a major score drop which will cause a lost games in one iteration, play the move you would have played on the previous iteration.

I have no idea how helpful that idea is.

Don

I think that was Berliner's idea. Works against humans. Not so well against computers...