Simple way to beat most go engines

[Isaac]

Hello,
I have been censored on KGS (and also on computer go mailing list apparently) because I described a strategy to win against almost any go program. In fact I only know one program that could be immune to it (Zen, not Alphago, mind you) but until now it showed to bug against me and I could win against it.
I must say a few things, first, that I am a 3kyu strength player which is equal to the average player on KGS. Second, that the purpose of this thread is not to exploit KGS ranking system, but just to expose a "stupid" weakness of go engines with the eventual goal in mind that programmers take some time to fix it. It is fixable since Zen is having some success about it.

Ok, here is the strategy:
If the game uses Chinese rules, take black (which is automatic if you're a kyu and you play vs a strong dan bot) and take a komi not greater than 0.5 (0.5 is the default on KGS in such a case). Do not use handicap stones. First move must be K10. Then do mirror unless the K10 group is in atari in which case you have to break the mirror and winning is not 100% guaranteed, though in my own experience it is easier to win than lose.

If the game uses Japanese rules, do the same, except that the komi must be negative. On KGS this means at least -4.5 komi. The game can still be ranked.
I played one game against Crazystone and lost because I did not realize it was using Japanese rules so I hadn't set a negative komi. But I could realize that, unlike Zen, Crazystone has no anti mirror strategy implemented.

Overall, Hirabot 6 dan, Zen19A 8 dan, hiraMC 4 dan, several hira 3 dan, darkgo 1 dan and golois 1 kyu have lost against me using this tactics. I could maintain a solid 6 dan rank on KGS, being only 3 kyu in strength.

P.S.: I have been thanked by Yamato, who's Zen's author, because apparently I exposed a bug and Zen should have countered the mirror (which it did against the opponent after me who tried the strategy) but it failed (also filling both dame and eyes in middle game which is not normal). So the author told me he would work on a fix.

[zamar]

Maybe it's just me, but I consider these kind of tricks as unfair play. One purpose of (positive) komi is to make playing mirror go not-so-easy.

In computer chess one way to win against almost any chess engine is use to very short time controls, head for blocked positions and pre-move like crazy...

Negative komi in go?! Please, give me a break.

[Isaac]

Maybe it's just me, but I consider these kind of tricks as unfair play. One purpose of (positive) komi is to make playing mirror go not-so-easy.

In computer chess one way to win against almost any chess engine is use to very short time controls, head for blocked positions and pre-move like crazy...

Negative komi in go?! Please, give me a break.

I agree that this is unfair in the sense that it's a bit like using handicap stones and such settings would never be allowed in any serious tournament between even players I suppose. But it's still fair enough to be allowed for ranked games on go servers.
With Chinese rules you don't need negative komi (komi 0.5 is enough).
But without handicap mirror strategy, it's possible that a newcomer to go or very weak player buy strong programs, take black and either set 0.5 komi (cn rules) or -1.5 komi (jp rules) and still win games.
While without mirror these people would be unable to win, even given 9 stones as handicap.
So I consider that this is a weakness in the programs, I mean not having an anti mirror strategy. Defeating a pure mirror script should be easy to code (just circle K10 group) although things get more complicated when the human starts with mirror and then breaks it to avoid losing K10.

[Leto]

This is valuable for Go programmers. I do doubt that this would work against Alphago.

[matthewlai]

I agree that this is unfair in the sense that it's a bit like using handicap stones and such settings would never be allowed in any serious tournament between even players I suppose. But it's still fair enough to be allowed for ranked games on go servers.

I would say that makes it a bug in KGS rules instead of Go engines. Obviously with enough komi in the wrong direction, anyone can win against anyone else. There are different types of handicap (stones and komi), and some may be more easily exploitable than others. That doesn't mean it's a problem with Go engines.

Go engines shouldn't need to be able to deal with mirror positions, because in games with reasonable komi, mirroring is a losing strategy.

It's like saying chess engines should be optimized for playing with queen odds. They aren't, because they are designed to play chess, and playing with queen odds makes it another game (that is similar enough that most engines will still do reasonably well, but is another game nonetheless).

[Isaac]

I agree that this is unfair in the sense that it's a bit like using handicap stones and such settings would never be allowed in any serious tournament between even players I suppose. But it's still fair enough to be allowed for ranked games on go servers.

I would say that makes it a bug in KGS rules instead of Go engines. Obviously with enough komi in the wrong direction, anyone can win against anyone else. There are different types of handicap (stones and komi), and some may be more easily exploitable than others. That doesn't mean it's a problem with Go engines.

Go engines shouldn't need to be able to deal with mirror positions, because in games with reasonable komi, mirroring is a losing strategy.

It's like saying chess engines should be optimized for playing with queen odds. They aren't, because they are designed to play chess, and playing with queen odds makes it another game (that is similar enough that most engines will still do reasonably well, but is another game nonetheless).

I don't consider that, under Chinese rules, a komi of 0.5 (thus favoring white) is equivalent to a Queen odd in chess. In fact it's less of a handicap than 2 stones handicap. So the handicap is only very slight. Nonetheless it's possible to defeat hira 6d with such a small handicap, regardless of your strength. To me, this would be equivalent to roughly a pawn or less, in chess.
Meanwhile, I have been told that some Android versions of Aya (a famous go engine), have anti mirror strategy. I didn't test it though.

[matthewlai]

I agree that this is unfair in the sense that it's a bit like using handicap stones and such settings would never be allowed in any serious tournament between even players I suppose. But it's still fair enough to be allowed for ranked games on go servers.

I would say that makes it a bug in KGS rules instead of Go engines. Obviously with enough komi in the wrong direction, anyone can win against anyone else. There are different types of handicap (stones and komi), and some may be more easily exploitable than others. That doesn't mean it's a problem with Go engines.

Go engines shouldn't need to be able to deal with mirror positions, because in games with reasonable komi, mirroring is a losing strategy.

It's like saying chess engines should be optimized for playing with queen odds. They aren't, because they are designed to play chess, and playing with queen odds makes it another game (that is similar enough that most engines will still do reasonably well, but is another game nonetheless).

I don't consider that, under Chinese rules, a komi of 0.5 (thus favoring white) is equivalent to a Queen odd in chess. In fact it's less of a handicap than 2 stones handicap. So the handicap is only very slight. Nonetheless it's possible to defeat hira 6d with such a small handicap, regardless of your strength. To me, this would be equivalent to roughly a pawn or less, in chess.
Meanwhile, I have been told that some Android versions of Aya (a famous go engine), have anti mirror strategy. I didn't test it though.

It's not a question of amount of handicap. Any handicap changes the game, and engines being able to play games they aren't designed for (no matter how small the difference is) is just a nice bonus, and failure should not be considered a bug in the engine.

If there's an easy way to draw Stockfish with a pawn odd, would that be considered a bug in Stockfish? I don't think so, because Stockfish was designed to play chess, not chess with pawn odds. What about knight odds? What about rook odds? Where do we draw the line?

[Isaac]

I agree that this is unfair in the sense that it's a bit like using handicap stones and such settings would never be allowed in any serious tournament between even players I suppose. But it's still fair enough to be allowed for ranked games on go servers.

I would say that makes it a bug in KGS rules instead of Go engines. Obviously with enough komi in the wrong direction, anyone can win against anyone else. There are different types of handicap (stones and komi), and some may be more easily exploitable than others. That doesn't mean it's a problem with Go engines.

Go engines shouldn't need to be able to deal with mirror positions, because in games with reasonable komi, mirroring is a losing strategy.

It's like saying chess engines should be optimized for playing with queen odds. They aren't, because they are designed to play chess, and playing with queen odds makes it another game (that is similar enough that most engines will still do reasonably well, but is another game nonetheless).

I don't consider that, under Chinese rules, a komi of 0.5 (thus favoring white) is equivalent to a Queen odd in chess. In fact it's less of a handicap than 2 stones handicap. So the handicap is only very slight. Nonetheless it's possible to defeat hira 6d with such a small handicap, regardless of your strength. To me, this would be equivalent to roughly a pawn or less, in chess.
Meanwhile, I have been told that some Android versions of Aya (a famous go engine), have anti mirror strategy. I didn't test it though.

It's not a question of amount of handicap. Any handicap changes the game, and engines being able to play games they aren't designed for (no matter how small the difference is) is just a nice bonus, and failure should not be considered a bug in the engine.

If there's an easy way to draw Stockfish with a pawn odd, would that be considered a bug in Stockfish? I don't think so, because Stockfish was designed to play chess, not chess with pawn odds. What about knight odds? What about rook odds? Where do we draw the line?

I agree that this is unfair (as I said in post 3) and that such games would never be allowed in serious tournaments.
However I do think that it's a problem, mainly because without mirror a 30 kyu wouldn't win even given 9 stones as handicap whilst if the komi is 0.5 which is much less of a handicap, he can win against top bots.
Apparently both Zen and Aya authors also thought it was a problem, else they wouldn't have bothered writing some code about it.

[Isaac]

It's not a question of amount of handicap. Any handicap changes the game(...)

I also disagree here for another reason. A very slight amount of handicap is no harm, indeed all players agree to play with non integer komi even in serious tournaments. However, if the game was fair, neither black or white should win with perfect play, there should be a jigo (draw) instead, which cannot be achieved with non integer komi.

So the amount of handicap really does matter.