Artificial stupidity - making a program play badly

[Jose Carlos]

I've been trying to remember what kind of mistakes I made more often when I used to play OTB.
-Sometimes I just forgot about a certain piece. That piece had a strong move at the end of the variation, but I forgot the piece was there (from the begining, or since a certain move in the middle of the variation).
-Other typical mistake is to forget that a certain piece was captured. At some point in my calculations I think the piece is still on the board, and use it.
-Less frequent, but also happens: just forget about a certain area of the board, typically where a passed pawn can emerge. It used to happen with corners of the board, not with the center.
-Positional mistakes happen when you forget about some important factor of the position. For example, you're very happy with your pawn structure, but forget to evaluate piece activity of the opponent, which happens to be deadly.
-Other times you make a positional mistake because you think you're smarter than then general rule. You think 'I know pawn weakness is bad, but this is an exception to the rule'. And your clever exception proves wrong.
-Sometimes you feel like Tahl and decide to make the board burn in tactical fire. Of course you discover you're not Tahl, and blunder a piece for nothing.
-Or you feel the king of attack, forget everything you know about evaluating positions and just try to the opponent's king killed.

So, in summary, it's all about forgetting things

[CThinker]

I have done this in Thinker.

Here are a few things that I have tried:

1. Make eval do piece counting only. This really, really works.

2. Avoid the best root move. Pick the second best (and sometimes the third best).

3. Use-up a small time, but don't move quickly. For example, search for only 1 second, and then sleep for 5 more seconds before making the move. This way your engine does not make quick moves, and gives the human opponent some time to ponder.

4. Never make a move in less than 2 seconds, even in bullet games.

5. Randomize the time allocation.

6. Offer-up draws when you are behind in score (the obnoxious human).

7. Disconnect randomly.

Note that the goal is for the engine to be 'weak', and not 'stupid'. So, you can violate (2) and (3) when the best move is a re-capture, a capture of a hanging piece, a promotion, or when evading a check.

[Zach Wegner]

6. Offer-up draws when you are behind in score (the obnoxious human).

7. Disconnect randomly.

LOL!

At any rate, I'm giving up for tonight. I'll log off after my next game, log on again with Glaurung running at full strength (on a Core 2 Duo 2.8 GHz), and open for all challengers, in case some of you would like to try a game or two against the latest Glaurung development version. Please be warned that it is terribly underrated, though. Smile

Personally, I'd like to give a crack at the weak version. I'll match you some though, just for kicks.

[michiguel]

I've been trying to remember what kind of mistakes I made more often when I used to play OTB.
-Sometimes I just forgot about a certain piece. That piece had a strong move at the end of the variation, but I forgot the piece was there (from the begining, or since a certain move in the middle of the variation).
-Other typical mistake is to forget that a certain piece was captured. At some point in my calculations I think the piece is still on the board, and use it.
-Less frequent, but also happens: just forget about a certain area of the board, typically where a passed pawn can emerge. It used to happen with corners of the board, not with the center.
-Positional mistakes happen when you forget about some important factor of the position. For example, you're very happy with your pawn structure, but forget to evaluate piece activity of the opponent, which happens to be deadly.
-Other times you make a positional mistake because you think you're smarter than then general rule. You think 'I know pawn weakness is bad, but this is an exception to the rule'. And your clever exception proves wrong.
-Sometimes you feel like Tahl and decide to make the board burn in tactical fire. Of course you discover you're not Tahl, and blunder a piece for nothing.
-Or you feel the king of attack, forget everything you know about evaluating positions and just try to the opponent's king killed.

So, in summary, it's all about forgetting things

"Forget" is what make us intelligent. I should name my engine "Funes" after the great J.L. Borges character.

"Sospecho, sin embargo, que no era muy capaz de pensar. Pensar es olvidar diferencias, es generalizar, abstraer. En el abarrotado mundo de Funes no había sino detalles, casi inmediatos."
Funes, el memorioso de Jorge Luis Borges.

"I suspect, nevertheless, that he was not very capable of thought. To think is to forget a difference, to generalize, to abstract. In the overly replete world of Funes there were nothing but details, almost contiguous details."
From "Funes the memorious", Jorge Luis Borges.

Miguel

[CRoberson]

There are several reasons for the problem:

1) many humans on ICC cheat and use a 2nd computer with a program.
I noticed (myself playing) that many people I played against
played normally (for their rating) or worse for a few games then
they played perfectly in the 4th game. An 1100 player never plays
perfectly. Many months later, I downloaded Dasher which came
with several rated bots. So, I played the 1000 - 1400 bots. They
played exactly like the "ICC humans" at those ratings. So, there
is a significant amount of cheating going on.

2) The instant move trick is something you can do in many of the ICC
interfaces. It is called a premove. Its a time gaining technique.
Don't use it myself, but lots of others do. Essentially, you tell the
interface make this move as soon as my opponent moves. So,
the opponent hasn't moved yet but I can make a premove which
says I want to make this move regardless of opponents move.

3) I am with Bob, Lance and some of the others. You can't linearly
weaken the programs strength without weakening the eval. Especially,
the pawn structure and king safety knowledge. Just consider the
order in which humans learn.

Code: Select all

          below 1000 - people don't know how to count material
          below 1200 - people drop pieces about 4 times a game if the
                             game lasts that long.
          below 1500 - people don't analyze pawn structure well
  

4) You must consider the machine speed. WBEC uses a P3 1.2Ghz
machine and claims TSCP is about 1650. So, if you have a much
faster machine then you have to up the rating. Also, I am not sure
if Leo adjusted his ratings to account for machine speed over the
years. If that is so, then TSCP may be even stronger compared to
humans.

I'll be glad to run some tests for you. I have an old computerized chess
board which holds a 1700 rating. I could match it against a Glaurung
with the parameters of your choice. It is a very good for static strength
testing - the HW and the SW never change.

Also, I'd be happy to try it out on ICC myself. My ICC rating is low
enough. However, it is 500 pts below my club rating and nearly 300
pts below my USCF rating. On a good day, I get real humans - these
days don't happen much.

[Aleks Peshkov]

Just short thought.
People experience tunnel vision effect opposite to computers horizon effect.

Many people discover some interesting variation and begin to implement it in blitz almost totally ignoring opponent silent moves.

I think that depth extensions and reductions should be extremely increased.
Reducing depth to any line except PV will help to graduate strength against humans better then fixed flat n-ply search.

[Michael Sherwin]

Hi Tord;

In the search you could try something like:

if(eval + margine >= beta) return beta;

Then the bigger the margine the weaker the play.

The margine can be varied to depth so you can control the tactics seperate from the strategy.

Just a vauge idea.

Anyway I could not help make Glaurung any stronger with my ideas.

Maybe I can help make it weaker!

[Zach Wegner]

Strange. Glaurung instantly became extremely popular among human players after I registered yesterday night. There are rarely more than a few seconds of idle time between the games. Perhaps having "Glaurung 080519, Elo = 1000" in the finger notes help to attract players, but I doubt they are fooled when the program's ICC rating is around 2100.

After I was reading today, I decided to give it another try. It was pretty nice this time: I have consistently gotten games, many against humans. What's better, ZCT has won every game except for the two against Glaurung it played (our first matchup, it seems) and a draw against Symbolic. Most of the players were sub-2000, but at least my rating goes up.

ZCT is playing against Glaurung now too. Maybe it won't be a catastrophe, it's an almost symmetrical position. The games are pretty interesting though.

[Uri Blass]

The last few days, I've been working on the most important missing feature in my chess program: Adjustable playing strength. Strange though it might seem, making my program play very badly is by far the most difficult and frustrating thing I have attempted to do in computer chess, and I am now close to giving up in despair.

At ratings above 2200, I achieve limited strength simply by reducing the speed of calculation. This works fairly well, as one would expect. Below 2200, I try to emulate typical human blunders and tactical mistakes. This is where the problems begin. My approach seems very reasonable to me: I just prune random moves everywhere in the tree, and the probability that a move is pruned depends on how hard the move would be to see for a human player. Underpromotions and moves with negative SEE value are pruned with very high probability, long diagonal moves also have quite high probability, obvious recaptures have very low probability of being pruned, and so on. Finally, the frequency of pruning of course depends on the playing strength.

Tuning this turned out to be much trickier than I thought. I used TSCP as my sparring partner. The simple task of adjusting the blunder frequency so that my program scored somewhere above 0% and below 100% took a lot of time. After days of work, I finally began to hit close to the mark. I managed to find various settings which scored around 10%, 25%, 50%, 75% and 90% against TSCP. I was also quite pleased with the look of the games: Glaurung played positionally stronger than TSCP, but lost by making quite human-looking blunders. Many of the games looked almost like I would expect a game between TSCP and a similarly rated human to look.

Proud and happy with my work, I started an account on the ICC last night, in order to test against human players. I started with the settings which scored 50% against TSCP, which I thought (based on the WBEC ratings) should have a strength around 1700. At this level, the programs plays positionally ugly chess, and makes plenty of tactical blunders, but rarely hangs a piece, or misses to capture a hanging piece. The result was terribly disappointing: Glaurung played about a dozen games against players around 1900-2100, and won all games except for a single draw. Apparently, 2000 rated players on the ICC make elementary tactical blunders all the time.

I then adjusted the rating down to 1300, and tried again. At this level, the program drops a piece about once or twice per game, on average (at blitz time controls). It turned out that this hardly made any difference: Glaurung still scored close to 100%. Glaurung was frequently hanging pieces, but half the time the human opponents didn't see it, and half the time they quickly paid back the favor by blundering a piece themselves. With a blitz rating of around 2200, I gave up in disgust, logged off and went to bed.

Today, I logged on with the strength set to 1000 -- the lowest implemented level, which scores 0% against TSCP. Glaurung makes several horrible blunders in every single games. It is painful to watch, and it is difficult to imagine how it is possible to play much weaker without playing completely random moves. To my immense frustration, Glaurung still wins most of its games. The current blitz rating, after 37 games, is 2098.

How is this possible? TSCP is rated around 1700, and even when I make my program weak enough to lose every single game against TSCP, it still wins easily against most human players on the ICC. Are the ICC ratings 1000 points too high, or something? How do I manage to lose against average human players, without playing completely random moves?

I'm not sure what the purpose of this post is, apart from venting my frustration, but any advice about how to achieve weak, but realistic-looking play by a computer program would be welcome.

Tord

1)Tscp is clearly better than 2000 at blitz.

2)I think that you are wrong if you assume that humans are positionally better than computers.

Here is an example of positional error that cannot happen to computers and happened to me in my last tournament game(90+30 time control).

In my last tournament game I simply did not pay attention to the fact that the opponent has a passed pawn and I considered the pawn d4 as a weak pawn when it is both weak pawn and passed pawn.

position is equal but I evaluated black as better because I did not see that d4 is a passed pawn and only some moves later in the game I suddenly saw that d4 is a passed pawn.

Analysis shows that this did not cause me to make mistakes in the relevant game(I made mistakes because of different reasons) but this type of mistake can also cause positional mistakes.

[d]r2r2k1/pp3pp1/4bn1p/3q4/2pP4/6NP/PPBQ1PP1/3RR1K1 b - - 0 1

Analysis by Rybka 2.3.2a 32-bit :

20...Qd5-b5 21.Ng3-e4
= (0.00) Depth: 5 00:00:00
20...Qd5-b5 21.Ng3-e4 Nf6xe4
= (-0.06) Depth: 6 00:00:00 7kN
20...Qd5-b5 21.Ng3-e4 Nf6xe4 22.Bc2xe4 Rd8-d7
= (0.06) Depth: 7 00:00:00 10kN
20...Qd5-b5 21.Ng3-e4 Nf6xe4 22.Bc2xe4 Rd8-d7 23.Qd2-c3
= (0.01) Depth: 8 00:00:00 26kN
20...Qd5-b5 21.Ng3-e4 Nf6xe4 22.Bc2xe4 Be6-d5 23.Be4xd5 Rd8xd5 24.Re1-e7 Ra8-e8
= (0.05) Depth: 9 00:00:00 44kN
20...Qd5-b5 21.Re1-e5 Nf6-d5 22.Bc2-e4 Qb5-b6 23.Ng3-f5 Nd5-f6
= (0.07) Depth: 10 00:00:03 225kN
20...Qd5-c6 21.Ng3-e2 Nf6-d5 22.Ne2-f4 Rd8-e8 23.Re1-e5 f7-f6
= (-0.01) Depth: 10 00:00:07 524kN
20...Qd5-c6 21.Ng3-e2 Nf6-d5 22.Ne2-f4 Rd8-e8 23.Re1-e5 Nd5xf4 24.Qd2xf4 Be6-d5
= (-0.03) Depth: 11 00:00:08 593kN
20...Qd5-c6 21.Ng3-e4 Nf6xe4 22.Bc2xe4 Be6-d5 23.Qd2-e3 Rd8-d6 24.Qe3-f4 Rd6-f6 25.Qf4-e5
= (-0.07) Depth: 12 00:00:18 1274kN
20...Qd5-c6 21.Ng3-e4 Nf6xe4 22.Bc2xe4 Be6-d5 23.Qd2-e3 Rd8-d6 24.Qe3-f4 Bd5xe4 25.Re1xe4 Rd6-f6 26.Qf4-e3 Ra8-d8
= (-0.06) Depth: 13 00:00:23 1635kN
20...Qd5-c6 21.Ng3-e4 Nf6-d5 22.Ne4-c5 Be6-c8 23.Nc5-a4 Bc8-e6 24.Na4-c5 Be6-c8 25.Nc5-a4 Bc8-e6 26.Na4-c5 Be6-c8 27.Nc5-a4
= (0.00) Depth: 14 00:00:39 2739kN
20...Qd5-c6 21.Ng3-e4 Nf6-d5 22.Ne4-c3 b7-b5 23.Bc2-e4 Ra8-b8 24.a2-a3 Qc6-b6 25.Be4xd5 Be6xd5 26.Qd2-f4 Qb6-b7
= (0.07) Depth: 15 00:01:05 4314kN
20...Qd5-b5 21.Re1-e5 Qb5-b6 22.Ng3-e4 Nf6xe4 23.Bc2xe4 f7-f6 24.Re5-c5 Rd8-d7 25.Qd2-c3 Ra8-d8 26.g2-g3 Be6-f7
= (0.04) Depth: 15 00:01:33 6528kN
20...Qd5-b5 21.Re1-e5 Qb5-b6 22.Bc2-f5 Be6-d5 23.Qd2-c3 g7-g6 24.Bf5-c2 Qb6-c6 25.Rd1-e1 Rd8-e8 26.f2-f3 Ra8-d8
= (0.01) Depth: 16 00:03:38 14184kN
20...Qd5-b5 21.Re1-e5 Qb5-b6 22.Bc2-f5 Be6-d5 23.Qd2-c3 g7-g6 24.Bf5-c2 Qb6-c6 25.Rd1-e1 Rd8-e8 26.f2-f3 Ra8-d8 27.Ng3-e2
= (0.05) Depth: 17 00:04:32 17302kN
20...Qd5-b5 21.Re1-e5 Qb5-b6 22.Bc2-f5 Be6-d5 23.Ng3-e4 Bd5xe4 24.Bf5xe4 Nf6xe4 25.Re5xe4 Rd8-e8 26.Rd1-e1 Re8xe4 27.Re1xe4
= (0.05) Depth: 18 00:07:37 29141kN
20...Qd5-c6 21.Ng3-e4 Be6-d5 22.Ne4xf6+ Qc6xf6 23.Re1-e5 Qf6-c6 24.f2-f3 f7-f6 25.Re5-e1 Rd8-e8 26.Qd2-b4 a7-a5 27.Qb4-a3
= (0.00) Depth: 18 00:10:49 44655kN

(so k, 21.05.2008)

[Uri Blass]

I can add that the position 2 plies earlier was

[d]r2r2k1/pp3pp1/4bn1p/2bq4/2pB4/2P3NP/PPBQ1PP1/3RR1K1 b - - 0 19

Bxd4 is probably the best move inspite of the fact that white get a passed pawn at d4.

I believe that a typical human mistake is not to pay attention to positional factors that were not in the root position(but later in the game the human may pay attention to them).

Uri