Moderator: Ras
After I was reading today, I decided to give it another try. It was pretty nice this time: I have consistently gotten games, many against humans. What's better, ZCT has won every game except for the two against Glaurung it played (our first matchup, it seems) and a draw against Symbolic. Most of the players were sub-2000, but at least my rating goes up.Tord Romstad wrote:Strange. Glaurung instantly became extremely popular among human players after I registered yesterday night. There are rarely more than a few seconds of idle time between the games. Perhaps having "Glaurung 080519, Elo = 1000" in the finger notes help to attract players, but I doubt they are fooled when the program's ICC rating is around 2100.
1)Tscp is clearly better than 2000 at blitz.Tord Romstad wrote:The last few days, I've been working on the most important missing feature in my chess program: Adjustable playing strength. Strange though it might seem, making my program play very badly is by far the most difficult and frustrating thing I have attempted to do in computer chess, and I am now close to giving up in despair.
At ratings above 2200, I achieve limited strength simply by reducing the speed of calculation. This works fairly well, as one would expect. Below 2200, I try to emulate typical human blunders and tactical mistakes. This is where the problems begin. My approach seems very reasonable to me: I just prune random moves everywhere in the tree, and the probability that a move is pruned depends on how hard the move would be to see for a human player. Underpromotions and moves with negative SEE value are pruned with very high probability, long diagonal moves also have quite high probability, obvious recaptures have very low probability of being pruned, and so on. Finally, the frequency of pruning of course depends on the playing strength.
Tuning this turned out to be much trickier than I thought. I used TSCP as my sparring partner. The simple task of adjusting the blunder frequency so that my program scored somewhere above 0% and below 100% took a lot of time. After days of work, I finally began to hit close to the mark. I managed to find various settings which scored around 10%, 25%, 50%, 75% and 90% against TSCP. I was also quite pleased with the look of the games: Glaurung played positionally stronger than TSCP, but lost by making quite human-looking blunders. Many of the games looked almost like I would expect a game between TSCP and a similarly rated human to look.
Proud and happy with my work, I started an account on the ICC last night, in order to test against human players. I started with the settings which scored 50% against TSCP, which I thought (based on the WBEC ratings) should have a strength around 1700. At this level, the programs plays positionally ugly chess, and makes plenty of tactical blunders, but rarely hangs a piece, or misses to capture a hanging piece. The result was terribly disappointing: Glaurung played about a dozen games against players around 1900-2100, and won all games except for a single draw. Apparently, 2000 rated players on the ICC make elementary tactical blunders all the time.
I then adjusted the rating down to 1300, and tried again. At this level, the program drops a piece about once or twice per game, on average (at blitz time controls). It turned out that this hardly made any difference: Glaurung still scored close to 100%. Glaurung was frequently hanging pieces, but half the time the human opponents didn't see it, and half the time they quickly paid back the favor by blundering a piece themselves. With a blitz rating of around 2200, I gave up in disgust, logged off and went to bed.
Today, I logged on with the strength set to 1000 -- the lowest implemented level, which scores 0% against TSCP. Glaurung makes several horrible blunders in every single games. It is painful to watch, and it is difficult to imagine how it is possible to play much weaker without playing completely random moves. To my immense frustration, Glaurung still wins most of its games. The current blitz rating, after 37 games, is 2098.
How is this possible? TSCP is rated around 1700, and even when I make my program weak enough to lose every single game against TSCP, it still wins easily against most human players on the ICC. Are the ICC ratings 1000 points too high, or something? How do I manage to lose against average human players, without playing completely random moves?
I'm not sure what the purpose of this post is, apart from venting my frustration, but any advice about how to achieve weak, but realistic-looking play by a computer program would be welcome.
Tord
First of all, with all due respect to your frustration, I must admit this thread is one of the funniest I've ever readTord Romstad wrote: That's some relief. Glaurung's ICC rating when playing at an Elo setting of 1000 seems to have stabilized around 2100. Right now, it's 2094, after 44 games (+33,-10,=1). It has a lost position against a 1400 rated player in the currently running game, but I have lost all faith in human players and expect Glaurung to win in the end.
This is _very_ difficult to make work. The trees are huge, and just choosing random moves to not search doesn't work very well in trying to weaken the engine. When I started trying to do this, that was my first idea, to weaken the search and leave everything else along. I turned off all extensions/reductions, and then started to just pick a random point at each node to say "no more moves are available". And I could not get it to play (and look) like a 1200 or 1400 player. Beginners will push pawns without regard to creating weak pawns, weak squares, or weak king safety. Etc. I could not get below 2000 in testing on my cluster or ICC, and in watching the games, they just didn't look right. I began to realize that a crippled search would affect one aspect ot the game (tactics) but not do a thing for positoinal judgement. Hence my later effort at cripling the evaluation. But just toning down the positional scoring was not enough without factoring in the random value I ended up adding.Marek Soszynski wrote:Tord,
Here's my idea...
Randomly exclude a number of moves from consideration altogether. For very weak play half the number of possible moves (or more) could be excluded. For a very slight reduction in playing level a single move (selected at random) could be excluded on every other turn. In addition, for a realistically reduced level of play, the engine should start its calculations after an interval.