Lemming Poll

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
BubbaTough
Posts: 1154
Joined: Fri Jun 23, 2006 3:18 am

Lemming Poll

Post by BubbaTough » Mon Sep 22, 2008 10:58 pm

Howdy,

LearningLemming has been off and on ICC for a couple years now without a learning reset, and has played A LOT of engine - engine matches...probably 100+ against a number of engines including multiple crafty's, tinker, arasan, amateur, telepath, multiple rybka's, and more). I would be interested if I could get anyone who has been involved in these matches (author or operator of opposing program, or observer) to give me a little feedback if possible on how the learned "book" is going. This is a completely unscientific poll, I am just looking for gut feel.

1. Do you think the "book" has been getting better of the months/years?
2. Do you think the results against your engine (or other engines or humans) is getting better in part because of its "book" is improving its play against your book?
3. Do you think Lemming is getting on average an advantage, disadvantage, or neutral position when you both exit book?
4. Is your program or Lemming getting a time advantage on average by one of the programs exiting book early?
5. Do you think the learning is overly specific to your book, and if you switched to a random book your results would improve?
6. Anything else that comes to mind.

Anyone who has played or watched or looked at results of LearningLemming games on ICC is encouraged to respond.

-Sam

bob
Posts: 20358
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Lemming Poll

Post by bob » Mon Sep 22, 2008 11:59 pm

BubbaTough wrote:Howdy,

LearningLemming has been off and on ICC for a couple years now without a learning reset, and has played A LOT of engine - engine matches...probably 100+ against a number of engines including multiple crafty's, tinker, arasan, amateur, telepath, multiple rybka's, and more). I would be interested if I could get anyone who has been involved in these matches (author or operator of opposing program, or observer) to give me a little feedback if possible on how the learned "book" is going. This is a completely unscientific poll, I am just looking for gut feel.

1. Do you think the "book" has been getting better of the months/years?
2. Do you think the results against your engine (or other engines or humans) is getting better in part because of its "book" is improving its play against your book?
3. Do you think Lemming is getting on average an advantage, disadvantage, or neutral position when you both exit book?
4. Is your program or Lemming getting a time advantage on average by one of the programs exiting book early?
5. Do you think the learning is overly specific to your book, and if you switched to a random book your results would improve?
6. Anything else that comes to mind.

Anyone who has played or watched or looked at results of LearningLemming games on ICC is encouraged to respond.

-Sam
I have not paid any attention at all. We have been re-writing Crafty's eval to use the "fruit interpolation" method in order to get a feel for how that works. As a result, I have not paid any attention at all to results on ICC except for the rare occasions where I am on and watch a game in progress. If you want to set up a time to play a few games where I can actually watch, I'd be willing, and could try to make sure I have a reasonably sane version running. I always take the most recent changes and run on ICC whether they have been tested on the cluster or not...

BubbaTough
Posts: 1154
Joined: Fri Jun 23, 2006 3:18 am

Re: Lemming Poll

Post by BubbaTough » Tue Sep 23, 2008 12:51 am

I have not paid any attention at all. We have been re-writing Crafty's eval to use the "fruit interpolation" method in order to get a feel for how that works. As a result, I have not paid any attention at all to results on ICC except for the rare occasions where I am on and watch a game in progress. If you want to set up a time to play a few games where I can actually watch, I'd be willing, and could try to make sure I have a reasonably sane version running. I always take the most recent changes and run on ICC whether they have been tested on the cluster or not...
Hey, thanks for the offer! LearningLemming is on ICC right now, and is set for autorematch. It should be on for the next few days hopefully without interruption. It sounds like you won't be able to answer much in terms of how its been progressing long term, but some feedback on whether it is getting decent positions, and whether it is getting on average and advantage or disadvantage against your book sounds possible and helpful. Though I guess that type of feedback is somewhat relevant to long term progress since assumably when LearningLemming had a book with 0 moves in it we can assume it was inferior in most ways....

-Sam

BubbaTough
Posts: 1154
Joined: Fri Jun 23, 2006 3:18 am

Re: Lemming Poll

Post by BubbaTough » Wed Sep 24, 2008 7:16 pm

Timea and TwistedLogic have been playing marathon matches on ICC with LearningLemming recently...hopefully to develop comments for this thread? :o

-Sam

Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 8:19 pm
Location: Oslo, Norway

Re: Lemming Poll

Post by Tord Romstad » Wed Sep 24, 2008 7:57 pm

bob wrote:We have been re-writing Crafty's eval to use the "fruit interpolation" method in order to get a feel for how that works.
What's the "fruit interpolation method"?

Tord

User avatar
Zach Wegner
Posts: 1922
Joined: Wed Mar 08, 2006 11:51 pm
Location: Earth
Contact:

Re: Lemming Poll

Post by Zach Wegner » Wed Sep 24, 2008 8:55 pm

Tord Romstad wrote:What's the "fruit interpolation method"?

Tord
The same as the "Glaurung interpolation method". ;)

That is, keeping an opening and endgame evaluation and interpolating between them based on the phase. I use it too, but I don't really like it. When I get around to rewriting my eval, I'm planning on interpolating for individual eval terms that need it, but keeping the overall eval clean with just one score.

bob
Posts: 20358
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Lemming Poll

Post by bob » Wed Sep 24, 2008 9:26 pm

Tord Romstad wrote:
bob wrote:We have been re-writing Crafty's eval to use the "fruit interpolation" method in order to get a feel for how that works.
What's the "fruit interpolation method"?

Tord
score = (ph * mgscore + (62 - ph) * egscore) / 62

where ph is a number between 0 and 62 and reflects the material point count on the board (31 is all pieces (pawns execluded) for one side.

Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 8:19 pm
Location: Oslo, Norway

Re: Lemming Poll

Post by Tord Romstad » Wed Sep 24, 2008 9:38 pm

Zach Wegner wrote:
Tord Romstad wrote:What's the "fruit interpolation method"?

Tord
The same as the "Glaurung interpolation method". ;)

That is, keeping an opening and endgame evaluation and interpolating between them based on the phase.
Aha!

That technique doesn't deserve to be named after Fruit or Glaurung, because it was common and widespread long before both programs existed. I know that Phalanx used it back in the last millennium, and I am sure you can find many earlier examples.

Tord

User avatar
Zach Wegner
Posts: 1922
Joined: Wed Mar 08, 2006 11:51 pm
Location: Earth
Contact:

Re: Lemming Poll

Post by Zach Wegner » Wed Sep 24, 2008 9:43 pm

Tord Romstad wrote:
Zach Wegner wrote:
Tord Romstad wrote:What's the "fruit interpolation method"?

Tord
The same as the "Glaurung interpolation method". ;)

That is, keeping an opening and endgame evaluation and interpolating between them based on the phase.
Aha!

That technique doesn't deserve to be named after Fruit or Glaurung, because it was common and widespread long before both programs existed. I know that Phalanx used it back in the last millennium, and I am sure you can find many earlier examples.

Tord
Of course. Check the wiki.

http://chessprogramming.wikispaces.com/Tapered+Eval
http://chessprogramming.wikispaces.com/ ... al/4123771

;)

Post Reply