Endgame Evaluation help!

[Edsel Apostol]

If you may have played Twisted Logic or watched its games, you may notice that it almost has no endgame knowledge. The only knowledge it has is concerning passed pawns. I'm planning to add some endgame knowledge to it.

My problem is that I don't know what are the basic endgame knowledge that should be added. I don't want yet to support EGTB and EGBB.

Do I have to follow Fruit's example of having code for specific positions like KPK, KRPK, etc, or there are some general rules specific for and applicable only for the endgame?

[Kempelen]

If you may have played Twisted Logic or watched its games, you may notice that it almost has no endgame knowledge. The only knowledge it has is concerning passed pawns. I'm planning to add some endgame knowledge to it.

My problem is that I don't know what are the basic endgame knowledge that should be added. I don't want yet to support EGTB and EGBB.

Do I have to follow Fruit's example of having code for specific positions like KPK, KRPK, etc, or there are some general rules specific for and applicable only for the endgame?

I have included some basic endgame knowledge in my (not released yet) engine. I give you some tips:

- passed pawn, doubled pawn, .... all kind of pawns formations....
- In pawn endgames: square rule.
- Of course: king in center has bonus.
- Bonus rooks always behind pawns.
- passed defended pawns get a little bonus.
- if only one pieze per side, and if they are opposed bishops, subtract a penalty (I subtract 80) if score-80 < 0 then score = 0
- rook pawn subtract a penalty when less pawns are in play, becuase this pawn is less valuable
- knight post (encourage been in center).
- winner side, bonus for having pawns. losser side bonus for having pieces. Winner side penalty for having pieces, losser side penalty for having pawns. This encourage "correct" exchanges.
- if no pawns, try to evaluate if winner side can win (KNN vs K , KRB vs KR, etc...)
- if side only have a minor pieces agains pawn, it can't win, so reduce the piece value in order to try to avoid that endgame.
- evaluate king movility if opponet side has more than one piece, because is possible to been in a net mate.
- rook is more valuable as less pawns are in the board.
- bishops pair has a huge aventage (I give them 50 centipawns) in open possitions. I use the formula (from memory) 50-(number_pawns*3)
- bishop situation from piece/square table
- knight get bonus if possition is closed (as more pawn are in the board, the better).
- Bonus for attacking an enemy piece
- Bonus for deffending an own piece.

I am near to release my engine and would like betatester. Volunteers are wellcome.

I hope you find this list valuable. Let me know.

Fermin

[Edsel Apostol]

Hi Fermin,

Thanks!

I have most of them, except for a few. My pawn structure and passed pawn evaluation is already good.

I just don't have code for material imbalance yet, maybe it would help.

May I know what's the name of your engine? Does it support UCI? How strong it is?

[Kempelen]

Hi Fermin,

Thanks!

I have most of them, except for a few. My pawn structure and passed pawn evaluation is already good.

I just don't have code for material imbalance yet, maybe it would help.

May I know what's the name of your engine? Does it support UCI? How strong it is?

The name is 'Rodin'. It does not support UCI yet. It is not very strong, dont know exactly how many. I.e. it scores 50% against Matheus 2.6. It solves around 260/270 wac in 5 secs. I'm trying to improve his tactical ability because I think is where is less strong. It has around 300.000 nps which I dont know if it is few or much (comparing to Crafty I feel an idiot I plan to release in two or there weeks. I am currently not making changes (at less important ones) and only doing intensive tests in my two computers.

[bob]

If you may have played Twisted Logic or watched its games, you may notice that it almost has no endgame knowledge. The only knowledge it has is concerning passed pawns. I'm planning to add some endgame knowledge to it.

My problem is that I don't know what are the basic endgame knowledge that should be added. I don't want yet to support EGTB and EGBB.

Do I have to follow Fruit's example of having code for specific positions like KPK, KRPK, etc, or there are some general rules specific for and applicable only for the endgame?

For starters I think the critical pieces of endgame information needed are as follows, not in any particular order:

1. passed pawns are more valuable as they advance, and they are more valuable if they are connected to a friendly pawn for support, and they are move valuable if accompanied by their king for additional support.

2. where does the king belong? In the center unless there are passed pawns to escort (friendly) or blockade/stop (enemy). This is one of the more important aspects as often endgames are won by a single tempo.

3. outside/distant passed pawns and the fact that they become more valuable as material is reduced.

4. candidate passed pawns because the search might not go deeply enough to see the candidate convert to a real passer.

5. blockade passed pawns, which is a sub-component of step 1 above. If passed pawns can't advance, they can't increase in value. This will also cause your program to attempt to remove an enemy blockade for the same reason.

There are lots of other ideas dealing with pieces. For example, sometimes two isolated passers are much better than two connected passers, because a lone king can't stop two passers on opposite wings while it slow two connected passers down and let its split passers advance. A knight favors the side with pawns on one wing, while a bishop is more effective with pawns on both wings due to its better mobility.. Etc...

[Edsel Apostol]

You're lucky considering the strength of your engine. When I'm just starting to learn computer chess programming and I released the first version of my engine, it is only rated at about same level as TSCP and that is 1650.

300000nps seems slow. What's your hardware? Crafty is a speed demon, it is optimized well.

My advice on your eval, try to support simple things for now, maybe material and piece square tables only. Then test your engine and watch its games. Add little by little the knowledge depending on which your engine needs as you seen it play. This way, you are really sure that the knowledge you are putting in your engine really helps.

I have done the mistake before that if I add all of the chess knowledge in my engine that it would be strong. What I didn't realize that most of them conflicts with each other. The secret of strong engine nowadays lies on simple and efficient eval factors that are in harmony with each other.

I hope it helps you and other new programmers out there.

[Edsel Apostol]

Thanks Bob,

The first engine I encountered before is Crafty and Rebel/ProDeo. That was about 5 to 6 years ago. I'm just discovering computer chess then. I don't even know how to program in C that time.

For starters I think the critical pieces of endgame information needed are as follows, not in any particular order:

1. passed pawns are more valuable as they advance, and they are more valuable if they are connected to a friendly pawn for support, and they are move valuable if accompanied by their king for additional support.

I have this already, though I don't have friendly pawn support yet. I think my general code for passed pawn evaluation considers it also but I will try to add specific code for that friendly pawn and see in tests if it might help.

2. where does the king belong? In the center unless there are passed pawns to escort (friendly) or blockade/stop (enemy). This is one of the more important aspects as often endgames are won by a single tempo.

I already have this one.

3. outside/distant passed pawns and the fact that they become more valuable as material is reduced.

I have this but I don't know if it becomes valuable when material is reduced. Have you tried doing the opposite? I have seen in the source of a strong engine (2900+) that it is doing the opposite.

4. candidate passed pawns because the search might not go deeply enough to see the candidate convert to a real passer.

I have this also.

5. blockade passed pawns, which is a sub-component of step 1 above. If passed pawns can't advance, they can't increase in value. This will also cause your program to attempt to remove an enemy blockade for the same reason.

I can't think of a way to score this. Isn't it better to leave this to the search, or maybe having a static eval is much better?

There are lots of other ideas dealing with pieces. For example, sometimes two isolated passers are much better than two connected passers, because a lone king can't stop two passers on opposite wings while it slow two connected passers down and let its split passers advance. A knight favors the side with pawns on one wing, while a bishop is more effective with pawns on both wings due to its better mobility.. Etc...

I think I don't have this one. These are the kind of things I'm looking for. I will try to implement this.

[Kempelen]

300.000nps seems slow. What's your hardware? Crafty is a speed demon, it is optimized well.

Well, this is actually my bottleneck I am crazy try to know why I dont get more nps. I have optimized a lot my program to make algorithms faster and also speed code in some places, but I think (and experts here can say if I may be correct) that I have some kind of "structural" problem which dont let me raise 300.000nps in my Core 2 Duo. If I change my eval func to only material + pieze_square_tables, I only get around 120.000 nps more, so it appear that eval is not my problem.
Using my own profile, I know that getFirstBit,getLastBit, popCound and is_attacked routines consume around 25/30% of totalTime. That's maybe because my engine is Bitboard based with no rotation or magic, but hard way calculation (as Robert explain in his site). I think that's is where I have my "structural" problem. does this sound plausible, or do you think that I would have to get more nps? Maybe if I want more nps I need to update my bitboard engine to magic, rotated or a more sophisticated method, with if my supposition is true then will do in future version.

My advice on your eval, try to support simple things for now, maybe material and piece square tables only. Then test your engine and watch its games. Add little by little the knowledge depending on which your engine needs as you seen it play. This way, you are really sure that the knowledge you are putting in your engine really helps.

Well, it sounds that I have been adding knowledge here and there as I program, but no. I started to code it around november/dicember of last year (9 month ago), and playing hundreds of games in 2 computers (5 during some months). I have been tunning the eval based on game results. The only problem, maybe, is that I have based each improvement result on 400 to 600 games, which for a "correct" interpretation of the some result would not be enought.

I hope it helps you and other new programmers out there.

yes , it does, thx a lot.

[Kempelen]

If you may have played Twisted Logic or watched its games, you may notice that it almost has no endgame knowledge. The only knowledge it has is concerning passed pawns. I'm planning to add some endgame knowledge to it.

My problem is that I don't know what are the basic endgame knowledge that should be added. I don't want yet to support EGTB and EGBB.

Do I have to follow Fruit's example of having code for specific positions like KPK, KRPK, etc, or there are some general rules specific for and applicable only for the endgame?

For starters I think the critical pieces of endgame information needed are as follows, not in any particular order:

1. passed pawns are more valuable as they advance, and they are more valuable if they are connected to a friendly pawn for support, and they are move valuable if accompanied by their king for additional support.

2. where does the king belong? In the center unless there are passed pawns to escort (friendly) or blockade/stop (enemy). This is one of the more important aspects as often endgames are won by a single tempo.

3. outside/distant passed pawns and the fact that they become more valuable as material is reduced.

4. candidate passed pawns because the search might not go deeply enough to see the candidate convert to a real passer.

5. blockade passed pawns, which is a sub-component of step 1 above. If passed pawns can't advance, they can't increase in value. This will also cause your program to attempt to remove an enemy blockade for the same reason.

There are lots of other ideas dealing with pieces. For example, sometimes two isolated passers are much better than two connected passers, because a lone king can't stop two passers on opposite wings while it slow two connected passers down and let its split passers advance. A knight favors the side with pawns on one wing, while a bishop is more effective with pawns on both wings due to its better mobility.. Etc...

Well, this advices are actually wisdom more that what I wrote.... in fact passed pawns are the soul of endgames.....

[Edsel Apostol]

25% to 30% consumed in the is_attack functions are way too high. The most expensive of my routines is eval_piece_activity, composed mostly of mobility, king attacks and passed pawn eval. My movegen and other attack functions are each less than 5% of execution time.

There really is something wrong with your data structures or the way you did your program. Have you tried inlining your functions such as get_first_bit? There are efficient algorithms for get_first_bit, just look it up in the chess programming wiki. Have you tried using a better compiler like Intel or MSVC?

Try to replace your movegen with that of Olithink, its small and fast.

About the number of games, if you are testing scientifically, you can determine if your engine version is better by just 60 games.