what is the best materialistic engine and what is the rating of it?

[Uri Blass]

I define materialistic engine as engine that always prefer more material relative to less material.
It can have a positional evaluation but if we use 1,3,3,5,9 for pieces the positional evaluation has to be less than 0.5 pawn so I can know based on the evaluation what is the material that it see in the search.

Note that I asked AI about it and the AI suggested to use stockfish and increse materialweight parameter but I do not know which version to use.

Maybe somebody can modify stockfish to use only material evaluation plus 1% of the positional nnue evaluation that it has snd test it when poaitional nnue evaluation is simply the difference between nnue evaluation and only material evaluation and I assume this difference is less than 50 pawns.

reason I would like only material evaluation is to analyze if a mistake is a tactical blunder or a positional blunder and today I do not know if big change in the evaluation is because a move lose material or because of a different reason.

[Eelco de Groot]

Uri, it does not make much sense to me what you are trying to do or rather the way you are proposing. Every chessplayer that also uses a chessprogram has faced the same question since chessprograms or computers were more than the toys that Anatoli Karpov thought of them back in those early days when GMs still played them. How do the chessplayers do it then if not your way, you should ask them, there are enough still here but it is maybe the slow forum stopping them. I should not criticize too much, I did learn in the Shredder GUI how to visualize the PV and then look at the little chessboard at the end of the PV, but in maybe 20 years I did no more than that and occasionally played out all those moves, by hand in the GUI so I could get to that position at the end of the PV. Good practice for a bit of memorization and visualization in my brain but not very efficien and often prone to mistakes. Then I learned I could insert the whole PV, in one click in the pgn. It was miracle! All automated. But I never knew that before Thank Caissa I am not much of a chessplayer so I never had to analyze a game well not since a long time I played.

But hat is how I would do it. Even Chessbase can do it, ha just kidding, use any modern GUI but I know Shredder can do it. It will show you the material difference at the end of any PV. What has changed will be the material invested by best play. Then take a less material program, like Rebel EAS or CSTal, whichever you prefer and do the same. Insert the PV it shows you and Shredder will show you the material invested by White or Black. Then you can compare the two values but it is not really all that important anymore. If CSTal shows an evaluation 25 % higher than SF then you can go to SF and subtract 25 % of the SF number and that will be the evaluation of a bean counter compared to SF. But it will be pretty meaningless to have this number; if you really need it it teaches you not much; not why the program played not materially.

[Uri Blass]

looking at the end of the pv does not help because it is possible that the pv win a pawn for white but it is a positional problem because black can avoid losing material but get worse position based on the evaluation of the engine.

[Eelco de Groot]

looking at the end of the pv does not help because it is possible that the pv win a pawn for white but it is a positional problem because black can avoid losing material but get worse position based on the evaluation of the engine.

Sure, if I understand a bit what you are saying here Uri, the PV will mainly show you where the program thought the game would go, depending on the time it had to think it (the PV) may be of little value. The opponent program will have had different ideas about where the game will go if at least it was a different program. But if there was a material loss or gain that had no compensation the PV will show that. The materialistic engine will also show that but it has no clue whatsoever about any compensation, by definition. So you can not learn about positional compensation from a materialistic engine. If the positional compensation is what you are after, you are far better off with the non materialistic engine to compare, even if it shows an enlarged value for it. In my opinion

Positional compensation is not an absolute. It depends on the strength of the engine and of the opponent engine both. Euwe had in the past these books, I forget the titles but it was something like 'Master against amateur', 'Master against strong clubplayer', etc. Each book would teach you different things. The positional rules are rules of thumb, only. They will not work in all cases. They are relative.


Totally off topic rambles about AI:

By the same token, AIs with "superhuman intelligence" are a bit of a nonsense. Because knowledge is, in most cases, relative. The expert system will use always different rules from even the greatest human expert, in for instance medicine. Because it has different sensors to use, for instance. The best system will always be if it complements the human expert that it works with and vice versa the human that best knows the limitations of both the expert system and his own. 'Gnothi seauton' (γνῶθι σεαυτόν) as the Greeks taught us.

https://en.wikipedia.org/wiki/Know_thyself

The Centaur will always be better than the expert system on its own and the human will/should for many reasons still have the end responsibility. If only for reasons of trust like in a doctor-patient relationship, or passengers in a plane. At least that is where part of the regulation should go IMO, but it will also be more expensive than jet planes flying fully automatic. Well there are people thinking differently, if it is up to the Ministry of War in the USA, soldiers do not need brains at all, they just need lots of testosterone. Maybe that is a bit of a projection. In the case of the American government, the stupider the better because it confuses your opponents. Maybe I'm digressing too much now. Ha.

[Uri Blass]

looking at the end of the pv does not help because it is possible that the pv win a pawn for white but it is a positional problem because black can avoid losing material but get worse position based on the evaluation of the engine.

Sure, if I understand a bit what you are saying here Uri, the PV will mainly show you where the program thought the game would go, depending on the time it had to think it (the PV) may be of little value. The opponent program will have had different ideas about where the game will go if at least it was a different program. But if there was a material loss or gain that had no compensation the PV will show that. The materialistic engine will also show that but it has no clue whatsoever about any compensation, by definition. So you can not learn about positional compensation from a materialistic engine. If the positional compensation is what you are after, you are far better off with the non materialistic engine to compare, even if it shows an enlarged value for it. In my opinion

Positional compensation is not an absolute. It depends on the strength of the engine and of the opponent engine both. Euwe had in the past these books, I forget the titles but it was something like 'Master against amateur', 'Master against strong clubplayer', etc. Each book would teach you different things. The positional rules are rules of thumb, only. They will not work in all cases. They are relative.


Totally off topic rambles about AI:

By the same token, AIs with "superhuman intelligence" are a bit of a nonsense. Because knowledge is, in most cases, relative. The expert system will use always different rules from even the greatest human expert, in for instance medicine. Because it has different sensors to use, for instance. The best system will always be if it complements the human expert that it works with and vice versa the human that best knows the limitations of both the expert system and his own. 'Gnothi seauton' (γνῶθι σεαυτόν) as the Greeks taught us.

https://en.wikipedia.org/wiki/Know_thyself

The Centaur will always be better than the expert system on its own and the human will/should for many reasons still have the end responsibility. If only for reasons of trust like in a doctor-patient relationship, or passengers in a plane. At least that is where part of the regulation should go IMO, but it will also be more expensive than jet planes flying fully automatic. Well there are people thinking differently, if it is up to the Ministry of War in the USA, soldiers do not need brains at all, they just need lots of testosterone. Maybe that is a bit of a projection. In the case of the American government, the stupider the better because it confuses your opponents. Maybe I'm digressing too much now. Ha.

The target of a materialistic engine is not to learn about positional compensation and I do not plan to use it to define my mistakes.
The idea is that after I defined my mistakes by a good engine I want to divide them to tactical mistakes and positional mistakes.

Tactical mistake is a move that is a mistake because it lose material relative to a better move(example it can win a pawn when the opponent have draw chances when the better move win a rook and easily the game).

lose material relative to other moves is not always really losing material but to simplify my post I am going to use later in the post simply lose material.

if my move lose material against the materialistic engine then I guess usually my move is also a tactical mistake(I say usually because I need to check that the moves of the materialistic engine are correct and I can imagine cases that the materialistic engine win material by bad moves and the way to punish my mistake is not the way the materialistic engine suggest).

For the minority of cases when the materialistic engine punish my move by a bad move,
I need to look for second best move by the materialistic engine maybe it also win material so the move is still a tactical mistake and maybe not.

[petero2]

I define materialistic engine as engine that always prefer more material relative to less material.
It can have a positional evaluation but if we use 1,3,3,5,9 for pieces the positional evaluation has to be less than 0.5 pawn so I can know based on the evaluation what is the material that it see in the search.

I implemented this in Texel 1.13a7. To use it you have to set the UCI option Materialistic to true. It plays around 500 Elo weaker than the default evaluation function at time control 6 seconds per game + 0.06 seconds per move. It still beats crafty 25.2 by around 200 Elo at the same time control though, so it might be good enough for analyzing human mistakes anyway.

The evaluation score is transformed to:

mtrl_score = M * 100 + compress((score - mean(M)) / stdDev(M))

where:

- M is the material score using 1,3,3,5,9 piece values.
- mean(M) is the average evalation score for material balance M, measured on an NNUE validation data set.
- stdDev(M) is the standard deviation of the evaluation score for material balance M.
- compress(x) is a function that maps [-inf, inf] to [-49, 49] in a way that tries to preserve as much information as possible.

I guess the main reasons for the Elo drop are:

* The engine is forced to accept all piece sacrifices and gambits, unless it can search deep enough to see it cannot keep the captured material.
* The search parameters are no longer tuned to the evaluation, since the evaluation scale changes.
* The granularity of the positional part of the score is not great. Changing the score internally to mtrl_score = M * 400 + corr, -199 <= corr <= +199, would probably help.

Code

[Eelco de Groot]

Thanks Peter!

[Uri Blass]

I define materialistic engine as engine that always prefer more material relative to less material.
It can have a positional evaluation but if we use 1,3,3,5,9 for pieces the positional evaluation has to be less than 0.5 pawn so I can know based on the evaluation what is the material that it see in the search.

I implemented this in Texel 1.13a7. To use it you have to set the UCI option Materialistic to true. It plays around 500 Elo weaker than the default evaluation function at time control 6 seconds per game + 0.06 seconds per move. It still beats crafty 25.2 by around 200 Elo at the same time control though, so it might be good enough for analyzing human mistakes anyway.

The evaluation score is transformed to:

mtrl_score = M * 100 + compress((score - mean(M)) / stdDev(M))

where:

- M is the material score using 1,3,3,5,9 piece values.
- mean(M) is the average evalation score for material balance M, measured on an NNUE validation data set.
- stdDev(M) is the standard deviation of the evaluation score for material balance M.
- compress(x) is a function that maps [-inf, inf] to [-49, 49] in a way that tries to preserve as much information as possible.

I guess the main reasons for the Elo drop are:

* The engine is forced to accept all piece sacrifices and gambits, unless it can search deep enough to see it cannot keep the captured material.
* The search parameters are no longer tuned to the evaluation, since the evaluation scale changes.
* The granularity of the positional part of the score is not great. Changing the score internally to mtrl_score = M * 400 + corr, -199 <= corr <= +199, would probably help.

Code

Thanks
I understand that default Texel is around 700 elo stronger than Crafty25.2

I see in the CCRL 40/15 smaller difference but maybe the difference is higher at the fast time control that you used and also Texel1.13 is stronger than 1.12

Texel 1.12 64-bit 4CPU 3532 (not sure how much stronger is texel1.13 relative to 1.12)
241 Crafty 25.2 64-bit 4CPU 3040

[petero2]

Thanks
I understand that default Texel is around 700 elo stronger than Crafty25.2

I see in the CCRL 40/15 smaller difference but maybe the difference is higher at the fast time control that you used and also Texel1.13 is stronger than 1.12

Texel 1.12 64-bit 4CPU 3532 (not sure how much stronger is texel1.13 relative to 1.12)
241 Crafty 25.2 64-bit 4CPU 3040

Texel 1.13a7 is roughly the same strength as 1.12, so I think the larger rating difference is caused by the very short time control. For CCRL 40/15 1 core, I would guess the materialistic Texel has rating:

2952+(3484-2952)*200/700 = 3100

[Uri Blass]

I dd some analysis with it.

I think that one problem is with the value of the pieces 1,3,3,5 for pawn knight bishop and rook.

The value of 2 minor pieces is closer to 7 pawn relative to 6 pawns and I can consider losing bishop and knight for rook and pawn as a tactical mistake.

My guess is that the new materialistic texel should be relatively weaker at longer time control because of weaknesses in evaluation of material but it is only a guess and I did not do testing.