Help me to test an idea for Stockfish (I don't know programm

[Lyudmil Tsvetkov]

But I never meant disabling the king psqt, as well as pawn psqt, probably I was not specific, my intention was to disable only NBRQ tables in the endgame. Now I think that might have been the main reason for Ronald's results. Did you disable all tables, Ronald? King and pawn tables are of course useful for the endgame.

According to the link Ralph gave above, for SF there are two PSQ tables that are very significant for playing strength: Knight and King tables (see bottom of that page). So I would assume that setting Knight values to zero for the endgame would hurt strength.

Pawn and Rook PSQ tables are almost switched off for the endgame already in SF (at least according to that link above), both have constant values for all squares (Pawn = -8, Rook = +3) which could as well be replaced by modifying the endgame material value by that score and setting the endgame PSQ value to 0. Doing so would also be the better choice for other PSQ tables when experimenting with "switching them off": instead of only setting a table to zero, the material value should be adjusted accordingly (e.g. by adding some intelligently weighted average of the PSQ values) as well so that "switching off" only means not to get different values per square but keeps the overall scoring balanced. This problem would disappear if all PSQ values were in fact already averaged around zero and the difference were already part of the material score (but that is another issue).

Thanks Sven.
Competent as always.

[Lyudmil Tsvetkov]

My suggestion was not that this will work automatically for Stockfish, but that it is an idea worth trying.

This is what you wrote:

My guess is that engines will not suffer without endgame psqt

Very clearly SF does suffer.

You know very well that between a crude idea and implementation there is a looong way to go. You should adjust some things, draw your conclusions, etc.

Sure, so after setting all values to 0 as per your suggestion we should now adjust each of the values until SF is back at its old level, right?

I suggest instead to just stick to the tried and tested endgame values. Maybe it can help to tweak some of them, but setting them all to 0 and expect that a well-tuned engine such as SF would not suffer is... naive...

Words have literal and deeper meaning within a context.

You should not adjust anything, but you had only to rerun your test with king psqt of all tables switched on. The results would be very different. Besides, the main idea we wanted to test is which of the 2 tables, the middlegame or the endgame, has bigger impact on strength. I would say the middlegame is much more important.

Rerun the test with king psqt switched on, find a way to avoid disrupting interpolation, and report again how important endgame psqt are.

[syzygy]

My suggestion was not that this will work automatically for Stockfish, but that it is an idea worth trying.

This is what you wrote:

My guess is that engines will not suffer without endgame psqt

Very clearly SF does suffer.

You know very well that between a crude idea and implementation there is a looong way to go. You should adjust some things, draw your conclusions, etc.

Sure, so after setting all values to 0 as per your suggestion we should now adjust each of the values until SF is back at its old level, right?

I suggest instead to just stick to the tried and tested endgame values. Maybe it can help to tweak some of them, but setting them all to 0 and expect that a well-tuned engine such as SF would not suffer is... naive...

Words have literal and deeper meaning within a context.

You should not adjust anything, but you had only to rerun your test with king psqt of all tables switched on.

Aha....
Look, you are simply redefining your requests with every new post. When you wrote the one I replied to, you had not yet thought for a moment about the king psqt. And now you are suddenly pretending that this is what you meant all along.

The results would be very different. Besides, the main idea we wanted to test is which of the 2 tables, the middlegame or the endgame, has bigger impact on strength. I would say the middlegame is much more important.

Here you are again changing your position. Nobody in this thread ever contested that the middlegame psqt values are more important. Your point from the beginning was that the endgame psqt should be removed.

Just for fun I'll quote from an earlier post of yours:

Maybe Luydmil means that the endgame values of the piece square tables should all be set to 0. That would be easy to test. But I highly doubt that this would be better than the (I assume) well-tuned values that SF is using now. One thing is certain: that SF plays the endgame well is in no way evidence that setting the endgame values to 0 would have any benefit.

That was exactly what I was referring to, Ronald. I think psqt in endgames is fully useless in top engines (but only in top engines, as they have all other things that would allow them play endings without psqt), except probably some bonus for minors on the 5th and 6th ranks.

This is pretty precise, but apparently not at all what you meant and not at all how it should have been understood. Duh. How about carefully wording your posts next time? Then you don't have to repeat the nonsensical blabbing about "literal deeper meaning in the context".

[michiguel]

My suggestion was not that this will work automatically for Stockfish, but that it is an idea worth trying.

This is what you wrote:

My guess is that engines will not suffer without endgame psqt

Very clearly SF does suffer.

You know very well that between a crude idea and implementation there is a looong way to go. You should adjust some things, draw your conclusions, etc.

Sure, so after setting all values to 0 as per your suggestion we should now adjust each of the values until SF is back at its old level, right?

I suggest instead to just stick to the tried and tested endgame values. Maybe it can help to tweak some of them, but setting them all to 0 and expect that a well-tuned engine such as SF would not suffer is... naive...

Words have literal and deeper meaning within a context.

You should not adjust anything, but you had only to rerun your test with king psqt of all tables switched on.

Aha....
Look, you are simply redefining your requests with every new post. When you wrote the one I replied to, you had not yet thought for a moment about the king psqt. And now you are suddenly pretending that this is what you meant all along.

The results would be very different. Besides, the main idea we wanted to test is which of the 2 tables, the middlegame or the endgame, has bigger impact on strength. I would say the middlegame is much more important.

Here you are again changing your position. Nobody in this thread ever contested that the middlegame psqt values are more important. Your point from the beginning was that the endgame psqt should be removed.

Both are equally important by definition. When two points determine a straight line, each of the points are critical. What is important is setting the straight line, and middlegame and endgame (terrible names) PSTs contain 64 pairs of points that determine 64 straight lines for each square.

The problem is that Lyudmil does not understand how this work. In fact, to translate his intuitive concept better to a real experiment, the endgame PSTs should not be set to zero, but set to _the_same_values_ contained in the middle game PSTs (i.e. copy the midPSTs onto the endPSTs). In other words, let the middle game values dominate the whole game. However, this has a huge number of other colateral problems. The material values will have to be adjusted, for instance.

Miguel

[Lyudmil Tsvetkov]

My suggestion was not that this will work automatically for Stockfish, but that it is an idea worth trying.

This is what you wrote:

My guess is that engines will not suffer without endgame psqt

Very clearly SF does suffer.

You know very well that between a crude idea and implementation there is a looong way to go. You should adjust some things, draw your conclusions, etc.

Sure, so after setting all values to 0 as per your suggestion we should now adjust each of the values until SF is back at its old level, right?

I suggest instead to just stick to the tried and tested endgame values. Maybe it can help to tweak some of them, but setting them all to 0 and expect that a well-tuned engine such as SF would not suffer is... naive...

Words have literal and deeper meaning within a context.

You should not adjust anything, but you had only to rerun your test with king psqt of all tables switched on.

Aha....
Look, you are simply redefining your requests with every new post. When you wrote the one I replied to, you had not yet thought for a moment about the king psqt. And now you are suddenly pretending that this is what you meant all along.

The results would be very different. Besides, the main idea we wanted to test is which of the 2 tables, the middlegame or the endgame, has bigger impact on strength. I would say the middlegame is much more important.

Here you are again changing your position. Nobody in this thread ever contested that the middlegame psqt values are more important. Your point from the beginning was that the endgame psqt should be removed.

Just for fun I'll quote from an earlier post of yours:

Maybe Luydmil means that the endgame values of the piece square tables should all be set to 0. That would be easy to test. But I highly doubt that this would be better than the (I assume) well-tuned values that SF is using now. One thing is certain: that SF plays the endgame well is in no way evidence that setting the endgame values to 0 would have any benefit.

That was exactly what I was referring to, Ronald. I think psqt in endgames is fully useless in top engines (but only in top engines, as they have all other things that would allow them play endings without psqt), except probably some bonus for minors on the 5th and 6th ranks.

This is pretty precise, but apparently not at all what you meant and not at all how it should have been understood. Duh. How about carefully wording your posts next time? Then you don't have to repeat the nonsensical blabbing about "literal deeper meaning in the context".

So, if there is an agreement about how much more useful middlegame psqt tables are than endgame psqt tables, why did not you point that out at the very beginning. Are you aware of some tests, precise data, or just speculating. I must say that I have never read till now what is the performance difference between middlegame and endgame psqt.

I guess you feel uncomfortable about king psqt, because it is this table that influences endgame most, as well as your provisional results. I guess you might even have some tests ready with only king endgame psqt enabled, but are reluctant to share them. Btw., if you were so sure of the outcome, why did you run the test?

Endgame psqt in endgames with 6 pieces or less is really a waste of resources, for the simple reason that with such a low number of pieces the interaction of the pieces themselves becomes much more important than some general good placements. You need some more endgame-specific knowledge there, rather than general positioning concepts. Thus, as based on chess knowledge, I would say that, apart from king and pawn psqt, other tables are pretty much irrelevant in simpler endgames. If you follow chess knowledge, you should either leave aside the other 4 endgame tables, or simplify them.

Now, the real problem, as Miguel says, is interpolation. But maybe a work-around could be found, for example interpolate only until a certain mimimum material is left, and then stop using the other 4 tables? Why not, if chess knowledge would suggest this is the better way to do it?

Now, I would not focus on your meticulous analysis of my wording, and even less on the judgements you make of me, people understand according to their intellectual level, I can do nothing about that. Some people read between the lines, others split hairs, etc., I do not mind this. What I am concerned is that you lack benevolence, and that is a quality I would appreciate above many others.

[Lyudmil Tsvetkov]

My suggestion was not that this will work automatically for Stockfish, but that it is an idea worth trying.

This is what you wrote:

My guess is that engines will not suffer without endgame psqt

Very clearly SF does suffer.

You know very well that between a crude idea and implementation there is a looong way to go. You should adjust some things, draw your conclusions, etc.

Sure, so after setting all values to 0 as per your suggestion we should now adjust each of the values until SF is back at its old level, right?

I suggest instead to just stick to the tried and tested endgame values. Maybe it can help to tweak some of them, but setting them all to 0 and expect that a well-tuned engine such as SF would not suffer is... naive...

Words have literal and deeper meaning within a context.

You should not adjust anything, but you had only to rerun your test with king psqt of all tables switched on.

Aha....
Look, you are simply redefining your requests with every new post. When you wrote the one I replied to, you had not yet thought for a moment about the king psqt. And now you are suddenly pretending that this is what you meant all along.

The results would be very different. Besides, the main idea we wanted to test is which of the 2 tables, the middlegame or the endgame, has bigger impact on strength. I would say the middlegame is much more important.

Here you are again changing your position. Nobody in this thread ever contested that the middlegame psqt values are more important. Your point from the beginning was that the endgame psqt should be removed.

Both are equally important by definition. When two points determine a straight line, each of the points are critical. What is important is setting the straight line, and middlegame and endgame (terrible names) PSTs contain 64 pairs of points that determine 64 straight lines for each square.

The problem is that Lyudmil does not understand how this work. In fact, to translate his intuitive concept better to a real experiment, the endgame PSTs should not be set to zero, but set to _the_same_values_ contained in the middle game PSTs (i.e. copy the midPSTs onto the endPSTs). In other words, let the middle game values dominate the whole game. However, this has a huge number of other colateral problems. The material values will have to be adjusted, for instance.

Miguel

I think the main problem with his tests was that the king and pawn psqts were switched off. Imagine a standard ending of 4 pieces: what do you need the other 4 tables for, when the position will be so concrete that piece interactions and general endgame knowledge will be much more important than a square that is better but only in an abstract way?

The problem remains how to interpolate, but maybe you might stop interpolating when material decreases below a certain point.

Of course, this is just useless speculation. One thing I would appreciate is some data of the relative importance of middlegame and endgame psqt, but, unfortunately, no one has such data to enlighten us.

[syzygy]

So, if there is an agreement about how much more useful middlegame psqt tables are than endgame psqt tables, why did not you point that out at the very beginning. Are you aware of some tests, precise data, or just speculating. I must say that I have never read till now what is the performance difference between middlegame and endgame psqt.

It is incorrect to speak about performance difference of the two sets of values, because they are really one set of lines, as Miguel explained. What I meant is that I did not dispute that setting all midgame values to 0 probably hurts more than setting all endgame values to 0. On second thoughts, this might not even be right so I will now conveniently withdraw this statement.

Far more important however is that it hardly makes sense to speak about the contribution of one versus the other. They are just one set of lines. So what if setting one half of the values to zero has more impact than setting the other half of the values to zero?

I guess you feel uncomfortable about king psqt, because it is this table that influences endgame most, as well as your provisional results. I guess you might even have some tests ready with only king endgame psqt enabled, but are reluctant to share them.

No I just get annoyed with people that keep twisting and turning and show to be immune against reasoned thought. You come up with your own psq tables that turn out to give a massive loss of Elo and happily shout out "they are optimal!". It is the Black Knight's logic... Just a flesh wound!

(Heh, the new twist is "not even 100 Elo lower, so there must be something good in it!")

Btw., if you were so sure of the outcome, why did you run the test?

It is not easy to reason with him (try to find an argument against "don't you see, it is obvious"), but this time the idea was simple to test and refute.

[Lyudmil Tsvetkov]

So, if there is an agreement about how much more useful middlegame psqt tables are than endgame psqt tables, why did not you point that out at the very beginning. Are you aware of some tests, precise data, or just speculating. I must say that I have never read till now what is the performance difference between middlegame and endgame psqt.

It is incorrect to speak about performance difference of the two sets of values, because they are really one set of lines, as Miguel explained. What I meant is that I did not dispute that setting all midgame values to 0 probably hurts more than setting all endgame values to 0. On second thoughts, this might not even be right so I will now conveniently withdraw this statement.

Far more important however is that it hardly makes sense to speak about the contribution of one versus the other. They are just one set of lines. So what if setting one half of the values to zero has more impact than setting the other half of the values to zero?

I guess you feel uncomfortable about king psqt, because it is this table that influences endgame most, as well as your provisional results. I guess you might even have some tests ready with only king endgame psqt enabled, but are reluctant to share them.

No I just get annoyed with people that keep twisting and turning and show to be immune against reasoned thought. You come up with your own psq tables that turn out to give a massive loss of Elo and happily shout out "they are optimal!". It is the Black Knight's logic... Just a flesh wound!

(Heh, the new twist is "not even 100 Elo lower, so there must be something good in it!")

Btw., if you were so sure of the outcome, why did you run the test?

It is not easy to reason with him (try to find an argument against "don't you see, it is obvious"), but this time the idea was simple to test and refute.

On the one hand you say that those are one set of lines, on the other that setting the middlegame values to zero probably hurts more. What is the true statement?

If, when changing mobility and a couple of other very important terms in your engine, or when changing 5 of the 6 psqts of your engine beyond recognition, you get a result for the modified version that is less than 100 elo weaker than the original, I would acknowledge you are right. Otherwise, I would say that the modified psqt are indeed promising.

[syzygy]

On the one hand you say that those are one set of lines, on the other that setting the middlegame values to zero probably hurts more. What is the true statement?

You suggest there is a logical contradiction, but there is none. Try again.

If, when changing mobility and a couple of other very important terms in your engine, or when changing 5 of the 6 psqts of your engine beyond recognition, you get a result for the modified version that is less than 100 elo weaker than the original, I would acknowledge you are right. Otherwise, I would say that the modified psqt are indeed promising.

What were the contributions of SF's psq tables again. Removing them leads to the following Elo losses:

Code: Select all

PST ELO GAMES & SCORE
Pawn: -1 [2029 - 2047 - 5924 [0.499] 10000]
Knight: -20 [1875 - 2445 - 5680 [0.471] 10000]
Bishop: -4 [1945 - 2051 - 6004 [0.495] 10000]
Rook: -2 [1934 - 1981 - 6085 [0.498] 10000]
Queen: -2 [1899 - 1950 - 6151 [0.497] 10000]
King: -32

So together they are worth 61 Elo.
You modified them by setting them to "optimal"(!!!) values and lost almost 100 Elo.
Good job.

[Lyudmil Tsvetkov]

On the one hand you say that those are one set of lines, on the other that setting the middlegame values to zero probably hurts more. What is the true statement?

You suggest there is a logical contradiction, but there is none. Try again.

If, when changing mobility and a couple of other very important terms in your engine, or when changing 5 of the 6 psqts of your engine beyond recognition, you get a result for the modified version that is less than 100 elo weaker than the original, I would acknowledge you are right. Otherwise, I would say that the modified psqt are indeed promising.

What were the contributions of SF's psq tables again. Removing them leads to the following Elo losses:

Code: Select all

PST ELO GAMES & SCORE
Pawn: -1 [2029 - 2047 - 5924 [0.499] 10000]
Knight: -20 [1875 - 2445 - 5680 [0.471] 10000]
Bishop: -4 [1945 - 2051 - 6004 [0.495] 10000]
Rook: -2 [1934 - 1981 - 6085 [0.498] 10000]
Queen: -2 [1899 - 1950 - 6151 [0.497] 10000]
King: -32

So together they are worth 61 Elo.
You modified them by setting them to "optimal"(!!!) values and lost almost 100 Elo.
Good job.

You can't take such a simplistic view, really. Those were tables that were more or less tuned to the whole, while mine is out of tune with almost everything else. Besides, if you had taken an even superficial look at Stockfish tables, you would see that they largely exclude space advantage, so they are very quiet tables, less likely to clash with other existing terms, while mine is very agressive on the other hand, with big clash possibilities unless well tuned.

Other thing is you still do not know what the precise elo difference is, Arjun gave 96 elo for a very short TC, and 65 elo for a bit longer TC.

But that leads us nothing, a discussion is worth it when there is a point to investigate. In the present case I would be interested what elo optimally written tables for different pieces could add. If you take the Stockfish example, would you think that king and knight tables are worth much more because of their specificities than other tables, or maybe the 2 tables with biggest elo impact are written significantly better than other tables? I think it is a mix of the 2, and I can never believe that pawn tables are worth just 1 elo, or that bishop tables are worth 5 times less than knight tables. A bishop is a more mobile piece and it makes sense that its psqt is less important, but 5 times less important than the knight table? Same with queen and rook tables, they give equal elo boost, however, taking into account how different both pieces are, it is difficult to believe the impact would be absolutely the same.

Anyone having some additional stats on how different tables perform for their engine? Are there comparable to Stockfish numbers, or there are specificities?