Policy determining quiet early opening preferences of Leela

[MMarco]

Neither engine should get TBs. It's supposed to be a test of their endgame play, not whether they can look up TBs.

It depens on what you mean by endgame play. For me endgame play in much larger than converting TBs positions. It doesn't matter to me if the engine cannot mate with N and B against a bare king, if it plays well before, if it is able to supress opponent counterplay, transform its positional advantage or material advantage to reach an absolutely won position given in the TBs."

The TBs aren't just converting what the engine (possibly) cannot. The TBs are guiding its earlier play, in the TB hits from the engine search.

If you really take your position (I certainly do not) that conversion skills don't matter, you should just run engine matches without TBs and adjudicate when they get down to 5 pieces. (The argument is weak, anyway, because why should we believe that 5-piece endgames are just "conversion" and don't need "ability to suppress opponent counterplay, trainsform its advantage", etc.?)

I was answering you, when you said:

It's supposed to be a test of their endgame play, not whether they can look up TBs"

That sounds to me that you're saying that to test engine endgame skills with TBs on is akin to test whether the engines can correctly look up TBs for conversion. Now you are suggesting me the opposite. I'm not sure what you really want to say.

I'm aware that the TBs are guiding (sometimes... partially...) the engine play, that is exacty why I keep them on. When I use my engine for analysis, the TBs are on. They are part of what make the engine play well in the endgame. Why would I want to test them without? I'm not interested to know how bad a weakened version of Leela or Stockfish can play because it can't access the TBs.

If you really take your position (I certainly do not) that conversion skills don't matter,

They don't matter when the TBs are on. I'm not interested to know whether is an engine can convert a given position without tablebases when I know it can with the TBs on.

...you should just run engine matches without TBs and adjudicate when they get down to 5 pieces.

Apart for conversion use, I keep them on also for their partial guidance in engine's play, but tablebases adjudication is on when I run engine matches. It just saves time. I don't want to have an extra 10 or 20 moves every game to make sure the engine can mate rook-king vs king. I don't want to see that.

Feel free to test or use your engines without TBs.

[jp]

I was answering you, when you said:

It's supposed to be a test of their endgame play, not whether they can look up TBs"

That sounds to me that you're saying that to test engine endgame skills with TBs on is akin to test whether the engines can correctly look up TBs for conversion. Now you are suggesting me the opposite. I'm not sure what you really want to say.

If you use TBs and there are TB hits, then the TBs are helping the engine. That does not mean the engine does nothing at all except look up the TBs, but of course it's a huge help, especially if the engine is weak at the endgames it is looking up. It doesn't have to be 100% help to be too much help.

You speculated "if it is able to supress opponent counterplay, transform its positional advantage or material advantage", but if it could really do those things but just couldn't "convert", then it should not perform any worse if you don't give it TBs and just cut it off when there are 5 pieces. (But calling it "conversion" is misleading, making it sound like it's unimportant, which lets an engine that cannot do it off the hook.)


Would you really test engines' opening play by giving them 10-move opening books? Because your "testing" of engines' endgame play is pretty similar to that. Of course, it sounds like you don't care about the engines' real endgame strength, and you are free not to care, but it shouldn't shock you if testing gives different results from your TB-full results.

[jp]

Do not use TBs and do not use any adjudications, it's important here, in this particular set-up.

[MMarco]

I was answering you, when you said:

It's supposed to be a test of their endgame play, not whether they can look up TBs"

That sounds to me that you're saying that to test engine endgame skills with TBs on is akin to test whether the engines can correctly look up TBs for conversion. Now you are suggesting me the opposite. I'm not sure what you really want to say.

If you use TBs and there are TB hits, then the TBs are helping the engine. That does not mean the engine does nothing at all except look up the TBs, but of course it's a huge help, especially if the engine is weak at the endgames it is looking up. It doesn't have to be 100% help to be too much help.

You speculated "if it is able to supress opponent counterplay, transform its positional advantage or material advantage", but if it could really do those things but just couldn't "convert", then it should not perform any worse if you don't give it TBs and just cut it off when there are 5 pieces. (But calling it "conversion" is misleading, making it sound like it's unimportant, which lets an engine that cannot do it off the hook.)


Would you really test engines' opening play by giving them 10-move opening books? Because your "testing" of engines' endgame play is pretty similar to that. Of course, it sounds like you don't care about the engines' real endgame strength, and you are free not to care, but it shouldn't shock you if testing gives different results from your TB-full results.

If the engine plays fine with TBs on, it is not a concern for me what happens if TBs are off, like it or not.

Besides, apart from critisizing others test, do you sometimes post your own over here?

If you want to prove something about "real engine strenght" without TBs, just run your own test.

[jp]

Besides, apart from critisizing others test, do you sometimes post your own over here?

If you want to prove something about "real engine strenght" without TBs, just run your own test.

It was you who posted responding to Kai's tests. Look at your first post on page 2. You sounded confused about the reason why he got the results he got. It's pretty clear what he's been trying to test. Since you're so agitated by my posts that simply state the obvious, why not just read his reply to you.

I don't want to prove anything. I am fine just reading Kai's testing results. Of course, anyone else's sensible testing results are also welcome.