TalkChess.com

Ovyron wrote: ↑Wed Mar 18, 2020 4:33 am

Zenmastur wrote: ↑Wed Mar 18, 2020 4:23 am If you don't understand the post, why don't you just say so and I'll explain any part you have a problem with.

I don't understand.

Can you explain why are you modifying an engine's source code so that it finds a mate faster in a 5-men position instead of just installing 5-men TBs so it finds the mate instantly?

What's the goal, and what's the problem?

This has NOTHING to do with 5-man TB positions. It just happens that a test position has 5 men in it. This isn't a requirement it's incidental. It doesn't even have to be an endgame position. It just happens that the position I used to "prove" this is a problem has 5 men in it. To analyze the position TB's SHOULD NOT BE USED for this very reason. The concept doesn't rely on the position being an endgame. If I would have had a non-endgame example position I would have certainly used it.

The problem is anytime there is a position (endgame or non-endgame) where the move that radically changes the evaluation of the position is close to 50-move limit the engine is VERY hard pressed to find the "correct" solution. When the critical move is close to the 50-move limit it will likely first be discovered by the search well beyond the 50-move limit. When this happens and the 50-move rule is strictly enforced the engine evaluates the result as 0.00 and it will be discarded if the current evaluation is better than 0.00. When it discards the line of play through an A/B cut off it throws away a "HUGE" indicator that a solution is near. If you turn the 50-move-rule enforcement off this problem goes away. The program will find the "first" solution outside the 50-move limit and then VERY quickly finds a near optimal solution where the critical move happens inside the 50-move limit.

This test case "PROVES" there is a problem and that it can be remedied.

I think there is a solution that would allow the engine to find solutions to similar problems during tournament play without disabling the 50-move-rule.

In the meantime I think a UCI option to disable 50-move-rule checking is warranted.


Regards,

Zenmastur

Yes, we are first testing. To test something, we need to know the correct answer. We then ramp up. We have not even tested the original position yet because even with the hack it'll be a challenge for engines. It's possible that SF still won't be able to solve it (but my prediction is it'll find it at depths >=180, and it can reach those depths on fast machine because there are so few pieces).

We can test non-TB positions by looking for some good endgame studies, but if the engine is struggling already with 5 pieces, it'll struggle even more with more pieces.

Zenmastur wrote: ↑Wed Mar 18, 2020 12:23 am Those moves came from Nalimov TB's.

Did you purchase these or can they be downloaded from somewhere (in what format?)?

jp wrote: ↑Wed Mar 18, 2020 7:47 am Yes, we are first testing. To test something, we need to know the correct answer. We then ramp up. We have not even tested the original position yet because even with the hack it'll be a challenge for engines. It's possible that SF still won't be able to solve it (but my prediction is it'll find it at depths >=180, and it can reach those depths on fast machine because there are so few pieces).

We can test non-TB positions by looking for some good endgame studies, but if the engine is struggling already with 5 pieces, it'll struggle even more with more pieces.

Zenmastur wrote: ↑Wed Mar 18, 2020 12:23 am Those moves came from Nalimov TB's.

Did you purchase these or can they be downloaded from somewhere (in what format?)?

I have a copy of 6-man from years ago. The 6-man Nalimov are 1.2TB. I doubt you can download them any more.

You don't have to test the full length mate!

That was the whole point of the last two PGNs in my post. The half move clocks are set so you can test a shorter version. It demonstrates the exact same problem in the same way without the HUGE time expenditure of the full length test. Quote that post and then copy the PGN's from it. Then you can run the tests in a few minutes.

Zenmastur wrote: ↑Wed Mar 18, 2020 9:08 am You don't have to test the full length mate!

That was the whole point of the last two PGNs in my post. The half move clocks are set so you can test a shorter version.

Yes, MonteCarlo suggested it in an earlier post on p.2, and my post at the top of p.3 showed a test of it for DTZ 23.

But that does not mean that this has to be the only problem for engines with the original position. SF might still fail with that position. (We are not asking it to mate, just to take the piece. After that, mate is of course very easy.) Succeeding with a short version doesn't mean it must succeed with the full version. e.g. the normal pruning might cut away the solution with a probability increasing with PV length.

Zenmastur wrote: ↑Wed Mar 18, 2020 9:08 am without the HUGE time expenditure of the full length test

I don't think it'll necessarily take a huge amount of time. I'm confident fast machines can get to depth 100 in 10 minutes (because it's only 5 pieces). How much more for depth 120 or 150 or whatever, I'm not sure, but possibly manageable.

jp wrote: ↑Wed Mar 18, 2020 9:50 am

Zenmastur wrote: ↑Wed Mar 18, 2020 9:08 am You don't have to test the full length mate!

That was the whole point of the last two PGNs in my post. The half move clocks are set so you can test a shorter version.

Yes, MonteCarlo suggested it in an earlier post on p.2, and my post at the top of p.3 showed a test of it for DTZ 23.

But that does not mean that this has to be the only problem for engines with the original position. SF might still fail with that position. (We are not asking it to mate, just to take the piece. After that, mate is of course very easy.) Succeeding with a short version doesn't mean it must succeed with the full version. e.g. the normal pruning might cut away the solution with a probability increasing with PV length.

Zenmastur wrote: ↑Wed Mar 18, 2020 9:08 am without the HUGE time expenditure of the full length test

I don't think it'll necessarily take a huge amount of time. I'm confident fast machines can get to depth 100 in 10 minutes (because it's only 5 pieces). How much more for depth 120 or 150 or whatever, I'm not sure, but possibly manageable.

There is no point to a longer test. The problem and it's solution are adequately displayed by the two shorter tests.

It's completely irrelevant if it can or can't find the long mate. That's a function on CPU speed, # of cores, TT size, and time allowed for the mate etc. It has nothing to tell us about the 50-move rule that's not demonstrated by the shorter tests.

Your just making it more complicated than it needs to be.

Zenmastur wrote: ↑Wed Mar 18, 2020 11:01 am It's completely irrelevant if it can or can't find the long mate. That's a function on CPU speed, # of cores, TT size, and time allowed for the mate etc. It has nothing to tell us about the 50-move rule that's not demonstrated by the shorter tests.

You may care only about the 50-move rule, but some of us are interested in what engines can or cannot solve, not just whether the coding of the 50-move rule makes it harder for them to do so.

We weren't originally trying to study the effect of the 50-move rule coding on engines. Whether they can find long conversions is entirely the point.

This is not about hardware either. (Faster hardware of course does let us test more things.) We are judging engines on depth, number of nodes, etc., so it's the software that matters.

Zenmastur wrote: ↑Wed Mar 18, 2020 5:35 am The problem is anytime there is a position (endgame or non-endgame) where the move that radically changes the evaluation of the position is close to 50-move limit the engine is VERY hard pressed to find the "correct" solution.

What if using TBs, as low as 5men-TB, made the problem disappear? Then all the testing without those TBs would be meaningless because any sane person would at least have that.

You have proved NOTHING because you insist on making tests on irrelevant positions (in this case one that has been fully solved), to show that a problem exists you have to come out of the laboratory and test a relevant position (one where the engine plays a drawing move in a winning position) allowing the engine to use TBs (as anybody wanting to find the win ASAP would do) and show that turning off 50-move draw detection helps, because I have yet to see such a case.

Ovyron wrote: ↑Wed Mar 18, 2020 7:30 pm What if using TBs, as low as 5men-TB, made the problem disappear? Then all the testing without those TBs would be meaningless because any sane person would at least have that.

If you have more than 7 pieces, there'll be positions where the problem is much worse. 5 men makes it much easier for engines, not harder.

jp wrote: ↑Wed Mar 18, 2020 7:42 pm If you have more than 7 pieces, there'll be positions where the problem is much worse. 5 men makes it much easier for engines, not harder.

Let's start from scratch: what is the practical problem that would be solved with this solution?

Treat me like a 5 year old that has just found out there's magical programs that somehow come up with chess moves, this problem makes them come up with bad chess moves, turning off 50-move draws make them come up with better moves, and I'd want to know why, and how in the entire history of computer chess nobody has noticed how such an easy hack can improve those move choices.

Zenmastur wrote: ↑Wed Mar 18, 2020 5:35 am I think there is a solution that would allow the engine to find solutions to similar problems during tournament play without disabling the 50-move-rule.

In the meantime I think a UCI option to disable 50-move-rule checking is warranted.

When would one choose to disable it and when not?