question to prof Hyatt re kaufman and best positional moves

[bob]

Here is my opinion. If (really big if) you can design an interface that does not cost the human time, then the human would win hands down. If he makes no tactical errors that the opponent could see, and if he can use a real GM evaluation to choose between moves that are equal tactically, then the computer would be helpless.

But the interface is the issue and does not exist today... Correspondence would be interesting, where the human could choose the objectively best move from a list of moves known to be completely safe tactically. Then the interface would not be such an issue.

Here is Larry kaufman's reply from the rybka forum:

http://rybkaforum.net/cgi-bin/rybkaforu ... 7#pid97067

Uri

I think anyone that believes otherwise is a bit "off". Note that I am _not_ talking about a program like gnuchess with material only. I am talking about a program equal to rybka 3 tactically. Which might only be Rybka 3 with just a material-only evaluation. My original premise was "no tactical errors" which means this program would necessarily see everything Rybka could see. Then the only difference between the human and computer would be "evaluation". There I would give an _enormous_ edge to the GM. And I do mean "enormous".

[bob]

We need to define exact conditions that Bob Hyatt expect the GM to win to be able to test if he is right or wrong.

The main problem with only material engine is that Bob may think that it is weaker tactically than rybka and not only positionally.

Uri

1. Most likely, Rybka 3, material only evaluation. Same search, extensions, reductions, whatever else it does.

2. An interface that is designed for this purpose. I have not given any thought to how this would look, but one possible idea might be to list the root moves on the screen, and either include or exclude moves of interest from the search by checking boxes beside them. This interface would probably want to employ multiple computers. One doing a traditional search, but with just material evaluation, picking the best tactical reply and showing the score. One (or more) that only search the specific move(s) the GM wants analyzed, although he would want the first version as well in case he overlooks some tactical win with his first positional choice.

3. Enough time to compensate for the interaction with the computer that he would have to do, in addition to his normal "searching" himself.

Given a GUI that shows the best tactical move (so that the GM won't overlook a winning move he missed in his own search) plus the ability to choose specific moves to search (since a material only search) will only show the best move, and he might want to know if a different move is tactically equal to the best move or does it lose material. And I could see wanting information on several such moves as a "screening process".

That is all I believe he would need.

[Karlo Bala]

Here is my opinion. If (really big if) you can design an interface that does not cost the human time, then the human would win hands down. If he makes no tactical errors that the opponent could see, and if he can use a real GM evaluation to choose between moves that are equal tactically, then the computer would be helpless.

But the interface is the issue and does not exist today... Correspondence would be interesting, where the human could choose the objectively best move from a list of moves known to be completely safe tactically. Then the interface would not be such an issue.

Here is Larry kaufman's reply from the rybka forum:

http://rybkaforum.net/cgi-bin/rybkaforu ... 7#pid97067

Uri

I think anyone that believes otherwise is a bit "off". Note that I am _not_ talking about a program like gnuchess with material only. I am talking about a program equal to rybka 3 tactically. Which might only be Rybka 3 with just a material-only evaluation. My original premise was "no tactical errors" which means this program would necessarily see everything Rybka could see. Then the only difference between the human and computer would be "evaluation". There I would give an _enormous_ edge to the GM. And I do mean "enormous".

Yes, but rybka will have all knowledge in leafs, while GM will have knowledge only near root. GM + tactical engine will work similar to preprocessor engine and we know that they are outdated.

[bob]

Here is my opinion. If (really big if) you can design an interface that does not cost the human time, then the human would win hands down. If he makes no tactical errors that the opponent could see, and if he can use a real GM evaluation to choose between moves that are equal tactically, then the computer would be helpless.

But the interface is the issue and does not exist today... Correspondence would be interesting, where the human could choose the objectively best move from a list of moves known to be completely safe tactically. Then the interface would not be such an issue.

Here is Larry kaufman's reply from the rybka forum:

http://rybkaforum.net/cgi-bin/rybkaforu ... 7#pid97067

Uri

I think anyone that believes otherwise is a bit "off". Note that I am _not_ talking about a program like gnuchess with material only. I am talking about a program equal to rybka 3 tactically. Which might only be Rybka 3 with just a material-only evaluation. My original premise was "no tactical errors" which means this program would necessarily see everything Rybka could see. Then the only difference between the human and computer would be "evaluation". There I would give an _enormous_ edge to the GM. And I do mean "enormous".

Yes, but rybka will have all knowledge in leafs, while GM will have knowledge only near root. GM + tactical engine will work similar to preprocessor engine and we know that they are outdated.

You overlook two missing adjectives. Let me add:

Rybka will have all _primitive_ knowledge in leafs, while GM will have all _relevant_ knowledge in the tree he searches. The difference between "primitive" and "relevant" is enormous. But as I said, until you have spent a lot of time talking with GM players, you will not be able to grasp the difference.

It is not going to be trivial for the GM to win. He still will have to work at it. But with the assurance that he won't make some brilliant plan and overlook some simple tactical move, the human has to be favored by a significant margin.

Testing the hypothesis is non-trivial however.

[duncan]

Is it possible to test who does the better positional moves, by allowing the gm takebacks only in the case where he loses material to some tactical combination.

I think Mr Kaufman said that allowing one move takebacks is 'only' worth 150 elo and is confident rybka would win a gm under such conditions.

of course more may be necessary.

[bob]

Is it possible to test who does the better positional moves, by allowing the gm takebacks only in the case where he loses material to some tactical combination.

I think Mr Kaufman said that allowing one move takebacks is 'only' worth 150 elo and is confident rybka would win a gm under such conditions.

of course more may be necessary.

I think that is too subjective a mechanism. Perhaps it could work if it goes such that if on any move, after the GM moves, if the program sees a win of any material, then the GM will have to take back his previous move, unless he is making an intentional positional sacrifice of some sort. But I am not sure that is the best way to proceed, On the human's move, he might really like a move and when he sees that it loses a pawn, he might investigate further to see if he still wants to play it. I'd rather have him make his move on his own time/clock. Otherwise he would either get a time advantage if a move gets rejected and he thinks "off the clock" or he might tend to move too quickly to see if the opponent will reject the move, so that he has more time to find another candidate. I'd rather just see him work everything out, then make a move and have to stick with it.

[Uri Blass]

Here is my opinion. If (really big if) you can design an interface that does not cost the human time, then the human would win hands down. If he makes no tactical errors that the opponent could see, and if he can use a real GM evaluation to choose between moves that are equal tactically, then the computer would be helpless.

But the interface is the issue and does not exist today... Correspondence would be interesting, where the human could choose the objectively best move from a list of moves known to be completely safe tactically. Then the interface would not be such an issue.

Here is Larry kaufman's reply from the rybka forum:

http://rybkaforum.net/cgi-bin/rybkaforu ... 7#pid97067

Uri

I think anyone that believes otherwise is a bit "off". Note that I am _not_ talking about a program like gnuchess with material only. I am talking about a program equal to rybka 3 tactically. Which might only be Rybka 3 with just a material-only evaluation. My original premise was "no tactical errors" which means this program would necessarily see everything Rybka could see. Then the only difference between the human and computer would be "evaluation". There I would give an _enormous_ edge to the GM. And I do mean "enormous".

Yes, but rybka will have all knowledge in leafs, while GM will have knowledge only near root. GM + tactical engine will work similar to preprocessor engine and we know that they are outdated.

You overlook two missing adjectives. Let me add:

Rybka will have all _primitive_ knowledge in leafs, while GM will have all _relevant_ knowledge in the tree he searches. The difference between "primitive" and "relevant" is enormous. But as I said, until you have spent a lot of time talking with GM players, you will not be able to grasp the difference.

It is not going to be trivial for the GM to win. He still will have to work at it. But with the assurance that he won't make some brilliant plan and overlook some simple tactical move, the human has to be favored by a significant margin.

Testing the hypothesis is non-trivial however.

I think that it is not correct to consider rybka's evaluation as primitive.

Rybka3's evaluation is at least better than other programs based on watching games.

If rybka and another program disagree about the evaluation at small depths then in most cases rybka is right.

My opinion is that
better evaluation is not the only reason that rybka is stronger than the opponents but it is one of the reasons.

Uri

[BubbaTough]

I think that it is not correct to consider rybka's evaluation as primitive.

Rybka3's evaluation is at least better than other programs based on watching games.

Maybe compared to other programs, but compared to strong humans Rybka eval is still hella-primitive. My program has drawn a number of completely lost games to Rybka 3 with very simple knowledge all strong humans know. Its even won one due to some fancy eval heuristic against superior hardware. My point is not that LearningLemming is super cool (which it is) but that there is much VALUABLE human knowledge not present in Rybka, including some that superior search cannot always replace.

That does not negate the fact that the question is still an interesting one, and difficult to test (impossible?). There are many situations at the end of trees that a grandmaster would immediately know were won or lost, but would not be known to a "material only" machine. For example, an obviously won pawn endgame or an obviously drawn rook endgame may look the same. Knowing which is forced requires the grandmaster to do his own lengthy analysis, not rely on a "material only" engine. 2 pawns on the 6th vs. a rook in endgame is another example. Trapped pieces, another trivial example. While a machine aide will help a grandmaster avoid some "blunders", many blunders will be unseen because decent eval of leafs is necessary. I think the experiment would be interesting (to computer chess junkies, probably not the general public) and fun to watch, and the answer is very difficult to know ahead of time.

To be fair, I think the "material only" eval should not use the default material eval of Rybka (which would probably have embedded probability of win information that would judge R+P vs. N as different than Q+R+P vs. Q+N). The value of pieces could use whatever the grandmaster wanted (or the default Kaufman values) with or without including the bishop pair as material (grandmaster choice...some people feel it is a material advantage not a positional one). An interesting side-effect would be that the search for the "material only" version of Rybka would possibly end up being significantly faster than the normal version (both because of less code executed and because the many eval score ties would cause more aggressive pruning).

By the way, it is unclear Kaufman would agree to such as match; I think he would prefer Grandmaster + NON-RYBKA vs. RYBKA match.

-Sam

[bob]

Here is my opinion. If (really big if) you can design an interface that does not cost the human time, then the human would win hands down. If he makes no tactical errors that the opponent could see, and if he can use a real GM evaluation to choose between moves that are equal tactically, then the computer would be helpless.

But the interface is the issue and does not exist today... Correspondence would be interesting, where the human could choose the objectively best move from a list of moves known to be completely safe tactically. Then the interface would not be such an issue.

Here is Larry kaufman's reply from the rybka forum:

http://rybkaforum.net/cgi-bin/rybkaforu ... 7#pid97067

Uri

I think anyone that believes otherwise is a bit "off". Note that I am _not_ talking about a program like gnuchess with material only. I am talking about a program equal to rybka 3 tactically. Which might only be Rybka 3 with just a material-only evaluation. My original premise was "no tactical errors" which means this program would necessarily see everything Rybka could see. Then the only difference between the human and computer would be "evaluation". There I would give an _enormous_ edge to the GM. And I do mean "enormous".

Yes, but rybka will have all knowledge in leafs, while GM will have knowledge only near root. GM + tactical engine will work similar to preprocessor engine and we know that they are outdated.

You overlook two missing adjectives. Let me add:

Rybka will have all _primitive_ knowledge in leafs, while GM will have all _relevant_ knowledge in the tree he searches. The difference between "primitive" and "relevant" is enormous. But as I said, until you have spent a lot of time talking with GM players, you will not be able to grasp the difference.

It is not going to be trivial for the GM to win. He still will have to work at it. But with the assurance that he won't make some brilliant plan and overlook some simple tactical move, the human has to be favored by a significant margin.

Testing the hypothesis is non-trivial however.

I think that it is not correct to consider rybka's evaluation as primitive.

Rybka3's evaluation is at least better than other programs based on watching games.

That is just as truthful as saying "a monkey's intelligence is greater than that of a dog." But it says _nothing_ about the monkey compared to a man.

If rybka and another program disagree about the evaluation at small depths then in most cases rybka is right.

My opinion is that
better evaluation is not the only reason that rybka is stronger than the opponents but it is one of the reasons.

Uri

And I would not disagree. But that has _nothing_ to do with comparing Rybka's evaluation to that of a Grandmaster chess player. They are in a different class.

[duncan]

Is it possible to test who does the better positional moves, by allowing the gm takebacks only in the case where he loses material to some tactical combination.

I think Mr Kaufman said that allowing one move takebacks is 'only' worth 150 elo and is confident rybka would win a gm under such conditions.

of course more may be necessary.

I think that is too subjective a mechanism. Perhaps it could work if it goes such that if on any move, after the GM moves, if the program sees a win of any material, then the GM will have to take back his previous move, unless he is making an intentional positional sacrifice of some sort. But I am not sure that is the best way to proceed, On the human's move, he might really like a move and when he sees that it loses a pawn, he might investigate further to see if he still wants to play it. I'd rather have him make his move on his own time/clock. Otherwise he would either get a time advantage if a move gets rejected and he thinks "off the clock" or he might tend to move too quickly to see if the opponent will reject the move, so that he has more time to find another candidate. I'd rather just see him work everything out, then make a move and have to stick with it.

Are you saying that If a game was played that after the GM moves, if the program sees a win of any material, then the GM will have to take back his previous move, unless he is making an intentional positional sacrifice of some sort.

and rybka won (which Mr kaufman is confident would happen) it is evidence that Rybka plays better positional moves than a gm.

while if it lost, it would not be in itself evidence that the gm has better positional understanding, as the conditions are advantageous to the human.