What is the purpose of chess engines?

[Ovyron]

It means that there is no clear definition if A is better than B or B is better than A.

I'm talking about playing the position, not the opponent. If you didn't know who was your opponent, what move would you play?

I like chrisw's concept of a chess map, chess engines would help you see the roads and where they lead to, the purpose would be where you want to go. Tablebases would just be a full map where you already know where every place is located, but parts leading to them remain dark.

There are white places (where getting the black king checkmated is unavoidable), black places (where getting the white king checkmated is unavoidable), the game starts in a grey place (slightly lighter, so it takes effort from black or misplaying from white to get to a darker zone), and players take turns going into roads. Once a side allows the other to go into one that forcibly leads to shades of grey that are very light or very dark the other is in troble and it's hard to go to the other side. There are places that are mostly grey and simple, and others that resemble zebra skins full of white and black places, very complex.

And finally, people usually go by foot, and can't see very far, so against a superior player like an engine that drives you into a dark alley thee's no much hope, while engine assisted chess would be like car racing where you can go as deep as you can to explore the terrain before committing to a road.

It's clear there must be a road that is the best one to get into one's destination, but people that just analyze and never play would be like people analyzing the best routes in Google Maps but never go out of their homes, so they'll never know if being there means they have to improvise because in real life, the path is blocked. Chess truth can only be experienced in a game and nobody in their right mind would play from tablebase positions.

[Uri Blass]

It means that there is no clear definition if A is better than B or B is better than A.

I'm talking about playing the position, not the opponent. If you didn't know who was your opponent, what move would you play?

Without knowing your opponent it is dependent on the statistical distribution of the possible opponents that I assume
and I can assume different distributions.

There is sometimes no clear solution to the question if A is better than B or B is better than A when both lead to draw against the perfect player.
Even saying they are equal is wrong when A is better against X and B is better against Y.

I can add that people may play from tablebases position and chess is not only engine-engine games.
In chesstempo you can train in tablebases positions if you choose endgame training.

[Ovyron]

A perfect player is theoretical. I even remember testing engines, one engine when reaching a drawn endgame would still try to win it and succeeded against opponents that didn't know how to play the positions. The next version had bitbases or something that allowed it to know those positions were drawn, and it'd play random moves that kept the draw, but it could no longer beat the opponents that the previous version defeated.

If we get this unattainable perfect player out of the equation, then we are left with a bunch of theoretical imperfect players. You can get rid of the bottom ones because they're so weak you wouldn't need a chess map to defeat them. The ones in the middle are probably also going to be defeated by a chess map that would beat the ones at the top.

As you sort imperfect players by strength, there will be at the very top the strongest possible theoretical player, one that would blunder only in the most complex positions when even yourself have the most difficulty finding the saving move. A chess map would rank first the moves that beat this player, unless somehow more weaker players would have a better performance against those ranked moves, but then you do a bit of tweaking for the move ranks.

In practice you yourself are an imperfect player in the scale, so you're unable to know how stronger players than you would play. Some people think that if you're against such a superior player you should just assume they're perfect (their superiority is enough that you wouldn't be able to get a position in the board where they blunder) and play for a draw, where the ranking of the moves is very different. Others think that you should just assume the opponent is superior when you're black and don't bother trying to win, but my wins as black would prove them wrong.

Taking all of these into account would indeed mean that building a chess map that ranks chess moves in all positions in a way that maximizes performance against all possible opponents is impossible, but it's very easy to prove it must be possible.

And for this I bring out the theoretical Quantum Player. If you think about it, the QP doesn't make sense, because it's an entity that can't exist. It's still interesting to discuss it just like people discuss "best play" even though "best play" remains vague an unattainable.

The Quantum Player is a random player that plays any move with the same probability, but it plays all possible games against you at the same time. When it wins a game (and it will, unless you're a perfect player) it takes note of the moves that were able to crack you (you would be like a password and it'd bruteforce chess moves until it finds the combination that beats you) and move to its next game of its life.

By its nature QP is a chess entity that would win all its games. Now, since it basically can travel back in time to try a different move, it would be able to produce a chess map that has a move ranking for all the positions it'll play on its life to maximize its performance against its opponents.

If instead of its opponents it knew what opponents would you face, then it'd be able to build a map that would maximize your life performance.

If instead of your opponents it knew what everyone on the world would face, then it'd be able to build a map that would maximize everyone's performance.

The chess map in this last layer exists, in the abstract, just like the numbers of PI beyond 3. exist and continue over 10^10^10^10 digits, even if we can't enumerate them.

After this chess map is proven to exist, you could build an algorithm that builds a random chess map, and by chance, on your first time using it, it could produce it. It'll most likely produce a garbage unusable map but the point is that if the chess map exists and you could build it at random, it could be constructed in some other way. And there's nothing philosophical about this.

[Uri Blass]

A perfect player is theoretical. I even remember testing engines, one engine when reaching a drawn endgame would still try to win it and succeeded against opponents that didn't know how to play the positions. The next version had bitbases or something that allowed it to know those positions were drawn, and it'd play random moves that kept the draw, but it could no longer beat the opponents that the previous version defeated.

If we get this unattainable perfect player out of the equation, then we are left with a bunch of theoretical imperfect players. You can get rid of the bottom ones because they're so weak you wouldn't need a chess map to defeat them. The ones in the middle are probably also going to be defeated by a chess map that would beat the ones at the top.

As you sort imperfect players by strength, there will be at the very top the strongest possible theoretical player, one that would blunder only in the most complex positions when even yourself have the most difficulty finding the saving move. A chess map would rank first the moves that beat this player, unless somehow more weaker players would have a better performance against those ranked moves, but then you do a bit of tweaking for the move ranks.

In practice you yourself are an imperfect player in the scale, so you're unable to know how stronger players than you would play. Some people think that if you're against such a superior player you should just assume they're perfect (their superiority is enough that you wouldn't be able to get a position in the board where they blunder) and play for a draw, where the ranking of the moves is very different. Others think that you should just assume the opponent is superior when you're black and don't bother trying to win, but my wins as black would prove them wrong.

Taking all of these into account would indeed mean that building a chess map that ranks chess moves in all positions in a way that maximizes performance against all possible opponents is impossible, but it's very easy to prove it must be possible.

And for this I bring out the theoretical Quantum Player. If you think about it, the QP doesn't make sense, because it's an entity that can't exist. It's still interesting to discuss it just like people discuss "best play" even though "best play" remains vague an unattainable.

The Quantum Player is a random player that plays any move with the same probability, but it plays all possible games against you at the same time. When it wins a game (and it will, unless you're a perfect player) it takes note of the moves that were able to crack you (you would be like a password and it'd bruteforce chess moves until it finds the combination that beats you) and move to its next game of its life.

By its nature QP is a chess entity that would win all its games. Now, since it basically can travel back in time to try a different move, it would be able to produce a chess map that has a move ranking for all the positions it'll play on its life to maximize its performance against its opponents.

If instead of its opponents it knew what opponents would you face, then it'd be able to build a map that would maximize your life performance.

If instead of your opponents it knew what everyone on the world would face, then it'd be able to build a map that would maximize everyone's performance.

The chess map in this last layer exists, in the abstract, just like the numbers of PI beyond 3. exist and continue over 10^10^10^10 digits, even if we can't enumerate them.

After this chess map is proven to exist, you could build an algorithm that builds a random chess map, and by chance, on your first time using it, it could produce it. It'll most likely produce a garbage unusable map but the point is that if the chess map exists and you could build it at random, it could be constructed in some other way. And there's nothing philosophical about this.

The Quantum Player can always beat deterministic non perfect players that are not strong enough.

It cannot beat everyone and it even cannot score 100% against the random move player with enough games because the random move player may be lucky to play the right moves.

It also cannot beat every deterministic non perfect player and for example it may be unable to beat and opponent that is not perfect but it is not perfect only in the meaning that it can miss a win against other non perfect players and not in the meaning that it can practically get a position when it does a losing mistake(it may do a losing mistake in analysis of some position but it never get it in a game).

[Ovyron]

All the concept of a Quantum Player is a fancy way of saying "a random mover that is lucky and plays the winning line that defeats an imperfect opponent." If an opponent is fallible it means that with enough games a random mover would beat it, QP would just play that game on its first try "by chance."

The point is that if a player can lose a game then there's a line that exists that beats them, so lines exist for every player, and those lines could be used to build a chess map to maximize performance against all imperfect opponents.

[chrisw]

All the concept of a Quantum Player is a fancy way of saying "a random mover that is lucky and plays the winning line that defeats an imperfect opponent." If an opponent is fallible it means that with enough games a random mover would beat it, QP would just play that game on its first try "by chance."

The point is that if a player can lose a game then there's a line that exists that beats them, so lines exist for every player, and those lines could be used to build a chess map to maximize performance against all imperfect opponents.

I think the reference frame is simply being moved around and going nowhere at the same time. The fundamental situation is just being rephrased.
1. We know that the design and also part-build of a humongous lookup table which stores everything for EGTB32, exists. But we can’t build such a thing, it’s too large. Time of creation also problematic.
2. We know that chess is solved by the minimax algorithm. But we get to the solution, too much time is required.

Postulating a “map”, whose dimensionality will be a lot more than 2D, upper bound unknown, just restates the problem from EGTB32 to N-dimensional map (equally unreadable) and equally unsolvable for the same reason as the EGTB. Not enough space.

Postulating a Quantum Player is just restating the minimax algorithm, hoping to get round the lack of available time. The argument “then build a map based on the lines created by imperfect opponents” fails on a) insufficient space (that map again) and b) insufficient exploratory imperfect opponents (not enough space nor time) and c) there is no Quantum Player and if there was, it would not be able to express itself within the available space.

Whichever which way you want to turn you run into the same space-time problem. Which changing the subject slightly, is the same reason humans and animals and life in general work to the good-enough heuristic. Good-enough works. Until somebody finds something gooder, so to speak. Then the gooder-enough takes over until that’s overthrown too. Conventionally known as progress. All done on heuristics. So, what makes you think you can overcome space, time and the good-enough heuristic with some (unattainable) scheme of perfection? Probably we would find the answer in psychology, or even encoded in ancient texts (Tower of Babel fable/myth).

[Ovyron]

So, what makes you think you can overcome space, time and the good-enough heuristic with some (unattainable) scheme of perfection?

Because you don't need the entire solution of chess for this, all you need is that what with you do, the move you rank at the top of the position you're playing is the same as the one you'd see ranked first on the theoretical chess map, and that's the one you play.

All the chess positions that I have seen, and will see on my life could easily fit in my hard drive, because storing a best move for them doesn't use too much space. And that's all I would need. Personally.

This would mean that the Tablebase approach is a huge waste of time and resources, because it wants to solve everything, but >99.999% of what it contains will never be checked and will never be used by anyone for anything. So instead, they should have been solving only the 7men positions that people actually reach, and need, which is a fraction of the total. For those you need 6men TBs, but again, not all of them. And you don't need all 5men ones. You only need the ones that can be reached from the position you're investigating, and for instance, this position again:

[d]3rk3/3p4/8/8/8/8/3P4/3RK3 w - -

You only need one move if you are white, and one reply for every move if you are black, and so on. Investigating and storing everything else is a waste of time. A line that wins against a blunder if played for a side, and that's it. If you deviate of what you have and reach a position that is outside of the solution, well, too bad, now you have left the map and have nothing, but the map would assume you know how to follow directions.

And who knows if white has a strategy where black is unable to trade pieces until they're less than 16 on the board, if such a thing happens then you don't need EGTB15 (or anything below) because you just follow the instructions that always keep 16 or more on the board.

Proponents of Draw Death (who claim the death of chess has already happened some years ago) claim that they can play a non-losing move on the fly from any chess position from the opening position (not from an arbitrary position you gave them, but they could guide you into positions that they know how to avoid losing every time), so they have attained "infallibility", and they could do it on less than 6 days per position. If what they are doing could be coded in a machine, the machine's moves would be indistinguishable from those of EGTB32 (and probably stronger, since stupid EGTB32 would play 1.f3 at random.)

The code to produce perfect moves is likely to be small, we'll probably see it in our lifetime, but I don't think it'll be achieved by squeezing elo points like the Stockfish project is doing or by learning how to improve results like NNs are doing (because it's clear they're not getting closer to the truth on certain positions), we need a genius new approach that will probably come from someone that doesn't even check what is the state of the art of chess programming (which is the first stumbling block, as people just regurgitate the same methods once they know about them.)

I'm actually playing as much corr chess as I can before this happens, because once that appears, we're done. But the point is, producing all solutions of all chess positions and having to store them is nowhere as necessary. If a random mover can get lucky and beat every non-perfect opponent, a method that finds those moves in reasonable time without having to check anything stored should be possible.

[chrisw]

So, what makes you think you can overcome space, time and the good-enough heuristic with some (unattainable) scheme of perfection?

Because you don't need the entire solution of chess for this, all you need is that what with you do, the move you rank at the top of the position you're playing is the same as the one you'd see ranked first on the theoretical chess map, and that's the one you play.

All the chess positions that I have seen, and will see on my life could easily fit in my hard drive, because storing a best move for them doesn't use too much space. And that's all I would need. Personally.

This would mean that the Tablebase approach is a huge waste of time and resources, because it wants to solve everything, but >99.999% of what it contains will never be checked and will never be used by anyone for anything. So instead, they should have been solving only the 7men positions that people actually reach, and need, which is a fraction of the total. For those you need 6men TBs, but again, not all of them. And you don't need all 5men ones. You only need the ones that can be reached from the position you're investigating, and for instance, this position again:

[d]3rk3/3p4/8/8/8/8/3P4/3RK3 w - -

You only need one move if you are white, and one reply for every move if you are black, and so on. Investigating and storing everything else is a waste of time.

you’ve done no more that achieve alpha-beta algorithm improvement over and above minimax (eliminating the requirement to rank replies in order).

An AB tree search can’t be done in time, it delivers a result faster than minimax, but not the gazillions of orders of orders of magnitude required.

Pedantically, while white may play one move only, e4, he must have a move stored for all of blacks replies (20 moves), and after the White stored move, another move for all black replies. So now we are four ply deep or whatever and we need 500 moves stored. Keep multiplying by 30 or so every other ply, you’ll be in LaLa land shortly. No possible. Space not available.

And, btw, this oracle can tell you exactly nothing about “why”. No explanation. Which brings another question, why would you believe an oracle that can’t explain the why? It might just be bluffing you. How to tell? Play it forever, but, er, time problem again.

Chess has conspired with its own designers to render itself unsolvable in space and time. Rather obviously there would be a game that made this conspiracy, would there not? And if there wasn’t somebody would make one. Just as there have been made solvable games.
So what is purpose of chess (or a chess engine)? It’s to keep some people on the edge of their seats imagining conquering unconquerable space-time problem. You may as well, actually, be designing perpetual motion machines. Same process. Fundamental laws say it can’t be done, but some people like to challenge fundamental laws. Better to challenge fundamental laws that actually are “wrong” or broken.

A line that wins against a blunder if played for a side, and that's it. If you deviate of what you have and reach a position that is outside of the solution, well, too bad, now you have left the map and have nothing, but the map would assume you know how to follow directions.

And who knows if white has a strategy where black is unable to trade pieces until they're less than 16 on the board, if such a thing happens then you don't need EGTB15 (or anything below) because you just follow the instructions that always keep 16 or more on the board.

Proponents of Draw Death (who claim the death of chess has already happened some years ago) claim that they can play a non-losing move on the fly from any chess position from the opening position (not from an arbitrary position you gave them, but they could guide you into positions that they know how to avoid losing every time), so they have attained "infallibility", and they could do it on less than 6 days per position. If what they are doing could be coded in a machine, the machine's moves would be indistinguishable from those of EGTB32 (and probably stronger, since stupid EGTB32 would play 1.f3 at random.)

The code to produce perfect moves is likely to be small, we'll probably see it in our lifetime, but I don't think it'll be achieved by squeezing elo points like the Stockfish project is doing or by learning how to improve results like NNs are doing (because it's clear they're not getting closer to the truth on certain positions), we need a genius new approach that will probably come from someone that doesn't even check what is the state of the art of chess programming (which is the first stumbling block, as people just regurgitate the same methods once they know about them.)

I'm actually playing as much corr chess as I can before this happens, because once that appears, we're done. But the point is, producing all solutions of all chess positions and having to store them is nowhere as necessary. If a random mover can get lucky and beat every non-perfect opponent, a method that finds those moves in reasonable time without having to check anything stored should be possible.

[Ovyron]

Pedantically, while white may play one move only, e4, he must have a move stored for all of blacks replies (20 moves), and after the White stored move, another move for all black replies. So now we are four ply deep or whatever and we need 500 moves stored. Keep multiplying by 30 or so every other ply, you’ll be in LaLa land shortly. No possible. Space not available.

Heh, it's curious that you brought up the Tower of Babel in your other post, because I can use something similar to prove you wrong, let me introduce you to...

The Library of Babel.

This guy has encoded everything that could be said ever. So on there there's a copy of our entire conversation.

Here's a link of a page that contains the text you typed in that I just quoted, and note this page already contained your text before you typed it.

The Library of Babel contains already the solution to chess and any other game you could come up with, a series of instruction of how to make a program that plays EGTB32 moves on chess positions without having to store them, and anything you can think of.

Let me demonstrate.

Since we don't have special symbols for FEN I'll make up my own chess notation. Pieces are denoted as wr wn wb wk wq wp for white and br bn bb bk bq bp for black. ee would be empty spaces. Moves will be denoted with common coordinates as starting square and ending square. We can afford to use full names for numbers instead of numbers. We don't need to specify the side to move, because we can just store what would be white's best move and black's best move if they were to move.

Suppose that from the opening position d4 (d two d four) is white's best move, and for the sake of completion, d5 (d seven d five) was black's best move (if black had to move first from the starting position). It'd look encoded like this:

Code: Select all

br bn bb bq bk bb bn br
bp bp bp bp bp bp bp bp
ee ee ee ee ee ee ee ee
ee ee ee ee ee ee ee ee
ee ee ee ee ee ee ee ee
ee ee ee ee ee ee ee ee
wp wp wp wp wp wp wp wp 
wr wn wb wq wk wb wn wr
d two d four
d seven d five

Well, before I woke up this morning this page already contained this text.

What about that position where Gusev played Qxe5 and the fastest time to find it has been 3 seconds? It'd look encoded like this:

This page shows the solution instantly. Turns out if black was to move Rxd6 would be best, who knew?

So the solution to any chess position is encoded already in the Library of Babel.

Unfortunately, all chess positions and all their playable moves are also encoded there, so you can't tell them apart, but if someone can come up with an algorithm that deleted all pages with chess positions encoded like this from the Library of Babel, except for those that have the best move from white and for black, then you'd have your EGTB32. How ironic is it that instead of having to store info about the best moves, a chess map would need us to delete all non-best moves?

[mclane]

The problems are not the chess positions where you can find something,
The majority of chess positions has nothing to find within a range of x plies.
And if you try to find a move or plan via tree, you play lousy.