AI offer/agree to a draw like a human?

[zulban]

I am fairly sure that when he wrote score he meant win probability.

There is no such thing in a chess engine, at least none of the top ones. If there is, it's packed with magic and guesses.

Contempt is something else.

[Pio]

I am fairly sure that when he wrote score he meant win probability.

There is no such thing in a chess engine, at least none of the top ones. If there is, it's packed with magic and guesses.

Contempt is something else.

Look at https://www.chessprogramming.org/Pawn_A ... e,_and_Elo

[zulban]

Sorry, that totally does not apply here at all.

If you make custom pieces and a custom board, you don't know the ELO of the player or the computer. Also, during a match ELO doesn't change.

a pawn value was computed using conventional pawn-normalized material ratios that are close to those used in strong chess programs (P=1, N=4, B=4.1, R=6, Q=12)

Note how I also don't have those material values in advance. Though my engine comes up with its own the first time it sees new pieces/boards.

[hgm]

You might say that is fairly static. Well what if we multiply all values by 10:

This is why I said the changes have to be interpreted relative to the draw margin. Then multiplying does not change anything, as it would multiply the draw margin too. In orthodox Chess the draw margin is slightly larger than one Pawn.

I don't know whether piece values have to be given by the user when he defines a new game, or are calculated through an algorithm from how it moves. But you should require that the values have some meaning on an absolute scale, not just the proper ratio. E.g. a piece that would convey a winning advantage should not get value 0.05, but something like 3 or 10 (depending if there are pieces that are obviously much weaker and also give a winning advantage).

If the computer would be completely clueless as to what the scores mean, you cannot expect it to make a sensible judgement (by human or any other standards) on how large its winning chances still are. That is just a logical impossibility.

[Alexander Schmidt]

I am fairly sure that when he wrote score he meant win probability.

There is no such thing in a chess engine.

That's not true

Leela works with probabilities, not scores.

[zulban]

I don't know whether piece values have to be given by the user when he defines a new game, or are calculated

Calculated. It would be pretty silly entrusting the player to construct the AI. I've had children play this, for example

But you should require that the values have some meaning on an absolute scale

I suppose I kind of do. A piece that moves very similar to a rook will be worth roughly what a rook is. However on a board with say, only white tiles, a rook will be worth almost nothing. I think what is tricky here is when there's lots of those pieces - you might even have 40 rook-like pieces. So their material value is somewhat grounded as rook-like, but when you look at the whole board, one rook-like piece can be worth a lot or not much.

I suppose I'll share whatever I come up with when something passes all my unit tests (cases I described earlier).

Leela

I've read two of the Alpha Go/Zero papers... saying that it runs on probabilities is an enormous oversimplification. The engine does not operate by guessing its percent chance of winning, as I recall. It does have a NN for how worthwhile a search branch is though (a kind of probability). I think the first iteration had two NN, one did this. Now that aspect is combined into one NN.

[zulban]

Hmmm, on second thought, maybe Leela does incorporate that (MC?). I'm not sure now. Maybe someone here would care to look at the paper again

[Alexander Schmidt]

Hmmm, on second thought, maybe Leela does incorporate that (MC?). I'm not sure now. Maybe someone here would care to look at the paper again

The output of Leela Zero for Go looks like this:

Code: Select all

NN eval=0.440025
Playouts: 4602, Win: 47.12%, PV: D4 Q4 Q16 D16 R3 R4 Q3 P3 P2 O3 O2 N3 N2 M3 C17 D17 C16 C15
Playouts: 8902, Win: 47.13%, PV: D4 Q4 Q16 D16 R3 R4 Q3 P3 P2 O3 O2 N3 N2 M3 C17 D17 C16 C15 B15 C14
Playouts: 12433, Win: 47.27%, PV: D17 Q16 D15 Q4 D4 C3 D3 C4 C5 B5 C6 B6 C7 B7 C8 B8 C9 B9 C10 B10 C11 B11 C12 B12

 D17 ->    6422 (V: 47.58%) (N:  0.62%) PV: D17 D16 C16 E17 C17 D15 C15 D14 C14 D13 D4
  D4 ->    1970 (V: 47.04%) (N: 14.49%) PV: D4 Q4 D16 Q16 R3 Q3 R4 R5 S5 R6 S6 R7 S7 R8 R17 R16 Q17 P17 P18 O17 O18 N17 N18 M17
  Q4 ->    1669 (V: 46.99%) (N: 13.46%) PV: Q4 D4 Q16 D16 C17 C16 D17 E17 E18 F17 F18 G17 G18 H17 C3 C4 D3 E3 E2 F3 F2 G3 G2
 Q16 ->    1603 (V: 47.01%) (N: 12.77%) PV: Q16 Q4 D16 D4 C3 C4 D3 E3 E2 F3 F2 G3 G2 H3 R3 Q3 R4 R5 S5 R6
 D16 ->    1544 (V: 46.95%) (N: 13.35%) PV: D16 D4 Q16 Q4 C3 C4 D3 E3 E2 F3 F2 G3 G2 H3 R3 Q3 R4 R5 S5 R6
 C16 ->     707 (V: 47.57%) (N:  0.72%) PV: C16 Q16 Q4 D4 E16 R3 R4 Q3 P3 P2 O3 O2 N3 N2 M3 M2 L3 L2

There is a MC search which gives back winning probabilities, and there is an evaluation by the network.

[zulban]

Well, there you go. It doesn't help my situation at all... but it's a good reminder. Thanks.

In case anyone is interested, I did manage to code a solution which passes all my tests. I was going to paste some real code, but that is not very concise!

Code: Select all

function AgreeToDraw{
	if computer is very easy, return true
	if it's the first 20 moves, return false
	if DoingSlightlyBadOrWorse return true
	if not IsProgressHappening return true
	otherwise return false
}

Here's how I decide "slightly bad or worse":

Code: Select all

function DoingSlightlyBadOrWorse{
    set whiteStart to the average material value of white pieces in first 5 moves    
    set whiteRecent to the average material value of white pieces in last 5 moves    
    set whiteRatio to whiteRecent / whiteStart
    // example: if white has lost half its material value, whiteRatio is 0.5    
    set the same for black   
    set materialRatio to blackRatio / whiteRatio
    // also invert materialRatio if white's turn so >1 is always good for computer.
    
    if materialRatio < 0.8 return true    
    otherwise return false
}

Here's how I decide if progress is still happening:

Code: Select all

function IsProgressHappening{    
    // MRIM = most recent irreversible move. An irreversible move is like a capture, promotion, etc.
    
    if MRIM is close to start of game, return true    
    if MRIM is less than 10 moves ago, return true
    
    set stdBeforeIrreversible to standard deviation of AI evaluation scores before MRIM
    set stdAfterIrreversible to standard deviation of AI evaluation scores after MRIM
    set stdRatio to stdAfterIrreversible / stdBeforeIrreversible
    
    if stdRatio < 0.5 return false    
    otherwise return true
}

In the future I might make a few of those constants dynamic. For example, in a very small board the first 20 moves is well past the start of the game, and should probably be less for AgreeToDraw. For now though this combination seems to be working great.

Maybe others will find this logic and statistics outline useful... thanks for the help

Toodles

[zulban]

Ah, forgot to add: DoingSlightlyBadOrWorse also checks that the most recent AI score is worse than its AI score for the first several moves in the game. So it won't consider its material shortage to be a problem if it knows it can make up for it soon.