REBEL-Dev

[Rebel]

The NNUE development of REBEL has come to a halt because my available hardware has become insufficient to make further progress in an acceptable time. For that reason I like to document my work during the last 2 years in the hope it will be useful. Much thanks goes to Chris Whittington who introduced me to NNUE programming.

....

https://rebel7775.wixsite.com/rebel/kop ... rl-blitz-1

[Steve Maughan]

Brilliant — thanks Ed!

[Rebel]

NU 1.2 release

Added 2 new functions -

[F5] - EPD shuffler - one phase shuffle. Installation Python required. EDP's are shuffled and stored in parts of 100,000 and then the stored parts of 100,000 are randomly appended into one file back into the EPD folder as shuffled.epd. For the best results the trainer needs shuffled data.

[F6] - Extract imbalances - 4 new interesting data ideas.

1. Basic 5-men stuff, KPK, lone knight or bishop vs 2 pawns or more, lone rook vs 3 pawns or more, KBNK, KQKR.
2. Imbalance Queen, for instance QvsRR, QvsRB, QvsRN, QvsBBN etc.
3. Imbalance Rook, for instance RvsBN, RvsB, RvsN.
4. Bishop Pair, when one side has the bishop pair and the other side has no bishops.

Data will follow later.

[Rebel]

I started a blog, trying all sorts of NNUE data.

https://rebel7775.wixsite.com/rebel/kop ... rl-blitz-2

[Rebel]

NU 1.3 release

Added 4 new functions -

[F7] - A fun function, from the downloadable epds make a PGN and create a Polyglot book. It gives some insight about the openings that are played to create data.

[F8] - EPD sanity check. For the best results NNUE data should be in balance, equal white wins, equal black wins and a draw percentage not higher than 55%. Ideally 30 | 30 | 40 or something close. Furthermore a Material Division and Piece Count Overview statistic are created. Also here balance is important.

[F9] - The Bishop ending is notorious for programmers because there are 2 types, the same colored bishop ending and the unequal colored bishop ending each having its own dynamics. This function extracts 4 types.
1. Unequal bishop endings.
2. Equal bishop endings.
3. Equal and unequal bishop endings [1+2]
4. Also extracted are the so called bad-bishop-ending KBPK cases.

[F10] - Delete suspect Draws, Wins and Losses.
1. An epd that is a draw with a score >= abs(5.00) is removed.
2. An epd that is a win with a score <= -3.00 is removed.
3. An epd that is a loss with a score >= 3.00 is removed.
A new file is created without the suspect cases.

https://rebel7775.wixsite.com/rebel/kop ... rl-blitz-1
​

[Steve Maughan]

Ed,

Many thanks.

Quick question: I assume all the positions used for training need to be "quiet positions". This would avoid a shallow search seeing a tactical win of material and then trying to force the NN to learn this static position with an unrealistic score. My question is how do you detect quiet positions? Do you have any insights you're able to share?

Here are my thoughts...

It would be too much to delete all positions where the best move is a capture. It might simply be an exchange. There are also the occasions where the tactic might start with a quiet move — maybe a check. I thought the best way may be to do the following:

1. If the best move is a capture, look for the next best non-capture. Only keep the position if the score for the non-capture is within 0.4 pawns of the capture
2. If the best move is non-capture, look for the second best move. Only keep the position if the score for the non-capture is within 0.4 pawns of the capture

In other words, only keep positions where there is a quiet move that is within a small value of the best move. Is this over-thinking it? What do others do?

— Steve

[Rebel]

Hi Steve,

The NU util has an option Make EPD Quiet, it removes captures, king in check and promotions. I don't know the exact reason but I suspect that because of the lack of QS in the net captures will disturb the material balance in the net and you probably get many wrong evals.

What does work are captures that sacrifice, I recently tested it. I think it's possible to learn a net to sacrifice material. The problem is there are too few and so it will only work in a small net. I keep checks, removing them cost elo.

Regarding your capture idea, try everything, NNUE programming is full of surprises.

In an older Rebel version I checked the history of each quiet bm move for 5 moves and if 3 or more of them were non-quiet I extracted that position, it worked back then, I should try it again.

[Steve Maughan]

Thanks Ed — I'll give it a try!

[Rebel]

New NU version 1.4

[F12] - Tuning Piece Division,  this is how our 4B test data looks like when we make an overview from the 4B data with NU [F8]. As you see some are not well synchronized, specially the high odd piece counts. With NU [F12] we create extra data to fill the gaps in order that the data is better synchronized. Note the data before and after.



Operation
Edit the file balance-piece-count.txt and in millions enter the data that should be added for each of the low piece-counts. And good for ~10 elo, in this particular case.

Another use of NU [F12] is to set everything (for instance) on "1" in balance-piece-count.txt witch will create a new epd where every piece count has exactly 1.000.000 positions. In other words you can design any piece-count division data you wish, from midgame to endgame.

balance-piece-count.txt

Code: Select all

32 : 1
31 : 1
30 : 1
29 : 1
28 : 1
27 : 1
26 : 1
25 : 1
24 : 1
23 : 1
22 : 1
21 : 1
20 : 1
19 : 1
18 : 1
17 : 1
16 : 1
15 : 1
14 : 1
13 : 1	
12 : 1
11 : 1
10 : 1
9  : 1
8  : 1
7  : 1
6  : 1
5  : 1
4  : 1
3  : 1