TalkChess.com

How do you choose the positions for the training? Is it better to use only quiet positions?

I have tried to use quantized weights in the training, only when I calculate the error of the network and seems to work. I have to try with int8 but with int32 works well.
Thank you!

But the net calculations would be all on floating point values while stockfish use only integer arithmetic. It's an advantage to use only ints because the net calculation is faster. But I miss something because while training the net the weights differs a lot and int8_t gives low accuracy. But maybe...

But if I have understood correctly in this way you get an index for an array but the weights of the neural net are multiplied with the input and added together. If I have understood stockfish use a fixed point arithmetic because the output is shifted by 6 places so is like a division by 64. If it's ...

I have a neural network with floating point weights and I want them to fit in int8_t . The weights goes from +64 and -57 and some are very low like 1e-5. How can I translate all these values in int8_t without losing accuracy? Is there a way to train the net to easily translate the weights to int8_t?...

I am very familiar with Leela type networks and far less so with the NNUE nets. That said, your LR seems extremely low for initial training (although no batch size was mentioned). I suggest asking in the SF-NNUE Discord but I'm not sure how joining that works. I update the weights after every sampl...

Is unclear the range of centipawn values your training and test sets have, but, if you were training on win probability (0.0 to 1.0 range) then a completely untrained (random) net is going to come back with an average “error” of about 0.3 which could well translate to your 200 cp. So it’s entirely ...

More information about the network architecture and hyper-parameters (learning rate, etc) might provide some clues. Just a quick stab, but depth 4 does not seem very deep relative to what the SF-NNUE Discord posts talk about. It might not be enough improvement from your current eval. I'm trying a s...

I'm implementing a neural network that should replace the evaluation of my engine. I've made a dataset of 200M fens with associated the score of a 4 depth search of my engine. For the optimization algorithm I have used Adagrad and Gradient descent, the loss function is the sum squared error. I have ...

Neural nets give a better evaluation than an handcrafted one. I'm going to try to implement my own neural network with my own training method so the evaluations will be different from other implementations. Improve the evaluation is very difficult and if I can replace it with a better one I will be ...