Booot progress

[booot]

Second stage done!

Simple input-featured Keras model with the same NN structure of hidden layers is built, trained,quantized and measured.

I took "Circle" function f(x,y)=20*(sqrt(X)-sqrt(Y)) to train NN. X and Y are in range (0..1023) integer (10 digits of binary code each). This function gives result in range (-640,640) - quite similar as chess evaluation function does. I generated 1M train data easy with input featured layer [:,20].
I spent a good time to train - quantize - evaluate the model and finally my game with it ends with final score 1:1. I received working train code and clear understanding what quantization exactly does in NN, model received the last layer (last neuron weights) int16 instead of int8. So my NN structure will have final view:
1 layer (input feature layer) - int16 weights
2-3 2 hidden layers - int8 weights
4 output layer - int16 weights

Int16 weights in the last layer cost nothing in terms of NN probing code speed but helps much to train model efficiently.

Quantized model was brought to Delphi and showed exactly the same results with python! So my SIMD code in Delphi with AVX512 and AVX2 support works fine and ready for future chess evaluating function.

I was surprised to see how exact the quantized model is!
For example : after 2 epochs of training the float32 Keras model shoed average 'mae' metric (mean absolute error) -1,19
Correponding quantized model in Delphi shoed the same metric on level 1,97. More then promising as for me!

[lithander]

When you say Delphi do you really mean that oldschool pascal-like programming language? I loved that back in the day but after Borland sold it seems to have lost all practical relevance. Why are you using it to build your chess engine? Because you like it personally or are there some distinct advantages I'm not aware of?

[booot]

Yes - i write in pascal. There is only language i know well to write an engine.

[lithander]

Yes - i write in pascal. There is only language i know well to write an engine.

Makes the CCRL Rating even more impressive. Really awesome to hear that good old Delphi is on par with C/C++ there!

[Steve Maughan]

Delphi is doing quite well. It can now target Win32, Win64, MacOS, iOS, Android and Linux. There are even third-party frameworks that target the web. Even after 26 years, there really isn't anything better for creating Windows applications. And while my chess engine is written in C, I use Delphi daily. Here's a link to my mapping application written entirely in Delphi:

https://www.alignmix.com

When it comes to speed the Delphi EXEs are definitely slower than the best C EXEs — I'd estimate 30% slower. Which makes Booot's strength even more impressive. The Delphi roadmap shows that the compiler may get some speed related improvement in the next update (10.5) — fingers crossed.

Steve

When you say Delphi do you really mean that oldschool pascal-like programming language? I loved that back in the day but after Borland sold it seems to have lost all practical relevance. Why are you using it to build your chess engine? Because you like it personally or are there some distinct advantages I'm not aware of?

[booot]

Yes, Steve! Delphi is not so bad as it sounds . And if i will be able to pass all the way to NN eval with SIMD - Booot would be as fast as other C engines : SIMD works the same anywhere!

[booot]

SSSE3 and SS2 support added to Delphi for future NN forward pass. Now everything is ready for the next stage : 'Booot's move'. After some pause i will start to write a procedure , generating zillions of positions, evaluated by Booot. I still do not know exactly what will be my feature schema but i still have some time to think about it.

[Kotlov]

Yes, Steve! Delphi is not so bad as it sounds . And if i will be able to pass all the way to NN eval with SIMD - Booot would be as fast as other C engines : SIMD works the same anywhere!

Мне кажется Делфи (Паскаль) слишком высокоуровневый язык для таких, зависимых от быстродействия, проектов как шахматные движки.
Я сам когда-то начинал с Паскаля... но рано или поздно перешел к парадигме мультиязычности в программировании. Поверьте, это не так сложно как кажется, нужно просто сделать первый шаг.

[booot]

Наверное. Но у меня нет мотивации изучать С - я не программист. Я могу поиграться для себя в нейросети попутно выучив пайтон с фреймворками, но вот целенаправленно учить язык чтобы написать движок... Нет. С другой стороны : я уже реализовал SIMD - они будут в будущем основное время процессора занимать. И им все равно откуда их вызвали.

[booot]

Next stage done!
I wrote self-play FEN generator in Booot. I made reasonably simpler generation method just to continue the whole process. Some details:
1. Booot saves search score (uses Eval function) for each FEN after 6 ply Search.
2. Each game has random number (4-6) of random moves.
3. I try store only quiet positions : not under check, best move = quiet move, small difference between static evaluation and search evaluation.
4. Simple algorithm to avoid FEN doubles.

Booot is slow engine, so generation speed does not make me happy : on 20 cores it gives me about 8-10 Millions Fens per hour. So i need to wait so looooooong to reach 1Billion training set. I still have time to think about configuration of my input feature NN layer.