BrainLearn

[Dann Corbit]

With the transparent memory access that AMD is working on (and also having the CPU read and write HBM memory transparently) I think that rather than replacements, we will see hybrids. Perhaps the only hybridization will be a shared hash, but even that would be awfully nice. Imagine if every place where LC0 was stronger was written to the hash and Sf read it, and every place where LC0 was weak was written to the hash and LC0 read it and there was no need to copy to and from video memory because we can just *use it directly* what sort of chess that would produce. I think it would be revolutionary. I also think that an AND/OR proof search using GPUs searching for checkmates would be incredible. I guess in ten years we will see chess hardware/software combinations that would slap the current software and hardware silly.

Wonderful simile, by the way.

[Leo]

@amchess I rewrite it in more detail because maybe you haven't seen it

For information however:
in the infinity gui neither of the 2 engines works
an empty file is created in the installation folder of the gui and not the engine
The uci commands to activate the type of learning of Shashchess are not displayed in the options
Eman successfully creates the BIN file
both on Playchess and on Infinity
You are both programmers
I hope you will solve it

Eman does'n't have Q-learning.
Even in linux, for example, Arena doesn't display an uci option ("Petrosian").
Conversely, in windows, it works perfectly.
So, this is a gui bug, not the engine and this is never the first time.
For example, chessbase updates continually their guis also to solve this kind of problems.
Anyway, we can show you, after a match play in Fritz gui, experience files are created.
Sorry, but we can't do more than this.
Finally, as we wrote in the readme file, Q-learning is for selfplay: why to use it in playchess or infinitychess?

Andrea I refer to the standard mode
this problem was also present in version 11
I changed the code and deleted the new version drop down menu
have the possibility to compile a version with the standard default learning activated
in fritz gui it works well
while in the infinity GUI it creates an empty BIN file
also by installing the engine in the GUI installation folder

If InfinityChess gui can't even show the uci option, it's clear it's a gui problem, not the engine.
So, sorry, but we can't help you.

What is Q-learning? I never heard of it before.

[amchess]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

[Leo]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Thanks. Ill read it.

[peter]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?

[matejst]

Great comments, Uly and Dann!

My hope too is that soon we will learn to combine both approaches, and have a better understanding of the knowledge brought by NNs. A NN that could be incrementally trained -- just like the experience.bin of BrainLearn -- would be an excellent idea. Every engine could become unique that way, play some positions better, etc. It could be very interesting.

[amchess]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?

The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
There are some principles similar to NN-network in Q-learning, but the programming paradigm is different.
Q-learning bin files don't have depth information compared to the standard, but a more refined score, based on a gameLine vector.
NNUE-learning uses an neural network.

[peter]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?

The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.

You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?

[amchess]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?

The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.

You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?

I mean two times the same engine with different names in the GUI and each one adressing an experience.bin of it's own for self play.

[peter]

[What is Q-learning? I never heard of it before.

https://groups.google.com/forum/#!topic ... IzmiSCB8lg

Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?

The best is to use Q-learning for self play.
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.

You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?

I mean two times the same engine with different names in the GUI and each one adressing an experience.bin of it's own for self play.

Ok, thanks again, will give it a try.