With the transparent memory access that AMD is working on (and also having the CPU read and write HBM memory transparently) I think that rather than replacements, we will see hybrids. Perhaps the only hybridization will be a shared hash, but even that would be awfully nice. Imagine if every place where LC0 was stronger was written to the hash and Sf read it, and every place where LC0 was weak was written to the hash and LC0 read it and there was no need to copy to and from video memory because we can just *use it directly* what sort of chess that would produce. I think it would be revolutionary. I also think that an AND/OR proof search using GPUs searching for checkmates would be incredible. I guess in ten years we will see chess hardware/software combinations that would slap the current software and hardware silly.
Wonderful simile, by the way.
BrainLearn
Moderators: hgm, Rebel, chrisw
-
- Posts: 12541
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: BrainLearn
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 1080
- Joined: Fri Sep 16, 2016 6:55 pm
- Location: USA/Minnesota
- Full name: Leo Anger
Re: BrainLearn
What is Q-learning? I never heard of it before.amchess wrote: ↑Wed Jul 01, 2020 3:16 pmIf InfinityChess gui can't even show the uci option, it's clear it's a gui problem, not the engine.Zerbinati wrote: ↑Wed Jul 01, 2020 3:00 pmAndrea I refer to the standard modeamchess wrote: ↑Wed Jul 01, 2020 2:46 pmEman does'n't have Q-learning.Zerbinati wrote: ↑Wed Jul 01, 2020 2:16 pm @amchess I rewrite it in more detail because maybe you haven't seen it
For information however:
in the infinity gui neither of the 2 engines works
an empty file is created in the installation folder of the gui and not the engine
The uci commands to activate the type of learning of Shashchess are not displayed in the options
Eman successfully creates the BIN file
both on Playchess and on Infinity
You are both programmers
I hope you will solve it
Even in linux, for example, Arena doesn't display an uci option ("Petrosian").
Conversely, in windows, it works perfectly.
So, this is a gui bug, not the engine and this is never the first time.
For example, chessbase updates continually their guis also to solve this kind of problems.
Anyway, we can show you, after a match play in Fritz gui, experience files are created.
Sorry, but we can't do more than this.
Finally, as we wrote in the readme file, Q-learning is for selfplay: why to use it in playchess or infinitychess?
this problem was also present in version 11
I changed the code and deleted the new version drop down menu
have the possibility to compile a version with the standard default learning activated
in fritz gui it works well
while in the infinity GUI it creates an empty BIN file
also by installing the engine in the GUI installation folder
So, sorry, but we can't help you.
Advanced Micro Devices fan.
-
- Posts: 328
- Joined: Tue Dec 05, 2017 2:42 pm
-
- Posts: 1080
- Joined: Fri Sep 16, 2016 6:55 pm
- Location: USA/Minnesota
- Full name: Leo Anger
Re: BrainLearn
Thanks. Ill read it.
Advanced Micro Devices fan.
-
- Posts: 3186
- Joined: Sat Feb 16, 2008 7:38 am
- Full name: Peter Martan
Re: BrainLearn
Thanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
Last edited by peter on Fri Jul 03, 2020 9:19 pm, edited 2 times in total.
Peter.
-
- Posts: 365
- Joined: Mon May 14, 2007 8:20 pm
- Full name: Boban Stanojević
Re: BrainLearn
Great comments, Uly and Dann!
My hope too is that soon we will learn to combine both approaches, and have a better understanding of the knowledge brought by NNs. A NN that could be incrementally trained -- just like the experience.bin of BrainLearn -- would be an excellent idea. Every engine could become unique that way, play some positions better, etc. It could be very interesting.
My hope too is that soon we will learn to combine both approaches, and have a better understanding of the knowledge brought by NNs. A NN that could be incrementally trained -- just like the experience.bin of BrainLearn -- would be an excellent idea. Every engine could become unique that way, play some positions better, etc. It could be very interesting.
-
- Posts: 328
- Joined: Tue Dec 05, 2017 2:42 pm
Re: BrainLearn
The best is to use Q-learning for self play.peter wrote: ↑Fri Jul 03, 2020 9:14 pmThanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
There are some principles similar to NN-network in Q-learning, but the programming paradigm is different.
Q-learning bin files don't have depth information compared to the standard, but a more refined score, based on a gameLine vector.
NNUE-learning uses an neural network.
-
- Posts: 3186
- Joined: Sat Feb 16, 2008 7:38 am
- Full name: Peter Martan
Re: BrainLearn
You mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?amchess wrote: ↑Fri Jul 03, 2020 9:28 pmThe best is to use Q-learning for self play.peter wrote: ↑Fri Jul 03, 2020 9:14 pmThanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?
Peter.
-
- Posts: 328
- Joined: Tue Dec 05, 2017 2:42 pm
Re: BrainLearn
I mean two times the same engine with different names in the GUI and each one adressing an experience.bin of it's own for self play.peter wrote: ↑Fri Jul 03, 2020 9:35 pmYou mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?amchess wrote: ↑Fri Jul 03, 2020 9:28 pmThe best is to use Q-learning for self play.peter wrote: ↑Fri Jul 03, 2020 9:14 pmThanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?
-
- Posts: 3186
- Joined: Sat Feb 16, 2008 7:38 am
- Full name: Peter Martan
Re: BrainLearn
Ok, thanks again, will give it a try.amchess wrote: ↑Fri Jul 03, 2020 9:41 pmI mean two times the same engine with different names in the GUI and each one adressing an experience.bin of it's own for self play.peter wrote: ↑Fri Jul 03, 2020 9:35 pmYou mean define two times same engine (e.g.BrainLearn 12) with different names in the GUI and each one adressing an experience.bin of it's own?amchess wrote: ↑Fri Jul 03, 2020 9:28 pmThe best is to use Q-learning for self play.peter wrote: ↑Fri Jul 03, 2020 9:14 pmThanks for the link, Andrea, interesting stuff.
If I may ask another question again, what does Q-learning do with existing experience- files generated by "Standard" option in new BrainLearn and ShashChess? Overwrite old entries or change evals stored?
Is it better to create experience.bin- files of their own by Q-learning only and merge them with standard experience or is it ok to use the same for both kinds of learning?
Sorry for lack of fundamental knowledge about it still.
Can Q-learning be compared to NNUE- learning?
After the merge of the "two same engines" into one, you can start from this to play against another engine in standard way.
So copy the .exe into two directories with storing experience in each one separately to merge those two then?
Or adress same experience.bin by same .exe with only two different names of .eng- files for the GUI to organize engine- engine- tournament between two times same .exe?
Peter.