+1, Agreed.connor_mcmonigle wrote: ↑Thu Nov 12, 2020 5:09 pm Both Halogen and Seer are comparatively all original. Both just happen to rely on the "efficiently updatable" idea. They probably shouldn't be lumped into the same category as Komodo+NNUE.
Speculations about NNUE development
Moderators: hgm, Rebel, chrisw
-
- Posts: 1754
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: New engine releases 2020
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 512
- Joined: Tue Sep 29, 2020 4:29 pm
- Location: Dublin, Ireland
- Full name: Madeleine Birchfield
Re: New engine releases 2020
But maybe we might be able to add Ethereal to the list soon.AndrewGrant wrote: ↑Thu Nov 12, 2020 10:17 pm+1, Agreed.connor_mcmonigle wrote: ↑Thu Nov 12, 2020 5:09 pm Both Halogen and Seer are comparatively all original. Both just happen to rely on the "efficiently updatable" idea. They probably shouldn't be lumped into the same category as Komodo+NNUE.
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: New engine releases 2020
Your wild speculations are amusing, stating as fact or high likelihood things you wish to be true. It’s good that things in developer land are generally more friendly. I’ve been encouraging the SF devs to port their trainer to pytorch for a while and been giving them small suggestions in a few areas now that they are on the way. I was afraid they were going to run into a development roadblock without this port, but they are making good progress. I am happy about this.connor_mcmonigle wrote: ↑Thu Nov 12, 2020 5:09 pmYes. They claim to and are very likely using Komodo's games as training data, but this doesn't mean they implemented new training code + made improvements/changes to the network architecture. This is exceedingly improbable imho.Madeleine Birchfield wrote: ↑Thu Nov 12, 2020 10:04 am ...which I interpreted to be referring to the search used in training Dragon (Stockfish's trainer uses Stockfish search), but it could just refer to the fact that Dragon uses the same search as Komodo.
Likely, what they did for training is the same as DKappe has been doing for a while which involves converting separate data obtained from self play games of a different engine into the packed fen format used by the SF trainer. It seems rather likely they didn't even bother swapping out the SF qsearch code used by the trainer.
To then actually run the networks produced by this process in their engine, they presumably got someone to exactly rewrite just the inference code so they could circumvent the GPL restrictions. If this is the case, I would personally like to see the computer Shogi developers who invested a lot of effort into writing the incredibly optimized and clever training code added to the Dragon authors list. They are responsible for the large majority of the work involved in the increase in strength.
Both Halogen and Seer are comparatively all original. Both just happen to rely on the "efficiently updatable" idea. They probably shouldn't be lumped into the same category as Komodo+NNUE.
(Also see Vajolet's NNUE branch)
Just as a note, I’ve been training distilled, endgame and specialist nets in tensorflow and pytorch (and have started to use julia/flux) for several years. These aren’t new concepts to me. It’s my hobby. Don’t assume because you are helpless and out of your depth with regard to training neural nets that others are too.
P.S. On a more useful note, I’ve started using Tord Romstad’s excellent Chess.jl library (https://github.com/romstad/Chess.jl), though it has one major castling bug that I’m working to fix. Pretty speedy for stuff like qsearch.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1754
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: New engine releases 2020
Ill note that you failed to deny the claims.dkappe wrote: ↑Sat Nov 21, 2020 7:34 amYour wild speculations are amusing, stating as fact or high likelihood things you wish to be true. It’s good that things in developer land are generally more friendly. I’ve been encouraging the SF devs to port their trainer to pytorch for a while and been giving them small suggestions in a few areas now that they are on the way. I was afraid they were going to run into a development roadblock without this port, but they are making good progress. I am happy about this.connor_mcmonigle wrote: ↑Thu Nov 12, 2020 5:09 pmYes. They claim to and are very likely using Komodo's games as training data, but this doesn't mean they implemented new training code + made improvements/changes to the network architecture. This is exceedingly improbable imho.Madeleine Birchfield wrote: ↑Thu Nov 12, 2020 10:04 am ...which I interpreted to be referring to the search used in training Dragon (Stockfish's trainer uses Stockfish search), but it could just refer to the fact that Dragon uses the same search as Komodo.
Likely, what they did for training is the same as DKappe has been doing for a while which involves converting separate data obtained from self play games of a different engine into the packed fen format used by the SF trainer. It seems rather likely they didn't even bother swapping out the SF qsearch code used by the trainer.
To then actually run the networks produced by this process in their engine, they presumably got someone to exactly rewrite just the inference code so they could circumvent the GPL restrictions. If this is the case, I would personally like to see the computer Shogi developers who invested a lot of effort into writing the incredibly optimized and clever training code added to the Dragon authors list. They are responsible for the large majority of the work involved in the increase in strength.
Both Halogen and Seer are comparatively all original. Both just happen to rely on the "efficiently updatable" idea. They probably shouldn't be lumped into the same category as Komodo+NNUE.
(Also see Vajolet's NNUE branch)
Just as a note, I’ve been training distilled, endgame and specialist nets in tensorflow and pytorch (and have started to use julia/flux) for several years. These aren’t new concepts to me. It’s my hobby. Don’t assume because you are helpless and out of your depth with regard to training neural nets that others are too.
P.S. On a more useful note, I’ve started using Tord Romstad’s excellent Chess.jl library (https://github.com/romstad/Chess.jl), though it has one major castling bug that I’m working to fix. Pretty speedy for stuff like qsearch.
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: New engine releases 2020
You mean the baseless speculations? Note what you like Andrew, but your rage posts are somewhat tiring.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 532
- Joined: Sun Sep 06, 2020 4:40 am
- Full name: Connor McMonigle
Re: New engine releases 2020
Whoa. I believe you totally misintrepret my words (perhaps a little too much speculation on my part in all fairness) I wish precisely the opposite! I'm very hopeful that the networks used in Komodo NNUE were trained using original training code and a unique network architecture. In fact, a separate forum post describing the training process/unique features of the implementations as well as lessons learned from implementing the training code and inference code from the ground up (is this what you're claiming?) would be much appreciated...dkappe wrote: ↑Sat Nov 21, 2020 7:34 amYour wild speculations are amusing, stating as fact or high likelihood things you wish to be true. It’s good that things in developer land are generally more friendly. I’ve been encouraging the SF devs to port their trainer to pytorch for a while and been giving them small pointer in a few areas now that they are on the way. I was afraid they were going to run into a development roadblock without this, but they are making good progress. I am happy about this.connor_mcmonigle wrote: ↑Thu Nov 12, 2020 5:09 pmYes. They claim to and are very likely using Komodo's games as training data, but this doesn't mean they implemented new training code + made improvements/changes to the network architecture. This is exceedingly improbable imho.Madeleine Birchfield wrote: ↑Thu Nov 12, 2020 10:04 am ...which I interpreted to be referring to the search used in training Dragon (Stockfish's trainer uses Stockfish search), but it could just refer to the fact that Dragon uses the same search as Komodo.
Likely, what they did for training is the same as DKappe has been doing for a while which involves converting separate data obtained from self play games of a different engine into the packed fen format used by the SF trainer. It seems rather likely they didn't even bother swapping out the SF qsearch code used by the trainer.
To then actually run the networks produced by this process in their engine, they presumably got someone to exactly rewrite just the inference code so they could circumvent the GPL restrictions. If this is the case, I would personally like to see the computer Shogi developers who invested a lot of effort into writing the incredibly optimized and clever training code added to the Dragon authors list. They are responsible for the large majority of the work involved in the increase in strength.
Both Halogen and Seer are comparatively all original. Both just happen to rely on the "efficiently updatable" idea. They probably shouldn't be lumped into the same category as Komodo+NNUE.
(Also see Vajolet's NNUE branch)
Just as a note, I’ve been training distilled, endgame and specialist nets in tensorflow and pytorch (and have started to use julia/flux) for several years. These aren’t new concepts to me. It’s my hobby. Don’t assume because you are helpless and out of your depth with regard to training neural nets that others are too.
P.S. On a more useful note, I’ve started using Tord Romstad’s excellent Chess.jl library (https://github.com/romstad/Chess.jl), though it has one major castling bug that I’m working to fix. Pretty speedy for stuff like qsearch.
That said, what's with the personal attacks? Perhaps you interpreted what I had written as an attack? This was certainly not my intention. I'm "not hopelessly out of my depths" when it comes to training neural networks. In fact, this PyTorch NNUE training code you refer to largely originated from the PyTorch training code I wrote for training networks for my personal project (you'll find it referenced in Gary's repository's readme).
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: New engine releases 2020
My apologies. As you no doubt know, there are many very angry people in these forums. I read your post and jumped to an unwarranted conclusion.connor_mcmonigle wrote: ↑Sat Nov 21, 2020 7:55 am
Whoa. I believe you totally misintrepret my words (perhaps a little too much speculation on my part in all fairness) I wish precisely the opposite! I'm very hopeful that the networks used in Komodo NNUE were trained using original training code and a unique network architecture. In fact, a separate forum post describing the training process/unique features of the implementations as well as lessons learned from implementing the training code and inference code from the ground up (is this what you're claiming?) would be much appreciated...
That said, what's with the personal attacks? Perhaps you interpreted what I had written as an attack? This was certainly not my intention. I'm "not hopelessly out of my depths" when it comes to training neural networks. In fact, this PyTorch NNUE training code you refer to largely originated from the PyTorch training code I wrote for training networks for my personal project (you'll find it referenced in Gary's repository's readme).
I’d love to share my experiences with Dragon, but that’s not mine to share. When I return focus back on my personal projects — a0lite, Bad Gyal and Night Nurse — and the tools and techniques I’ve developed, I will as usual make them all public.
Thanks for your work with seer (I assume that’s yours). Without it, work on Stockfish was going to start to get rough.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 1754
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: New engine releases 2020
Rage? What. Also, interesting phrase, "baseless speculations". "baseless accusations" is a thing, but baseless speculations? That is new.
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 532
- Joined: Sun Sep 06, 2020 4:40 am
- Full name: Connor McMonigle
Re: New engine releases 2020
Correct. No worries. I understand that you might not be able to disclose details about the training process. In any case, I've always found your other projects pretty neat and hope to see you make more interesting contributions to computer chess going forwards.
However, hopefully it's reasonable to suggest that if the Shogi Developers' training code was directly used to produce a network for Dragon (not that this is wrong by any means), the Shogi developers be added to the author list. Again, completely ignore this if it's not the case
-
- Posts: 4606
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: New engine releases 2020
I seems this thread was hijacked for speculations about nnue.
(especially Komodo ones, which never was a matter in this thread before at all and shouldn't,
as I never announce commercial releases).
I suggest to spllt that part away from the original thread.
Somehow it started with some dropping in by 'Madeleine'.
Guenther
(especially Komodo ones, which never was a matter in this thread before at all and shouldn't,
as I never announce commercial releases).
I suggest to spllt that part away from the original thread.
Somehow it started with some dropping in by 'Madeleine'.
Guenther