Stockfish with new NNUE architecture and bigger net released

[Dann Corbit]

Well stated, Dann!

I see the Stockfish developers' latest work as validation that Albert Silver first had the correct methodology for NNUE architecture and net training. Chess engine progress is being made, and it's being done by following Albert's approach.

Bigger really is better.

All the best,
-Steve-

You do realize that HalfKaV2 is very different from Halfkp right?
Can you point out exactly where "Albert's methodology" helped make this possible.

Here's an image, edit it by pointing at the parts influenced by Albert.




I'll help you, Albert didn't influence anything at all.

You're right. This new try does not seem nearly so successful.

[Madeleine Birchfield]

Bigger nets are nothing new; jjoshua (Stein net trainer) was training 30x320 sized nets for nodchip's Stockfish NNUE back before it was merged into official stockfish in August of last year. I asked the Stockfish team around the same time in August whether they should use a net size larger than 20x256 after the merger with official stockfish, but they said that they were going to stick with 20x256 until improvement plateaus, before they plan on moving to larger nets.

Those who credit Albert Silver for this switch to larger nets have cause and effect reversed; it is Albert Silver who should be crediting the various Stockfish net trainers of the past for Fat Fritz, like jjoshua, Raphexion, Joerg Oster, dkappe, amongst others, for showing the way forwards in training larger nets. But we already know how Albert and Chessbase have behaved regarding the commercial Stockfish derivative...

[connor_mcmonigle]

...
You're right. This new try does not seem nearly so successful.

Surely you can't be serious, unless you're defining "success" in some nonsensical way. I don't see how any rational person could come to your conclusion given the facts. Hopefully you're confused rather than arguing in bad faith. This network is almost certainly stronger than the FF2 network. In fact, the FF2 network proved inferior to even the SF13 network H2H in all statistically significant independent tests I've seen. The FF2 network wouldn't have even passed FishTest. I think you pointed to CEGT showing FF2 ahead of SF 13 earlier, but the relative elo is well within error bars (NVM the fact that SF13 and SF12 won H2H there too, though the sample size is tiny).

As many others have pointed out and you're neglecting to acknowledge, Albert wasn't the first to try training wider networks. He didn't even succeed in demonstrating the superiority of larger networks. Albert trained a fairly strong network using the available tools, but this likely isn't even close to as difficult as you're imagining.

[Dann Corbit]

...
You're right. This new try does not seem nearly so successful.

Surely you can't be serious, unless you're defining "success" in some nonsensical way. I don't see how any rational person could come to your conclusion given the facts. Hopefully you're confused rather than arguing in bad faith. This network is almost certainly stronger than the FF2 network. In fact, the FF2 network proved inferior to even the SF13 network H2H in all statistically significant independent tests I've seen. The FF2 network wouldn't have even passed FishTest. I think you pointed to CEGT showing FF2 ahead of SF 13 earlier, but the relative elo is well within error bars (NVM the fact that SF13 and SF12 won H2H there too, though the sample size is tiny).

As many others have pointed out and you're neglecting to acknowledge, Albert wasn't the first to try training wider networks. He didn't even succeed in demonstrating the superiority of larger networks. Albert trained a fairly strong network using the available tools, but this likely isn't even close to as difficult as you're imagining.

Of course, I am trolling. But I am also trying to make a point. I think that Albert is being vilified in an unfair and (dare I say silly?) way. People are mad because he made money on a GPL project, and because he was proud of his work. There is nothing wrong with making money on a GPL project or of being proud of your work. He spent many long hours of work and many thousands of dollars of his own money on the project, doing something he thought was interesting enough to pursue. Now, I would like you to consider some facts:
The net and training Albert did with FF2.1 beats the SF from the time of his fork. The SF team has an ARMY of programmers and a MOUNTAIN of hardware at their beck and call that pound away day and night. Any commercial programmer has to compete against "free hardware" "free labor" and many of those SF programmers are really brilliant.

Sure, others talked about bigger nets before Albert did it, including me, but this was the first public example of the NNUE net attempted at double size which makes it both clever and interesting.

Something else of interest. I have about 1000 EPD positions that SF face-plants on. I tried LC0 on 100 of them at one hour per position. It solves less than 20% (not great, but a lot better than 0% -- and to be fair, despite the huge statistical data pile that tells me SF is wrong, surely some percent of the time SF will be right and the results are simply slanted). The solve rate for FF2.1 is much better than either (more than twice as good, but my experiment cut short at 70 positions when I should do the full thousand because I have other machine needs and promises to fulfill, and therefore even the 100 LC0 positions analyzed are a doubtful result). Now, it is an unfair test. The positions are chosen from a database where SF solved around a billion positions with years of calculations and dozens of machines participating around the clock. The 1000 not solved are therefore an insignificant drip in an ocean of data. But the high solve rate by FF2.1 is still interesting.

The belittling of Albert Silver makes me ashamed of the chess community and of being a part of it. Albert Silver has literally done nothing wrong, so far as I can see. Did he pridefully overestimate his contribution? Good thing none of us have ever done that, or we would be hypocrites.

There is something wrong with the way the computer chess community jumps down people's throats. Sometimes it is justified (there are blatant ripoffs that have been exposed and those people really are criminals). But {for example} the treatment given to Vas was WAY over the top harsh and the treatment given to Albert Silver is simply wrong, I would go so far as to say disgusting.
IMO-YMMV

[Raphexon]

...
You're right. This new try does not seem nearly so successful.

Surely you can't be serious, unless you're defining "success" in some nonsensical way. I don't see how any rational person could come to your conclusion given the facts. Hopefully you're confused rather than arguing in bad faith. This network is almost certainly stronger than the FF2 network. In fact, the FF2 network proved inferior to even the SF13 network H2H in all statistically significant independent tests I've seen. The FF2 network wouldn't have even passed FishTest. I think you pointed to CEGT showing FF2 ahead of SF 13 earlier, but the relative elo is well within error bars (NVM the fact that SF13 and SF12 won H2H there too, though the sample size is tiny).

As many others have pointed out and you're neglecting to acknowledge, Albert wasn't the first to try training wider networks. He didn't even succeed in demonstrating the superiority of larger networks. Albert trained a fairly strong network using the available tools, but this likely isn't even close to as difficult as you're imagining.

Of course, I am trolling. But I am also trying to make a point. I think that Albert is being vilified in an unfair and (dare I say silly?) way.

People are mad because he made false claims, tried to paint himself as an innovator and didn't abide by the GPL license.
He also made the claim of doing it on his own while also having asked SF-devs for help numerous times. If any real innovation happened, it's his programmer friends that did it. (Daniel Uranga and Dkappe)

this was the first public example of the NNUE net attempted at double size which makes it both clever and interesting.

It wasn't.

[Dann Corbit]

...
You're right. This new try does not seem nearly so successful.

Surely you can't be serious, unless you're defining "success" in some nonsensical way. I don't see how any rational person could come to your conclusion given the facts. Hopefully you're confused rather than arguing in bad faith. This network is almost certainly stronger than the FF2 network. In fact, the FF2 network proved inferior to even the SF13 network H2H in all statistically significant independent tests I've seen. The FF2 network wouldn't have even passed FishTest. I think you pointed to CEGT showing FF2 ahead of SF 13 earlier, but the relative elo is well within error bars (NVM the fact that SF13 and SF12 won H2H there too, though the sample size is tiny).

As many others have pointed out and you're neglecting to acknowledge, Albert wasn't the first to try training wider networks. He didn't even succeed in demonstrating the superiority of larger networks. Albert trained a fairly strong network using the available tools, but this likely isn't even close to as difficult as you're imagining.

Of course, I am trolling. But I am also trying to make a point. I think that Albert is being vilified in an unfair and (dare I say silly?) way.

People are mad because he made false claims, tried to paint himself as an innovator and didn't abide by the GPL license.
He also made the claim of doing it on his own while also having asked SF-devs for help numerous times. If any real innovation happened, it's his programmer friends that did it. (Daniel Uranga and Dkappe)

this was the first public example of the NNUE net attempted at double size which makes it both clever and interesting.

It wasn't.

Oh, the other famous and successful double sized net.
What was that again?

I resign as moderator from this forum.
It's time for me to take a break.
Maybe some photography.

[M ANSARI]

...
You're right. This new try does not seem nearly so successful.

Surely you can't be serious, unless you're defining "success" in some nonsensical way. I don't see how any rational person could come to your conclusion given the facts. Hopefully you're confused rather than arguing in bad faith. This network is almost certainly stronger than the FF2 network. In fact, the FF2 network proved inferior to even the SF13 network H2H in all statistically significant independent tests I've seen. The FF2 network wouldn't have even passed FishTest. I think you pointed to CEGT showing FF2 ahead of SF 13 earlier, but the relative elo is well within error bars (NVM the fact that SF13 and SF12 won H2H there too, though the sample size is tiny).

As many others have pointed out and you're neglecting to acknowledge, Albert wasn't the first to try training wider networks. He didn't even succeed in demonstrating the superiority of larger networks. Albert trained a fairly strong network using the available tools, but this likely isn't even close to as difficult as you're imagining.

Of course, I am trolling. But I am also trying to make a point. I think that Albert is being vilified in an unfair and (dare I say silly?) way. People are mad because he made money on a GPL project, and because he was proud of his work. There is nothing wrong with making money on a GPL project or of being proud of your work. He spent many long hours of work and many thousands of dollars of his own money on the project, doing something he thought was interesting enough to pursue. Now, I would like you to consider some facts:
The net and training Albert did with FF2.1 beats the SF from the time of his fork. The SF team has an ARMY of programmers and a MOUNTAIN of hardware at their beck and call that pound away day and night. Any commercial programmer has to compete against "free hardware" "free labor" and many of those SF programmers are really brilliant.

Sure, others talked about bigger nets before Albert did it, including me, but this was the first public example of the NNUE net attempted at double size which makes it both clever and interesting.

Something else of interest. I have about 1000 EPD positions that SF face-plants on. I tried LC0 on 100 of them at one hour per position. It solves less than 20% (not great, but a lot better than 0% -- and to be fair, despite the huge statistical data pile that tells me SF is wrong, surely some percent of the time SF will be right and the results are simply slanted). The solve rate for FF2.1 is much better than either (more than twice as good, but my experiment cut short at 70 positions when I should do the full thousand because I have other machine needs and promises to fulfill, and therefore even the 100 LC0 positions analyzed are a doubtful result). Now, it is an unfair test. The positions are chosen from a database where SF solved around a billion positions with years of calculations and dozens of machines participating around the clock. The 1000 not solved are therefore an insignificant drip in an ocean of data. But the high solve rate by FF2.1 is still interesting.

The belittling of Albert Silver makes me ashamed of the chess community and of being a part of it. Albert Silver has literally done nothing wrong, so far as I can see. Did he pridefully overestimate his contribution? Good thing none of us have ever done that, or we would be hypocrites.

There is something wrong with the way the computer chess community jumps down people's throats. Sometimes it is justified (there are blatant ripoffs that have been exposed and those people really are criminals). But {for example} the treatment given to Vas was WAY over the top harsh and the treatment given to Albert Silver is simply wrong, I would go so far as to say disgusting.
IMO-YMMV

Well said ... I agree 100% !!! I really don't get the incredible hate that comes out of some people for really ridiculous issues.

[Modern Times]

The position with this new net is less clear on chess960. I ran a match with all 960 positions reversed sides, 1CPU 40/2 repeating time control, and got this:



Score of Stockfish 22-May-2021 64-bit vs Stockfish 19-Mar-2021 64-bit: 173 - 175 - 1572 [0.499]
Elo difference: -0.4 +/- 6.6, LOS: 45.7 %, DrawRatio: 81.9 %

1920 of 1920 games finished.


So it is a tie more or less.

[jr66]

Oh, the other famous and successful double sized net.
What was that again?

I resign as moderator from this forum.
It's time for me to take a break.
Maybe some photography.

Or perhaps fishing
You will have a.....stock of fish !

[connor_mcmonigle]

...
Of course, I am trolling. But I am also trying to make a point. I think that Albert is being vilified in an unfair and (dare I say silly?) way. People are mad because he made money on a GPL project, and because he was proud of his work. There is nothing wrong with making money on a GPL project or of being proud of your work. He spent many long hours of work and many thousands of dollars of his own money on the project, doing something he thought was interesting enough to pursue. Now, I would like you to consider some facts:
The net and training Albert did with FF2.1 beats the SF from the time of his fork. The SF team has an ARMY of programmers and a MOUNTAIN of hardware at their beck and call that pound away day and night. Any commercial programmer has to compete against "free hardware" "free labor" and many of those SF programmers are really brilliant.

Sure, others talked about bigger nets before Albert did it, including me, but this was the first public example of the NNUE net attempted at double size which makes it both clever and interesting.

Something else of interest. I have about 1000 EPD positions that SF face-plants on. I tried LC0 on 100 of them at one hour per position. It solves less than 20% (not great, but a lot better than 0% -- and to be fair, despite the huge statistical data pile that tells me SF is wrong, surely some percent of the time SF will be right and the results are simply slanted). The solve rate for FF2.1 is much better than either (more than twice as good, but my experiment cut short at 70 positions when I should do the full thousand because I have other machine needs and promises to fulfill, and therefore even the 100 LC0 positions analyzed are a doubtful result). Now, it is an unfair test. The positions are chosen from a database where SF solved around a billion positions with years of calculations and dozens of machines participating around the clock. The 1000 not solved are therefore an insignificant drip in an ocean of data. But the high solve rate by FF2.1 is still interesting.

The belittling of Albert Silver makes me ashamed of the chess community and of being a part of it. Albert Silver has literally done nothing wrong, so far as I can see. Did he pridefully overestimate his contribution? Good thing none of us have ever done that, or we would be hypocrites.

There is something wrong with the way the computer chess community jumps down people's throats. Sometimes it is justified (there are blatant ripoffs that have been exposed and those people really are criminals). But {for example} the treatment given to Vas was WAY over the top harsh and the treatment given to Albert Silver is simply wrong, I would go so far as to say disgusting.
IMO-YMMV

Sorry, it was unclear to me that you were trolling which I guess is as much a reflection of the state of discussion on this forum as it is a personal failing. This was unexpected, originating from a moderator.

I agree that there is nothing inherently wrong with making money on a GPL project and I believe that the majority of people taking issue with ChessBase's actions in relation to FF2 would agree with this statement. I will point out that the FF2 network did not, in fact, beat the existing Stockfish network at the time of the fork. The FF2 network, while quite strong, was likely weaker than dozens of networks already trained by individuals working on the Stockfish project at the time of its release. Again, Albert trained a fairly strong network using the available tools, but this isn't a particularly noteworthy accomplishment. You seem to view this as some sort of David vs. Goliath story where our hero, Albert, surprisingly overcomes the Stockfish army. This is a pretty backwards way to view the situation as this "Stockfish army" you refer to is more accurately characterized as a small team of hard working developers actively donating their spare time to provide you with the best possible chess engine completely free of charge ("free labor" -> donated labor, "free hardware" -> donated hardware).

For the final time, Albert was not the first to attempt to train a double sized network nor was the sale of FF2 the first time such a network was released to the public. Doubling a layer's width is neither clever nor interesting to anyone with any understanding of this subject, but ChessBase wasn't even the first to sell such a product. Do you know why I'm so convinced that ChessBase could have profited by selling a network for Stockfish without any backlash? Because "JJoshua" has been selling his double sized "StockfiNN" networks via his Patreon page far prior to ChessBase's selling the FF2 network.

Pehaps you're not interested in listening, but people find ChessBase's actions in relation to FF2 problematic specifically because ChessBase marketed FF2 as a chess engine rather than a network and obscured its connection to Stockfish. ChessBase effectively took advantage of low information consumers by selling them an extremely expensive product which differed insignificantly by any reasonable measure from Stockfish 13. The consumers are the victims in this story. What ChessBase did was analogous to collecting a bunch of free food from a food bank and then selling the collected food for extortionist prices on the street in front of said food bank.


I don't think people are trying to belittle Albert so much as correct misconceptions about the nature of his accomplishments. You even write that he/ChessBase overestimates his own contributions. The only natural response is for the community to correct that overestimation which shouldn't be interpreted as belittling. I've not seen any disgusting personal attacks on Albert, but as a moderator, you would know better than I would. I'd agree that Vas's treatment was completely unjust. I hope it's clear that I, and many others, view ChessBase as the villain here and Albert the villain only insofar as ChessBase's actions can be attributed to him. He claims to have had little influence on ChessBase's marketing team and I don't see any reason not to believe him.