Fat Fritz 2

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
pohl4711
Posts: 2444
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Fat Fritz 2

Post by pohl4711 »

Gabor Szots wrote: Sat Feb 13, 2021 2:04 pm
pohl4711 wrote: Sat Feb 13, 2021 11:45 amI looked into both nets with a HEX-editor and they are definitly different!
How did you do that? The official release has no separate net, it's embedded.
There is an updated version of FF2 (called update1) (source code on github), where the net is a separate file. Because the first release with the nnue-net being part of the binaries was a gpl license violation.
This updated version of FF2 can use the commercial net or the weaker net from the github site of FF2.
Last edited by pohl4711 on Sat Feb 13, 2021 4:37 pm, edited 2 times in total.
User avatar
pohl4711
Posts: 2444
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Fat Fritz 2

Post by pohl4711 »

Sylwy wrote: Sat Feb 13, 2021 3:41 pm
Graham Banks wrote: Sat Feb 13, 2021 3:21 pm
Sylwy wrote: Sat Feb 13, 2021 12:04 pm :lol: This means that CCRL also tested an old version of the 20x512 net! :lol:

Image

FatFritz 2 x64.................3634 (493 games)
Stockfish 12 x64..............3631 (2699 games)
No. We tested the official release version.
Thanks ! This means that the net in the archive uploaded by Mr. Michael Taktikos is the correct one. My test is evolving similarly to the version tested by CCRL.
No. Only the commercial version of FF2 has the strong FF2 net. The ratinglists (CEGT, CCRL and me (SPCC)) got this version for testing. The net, which can be downloaded on the FF2-github website is weaker. Of course. Otherwise nobody would buy FF2 anymore.
twobeer
Posts: 20
Joined: Sat Feb 22, 2020 6:43 pm
Full name: Leif Aronsson

Re: Fat Fritz 2

Post by twobeer »

pohl4711 wrote: Sat Feb 13, 2021 4:32 pm No. Only the commercial version of FF2 has the strong FF2 net. The ratinglists (CEGT, CCRL and me (SPCC)) got this version for testing. The net, which can be downloaded on the FF2-github website is weaker. Of course. Otherwise nobody would buy FF2 anymore.
On what test have you confirmed it is "weaker"?... I seriously doubt they would produce a "false" additional net just to publish on github.. That would get them into more problems GPLv3 Wise if they try murky stuff like that.

Can you include test with this net for refernce? or is it just an assumption on your part that it is weaker and not equal...

Do you have both bin files? can you post the MD5 checksums? whould be good refrence with the checksum.. How do we even now that SPCC got the same net as CEGT, CCRL etc.. If those are "hand-picked" by mr. A.Silver..
Werewolf
Posts: 1797
Joined: Thu Sep 18, 2008 10:24 pm

Re: Fat Fritz 2

Post by Werewolf »

Can someone comment on this:

At 31:30 Albert S clearly says Fat Fritz 2 is "genuinely stronger than what's out there right now, without a question and clearly so" And SF Dev is referenced saying Fat Fritz 2 is measurably stronger.

I'm just not seeing this at all on my tests, and I used SF Dev which was released the same day as FF2 came out.

https://en.chessbase.com/post/interview ... -fritz-2-0

I can only think of 4 possible explanations:

a) My test results are inaccurate (Test run ongoing, only 500 games so far, but they are in line with others.)
b) the test criteria is very specific on a large number of cores (I'm testing single-threaded on Cutechess for time's sake)
c) the video is old, perhaps around Christmas time, and doesn't allow for SF Dev improvement
d) a more sinister conclusion.

Can someone comment as I'd prefer to give the benefit of the doubt if possible, I just can't replicate this result.
twobeer
Posts: 20
Joined: Sat Feb 22, 2020 6:43 pm
Full name: Leif Aronsson

Re: Fat Fritz 2

Post by twobeer »

Werewolf wrote: Sat Feb 13, 2021 5:12 pm Can someone comment as I'd prefer to give the benefit of the doubt if possible, I just can't replicate this result.
Option d Obviously ...
gaard
Posts: 447
Joined: Mon Jun 07, 2010 3:13 am
Location: Holland, MI
Full name: Martin W

Re: Fat Fritz 2

Post by gaard »

Werewolf wrote: Sat Feb 13, 2021 5:12 pm Can someone comment on this:

At 31:30 Albert S clearly says Fat Fritz 2 is "genuinely stronger than what's out there right now, without a question and clearly so" And SF Dev is referenced saying Fat Fritz 2 is measurably stronger.

I'm just not seeing this at all on my tests, and I used SF Dev which was released the same day as FF2 came out.

https://en.chessbase.com/post/interview ... -fritz-2-0

I can only think of 4 possible explanations:

a) My test results are inaccurate (Test run ongoing, only 500 games so far, but they are in line with others.)
b) the test criteria is very specific on a large number of cores (I'm testing single-threaded on Cutechess for time's sake)
c) the video is old, perhaps around Christmas time, and doesn't allow for SF Dev improvement
d) a more sinister conclusion.

Can someone comment as I'd prefer to give the benefit of the doubt if possible, I just can't replicate this result.
http://talkchess.com/forum3/viewtopic.p ... 09#p882009

"genuinely stronger" is about as honest as him saying that starting from scratch with FF2 was torture. FF2 performed less than 3 Elo better against SF12 than a SF-dev version I tested against SF12 two months ago.
User avatar
Ozymandias
Posts: 1535
Joined: Sun Oct 25, 2009 2:30 am

Re: Fat Fritz 2

Post by Ozymandias »

Werewolf wrote: Sat Feb 13, 2021 5:12 pm Can someone comment on this:

At 31:30 Albert S clearly says Fat Fritz 2 is "genuinely stronger than what's out there right now, without a question and clearly so" And SF Dev is referenced saying Fat Fritz 2 is measurably stronger.

I'm just not seeing this at all on my tests, and I used SF Dev which was released the same day as FF2 came out.

https://en.chessbase.com/post/interview ... -fritz-2-0

I can only think of 4 possible explanations:

a) My test results are inaccurate (Test run ongoing, only 500 games so far, but they are in line with others.)
b) the test criteria is very specific on a large number of cores (I'm testing single-threaded on Cutechess for time's sake)
c) the video is old, perhaps around Christmas time, and doesn't allow for SF Dev improvement
d) a more sinister conclusion.

Can someone comment as I'd prefer to give the benefit of the doubt if possible, I just can't replicate this result.
He's not providing any data to sustain his claim, that should give you a good idea as to where the wind is blowing.
nio
Posts: 6
Joined: Sun Jan 17, 2021 7:27 pm
Full name: Adi Fischet

Re: Fat Fritz 2

Post by nio »

He also said evaluation is changed to old fat fritz style. I cant find that in the source.
User avatar
Ozymandias
Posts: 1535
Joined: Sun Oct 25, 2009 2:30 am

Re: Fat Fritz 2

Post by Ozymandias »

He's just talking about the use of mcts/nn data rather than SF's nnue, but don't worry, he's been saying questionable things as far back as the original AlphaZero article he wrote for ChessBase:
A new paradigm

On December 5 the DeepMind group published a new paper at the site of Cornell University called "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm", and the results were nothing short of staggering. AlphaZero had done more than just master the game, it had attained new heights in ways considered inconceivable. The test is in the pudding of course, so before going into some of the fascinating nitty-gritty details, let’s cut to the chase. It played a match against the latest and greatest version of Stockfish, and won by an incredible score of 64 : 36, and not only that, AlphaZero had zero losses (28 wins and 72 draws)!

Stockfish needs no introduction to ChessBase readers, but it's worth noting that the program was on a computer that was running nearly 900 times faster! Indeed, AlphaZero was calculating roughly 80 thousand positions per second, while Stockfish, running on a PC with 64 threads (likely a 32-core machine) was running at 70 million positions per second. To better understand how big a deficit that is, if another version of Stockfish were to run 900 times slower, this would be equivalent to roughly 8 moves less deep. How is this possible?
The amount of BS in these two paragraphs alone are enough for a whole thread, and he didn't even have any other stake in the matter other than having the article published.
User avatar
pohl4711
Posts: 2444
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Fat Fritz 2

Post by pohl4711 »

twobeer wrote: Sat Feb 13, 2021 4:53 pm
pohl4711 wrote: Sat Feb 13, 2021 4:32 pm No. Only the commercial version of FF2 has the strong FF2 net. The ratinglists (CEGT, CCRL and me (SPCC)) got this version for testing. The net, which can be downloaded on the FF2-github website is weaker. Of course. Otherwise nobody would buy FF2 anymore.
On what test have you confirmed it is "weaker"?... I seriously doubt they would produce a "false" additional net just to publish on github.. That would get them into more problems GPLv3 Wise if they try murky stuff like that.

Can you include test with this net for refernce? or is it just an assumption on your part that it is weaker and not equal...

Do you have both bin files? can you post the MD5 checksums? whould be good refrence with the checksum.. How do we even now that SPCC got the same net as CEGT, CCRL etc.. If those are "hand-picked" by mr. A.Silver..
A.Silver said, that the github net is weaker. And it is obvious, that this must be true. If not, nobody needed to buy FF2, so it would be crazy, if Chessbase would provide the strong commercial net on github for free.
I have both nets,,of course, the commercial net and the github net. They are definitly different. As I said before, the bench is different, if FF2 uses one or the other net. And I looked into them with a HEX Editor. The nets are different.
A 3000 games testrun is running on one of my machines (a bullet RoundRobin with FF2 commercial, FF2 with free github net and Stockfish 210111 (the StockfishDev, FF2 was forked of)) - result in 2-3 days on my website.
The net is not part of the Stockfish sourcecode, so it is not under gpl, since the net is a separate file. Only the FF2 sourcecode is under gpl. Thats why the FF2 update separated the net from the binary.