ncm testing at its best

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Ozymandias
Posts: 1535
Joined: Sun Oct 25, 2009 2:30 am

ncm testing at its best

Post by Ozymandias »

The last SF patch introduces no functional change and yet, it's performance as tested at ncm is more than 12 Elo points behind the previous version. The combined error margins are below 11 Elo points.

Cleanup was detrimental, so... dirt was good?
RubiChess
Posts: 584
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: ncm testing at its best

Post by RubiChess »

People in SF discord discovered that the details of this last NCM test shows some horrible NPS variances/decrease in some workers. So there was probably some load on the machines that influenced the test.

Regards, Andreas
User avatar
Ozymandias
Posts: 1535
Joined: Sun Oct 25, 2009 2:30 am

Re: ncm testing at its best

Post by Ozymandias »

So much for "dedicated servers".
Jouni
Posts: 3286
Joined: Wed Mar 08, 2006 8:15 pm

Re: ncm testing at its best

Post by Jouni »

And what is this "Base NPS" thing? It's always like 1973582. But first NNUE version was 35% slower. Why can't You see any difference?
Jouni