Is Fruit strength related to NPS?

[sedicla]

I have been testing my engine, Tucano, against Fruit for a while.

I know fruit is very well written and well tuned, and even with many improvements tucano is still unable to beat it. I know I have a lot of work to do.

So I was wondering if ONE of the strengths of fruit is its nodes per second. I noticed my engine is doing sometimes half of fruit's NPS. Usually it does 75% NPS.

To be honest I never paid much attention to NPS in my engine, as I said I have lot of work to do, but I guess should also give NPS some attention.

What you guys think?

[kbhearn]

Unless your NPS is absolutely horrid, it's probably not that big a deal. I mean, if you see something that's an 'obvious' speed up and not too much work, there's no reason not to try it - but the large number of little performance tweaks needed to double nps would probably be better spent on shrinking the tree size.

[Steve Maughan]

Hi Alcides,

I think Fruit's strength comes from being extremely well written (bug free), minimalist (doing what counts), and doing it well. It isn't a super fast engine or a super slow engine, but I'm quite sure speed isn't its secret.

Steve

[hgm]

Every speed doubling should give you about 70 Elo. But 70 Elo would not make you lose every time. So the ELo difference currently is probably much larger, and the NPS is only responsible for part of it.

[nkg114mc]

I am also curious that what NPS is acceptable for an engine whose strength is in fruit level (2700+)? It seems that a 500k+ NPS is a minimal requirement in my opinion. My engine can only get a 800k in perft (notice that it is the NPS in perft), and in real search it decreases to less than 300k, which is really slow. Probably the NPS is not all the story, but is it worth to do something first on NPS (for example, double it to reach a 500k+ NPS) rather than improve the other parts at the beginning stage?

[hgm]

Well, you can easily calculate that, right? If Houdini is 350 Elo stronger than Fruit (say), that is 5 x 70 Elo. So you could reduce the speed of Houdini by a factor 2^5 = 32 to make it equal in strength to Frui. If Houdini does 2Mnps (say), that means it can be as strong as Fruit at 62,500 nps.

You could also try it out, by letting Houdini play a time-odds match against Fruit, and see how much time odds you have to give it to play even.

[sedicla]

I have a new release of tucano in a few days. I'm thinking to work on improving performance in general for next release. Let's say you improve something and get extra 1000 node here, another 2000 there and so on. It goes with what Steve said, to have just what is necessary in eval for example.

Just to clarify, I may add that tucano can't beat fruit in a match, let's say 100 games at 1m+1s, but of course can win some games...

I'm just wondering the importance of NPS...

[bob]

I have a new release of tucano in a few days. I'm thinking to work on improving performance in general for next release. Let's say you improve something and get extra 1000 node here, another 2000 there and so on. It goes with what Steve said, to have just what is necessary in eval for example.

Just to clarify, I may add that tucano can't beat fruit in a match, let's say 100 games at 1m+1s, but of course can win some games...

I'm just wondering the importance of NPS...

NPS in and of itself means exactly nothing. But for a given program, if you double its NPS, you just gained 70 Elo or so. So the importance of NPS is relative. Don't compare to other programs, but you do want to compare against yourself because the faster you go the stronger you play, assuming you are searching the exact same tree both times.

[jdart]

It is always a good idea to do a profiling run over your code periodically. You may find there is a cheap way to improve speed. (If you are on Intel h/w I recommend VTune, which measures cache and memory usage, not just CPU).

On the other hand it is usually really hard to double NPS and improving chess strength can be done w/o NPS increase. So I wouldn't make NPS improvement a primary goal.

--Jon