Stockfish 2.3 update available

[mcostalba]

Does the bug have any effect on normal chess ?

No, not at all.

I have sent you the test version.

[mcostalba]

What has changed? We noticed that there are significant differences in 2.2.2 and this version with respect to playing characteristic.

For example, on a 1 ply search version 23 is about 30 ELO stronger when playing head to head (only 8000 games) and is over twice as fast.

That implies some evaluation improvements and either quies search performance improvements or perhaps simply startup costs have been reduced. One version of Komodo was very slow on sub 5 ply searches because we spend a lot of time obsessing over the ply 1 move list.

However 2.3 seems to retains a big speed advantage although not the 2 to 1 I see in 1 ply searches. It searches at least 1/4 ply more.

Can you tell us what the most impactful changes are?

very small changes, mostly to evaluation.

I take this occasion to publicly thank Gary Linscott for his evaluation patches: if there is an increase in this version is due to him !

Sorry Gary for the delay in giving you the very well due credit.

Regrading your test of 1 ply search I don't have any experience because I have never done it, but I don't unerstand how your 1 ply search test result is bounded with speed increase: if it is a fixed depth search speed should not count, or am I missing something ?

[Don]

What has changed? We noticed that there are significant differences in 2.2.2 and this version with respect to playing characteristic.

For example, on a 1 ply search version 23 is about 30 ELO stronger when playing head to head (only 8000 games) and is over twice as fast.

That implies some evaluation improvements and either quies search performance improvements or perhaps simply startup costs have been reduced. One version of Komodo was very slow on sub 5 ply searches because we spend a lot of time obsessing over the ply 1 move list.

However 2.3 seems to retains a big speed advantage although not the 2 to 1 I see in 1 ply searches. It searches at least 1/4 ply more.

Can you tell us what the most impactful changes are?

very small changes, mostly to evaluation.

I take this occasion to publicly thank Gary Linscott for his evaluation patches: if there is an increase in this version is due to him !

Sorry Gary for the delay in giving you the very well due credit.

Regrading your test of 1 ply search I don't have any experience because I have never done it, but I don't unerstand how your 1 ply search test result is bounded with speed increase: if it is a fixed depth search speed should not count, or am I missing something ?

I run a few thousands 1 ply games and my tester tracks the total time spent each program spent thinking. I don't go by the time returned by the engine as that is not done by any GUI.

So when my tester sends "go depth 1" it starts the clock. When it returns with a move it stops the clock.

The difference can be accounted for by many factors, such as a more responsive I/O in the new stockfish, some pre-search calculations that have been sped up or removed, or other things. I was just wondering what it was.

[lkaufman]

If you send me a pm with an email address I can send to you directly a version compiled by me (not fast as Jim but enough to confirm bug is fixed).

P.S: For people testing in normal chess there is no need to wait, go with the fastest version and have fun

Non SSE compiles, ultra-bullet 2.5s + 0.04s games

Code: Select all

    Program                            Score       %     Elo    +   -    Draws

  1 Stockfish 2.3 JA 64bit         : 3227.5/6000  53.8   3213    7   7   42.9 %
  2 Stockfish 2.2.2 JA             : 2772.5/6000  46.2   3187    7   7   42.9 %

26 +/-7 Elo points improvement at this TC, probably 10-15 Elo points improvement at longer TC.

Kai

Using Linux, I show a huge gain in the direct matchup as you ran, at 5" + .05, about sixty elo. However when run against Komodo the gain drops to about thirty, and when run against Komodo at 15" + .15 it halves again to about 15. So I'll predict that it will show around 10 elo on the blitz rating lists.
I should point out that nodes per second is about 10% better for the new version. Can anyone tell us what they are getting as a nodes per second increase in Windows (over 2.2.2.)? It is also interesting that the new version is getting about 4/10s of a ply more depth at this level, far more than is due to the speed increase alone. But apparently this is not for free, as the elo gain I'm getting is fully explainable just by the speedup. Does anyone know the reason for the 10% speedup in Linux? Is it just a better compile, or has there been a true speedup in the code itself?

[Modern Times]

So I'll predict that it will show around 10 elo on the blitz rating lists.

I'd say better than that. I tested Stockfish VE_09 64-bit here, Stockfish 2.3 can only be better (hopefully)

http://www.computerchess.org.uk/ccrl/40 ... _length=30

[lkaufman]

So I'll predict that it will show around 10 elo on the blitz rating lists.

I'd say better than that. I tested Stockfish VE_09 64-bit here, Stockfish 2.3 can only be better (hopefully)

http://www.computerchess.org.uk/ccrl/40 ... _length=30

Actually, looking at your results, you show a negative progression for 2.1.1 to 2.2.1 to 2.2.2. But I think it is pretty clear that SF did make steady but small progress between those versions, so your 2.2.2 results is probably on the low end of the error bar. If so, a gain of 10 or perhaps 12 looks like a good prediction.
Can you tell us what is the percentage increase in nodes per second you get going from 2.2.2. to 2.3? I wonder if my 10% increase in Linux applies to the Windows version.

[Modern Times]

If so, a gain of 10 or perhaps 12 looks like a good prediction.

We shall see.

Can you tell us what is the percentage increase in nodes per second you get going from 2.2.2. to 2.3? I wonder if my 10% increase in Linux applies to the Windows version.

2.3 shows a decrease in kn/s for me vs 2.2.2

[Modern Times]

Does the bug have any effect on normal chess ?

No, not at all.

I have sent you the test version.

It complains about a missing DLL so I can't run it

[lkaufman]

If so, a gain of 10 or perhaps 12 looks like a good prediction.

We shall see.

Can you tell us what is the percentage increase in nodes per second you get going from 2.2.2. to 2.3? I wonder if my 10% increase in Linux applies to the Windows version.

2.3 shows a decrease in kn/s for me vs 2.2.2

My current elo gain at 15" for 2.3 is 18, which implies around 12 or so at blitz levels. But this is with a speedup on Linux (about 8% now). So I'll be surprised if SF 2.3 shows even +10 on Windows if you show a decrease in NPS. As you say, we'll see. Frankly we were hoping for a bigger gain by SF, as currently only Ippo-related programs are close to Komodo at hyperbullet speeds, and we could use an unrelated opponent closer in strength for testing.

[JuLieN]

Does the bug have any effect on normal chess ?

No, not at all.

I have sent you the test version.

It complains about a missing DLL so I can't run it

I guess it's the cygwin1 missing?

You can download it here:
http://www.dll-files.com/dllindex/dll-f ... ml?cygwin1

(and any other dll).

Unzip the file and put the dll in Stockfish's drawer.