Stockfish port to C# Complete

[bpfliegel]

Mark, what Mps do you get for perft in Amaia? Wondering what could that be in you get 4 Mps on a single core for searching...

Cheers, Balint

[diep]

Interesting - Peter Österlund has just released a C++ implementation of his strong (Elo 2682) Java engine CuckooChess. He says that it's about twice as fast as the Java version: http://talkchess.com/forum/viewtopic.php?t=42999

From experience, C# runs faster than Java. So any claim of a C/C# slowdown in excess of a factor of 2 needs some pretty strong evidence before I'm going to believe it.

Yeah about a factor of 4 it is effectively.

Just try to do the same things like the C engines ok. Multiproces/multithread, hashtables, and elo-optimizations rather than nps optimizations.

If i turn off hashtable for example in Diep for the last ply, diep's nps goes up 20%.

Years ago already we saw 20 mln nps engines, and that was with forward pruning and with elo optimization.

Every compare here is with GCC. A junk compiler i use daily, so the junk it produces i know everything about. For free and therefore everyone who has a commercial interest in throwing in persons to keep it slow, they do that (ever seen a $100 billion company not do something like that to keep a $100 billion company huh?). A compiler factor 2 slower already, and even then nothing is on par with it.

What will happen if you have a full blown engine is that your L1i will totally get overrun by C#, that the caches get stressed to the limit, and that everywhere penalties will be there for C#. Heavy branch mispredicts which the pgo (not working very well for GCC to say polite - it's losing already 30% there) of the C/C++ engines doesn't have at all, dozens of cycles just for loading a few bytes out of L1i, which no C/C++ engine has either.

And all that won't be there in a testlab comparing just 1 routine versus a C compile.

And now you still are in the proces of comparing to beancounters so far...

Diep has a huge evaluation function, yet it really suffers also from a 1.34% L1i missrate. That 1.34% might seem little to you, but it's a huge slowdown.

If diep would be in C#, even if you'd produce efficient code with C#, which C# won't do of course, then you're looking at a slowdown of factor 5 already *just because of the L1i*.

Also i've never seen a C# engine use effectively a core or 16, someone already referred to that here earlier that when he's doing things SMP that he slows down a lot more.

So you'll be factors slower than C/C++. Where we already can't take it serious when you're 20% slower, let alone factor 2, so far the evidence we have there from a bit more objective tests than you did do here so far is that the slowdown is more of a factor 4 than factor 2.

And that for a programming language from which just got designed to counter Java.

The real question is: why would you want to slowdown volunteerly factor 4?

That's like the difference between an i7 sixcore and my old dual opteron dual core from januari 2006. A six year difference, as they ain't following Moore very well...

It's a no brainer of course: go use C for a chess engine, as most also mess up with C++ versus C.

Wanting to run 6 years behind volunteerly just because of the language choice is something i'll never understand.

It's true you also can say that from assembler versus C; assembler is a lot faster, yet it's also way more work and not portable.

In reality in object oriented languages you type more lines of code than in an imperative programming language. Now i do understand those who favour C++ over C, as in theory you can reach the same speed.

So learning that language better in manner that you write code similar to C, i do understand the choice of some for C++.

Yet C# or Java. 6 years. Even a T-ford from start 20th century wasn't driving on the roads factor 4 slower than todays cars.

Over here it's getting towards 130 KM/hour (most areas still 100 or 120 caused by civil servants who all vote for the communists and don't have a car themselves). A t-ford could definitely reach 130 KM/hour within a factor of 4.

So you're running 100 years behind man.

Vincent

[mcostalba]

Every compare here is with GCC. A junk compiler i use daily, so the junk it produces i know everything about.

GCC has improved a bit since the 90s. Today this produces the fastest binary on Windows:

http://www.equation.com/servlet/equation.cmd?fa=fortran

[RoadWarrior]

Mark, what Mps do you get for perft in Amaia? Wondering what could that be in you get 4 Mps on a single core for searching...

Amaia doing perft 7 from the initial position produces 16.8M NPS on a single core of an i7-2600K over-clocked at 4.6 Ghz. That's without hashing or bulk-counting.

[RoadWarrior]

Interesting - Peter Österlund has just released a C++ implementation of his strong (Elo 2682) Java engine CuckooChess. He says that it's about twice as fast as the Java version: http://talkchess.com/forum/viewtopic.php?t=42999

Yeah about a factor of 4 it is effectively.

Vincent, just look at Peter's numbers for a clear and relevant comparison. Whatever you might say, all you have are words and what he has is working programs and an operational comparison.

Give me a couple of years to whip Amaia into shape, and then we can have a match between Diep (Spyker C12 Zagato) and Amaia (Model T Ford). That would be very interesting.

[bpfliegel]

Vincent, you are correct in a lot of things - but there are a couple of reasons left one might want to go C#: a) Silverlight for being browser based (what MS already killed, anyway) or b) using it in in Windows Phone 7 (okay, not that many users, but still), c) it's totally clear that C# is inferior in performance speed. The point here is the fun is to beat that as much as possible.

I'm terribly weak at chess programming (unlike you), but somewhat okay in .Net and really enjoy the job and the progress: by looking at the standard benchmarks for Stockfish there is a factor of 2,44 after the last pieces of optimization between the original and the port! Of course that took a lot more just to make the raw port - and .Net offered a couple possibilities which allowed significant speedups in some areas.

I'm really interested how far Mark would get and also sorry that David suspended his efforts. Mark, good luck with Amaia! (wish I would have more time aswell).

Cheers, Balint

[diep]

Every compare here is with GCC. A junk compiler i use daily, so the junk it produces i know everything about.

GCC has improved a bit since the 90s. Today this produces the fastest binary on Windows:

http://www.equation.com/servlet/equation.cmd?fa=fortran

Why do you claim this nonsense?

Every chessprogrammer knows intel c++ is up to factor 2 faster for some. For diep it's over 20% to 40% faster.

Just start up your i7, as that's the hardware where everyone plays at and start compiling...

You're the first, yet not a chessprogrammer, posting it stupidly like this.
All the chessprogrammers i know who tested the diff compilers - intel c++ is first.

I tend to remember Dan Corbitt over here compiled many open source programs, including Glaurung and Stockfish for intel c++.

Why have you have him compile SF and meanwhile post this nonsense?

[diep]

Vincent, you are correct in a lot of things - but there are a couple of reasons left one might want to go C#: a) Silverlight for being browser based (what MS already killed, anyway)

You change the discussion from chess engines to generic.

I prefer to discuss chess engines here. But anyway if you want to know some stuff as i had figured out silverlight a year or what ago...

Some media stations have signed a contract to only use silverlight. That gave them a monopolyposition for some media websites, under which at the dutch NOS, where 1 billion euro of tax money goes. NOS is the state funded media organisation. I'd say away with them. Save out that 1 billion euro.

Probably over there it's a typical communist who gave away a monopoly to 1 specific companies product at that time.

Some of his own managers do not agree with about anything that happens over them - obviously i can't quote names there - they'd get fired instantly by the commie.

Maybe microsoft bought him a few tickets for free to USA - who knows. Probably something like that will explain it.

I complained back then against what was happening there with silverlight having the monopoly. Not sure they did do anything with that complaint - probably they shreddered it - that's how commies usually work.

b) using it in in Windows Phone 7 (okay, not that many users, but still),

We were discussing the speed comparision of C# versus imperative languages like C (and C++ obviously also falls under that as it's C compatible so to speak). I completely miss why mentionning a phone has any relevance to that.

c)
it's totally clear that C# is inferior in performance speed. The point here is the fun is to beat that as much as possible.

Implications are larger than you might guess - as we see of course from m$ and from Sun, nonstop claims about how 'fast' java and C# are, which is total nonsense claims all.

The impression was shown that C# is only factor 2 slower.

The more measurements that show up here, the slower it gets. My experience is that this settles after some objective comparitions effectively at C# being factor 4+ slower.

Still a huge amount of cycles wasted for nothing in case of a chess engine that has to compete against other chess engines, where factor 2 is really too much to lose; yet my experiences and that of others is that it's really factor 4+ and not even close to factor 2.

Now someone is doing big effort to port StockFish to C# and then is doing a few crappy compares at a small search depth with 1 thread versus an executable that's GCC compiled - i'd say ask corbitt for an intel c++ native for i7 compiled one - and use hardware that the speedjunks use over here which is at least core2 and even better an i7. Plenty of testers you can get.

I'm terribly weak at chess programming (unlike you), but somewhat okay in .Net and really enjoy the job and the progress: by looking at the standard benchmarks for Stockfish there is a factor of 2,44 after the last pieces of optimization between the original and the port! Of course that took a lot

I doubt that 2.44 and also from the formulations done here it's obvious it's more than factor 2.44

You can just simply run 12 instances at an i7 at 12 logical cores, have it do some major plydepth and measure like that. Then take the average wall clocktime it took to finish. Do the same with SF and you will see what is about to happen if you try to compete elowise. Probably factor 5 slower then or so, as we can easily double each claim is the experience. 2.44 getting 4.88, hands down.

This test is easy to do.

If someone is doing 2 years of effort to port SF, i really fail to see why 1 afternoon of objective comparisions is not possible.

We so far have just seen very subjective comparisions.

more just to make the raw port - and .Net offered a couple possibilities which allowed significant speedups in some areas.

I'm really interested how far Mark would get and also sorry that David suspended his efforts. Mark, good luck with Amaia! (wish I would have more time aswell).

Cheers, Balint

Did he suspend his effort *after* hearing that at his port still the GPL applies, or before?

Note that it is David who did work for 2 years or so at this port, meanwhile his real speed comparision is not very good science.

You need to compare to the best there is simply at both sides. At the C# side as well as at the C/C++ side in this case.

That's an i7 with intel c++ for SF and whatever the hell he wants for his C# port.

Also doing a run to a ply or 12, which is far under 1 second, not even allowing the CPU to unidle and also you have to wonder in how far things get turboboosted or not and so on, all that is not very assuring way of measurement.

You need to measure objective. It was written for speed in C++ and you should use then the fastest you got.

On internet they're all using i7's with chess engines you know...

Note that it's not so popular to play with SF online (speaking of wasting a paid team).

[bpfliegel]

I was only arguing while someone wants to use a slower car or not - so if there are uses for a .Net engine aswell.

Just checked two things:
- For the original versions GCC is really half the speed at an actual gameplay as the Intel one. You are correct to say to use big depth searches - by plainly just checking the benchmark tests the GCC version performs in fact 20-30% better (maybe 16 * depth 12 is crappy, but I was scammed by only looking at the benchmark, it should be at similar ratio). So in fact the GCC is a garbage when it comes to actual gameplay. Wonder how that looks on i7.
- I ran a game between the 2.2.2 intel non-SSE version and the port at relative high depths (25+), but I saw no significant speed changes regarding nps after 20+. The performance ratio is between 2.8-3 in these cases (as I wrote earlier, 1:3 in gameplay, but lately I made small optimizations). This is an i5 laptop, original delivers in ranges 2600-3000 nps and the port around 880-960.

Obviously this should be just some bull**** as there is a huge amount of concurrent memory allocation and the garbage collection is under constant stress and the code is 100% safe, blablabla. But it isn't and this difference is what I put a lot of work into. I plan to tidy up things first before doing a release and I'm in no rush, so it might the wise to stop corresponding before there actual facts one is able to validate as it should be.

I don't care about the politics, just enjoy the task trying to deliver exceptional performance in this technology.

Cheers, Balint

[bpfliegel]

My experience is that this settles after some objective comparitions effectively at C# being factor 4+ slower.

By removing my heaviest optimization facility it is actually 6.8 times slower!