C# Performance

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

RoadWarrior
Posts: 73
Joined: Fri Jan 13, 2012 12:39 am
Location: London, England

Re: C# Performance

Post by RoadWarrior »

diep wrote:'cryptographically-secure pseudo-random'
that's a huge sentence, tough to parse for me, especially for Bill Gates toylanguage.

uhhhh?

What's the difference with a RNG?
The first difference is that a CSPRNG is designed to pass rigorous statistical randomness tests such as those in the Diehard and Dieharder suites. This gives you confidence that the numbers produced are useful in specific contexts like Zobrist hashing.

The second difference isn't applicable to chess programming, but is very important for cryptography. A CSPRNG is designed to hold up under serious attack, even when its initial or running state becomes available to an attacker.
diep wrote:Optimization in C# seems more complex than in C.

Not surprisingly. Usually microbenchmarks you can keep speed difference under control. Factor 4 or so slower than well written C.

The real problem comes of course when a program grows a tad bigger.

Yet if i see the struggle here, then i'm glad i program in C
A managed language like C# removes whole categories of bugs, such as the heap corruptions and array-index-out-of-bound errors that so often lead to frustrating late-night debugging sessions. So you can get more done with less effort, especially using the rich functionality provided by the .NET base class library.

The trade-off, however, is that you have to work further away from the hardware and with a relatively high-level language. Which means that your mental model of the language spec and the JIT compiler has to be fairly good.

Note that the list of optimisations performed by the JIT compiler is impressive given that it's done in "wall clock" time:

Code: Select all

Constant folding
Constant and copy propagation
Common subexpression elimination
Code motion of loop invariants
Dead store and dead code elimination
Register allocation
Method inlining
Loop unrolling (small loops with small bodies)
The result is comparable to traditional native code - at least in the same ballpark. I think your "factor of 4" is an exaggeration. If I'm faced with performance issues, I don't usually look at the language - I look in the mirror. :)
There are two types of people in the world: Avoid them both.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: C# Performance

Post by diep »

RoadWarrior wrote:
diep wrote:'cryptographically-secure pseudo-random'
that's a huge sentence, tough to parse for me, especially for Bill Gates toylanguage.

uhhhh?

What's the difference with a RNG?
The first difference is that a CSPRNG is designed to pass rigorous statistical randomness tests such as those in the Diehard and Dieharder suites. This gives you confidence that the numbers produced are useful in specific contexts like Zobrist hashing.

The second difference isn't applicable to chess programming, but is very important for cryptography. A CSPRNG is designed to hold up under serious attack, even when its initial or running state becomes available to an attacker.
diep wrote:Optimization in C# seems more complex than in C.

Not surprisingly. Usually microbenchmarks you can keep speed difference under control. Factor 4 or so slower than well written C.

The real problem comes of course when a program grows a tad bigger.

Yet if i see the struggle here, then i'm glad i program in C
A managed language like C# removes whole categories of bugs, such as the heap corruptions and array-index-out-of-bound errors that so often lead to frustrating late-night debugging sessions. So you can get more done with less effort, especially using the rich functionality provided by the .NET base class library.

The trade-off, however, is that you have to work further away from the hardware and with a relatively high-level language. Which means that your mental model of the language spec and the JIT compiler has to be fairly good.

Note that the list of optimisations performed by the JIT compiler is impressive given that it's done in "wall clock" time:

Code: Select all

Constant folding
Constant and copy propagation
Common subexpression elimination
Code motion of loop invariants
Dead store and dead code elimination
Register allocation
Method inlining
Loop unrolling (small loops with small bodies)
The result is comparable to traditional native code - at least in the same ballpark. I think your "factor of 4" is an exaggeration. If I'm faced with performance issues, I don't usually look at the language - I look in the mirror. :)
Mark, the chessprogrammers aren't beginners. Stuff like out of bounds you have boundscheckers for. The fact that default C# is doing automatic boundschecking slows down the language bigtime, i don't see it as an advantage of the language for low level codes like game tree search where you put in a lot of time per line of code and try to regurarly optimize what you wrote before.

In fact it's no secret my code runs in C on many different platforms, yet if i see how total inefficient most compilers, even in C, deal with the code, the only problem in low level development is the quality of the compilers.

Now where compared to all other languages C/C++ compilers are pretty decent by any standard, they are not for any other language.

Even then the mistakes i see the C/C++ compilers make is huge.

C++ code of c ourse i only mention because it CAN be similar to C. Not because it practically is. Maybe 1 or 2 coders on the planet manage to write C++ same speed, practical the rest is factors slower.

Just the nature of object oriented languages already implicitly forces you to subclass everything and make new classes, using a number of layers of classes already assures you of the huge slowdown of course, which imperative languages like C and Fortran do not have.

The thing that caused JAVA and C# to be there IMHO is the clumsy and total nerd manner how they've 'standardized' C++ at ISO. It took till 1999 or so and they have allowed too much and just have kept adding to it, with too many nerd features, making it worlds most complex language.

Just 0 business sense. Then Java came there by Sun years before they managed to finalize an iso in C++ (which was a total messed up ISO of course if you look from it from a company viewpoint that wants to get things done). Java so called had no pointers and only single inheritence.

The initial compilers for it were a crap.

It rescued Sun for a few years, and by now we know that java has outlived Sun.

m$ then responded with C# onto Java.

C# would never have existed without Java having seen the light.

Java would not have been there i guess if they would've been more decisive about C++.

That's why we have the mess right now as it is.

Vincent

p.s. i don't want this to end up in a yes/no between languages, but we have to face reality that C# and Java are of similar speed, about 3x slower than C if both get written by excellent programmers, for larger pieces of code.

That is - if your C# and Java isn't gonna defragment the memory while your proggie is running :)

Note that for company codes, which are massive the difference is bigger. Usually a 100+. However we also must note that C is not a nice language from company viewpoint to manage when you have many programmers of varying experience in programming.
RoadWarrior
Posts: 73
Joined: Fri Jan 13, 2012 12:39 am
Location: London, England

Re: C# Performance

Post by RoadWarrior »

diep wrote:p.s. i don't want this to end up in a yes/no between languages, but we have to face reality that C# and Java are of similar speed, about 3x slower than C if both get written by excellent programmers, for larger pieces of code.
Vincent, there's a Java program called CuckooChess that's about the same strength as Diep :). Demonic speed is very useful, but it isn't the be-all and end-all of engine strength or chess programming. My C programming would probably produce a chess engine at least 3 times as slow as your C programming. Given that existing constraint, my choice of C# looks eminently sensible. :D

Software is just a medium to get from here to somewhere else. IMX (35 years of programming) people who get deeply religious about their software language tend to be more excited about how they're going to do something, and not about what they're trying to accomplish. That's fine, but it's not my thing.
There are two types of people in the world: Avoid them both.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: C# Performance

Post by diep »

RoadWarrior wrote:
diep wrote:p.s. i don't want this to end up in a yes/no between languages, but we have to face reality that C# and Java are of similar speed, about 3x slower than C if both get written by excellent programmers, for larger pieces of code.
Vincent, there's a Java program called CuckooChess that's about the same strength as Diep :). Demonic speed is very useful, but it isn't the be-all and end-all of engine strength or chess programming. My C programming would probably produce a chess engine at least 3 times as slow as your C programming. Given that existing constraint, my choice of C# looks eminently sensible. :D

Software is just a medium to get from here to somewhere else. IMX (35 years of programming) people who get deeply religious about their software language tend to be more excited about how they're going to do something, and not about what they're trying to accomplish. That's fine, but it's not my thing.
I don't think there is anything on the market right now that can handle Diep :)

Maybe some programmer with a strong engine has at home a beta version of a new branch that's stronger, but we'll see some months from now.

Anyway you won't find anything that's written in anything else than C/C++/assembler above 3000.

btw for the curious you could start by taking a look at some photos i posted at my facebook, which will give you some insight :)

Losing factor 3 to 5 or so to C# or Java really is too much folks.

Try this. play your favourite engine at factor 5 slower hardware.
don't give it factor 5 less time at good hardware as it still ponders then
and fills HASHTABLES.

The todays beancounters factor 5 less in nps are worth nothing of course.
Last edited by diep on Sun Feb 19, 2012 10:24 pm, edited 1 time in total.
Richard Allbert
Posts: 792
Joined: Wed Jul 19, 2006 9:58 am

Re: C# Performance

Post by Richard Allbert »

Hi Vincent,

Not seen you for a while now, hope you are well!

As you've read on this thread, there has been quite a lot of performance testing.

I thought at first that C# was 'slow', but I must say that with AB search implemented, and a basic eval, I get withing 80% of the same C++ program, using as similar a structure as possible.

I agree that you do have to put some thought into how to structure things - such as flattening arrays, and profiling to check heap object creation, but there are also a lot of pitfalls in C++ - such as the behind the scenes copying of classes if you don't override the the copy constructor to stop it. Us ing a profiler has hepled a lot in making sure problems are avoided.

C is different, I know, and has fewer traps, but you need to be very good at it all the same - Crafty an example.

A key part of what you wrote is that an "expert C programmer" would get more performance than an expert C# programmer.

And remember the winner of the Facebook Hacker cup 2011 who used which language? Java! The other finalists all used C++/C.

Anyway, writing the engine in C# has (so far) been a nice experience, and due to the availability of good Logging, MySQL, String, Thread libraries, the more "mundane" programming is implemented much more easliy.


I don't want to creat a C vs C# row either ;) But both have advantages and disadvantages

Ciao,

Look forward to meeting again

Richard
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: C# Performance

Post by diep »

Richard Allbert wrote:Hi Vincent,

Not seen you for a while now, hope you are well!

As you've read on this thread, there has been quit
e a lot of performance testing.

I thought at first that C# was 'slow', but I must say that with AB search implemented, and a basic eval, I get withing 80% of the same C++ program, using as similar a structure as possible.

I agree that you do have to put some thought into how to structure things - such as flattening arrays, and profiling to check heap object creation, but there are also a lot of pitfalls in C++ - such as the behind the scenes copying of classes if you don't override the the copy constructor to stop it. Us ing a profiler has hepled a lot in making sure problems are avoided.

C is different, I know, and has fewer traps, but you need to be very good at it all the same - Crafty an example.

A key part of what you wrote is that an "expert C programmer" would get more performance than an expert C# programmer.

And remember the winner of the Facebook Hacker cup 2011 who used which language? Java! The other finalists all used C++/C.

Anyway, writing the engine in C# has (so far) been a nice experience, and due to the availability of good Logging, MySQL, String, Thread libraries, the more "mundane" programming is implemented much more easliy.


I don't want to creat a C vs C# row either ;) But both have advantages and disadvantages

Ciao,

Look forward to meeting again

Richard
you don't get within 80% of the speed of good C++. problem is you need to compare with the utmost best. default c++ is also losing factor 2 to 3 or so to real good c++.

ask some c++ programmers over here...

in itself what language you start with doesn't matter - the problem is that no one when their engine gets a tad stronger can easily switch languages as then they lose what they already wrote.

that's the real problem.

how about this richard, why not start c++.

you can use everything you use in C# as well kind of. just don't use any weirdo c++ constructs you don't know about. probably you'll end up with a real slow engine as well, yet you can still later on speed that up.

what i dislike of c++ is that basically you need a hack to do multithreading.

note i use c++ bigtime for GUI programming so i'm not exacty a stranger. also i did do some java/javascript programming in past (who didn't) so i know the sad feeling you get in java.

C# i kept away from - the salary to convince me programming C# is at least 6 digits and when you double it will approach 7 digits rapidly :)

the real advantage of C over C++ for a single programmer, as i would never recommend a company with many programmers using an imperative language, is that there is hard defined limits in the language where you know EXACTLY that the compiler can optimize it well and you do not know that with c++. to figure that out with c++ takes forever and is total compiler dependant and has nothing to do with the actual ISO standard.

besides you need to pay bigtime for c++ iso. to get it the legal way it's what is it $400 or so?

then you have a truckload of documentation that'll take you 30 years to figure out.

that's where java was doing better and initially C# as well. But by now i understood they added a lot of nonsense to C# as the language was too slow; a language that needs 100 tricks to not be ultra slow by default is not a good idea Richard!

what's the difference between C++ and C# and Java in the first place, except that C++ gives you the option to be one day REALLY FAST. Same speed like C.

Vincent
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: C# Performance

Post by diep »

Richard Allbert wrote:Hi Vincent,

Not seen you for a while now, hope you are well!

As you've read on this thread, there has been quite a lot of performance testing.

I thought at first that C# was 'slow', but I must say that with AB search implemented, and a basic eval, I get withing 80% of the same C++ program, using as similar a structure as possible.

I agree that you do have to put some thought into how to structure things - such as flattening arrays, and profiling to check heap object creation, but there are also a lot of pitfalls in C++ - such as the behind the scenes copying of classes if you don't override the the copy constructor to stop it. Us ing a profiler has hepled a lot in making sure problems are avoided.

C is different, I know, and has fewer traps, but you need to be very good at it all the same - Crafty an example.

A key part of what you wrote is that an "expert C programmer" would get more performance than an expert C# programmer.

And remember the winner of the Facebook Hacker cup 2011 who used which language? Java! The other finalists all used C++/C.

Anyway, writing the engine in C# has (so far) been a nice experience, and due to the availability of good Logging, MySQL, String, Thread libraries, the more "mundane" programming is implemented much more easliy.


I don't want to creat a C vs C# row either ;) But both have advantages and disadvantages

Ciao,

Look forward to meeting again

Richard
a small note on game tree search programming:

If your ambition is a strong chessprogram, then count on it that being a hell of a software engineer is requirement 1. The rest really is second place.

expert software engineer is not the requirement. 'book writing author' is the requirement.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: C# Performance

Post by diep »

Richard Allbert wrote:Hi Vincent,

Not seen you for a while now, hope you are well!
thx, but as you probably know the hit 'n run times at this forum i usually do when i'm sick as a dog at home and by sheer borement have seen the news 10 times after which i checkout with just at most a few mouseclicks 2 or 3 subjects at CCC, because you need to be really sick to be motivated to be busy with computerchess still :)
RoadWarrior
Posts: 73
Joined: Fri Jan 13, 2012 12:39 am
Location: London, England

Re: C# Performance

Post by RoadWarrior »

diep wrote:Anyway you won't find anything that's written in anything else than C/C++/assembler above 3000.
Vincent, neither of us is going to create the next Elo 3000+ monster no matter what programming language we choose. :)
There are two types of people in the world: Avoid them both.
Richard Allbert
Posts: 792
Joined: Wed Jul 19, 2006 9:58 am

Re: C# Performance

Post by Richard Allbert »

Out of laziness, I used my C Mersenne twister that I had to print the keys out and pasted them in :)

The C# version is now playing in FICS - with a skeleton search. I put it on a couple of days ago, after a hack job implementation of search. Since then it has played a couple of hundred games finishing at 1350 elo, which is not good! I wasn't expecting much more than 1700 - under that points to some big bugs, as even with just material evaluation, tactics should be ok.

I had a look, and found some massive bugs in there :) - qsearch wasn't working, the evaluation score was incorrect for certain material situations (ahem)... even and the matescore was inverted - causing some odd selfmates(!) plus a host of other things. Anyway, it was fun to see it play.

I fixed some of the major problems today, and it hasn't lost for a while, the rating is 1480

By morning I expect it to be a bit higher.

It's reaching depth 7 or so in the middlegame with no pruning or tricks, no null move or TTable. Just Alpha-Beta pruning.

Positive is the speed - it's not far behind my C++ version.

(but I'm nowhere near an expert;)

Richard