Why C++ instead of C#?

[lithander]

I tried fixing that next and the benefit wasn't clearly measurable and drowned in the variance between tests. My builds have JIT still enabled (instead of AoT) and I wouldn't be surprised if the runtime catches something like that and optimizes it. Did you use AoT, Roland? In your tests the benefit seemed pretty clear. Hmm...

To be honest, I don't know. I built against .NET 5.0, allowed optimizations and set the build to prefer 64bits. Basically what I did was use the Visual Studio template for a .NET 5.0 console app and replaced the autogenerated program.cs by qbb_perft.cs without making any changes to the project settings.

If you just run it from Visual Studio that way I think you always get the JIT Version. To get AoT (ahead of time compilation) you have to publish a release. There unter TargetRuntime (Zielruntime) you get publishing options and then you get a check-box where you can select that you want ReadyToRun compilation. What that means in detail is explained here: https://docs.microsoft.com/en-us/dotnet ... ady-to-run

But in my experience it doesn't make a huge difference. Still the problem remains that I couldn't reproduce the benefit of the LSB improvement on my system. :/

I think I will get rid of the yield-statements next and try the LSB changes again later.

[R. Tomasi]

The refactoring away from yielding moves to adding them to a move list is also something I want to do soon and maybe I should do that first. But I wouldn't have done it exactly like you suggested but would have tried to stay closer to the C code and use an array on the stack for each recursive call. But to be sure I can test both versions.

The way I did it was in essence how I did those things in the first versions of my engine (which was written in C# at the time). Mind you, that's quite some years ago, and some of the stuff (like allocating on the stack) were not available at the time (at least I did not know of them back then). The runtime has quite evolved since that, in any case.

[R. Tomasi]

I tried fixing that next and the benefit wasn't clearly measurable and drowned in the variance between tests. My builds have JIT still enabled (instead of AoT) and I wouldn't be surprised if the runtime catches something like that and optimizes it. Did you use AoT, Roland? In your tests the benefit seemed pretty clear. Hmm...

To be honest, I don't know. I built against .NET 5.0, allowed optimizations and set the build to prefer 64bits. Basically what I did was use the Visual Studio template for a .NET 5.0 console app and replaced the autogenerated program.cs by qbb_perft.cs without making any changes to the project settings.

If you just run it from Visual Studio that way I think you always get the JIT Version. To get AoT (ahead of time compilation) you have to publish a release. There unter TargetRuntime (Zielruntime) you get publishing options and then you get a check-box where you can select that you want ReadyToRun compilation. What that means in detail is explained here: https://docs.microsoft.com/en-us/dotnet ... ady-to-run

But in my experience it doesn't make a huge difference. Still the problem remains that I couldn't reproduce the benefit of the LSB improvement on my system. :/

I think I will get rid of the yield-statements next and try the LSB changes again later.

That reflects my experience as well. When I was working with C# on my engine you had to use some Microsoft tool to precompile, and it never pruduced any measurable improvements for me. But again, that was like eons ago...

EDIT: have you tried comparing the verbatim code that I posted on your machine? My Laptop might really be a suboptimal system for testing such things.

[lithander]

EDIT: have you tried comparing the verbatim code that I posted on your machine? My Laptop might really be a suboptimal system for testing such things.

With the verbatim code I could indeed measure a small improvement when doing the LSB fix. I have then made a crude mix of my changes and your changes and this mix completes the testsuite in less than 40s where the vanilla version took more than 60s. So it's pretty fast and a step in the right direction. But when I remove the LSB optimizations now it doesn't really make a difference or rather the difference it makes seems to drown in the measurment variance. At this point, when in doubt I'd rather pick whatever is closer to the original C version.

Here are some measurements. Sadly I didn't keep the earlier ones around but they were pretty much the same:

Calling LSB twice in move gen without a temporary variable to store the LSB result:

Code: Select all

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi no LSB Fix> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3667ms, 32467K NPS
OK! 5200ms, 37245K NPS
OK! 5923ms, 30154K NPS
OK! 19761ms, 35727K NPS
OK! 1ms, 34179K NPS
OK! 4415ms, 37156K NPS

Total Time: 38970 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi no LSB Fix> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3674ms, 32403K NPS
OK! 5197ms, 37263K NPS
OK! 5989ms, 29826K NPS
OK! 19918ms, 35446K NPS
OK! 1ms, 33781K NPS
OK! 4428ms, 37050K NPS

Total Time: 39209 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi no LSB Fix> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3705ms, 32132K NPS
OK! 5183ms, 37368K NPS
OK! 6031ms, 29614K NPS
OK! 19917ms, 35448K NPS
OK! 1ms, 33704K NPS
OK! 4405ms, 37241K NPS

Total Time: 39245 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi no LSB Fix> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3645ms, 32663K NPS
OK! 5178ms, 37402K NPS
OK! 5916ms, 30191K NPS
OK! 19682ms, 35871K NPS
OK! 1ms, 32868K NPS
OK! 4386ms, 37403K NPS

Total Time: 38811 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi no LSB Fix> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3681ms, 32336K NPS
OK! 5213ms, 37150K NPS
OK! 5940ms, 30071K NPS
OK! 20017ms, 35270K NPS
OK! 1ms, 33508K NPS
OK! 4512ms, 36362K NPS

Total Time: 39367 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi no LSB Fix> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3644ms, 32671K NPS
OK! 5175ms, 37421K NPS
OK! 5780ms, 30901K NPS
OK! 19612ms, 35999K NPS
OK! 1ms, 33035K NPS
OK! 4391ms, 37360K NPS

Total Time: 38606 ms

With the LSB fix (the move gen related code is exactly like you posted it)

Code: Select all

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3724ms, 31963K NPS
OK! 5281ms, 36676K NPS
OK! 6004ms, 29751K NPS
OK! 20103ms, 35120K NPS
OK! 1ms, 33741K NPS
OK! 4403ms, 37262K NPS

Total Time: 39518 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3704ms, 32143K NPS
OK! 5303ms, 36517K NPS
OK! 6026ms, 29641K NPS
OK! 20211ms, 34933K NPS
OK! 1ms, 33666K NPS
OK! 4436ms, 36979K NPS

Total Time: 39684 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3672ms, 32416K NPS
OK! 5194ms, 37286K NPS
OK! 5803ms, 30780K NPS
OK! 19807ms, 35644K NPS
OK! 1ms, 34107K NPS
OK! 4357ms, 37657K NPS

Total Time: 38837 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3679ms, 32358K NPS
OK! 5211ms, 37162K NPS
OK! 5852ms, 30523K NPS
OK! 19833ms, 35598K NPS
OK! 1ms, 33951K NPS
OK! 4441ms, 36939K NPS

Total Time: 39020 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3753ms, 31718K NPS
OK! 5306ms, 36500K NPS
OK! 6094ms, 29310K NPS
OK! 20310ms, 34762K NPS
OK! 1ms, 32348K NPS
OK! 4490ms, 36539K NPS

Total Time: 39957 ms
Press any key to quit

PS D:\Projekte\Chess\QBB\QBB-Perft\bin\Release\net5.0\publish\1.2 RTomasi> .\qbb_perft.exe
QBB Perft in C#
https://github.com/lithander/QBB-Perft/tree/v1.2

OK! 3759ms, 31670K NPS
OK! 5289ms, 36621K NPS
OK! 5856ms, 30504K NPS
OK! 20061ms, 35194K NPS
OK! 1ms, 33726K NPS
OK! 4460ms, 36787K NPS

Total Time: 39427 ms
Press any key to quit

[R. Tomasi]

Well, if we went from 60s to 40s and the difference between C and C# was a factor 3x initially, we should be at about a factor of 2x now, right? So the more pessimistic expactations are already met. Let's see if we can get even a little bit closer.

[mvanthoor]

Am I reading this correctly, that the C# code is running perft at about 37 million NPS?

That is faster than I've ever seen a C# program running. My engine runs at 32 million NPS, but it also does incremental updates for Zobrist, material, mid-game PST and end-game PST, all which are unnecessary for perft. Without those (in earlier versions of the engine), it ran at 41 million NPS. I'm also assuming that lithander's 6-core computer is newer / faster per core than mine.

But 37 mln NPS per second is very good for a C# program. As said, better than I've ever seen.

[klx]

This is awesome, thanks for doing this. I might not have a lot of time to contribute, but I'm following with great interest. Would be cool if someone felt up to the challenge of porting to Java, Rust and other languages as well.

Well, if we went from 60s to 40s and the difference between C and C# was a factor 3x initially, we should be at about a factor of 2x now, right?

Coincidentally, this was exactly my prediction!

[klx]

Code: Select all

         move.From = LSB(promo)-8;
         move.To = LSB(promo);

There's a lot of unnecessary LSB calls for creating pretty much every pawn move. Should be a speedup for both programs, but presumably moreso on the C# side of things.

Yeah, I can't speak for C#, but in C world the compiler should detect it's the same and only do it once. Might be a reason the author didn't even bother deduplicating it. I think it's perfectly fine to fix this in C#, though the compiler might be smart enough to handle it there too.

[lithander]

Coincidentally, this was exactly my prediction!

But we ain't done yet

Am I reading this correctly, that the C# code is running perft at about 37 million NPS?

The gap to the C version has shrunk by another 8 seconds.

Code: Select all

OK! 3031ms, 39275K NPS
OK! 4204ms, 46070K NPS
OK! 4735ms, 37718K NPS
OK! 16089ms, 43881K NPS
OK! 1ms, 31637K NPS
OK! 3556ms, 46129K NPS

Total Time: 31619 ms

I'll update the github repo tomorrow. not quite commit-ready yet

[pedrojdm2021]

I think that a Fair comparation is to run the exact same algorithm in the 2 languages, if you procced to super optimize the C# version and the C++ not, then is not an 1:1 comparation.

I don't want to use pointers and unsafe code (not in the mainline version at least) and there are no macros and so there will never be a 1:1 comparison even possible.
I think what is fair is to assume the C version as a quality reference implementation of the algorithms (if you got improvements there we can of course try to optimize that too) and to now create what we™ consider a quality reference implementation of the same algorithms in C#.

You can argue a lot about the details but I think that visual similarity (which was my approach when porting so you can easily map the contents of the two files to their respective counterpart) should not be an important criteria long term.

then is not a fair comparasion if you don't use the same algorithms. Also, you don't have to use pointers, if you move representation were binary encoded to int or short you wouldn't need even a struct for Moves

and as i said earlier in this post, is not needed to allocate memory anywhere on the run, you can just fill the moves from an array, or a jagged array.