Testing LazySMP

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

LazySMP

Re: New engine: LazySMP

Post by LazySMP »

JVMerlino wrote: Thu Oct 24, 2024 9:00 pm Given that Ed has said that your engine is between 1800-2000 elo, not only will nobody be interested in copying it, but releasing your code will only help to improve your engine because you will likely get useful feedback and bug reports.
Although my engine is 1800 elo, but it is very fast. My engine speed is about 10 million NPS on a single thread. Have you ever seen an engine like this before?
supernova wrote: Thu Oct 24, 2024 9:07 pm I have already reported the issue, and others have provided their advice. However, for now, no action has been taken from him. Regards.
The main issue is not the bugs of my engine. Even if I fix all the bugs in the engine, it won't test.
hgm wrote: Thu Oct 24, 2024 10:54 pm I think you miss the point on several issues here. For one, the version you originally wanted tested was buggy, and forfeited nearly every game. The fact that you had not even noticed such an obvious bug yourself suggested that the engine would be riddled with undiscovered bugs. New engines typically are, before they have undergone some very basic trials for discovering and ironing out the bugs. Serious testers would not want to test such an engine.
Thank you for your guidance, love and endless patience. As you mentioned, your engine (Joker) was tested even though it was not open source. They don't want to test my engine, not because of its bugs, but because of other reasons. I think Kryukov's philosophy of creating the CCRL organization was a fair competition and not what we see now. Even if I fix all the bugs in the engine, it won't test.
Rebel wrote: Thu Oct 24, 2024 8:52 pm LazySMP crashes at move 128, you must widen your game history, my advice is 1000 moves.
As I said before, this is a minor bug and it can be solved by changing the array size.
Ras wrote: Fri Oct 25, 2024 1:37 am Get real. There's nothing in a 2000 Elo engine to be copied that isn't already well-known anyway, and copying from your code would be more work than just improving from these well-known principles.
If there is nothing in the 2000 Elo engine, why the authors of the 3800 ELO engines (Clover, Rubi, etc) don't want my engine to be tested in the rating list?!
supernova
Posts: 87
Joined: Mon Apr 15, 2024 8:30 pm
Full name: Arthur Matheus

Re: New engine: LazySMP

Post by supernova »

LazySMP wrote: Fri Oct 25, 2024 2:32 am uote=supernova post_id=970637 time=1729796833 user_id=14208]
I have already reported the issue, and others have provided their advice. However, for now, no action has been taken from him. Regards.
The main issue is not the bugs of my engine. Even if I fix all the bugs in the engine, it won't test.
[/quote]

Your reasoning is flawed. The engine crashing at 128 is clearly a bug, yet you refuse to address it over a simple array issue.

Honestly, this whole situation is exhausting. I’ll complete the test, share the findings, and move on. This type of problem is what drives me away from computer chess. Since you keep leaving out crucial details, here they are again to counter the ongoing nonsense.
supernova wrote: Thu Oct 24, 2024 9:07 pm
Rebel wrote: Thu Oct 24, 2024 8:52 pm

LazySMP crashes at move 128, you must widen your game history, my advice is 1000 moves.
I have already reported the issue, and others have provided their advice. However, for now, no action has been taken from him. Regards.
Modern Times
Posts: 3708
Joined: Thu Jun 07, 2012 11:02 pm

Re: New engine: LazySMP

Post by Modern Times »

hgm wrote: Thu Oct 24, 2024 12:50 pm
Graham Banks wrote: Thu Oct 24, 2024 11:43 am
LazySMP wrote: Thu Oct 24, 2024 12:12 amUnfortunately, there are many people (lucametehau, noobpwnftw, RubiChess, Guenther, etc.) who do not want my engine to be tested for chess rating list. As Mr. Gabor mentioned, he does not want to test my engine. This is a negative approach.
If you can reach a rough rating of 2450 Elo, plus make your engine open source giving credit where it's due. I'll consider testing it.
Open source? Is that a new CCRL requirement or just your own interest? I remember you tested Joker, the source of which I never published.
Exactly. A decade or more ago, huge numbers of engines were closed source and no-one batted an eyelid. Today the trend is towards open source, but authors are fully entitled to choose if their engines are open source or closed source. Non-one should have to release their source, and furthermore the clear implication here is that Graham is suspicious that it may be a clone or derivative so he wants it released so that fellow authors can inspect it. No other reason for him to request that the source be released. Maybe it is and maybe it isn't a derivative. But the sad trend these days is one of mistrust - guilty until proven innocent. Computer chess is very nasty these days. In terms of him testing or not, that is entirely his decision, and he doesn't need to give reasons.
User avatar
Rebel
Posts: 7306
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: New engine: LazySMP

Post by Rebel »

LazySMP wrote: Fri Oct 25, 2024 2:32 am If there is nothing in the 2000 Elo engine, why the authors of the 3800 ELO engines (Clover, Rubi, etc) don't want my engine to be tested in the rating list?!
Please quote what the authors of Clover and Rubi exactly said.
90% of coding is debugging, the other 10% is writing bugs.
mar
Posts: 2656
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: New engine: LazySMP

Post by mar »

Modern Times wrote: Fri Oct 25, 2024 5:29 am Exactly. A decade or more ago, huge numbers of engines were closed source and no-one batted an eyelid. Today the trend is towards open source, but authors are fully entitled to choose if their engines are open source or closed source. Non-one should have to release their source, and furthermore the clear implication here is that Graham is suspicious that it may be a clone or derivative so he wants it released so that fellow authors can inspect it. No other reason for him to request that the source be released. Maybe it is and maybe it isn't a derivative. But the sad trend these days is one of mistrust - guilty until proven innocent. Computer chess is very nasty these days. In terms of him testing or not, that is entirely his decision, and he doesn't need to give reasons.
yes, anyone is free to stay closed source or even private - just ignore the bullies
in some cases closed source would even be a service to mankind :)

guilty until proven innocent - wasn't this always the case as far as I remember? Vas paid the price for being successful, for example

as for today (or the recent past) - people sell stockfish to other people (like Houdart, I guess I don't have to mention FF2 which also violated GPL), Munter releases a closed source Berserk clone, so it's not like the suspicion is unwarranted. it's those people who actually harm computer chess (and some of those engines still pollute rating lists to this day)

in this case however, the guy is an obvious troll (judging from the posts) and gets way more attention than he deserves. I suspect this is in part to annoy a certain group of people here
Modern Times
Posts: 3708
Joined: Thu Jun 07, 2012 11:02 pm

Re: New engine: LazySMP

Post by Modern Times »

mar wrote: Fri Oct 25, 2024 7:26 am in this case however, the guy is an obvious troll (judging from the posts) and gets way more attention than he deserves. I suspect this is in part to annoy a certain group of people here
If so, the strategy is clearly working !
User avatar
Ras
Posts: 2696
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: New engine: LazySMP

Post by Ras »

LazySMP wrote: Fri Oct 25, 2024 2:32 amAlthough my engine is 1800 elo, but it is very fast. My engine speed is about 10 million NPS on a single thread. Have you ever seen an engine like this before?
The last version of my 2700ish engine with full move make/unmake and in-check detection in perft had about 35M NPS in perft, single threaded. That's a simple mailbox approach with ray scanner loops, and it does already include MVV-LVA and mobility detection in the move generator.

Speed in itself doesn't mean much. I think 1800 Elo is a good achievement for a first release (so, congrats!), but you'll get tons of Elo by trading speed for features. Your raw NPS will go down anyway once you implement features like mobility detection, history, hash tables, and once you debug your move generator which will most likely fail in weird positions (that's where perft is really helpful). Not to speak of a refined eval which will drag down the speed even further. Just to illustrate that point - my engine also runs on a microcontroller where it has only about 30 kNPS, but that's still in the 2200 Elo ballpark.
If there is nothing in the 2000 Elo engine, why the authors of the 3800 ELO engines (Clover, Rubi, etc) don't want my engine to be tested in the rating list?!
That's not what was said to my understanding, at least not in this thread. The point is rather that it is your job to properly test and debug your engine beforehand. Also, the testing effort on CCRL is not evenly spread; the top engines naturally get more attention. A new Stockfish release is likely to be tested with top priority, but a midrange engine isn't. One or two releases per year should be fine, and expect some weeks delay before they get around to test that. That also implies that you don't want to burn through the scarce testing slots and also annoy testers with buggy releases.

CCRL is like your engine having a date, and you wouldn't attend dates unkempt in a dirty shirt, right?
Modern Times wrote: Fri Oct 25, 2024 5:29 ambut authors are fully entitled to choose if their engines are open source or closed source.
Sure. The original disagreement was not closed/open source, but public/private, and I think private engines shouldn't even be on CCRL unless they are somehow important. As of now, I can only see Torch in that role because of its commercial relevance on one of the largest online chess platforms, and people are surely curious whether its analysis can provide something that they can't get for free with Stockfish.
the clear implication here is that Graham is suspicious that it may be a clone or derivative
I don't think it would be that buggy in this case. :wink:
Rasmus Althoff
https://www.ct800.net
User avatar
Gabor Szots
Posts: 1442
Joined: Sat Jul 21, 2018 7:43 am
Location: Budapest, Hungary
Full name: Gabor Szots

New engine: LazySMP

Post by Gabor Szots »

LazySMP wrote: Sun Oct 20, 2024 6:54 pm Dear Mr. Gabor, May I ask you, what was your purpose in creating that issue on my GitHub repository?!
You wrote "Are you sure LazySMP is the best possible name" which means you were interested in testing my engine.
Also, you announced in this forum that if my engine bugs are fixed, you will test it.
I fixed issues discovered during matches: Castling, EnPassant, Promotion...
What happened that you changed your mind and don't want to test my engine?! At least I hope you explain why you don't want to test engine :(
Daniel,

I simply do not like the name.

And I don't remember ever having said that I was going to test your engine after the bugs have been fixed. What I remember is that I said I might test it in the future. I may be wrong, would you care to quote that announcement? You are very good at quoting.

Regards,
Gabor
Gabor Szots
CCRL testing group
LazySMP

Re: New engine: LazySMP

Post by LazySMP »

Modern Times wrote: Fri Oct 25, 2024 5:29 am Authors are fully entitled to choose if their engines are open source or closed source. Non-one should have to release their source, and furthermore the clear implication here is that Graham is suspicious that it may be a clone or derivative so he wants it released so that fellow authors can inspect it. No other reason for him to request that the source be released. Maybe it is and maybe it isn't a derivative. But the sad trend these days is one of mistrust - guilty until proven innocent.
Thank you for your support and interest. If Graham is suspicious that my engine may be a clone or derivative, prove it. But reaching 2450 Elo and publishing the source code is an irrational request.
mar wrote: Fri Oct 25, 2024 7:26 am as for today (or the recent past) - people sell stockfish to other people (like Houdart, I guess I don't have to mention FF2 which also violated GPL), Munter releases a closed source Berserk clone, so it's not like the suspicion is unwarranted. it's those people who actually harm computer chess (and some of those engines still pollute rating lists to this day) in this case however, the guy is an obvious troll (judging from the posts) and gets way more attention than he deserves. I suspect this is in part to annoy a certain group of people here
If someone claims that my engine is a copy of another engine, please provide proof. You are probably one of the friends who don't want my engine to be tested. Just for your information, I no longer have a decision to continue developing my engine, and you succeeded.
Ras wrote: Fri Oct 25, 2024 10:17 am The last version of my 2700ish engine with full move make/unmake and in-check detection in perft had about 35M NPS in perft, single threaded. That's a simple mailbox approach with ray scanner loops, and it does already include MVV-LVA and mobility detection in the move generator. Speed in itself doesn't mean much. I think 1800 Elo is a good achievement for a first release (so, congrats!), but you'll get tons of Elo by trading speed for features. Your raw NPS will go down anyway once you implement features like mobility detection, history, hash tables, and once you debug your move generator which will most likely fail in weird positions (that's where perft is really helpful).
Thank you for your excellent presentation with detailed information about NPS. I would like to test your engine speed. Please send me the download link. However, by adding new features to the engine, its speed should be reduced. As I said earlier, NPS doesn't mean anything, if you want high NPS go back to pure alpha beta but may lose accuracy of the evaluation.
Gabor Szots wrote: Sat Oct 26, 2024 11:56 am Daniel, I simply do not like the name. And I don't remember ever having said that I was going to test your engine after the bugs have been fixed. What I remember is that I said I might test it in the future. I may be wrong, would you care to quote that announcement? You are very good at quoting.
Dear Gabor, Thank you for your response. You didn't do anything wrong, but other people's words made you wrong. In any case, I am not interested in testing my engine in the future. I hope you will understand later why other people don't want my engine to be tested.
User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: New engine: LazySMP

Post by hgm »

LazySMP wrote: Fri Oct 25, 2024 2:32 amAlthough my engine is 1800 elo, but it is very fast. My engine speed is about 10 million NPS on a single thread. Have you ever seen an engine like this before?
In the mailbox trials I presented a search on a middle-game position that reached 13.5Mnps, single threaded.

Engines that do not use a Transposition Table tend to be very fast. Micro-Max 1.6, which has a very stupid move generator, which must scan the entire board to find its own pieces, already reached about 6Mnps. When I equiped it with a TT this dropped to 1-2Mnps. But of course it was much stronger.

You should also realize that rogrammers and testers might have a different perception of how serious a bug is. If you mistake a real bullet for a blank when loading a gun, you might argue it is just a small mistake because these hardly look different. To the one who gets killed by this it might not seem so insignificant, though. When an engine crashes during a game this is the worst possible outcome for a tester, and he won't care if it cause by you typing 128 instead of 256.