Preconditions

[Henk]

It often happens that when you implement a solution it does not work. Sometimes it is caused by bugs but other times it can be because you use other stuff in the remainder of your chess program.

It would be nice if descriptions of solutions would be more complete by listing some conditions that say this solution only works if these conditions are valid.

For instance SEE in quiescence search or Lazy evaluation does not work for my program because probably my evaluation is too minimal. Same holds for futility pruning below level two because these lower levels are often skipped by reductions in null move or LMR.

Perhaps multi cut does not work when you use null move pruning. Singular extensions also did not give much improvement for my program. Don't know why.

[lucasart]

Cut the crap. What is your question?

[Sven]

It is simple: some solutions only work if you apply them without bugs

I definitely know that bugs are by far the most common reason why something does not work that works for other people.

Another typical failure reason apart from bugs is the presence of unusual constructions in a program. Such issues can sometimes be found by starting to explain your program to someone else. That other person does not need to do much except listening and sometimes asking questions. Then you will most probably encounter parts of your program for which you have great difficulties to explain them, possibly caused by something that is really unusual. Look deeper into these parts and try to replace the unusual elements by something else.

You might also want to specify "it does not work" more precisely. Do you fail to implement it correctly in your program? Do you succeed in implementing it but it makes your program inconsistent and crashing? Does it work technically but makes your program weaker? Does it fail to improve your program at all? Does it fail to improve your program by a margin that you regard as minimum to accept the change? There are certainly different levels on which "does not work" can be seen.

Sven

[Henk]

What I mean with "Does not work" is that it fails to improve the chess program significantly. I am not interested in improvements of 10 ELO or less if I am at least five hundred ELO points behind.

So SEE pruning of bad captures does not work in my program at depth <= 2. But I don't have the fastest SEE implementation.

I also removed killer moves, singular extensions, mate distance pruning, multi cut and razoring for the same reasons.

I only play some games at Li-chess Stockfish level 5 (3 seconds time control) for testing. So my testing is very poor, for it costs me too much time.

Only fool proof descriptions of solutions seem to work for my program.

[Codesquid]

Only fool proof descriptions of solutions seem to work for my program.

What does that imply about you?

So my testing is very poor, for it costs me too much time.

You need proper testing to improve.

[ZirconiumX]

I am not interested in improvements of 10 ELO or less if I am at least five hundred ELO points behind.

50 Elo is 25 great ideas worth 2 Elo.

Equally, 500 Elo is 50 great ideas worth 10 Elo.

Matthew:out

[Sven]

What I mean with "Does not work" is that it fails to improve the chess program significantly. I am not interested in improvements of 10 ELO or less if I am at least five hundred ELO points behind.

So SEE pruning of bad captures does not work in my program at depth <= 2. But I don't have the fastest SEE implementation.

I also removed killer moves, singular extensions, mate distance pruning, multi cut and razoring for the same reasons.

If you have no history heuristic yet then a straightforward killer move implementation should usually result in at least 20 ELO points improvement. If you have no futility pruning yet then I think that razoring should improve strength by 30 or 40 ELO points at least. For singular extensions, mate distance pruning, multi cut and SEE pruning I am not sure but all of these will give *some* improvement at least.

What you are searching for does probably not exist, there is no "killer feature" in chess programming beyond alpha-beta, quiescence search and null move pruning from which you can expect 100+ ELO points improvement when adding it. Strong chess programs have no bugs, are tested and tuned really well, have lots of small things inside that give some small improvement, have all state-of-the-art search features, usually have a decent evaluation function, are fast, and are tested, tested and tested.

I only play some games at Li-chess Stockfish level 5 (3 seconds time control) for testing. So my testing is very poor, for it costs me too much time.

So in fact you know almost nothing about the effect of your changes. You *guess*, from admittedly poor testing, that your changes do not improve your program significantly. I suggest to stop guessing and to start measuring correctly instead.

Also try to get a rough idea of the strength of your engine relative to some other known engines (I would try to find a couple of opponents that score 40-60% against your engine in many games so that you get a reliable estimate). If it turns out that your engine has, say, a strength of ~2500 ELO (you wrote "at least 500 points behind") then don't worry, it will already be difficult on that level to improve it by simply adding killer moves or singular extensions. But I assume that isn't the case (I never heard of a 2500 engine without killer moves).

If the strength is more about 1500 ELO then don't bother with medium-level search improvements like those you enumerated above, but start hunting your bugs instead. This will bring you some hundreds of ELO and a lot of satisfaction when you are done with it.

Only fool proof descriptions of solutions seem to work for my program

Not proven

Sven

[bob]

It often happens that when you implement a solution it does not work. Sometimes it is caused by bugs but other times it can be because you use other stuff in the remainder of your chess program.

It would be nice if descriptions of solutions would be more complete by listing some conditions that say this solution only works if these conditions are valid.

For instance SEE in quiescence search or Lazy evaluation does not work for my program because probably my evaluation is too minimal. Same holds for futility pruning below level two because these lower levels are often skipped by reductions in null move or LMR.

Perhaps multi cut does not work when you use null move pruning. Singular extensions also did not give much improvement for my program. Don't know why.

I don't think that is possible. I am not sure I can think of all the things that might break a workable idea. for example, you implement LMR. And you see a +80 Elo improvement, which is in line with what most others see. And then you see someone mention null-move as a +80 idea. You try it and only get +40. And you think "I have a bug." In reality null-move and LMR are complementary concepts, adding either is a big gain. Adding the other is a much smaller gain.

Another examples. Rook mobility. You add this and get zero improvement. A bug? A bad idea? Or could it be that you have complementary evaluation concepts that together provide the same effect. For example, rook on open file certainly enhances mobility. Leaving the file hurts mobility. Doing both might be just a duplication of effort with no new knowledge added.

Every chess engine is a huge collection of bits and pieces of implementation, code written by the author. You might take care of some things in one way, while others do it another way. You add their way to your code and get nothing, or even a regression. I am not sure how you could enumerate all the possible pre-conditions that might prevent an idea from working. There are likely an infinite number of such pre-conditions for each unique idea. More importantly, most want to develop ideas that work, and then move on, not try to enumerate all the things others might have done that would make this idea ineffective if tried in their code...

[Henk]

That's bad news that there is no killer feature. This means that it's a very long way to go. Problem with these small ELO improvements is that it takes a lot of games to proof that it really is an improvement.


I removed killer moves for I thought that the first move (from hash or IID) should almost always be the best move. This means that killer moves should almost always be worse than the first move tried.

Second reason is the extra time needed for administrating killer moves.
And of course the less code the better.

At this moment I only use piece square table for move ordering of non captures. Perhaps I should add Killer moves again to my chess program.

[JVMerlino]

That's bad news that there is no killer feature. This means that it's a very long way to go. Problem with these small ELO improvements is that it takes a lot of games to proof that it really is an improvement.

I removed killer moves for I thought that the first move (from hash or IID) should almost always be the best move. This means that killer moves should almost always be worse than the first move tried.

Second reason is the extra time needed for administrating killer moves.
And of course the less code the better.

At this moment I only use piece square table for move ordering of non captures. Perhaps I should add Killer moves again to my chess program.

Speaking as somebody with a very mediocre engine, I can say that if you implement the basics (Alpha-Beta, PVS, qsearch, null move, hash table and a reasonable evaluation with king safety and pawn structure), and do that without any crippling bugs or a horrifically slow implementation, you should be able to achieve an engine that is at least 2000 ELO on CCRL. If it is very fast and well-written, it could easily be 2300 or more. Greko and Sungorus are good examples of a very small codebase providing surprising strength.

Note that it took me about one year to achieve an engine that rated 2000 CCRL, because I worked on Myrddin very irregularly and was very poor at testing. In the three years since then, again with very irregular effort but with at least some sort of formal testing procedure, I've only managed about a 350 ELO increase.

You don't make it clear how strong you EXPECT your engine to be. But my point is that it takes a long time to make even a decent engine. And once you get past a certain point it can be very difficult to make any improvement at all. In the last year I haven't been able to improve Myrddin at all, much to my dismay.

But it can be a fun ride....

jm