Looking for advices

[xr_a_y]

Dear all,
I've started writing my own chess engine nearly a year ago, just for fun. I really want to thank CPW contributors, forums members, ... for making this science freely accessible today. So, thank you all !

Today my engine has all the classic stuff (at least my own interpretations of them):

• alpha-beta framework (not negamax)
  • PVS (root and alpha-beta)
    Window
    IID
  TT (mainly for sorting, seems buggy if used for cut-off)
  QTT (idem)
  EvalTT
  LazyEval
  Sort with
  • Piece position evaluation
    MVV-LVA
    Killer
    History
    SEE (not working ...)
    Checks
  QSearch
  • SEE (not working)
    Delta prunning (not working)
  Futility pruning (extended with adaptative margin)
  Razoring (reduction)
  Static null move (reversed futility pruning) : drop to QSearch
  Null move prunning (adaptative)
  Extension
  • Single reply
    Check
    Promotion (and near promotion)
    End game
    Very end game
    PV
  LMR (adaptative)
  Book
  • Small internal
    Big, read from file (too slow !)
  Evaluation
  • material
    position
    bishop pair
    mobility (global and by piece) (too expensive)
    pawn push
    pawn shield
    castling
    rook connected
    rook on 7th rank
    center control (too expensive)
    king troppism (very expensive)
    pawn structure
    open file
    • double
      isolated
      passed
      protected passed
      connected
      king too far (buggy)
  XBOARD support (partial)

The engine is quite slow (around 500knps single thread, and by node I mean all node and qnode). The current bottleneck is the "getthreats" function (that many people called "attack"), especially if I activate more complexe evaluation (like king troppism, center control, ...)

I cleary see that my LMR + other pruning technics parameters are not optimal, as the engine often make mistakes when all of them are activated.

I also remark that my evaluation is quite unstable in the sense that is black see -45 then white see +80, then black -120, then white +20. I often see more than a pawn difference in the evaluation of move n and n+1.

At the end, the engine has a elo rating around 2200 in classic tests suite. And when playing agaist fairy-max, it lose more often than it win.

I thus can work on at least three things :
1°) make the engine faster (I mean sequentially using more bitboard and also using multiple threads)
2°) make my evaluation smarter : I cleary lack some more pawn structure knowledge for exemple
3°) try to find better parameters for my pruning

I really need some advices here. I really hope the engine can 2500 by the end of the year

Best regards

[AlvaroBegue]

I would start by simplifying the engine dramatically and working on making sure everything you add doesn't have bugs and adds performance. You should be able to beat Fairy-Max with half as many features as you listed.

Test that each of the features you implemented actually adds Elo, by playing quick games between versions with and without the feature. If it's unclear, take the feature out. You can always try it later, but having a bunch of half-working things in the program only makes progress harder.

2500 by the end of the year is ambitious, though...

[Dann Corbit]

Make sure it is correct before making it faster.
Then, when you want to make it faster, profile first.
After you profile, try to improve the algorithms that are slow.

If you post the code, I imagine people can critique it for you.

[lauriet]

I can only agree with the comments so far.
You need to start with a simple implementation and progressively adding each feature making sure you understand how it is working and it is doing what you think it is......bug free.
From your post, it is obviously not bug free. If you can't figure out the bugs then you are not understanding your code as well as you should. This is not a criticism of you, just advice.
So "incremental implementation" is the way to go.

Laurie

[Sven]

Hi Vivien,

welcome to TalkChess! It seems you've done a great amount of programming work so far, congrats! Now according to your description there are really a lot of things going wrong. That's quite normal at this stage

BUT: As soon as you have realized what is going wrong it would be "not normal" to continue in the same way as you did before. You should become aware that there is a better way to write a strong chess engine. The preferred way is not to have 50 features with 25 bugs (or so) but to have few features without bugs, then add the next one and remain bug-free, then again the next one, and so on.

Therefore I seriously suggest that you disable at least the following features from your list before doing anything else:

TT+QTT
SEE
Delta pruning
all extensions except check extension
opening book
LazyEval
*all* evaluation features except material and "position" (assuming this is what others often call "piece-square table")
and maybe also some of the search features (or are you really sure that null move, IID, LMR, Futility pruning, Razoring are working perfectly as intended?)

If your move generator is "proven" to be correct (based on "perft" tests) then establish a first rating estimate by playing 1000 very fast games against FairyMax. Then I would enable your TT and debug it carefully until it works for both TT cutoff and move ordering, this is crucial. Adding a working TT should give a substantial improvement, test it by playing - for instance - 1000 very fast games against the non-TT version.

Continue like that, adding only one feature at a time and doing a lot of tests and debugging until you fixed all bugs. I always found it easier to add and test search features with a minimalistic evaluation because even small, subtle evaluation bugs in complex code can have a huge effect and lead to being unable to get any improvement from adding some search features. In my view a correct basic engine with a decent search is a good starting point, a better evaluation can be added later.

Regarding Elo ratings, try to be realistic. Making less than 50% vs. FairyMax does not sound like a 2200 rating (and calculating a rating only based on solving some tactical positions may be quite misleading), more like clearly below 2000. If you follow most of the advice given here your engine will become stronger soon.

Playing many very fast test games also requires to have a bug-free time control implementation. Use a fixed set of opening positions, and use a tool like cutechess-cli for playing the games. Building a simple but working test environment is crucial and will actually decide about your success. At some point 1000 games will not be sufficient any longer due to the high error bars compared to the small improvements you are going to make. So be prepared to play even much more than these 1000 games.

I wish you good luck for your project!

[Henk]

Huh. No question about how to compute backpropagation step in a deep convolutional network ? No Monte Carlo Simulation ? Just going on in old fashioned way.

Or was Alpha zero just some kind of Aprils fool joke. Then they fooled me.

[xr_a_y]

Thank you all for the help. I've just started a tournament between different configurations to track down the potential issues. I'll post the code on github as soon as i'm not to ashamed of it

It seems there is something wrong in my pvs when applied at none root node.

Best regards

[xr_a_y]

I'm back !

after a disappointing Winter and a bottom place in March tournament, my engine "Weini" is now able to defeat fairymax quite well : 60-30-22 in a 12sec/40moves tournament (this is around +100elo).

But it takes LMR, SMP (4 threads), pondering and some bug fix of course as you have all advised me last year...

There is still two issues I don't understand well :
*why LMR at root is not good (at all: -75elo)
*why using TT at root is not good (at all: -60elo)

Another thing I don't get well (and it makes my engine slow) is the cost of "IsInCheck" function during threats detection. I know many engines pre-compute pinned pieces once and for all for a position but I found it's even slower in mine.

Still, thank you all for your advices, I now test everything with cutechess tournament.

Best regards

[Daniel Anulliero]

Huh. No question about how to compute backpropagation step in a deep convolutional network ? No Monte Carlo Simulation ? Just going on in old fashioned way.

Or was Alpha zero just some kind of Aprils fool joke. Then they fooled me.

It is far better to Hear some EXPERTS advices and write an old fashion engine for the begenning
We all saw Weini played at hgm's tournament , it is young but sure it can kill Skipper

So vI viens , good work anyway

[Henk]

Play a game against your own engine and against LCZero. Which one did you like best ?