Thoughts on Komodo 11, Free engines and the future

[leavenfish]

If you ask me (and no one has of course....) in furtherance of greater and greater elo, 'speedup's' and search technique seems to be about all any programmer/developer of the top engines are really going for at the moment. I mean...K11 is released with a 4 or 5 'elo improvement' over K10.4....perhaps though that was just because it was time to get a fully 'new' version to Chessbase (?). But it seems pretty clear we may be at an impass when it comes to any notable leap in improvement. True, we cannot see the future, but can we not at least acknowledge that this possible truth may just be staring us in the face?

I'm a Komodo subscriber and as such appreciate ANY overall improvements being made available to me, but I find this something interesting to speculate on....not just for Komodo but other top engines.

I know the trend for Team K seems to be to have less emphasis on knowledge and more on these 'speed-ups' and refined searches and the various tweaks made in furthering those. But, if in the end computer chess is only about as deep a search as one can obtain as fast as one can with as little 'knowledge' as need be...then are we not at a crossroads where developers are gravitating to that mean and we may only ever seen incremental improvments? Heck, we can always get 'more' simply by using faster processors and more cores, etc.

If that is likely to be so, then I feel those who own the (commercial) programs need to offer...something more. I am not sure what that is, but that is the point of this tread - to discuss what a commercial program could do to make itself more (dare I say) 'relevant' when free engines (Stockfish and its derivatives) continue to offer essentially the same and even outpace the commercial engines a bit.

One could argue that it is simply the programs which make use of these engines where one should look to make them more useful in non-game play areas. Lets say how they are used WITHIN programs like Chessbase or Chess Assistant (or Aquarium) or the nice training programs like Chess Position Trainer or Chess Openings Wizard - both VERY similar? Okay, maybe so...but a free engine is essentially as good as a commercial engine there, right? I mean, I'm using the 'Deep Analysis' feature in Chessbase as I type this, but even there, isn't a current development version of Stockfish every bit as good as Komodo 11.1?

Be that as it may, what about relevance at the engine level? I know Team Komodo has made attempts by offering interesting ideas one can tweak oneself. Items like 'White Contempt', but is that about all we will ever get when it comes to engines alone? Is that all we should expect for our $$? Just wondering...

[yanquis1972]

engine progress came to a complete standstill before fruit (to my knowledge, maybe there was a predecessor). i think it was years with no elo growth. after fruit came rybka, which (to my memory) was tactically inferior to other top engines but slaughtered all of them. by rybka 3 it was a perfect monster & nothing could touch it.

so incremental progress is where we're at now; at least it's still progress & a 3rd party is there to be nipping at the heels in k-team. from my perspective as an end user, i'm fully confident t hat eventually someone will find something in SF code or a new kid will show up out of nowhere (is this what happened w/ fruit? and then rybka following it...and i really don't mean to turn this into an ethical/legal debate...) with a radical idea that leads to massive gains & the goalposts will shift again.


(edit: either didn't read or somehow didn't see your actual question, which i think is a great one & something i've been meaning to really think about. apologies for the tangent)

[mjlef]

engine progress came to a complete standstill before fruit (to my knowledge, maybe there was a predecessor). i think it was years with no elo growth. after fruit came rybka, which (to my memory) was tactically inferior to other top engines but slaughtered all of them. by rybka 3 it was a perfect monster & nothing could touch it.

so incremental progress is where we're at now; at least it's still progress & a 3rd party is there to be nipping at the heels in k-team. from my perspective as an end user, i'm fully confident t hat eventually someone will find something in SF code or a new kid will show up out of nowhere (is this what happened w/ fruit? and then rybka following it...and i really don't mean to turn this into an ethical/legal debate...) with a radical idea that leads to massive gains & the goalposts will shift again.


(edit: either didn't read or somehow didn't see your actual question, which i think is a great one & something i've been meaning to really think about. apologies for the tangent)

I suppose it all depends upon what you mean my "incremental", but elo growth over the last few years has been amazing. I think one or two of my ideas made it into Komodo 6, but lets start with that as a starting point. On the CCRL 40/40 list, Komodo 10.4 is 172 elo stronger than Komodo 6:

http://www.computerchess.org.uk/ccrl/40 ... t_all.html

This is all 4 core/thread runs at a pretty long time control. I picked 4 threads since I think the average home user serious about chess probably has a quad. Stockfish and other programs show similar improvement. Looking over the last few years it looks like this is the golden age of chess, with elo improvements happening very quickly. Not long agao you would be happy to see a 20 or 30 elo improvement in a year. Note the CCRL matches try to calibrate the time controls so each machine uses a similar about of cpu, so those gains are not due to hardware improvements (which have slowed way down). And that is in less than 4 years. I do not think that is "incremental". Speaking of Komodo, the way it does most things has changed while I have worked on it. From move generator to eval to search/pruning. Some are small tweaks of existing terms, but those never seem to be worth much. Most are new ideas that we have no tried before. And I think that is why Larry and I like computer chess. It is fun to discover something new, especially if ti is something that you might be the first to find.

I agree someone could come along with some great new idea(s) and blow the other programs away. I think that would be great, even if we are left behind. And we certainly cannot predict when we will make an improvement. Sometime you hit 5 in a few, and other times it takes weeks to find a meaningful change. We all could hit a brick wall at any time. But not yet, hopefully.

[Vladimir Xern]

One of the avenues for exploration that might become increasingly relevant would be in parallelizability. I don't know how tenable entirely shifting the paradigm away from alpha-beta to something weaker but more parallelizable is, but it very well could be on the horizon with the way hardware is progressing.

[Ras]

I think the ELO race has become pointless, in a certain way. Of course, it is a challenge for the programmers, but what is the cash value for the customer? Does it even matter whether the engine is ELO 3000 or 3200?

I've bought DeepShredder 13 because I've been buying DeepShredder since version 11, and the time between new versions is long enough that it's more like "oh cool, shopping time" for me. I don't care about a hundred bucks every two or three years.

The user interface is very simple, I've always liked it. It's visually pleasing and easy on the eyes. I really dislike complex UIs that often are part of open source software. Every possible option crammed into even more config stuff. Not every open source software reaches the WTF-level of Git, but the tendency seems to be there.

I did try Arena once.. and uninstalled it. I found it too confusing. Getting the GUI right is not easy, mostly because developers have to make the shift from thinking in features to thinking in workflows. I can have various configuration options in the Shredder GUI, but I don't care to figure out stuff. The defaults work fine.

Of course, I can figure out things if I care - it's not that I'm stupid (I think). The release notes of GCC with every new compiler version are interesting read for me. Or the data sheets of the microcontroller for my chess system, including the errata sheets.

But as soon as I pay money for software, I expect that I don't have to think because the developer has been arranging everything so that I can just be fine. That's exactly what I'm willing to pay for - convenience, which not the same as ELO performance.

I couldn't care less whether I may configure the engine using -use-obscure-option=some-useless-label or not. But is there a convenient ELO slider to reduce the engine strength? And does the reduction work in a convincing way? I mean, not like playing 40 moves like Carlsen on steroids and then throwing away the queen? Does the GUI provide a decent eval plot after analysing a game? And can I install stuff with a double click and everything just works automagically?

Next big thing, copy protection. I understand this is an issue for commercial engines, but I will not accept any real hassle. Shredder's solution with a code tied to my name that I copy/paste and no limitations on my number of computers is OK. Crackers will crack everything anyway, so why annoy paying customers. Especially since I expect cracked software to contain all sorts of malware.

Since the days when Chessbase came up with the crazy idea of using malformatted floppy disks that keep track of how many times you install Fritz, I've sworn to never buy anything from this company, with so customer hostile priorities. Really, their copy protection worked as a permanent purchase protection.

Especially with the open source competition, I think there is no room anymore for copy protection that enerves paying customers.

[Lyudmil Tsvetkov]

If you ask me (and no one has of course....) in furtherance of greater and greater elo, 'speedup's' and search technique seems to be about all any programmer/developer of the top engines are really going for at the moment. I mean...K11 is released with a 4 or 5 'elo improvement' over K10.4....perhaps though that was just because it was time to get a fully 'new' version to Chessbase (?). But it seems pretty clear we may be at an impass when it comes to any notable leap in improvement. True, we cannot see the future, but can we not at least acknowledge that this possible truth may just be staring us in the face?

I'm a Komodo subscriber and as such appreciate ANY overall improvements being made available to me, but I find this something interesting to speculate on....not just for Komodo but other top engines.

I know the trend for Team K seems to be to have less emphasis on knowledge and more on these 'speed-ups' and refined searches and the various tweaks made in furthering those. But, if in the end computer chess is only about as deep a search as one can obtain as fast as one can with as little 'knowledge' as need be...then are we not at a crossroads where developers are gravitating to that mean and we may only ever seen incremental improvments? Heck, we can always get 'more' simply by using faster processors and more cores, etc.

If that is likely to be so, then I feel those who own the (commercial) programs need to offer...something more. I am not sure what that is, but that is the point of this tread - to discuss what a commercial program could do to make itself more (dare I say) 'relevant' when free engines (Stockfish and its derivatives) continue to offer essentially the same and even outpace the commercial engines a bit.

One could argue that it is simply the programs which make use of these engines where one should look to make them more useful in non-game play areas. Lets say how they are used WITHIN programs like Chessbase or Chess Assistant (or Aquarium) or the nice training programs like Chess Position Trainer or Chess Openings Wizard - both VERY similar? Okay, maybe so...but a free engine is essentially as good as a commercial engine there, right? I mean, I'm using the 'Deep Analysis' feature in Chessbase as I type this, but even there, isn't a current development version of Stockfish every bit as good as Komodo 11.1?

Be that as it may, what about relevance at the engine level? I know Team Komodo has made attempts by offering interesting ideas one can tweak oneself. Items like 'White Contempt', but is that about all we will ever get when it comes to engines alone? Is that all we should expect for our $$? Just wondering...

I guess for a couple of hours interior design(house painting) you will have to pay more, so Komodo is not that expensive after all.

who said Komodo and SF did not implement any knowledge? that is far from the case, for me, they offer seach and eval, speed and knowledge, in about equal measure. besides, seach is also necessarily knowledge-related.

main difficulties in implementing more knowledge in current top engines are:
- inherent evaluation redundancies, that are sometimes so nasty and difficult to trace, that I guess programmers simply mostly give up on doing that and instead tune the untunable in countless games
- the fundamental chess engine structure, that is basically the same for all engines around - from top to bottom - and that necessarily involves the interconnection of search and eval, making tuning even more problematic. In a pure evaluator, one can try getting rid of redundancies, without being afraid that somewhere there down the search tree, maybe at ply 10 or 15, we are pruning lines we are not supposed to prune based on our evaluation, etc., etc.

what can current top engines still offer?

well, of course, more and more ELO.

current top engines are still relatively weak, so plenty more to expect from them.

[Mike S.]

There is a continous Elo gain, but it has become artificial long since. It is measured by AI versus AI on strength levels beyond human grasp. Substantial progress would include to solve "intrinsic" problems of chess engines, like fortress detection, trapped pieces, desperado rooks...

Or I'm not up to date and some or all of these problems have already been solved, but by which engine?

[Lyudmil Tsvetkov]

There is a continous Elo gain, but it has become artificial long since. It is measured by AI versus AI on strength levels beyond human grasp. Substantial progress would include to solve "intrinsic" problems of chess engines, like fortress detection, trapped pieces, desperado rooks...

Or I'm not up to date and some or all of these problems have already been solved, but by which engine?

I mostly agree with this.

things that have not been solved until now are generally more difficult to solve, either because they involve deeper search, or much more sophisticated eval.

also, starting from an existing point, it is extremely difficult to add new evaluation features/knowledge, simply because redundancies would be big and it is not fully clear how to completely get rid of those.

I guess most engines would see desperados in their search, in most cases, as eg lines are shallower, fortresses are a big problem, as so various as needing about 2 billion lines of code to fully encompass them, and trapped pieces also are frequently associated with deeper search and at times intricate eval.

if all 3 of above are solved, I would expect 200 elo boost in a top engine at most.

there are however other, more salient knowledge features lacking, pertaining to pawn structures, outposts, king safety and closed positions structures, that could cumulatively add, only in eval, at least 2000 elo...

[Lyudmil Tsvetkov]

here probably one of the weightiest eval features, partaining to central pawn structures/king safety, that is so frequent, and usually found right out of the opening, that it can bring an enormous amount of elo.(almost no current engines recognises it fully, otherwise why would SF pick the French?)

[d]4k3/5pp1/4p3/4P3/8/8/8/8 w - - 0 1

rule might be as follows:

white pawn on e5, black pawns on e6,f7,g7, black king on e,f,g or h file, or, alternatively, on the queen side, white pawn on d5, black pawns on d6,c7,b7, black king on d,c,b or a file, some bonus for white(who knows how big, I can only estimate:) )

mirrored for black.

this configuration happens in around 1/3 of all games, can happen, I mean, right out of the opening. and yet engines fully do not consider it.

when will engines be able to implement it?

maybe never. why? because:

- there are redundancies and the engine is already very well tuned; the e5 white pawn might be any white pawn, suppoted or isolated, part of a chain or not, blocked by enemy pawn or not, passed pawn, lever, God knows what else, etc., etc.
- tests with thousands games at TC of 1 min., a bit more or less, will most probably fail, as this has to do with king safety, involving depths of 30-40 moves, and the engine search will usually point in another direction, preferring moves that are counter-productive for this particular feature
-etc., etc., etc.

so that, unless Daniel succeeds in implementing it, obviously no one is going to be able to do so...

so long for the future of computer chess.

[Lyudmil Tsvetkov]

[d]r1b1kb1r/pp1n1ppp/2n1p3/q1ppP3/N2P1P2/4BN2/PPP3PP/R2QKB1R w KQkq - 0 9

SF, having chosen the losing French, considers the position equal and would like to repeat here with Nc3 Qb6 Na4, etc. Instead, after c3, black is quite probably already lost.

SF does not recognise the e5,e6,f7,g7 pawn structure, it considers just any pawn on e5, some e5 passers, some e5 defended pawns, some e5 connected/duo pawns, and that is about all SF has in its code about the e5 pawn.

I pretty much guess, the e5 pawn would be scandalised, if it knew about this...

seemingly, the French is bad, because already on the 3rd move, after 1.e4 e6 2.d4 d5 3.e5, white can achieve the aforementioned central pawn structure.