Big new ideas in chess programming

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Steve Maughan
Posts: 1221
Joined: Wed Mar 08, 2006 8:28 pm
Location: Florida, USA

Re: Big new ideas in chess programming

Post by Steve Maughan »

gladius wrote:[...]But, reducing it down to ELO terms, how much comes from the idea vs the implementation choices?
You cannot separate the two.

Philosophically all value added by humans has two component - the thought in the mind and action which implements. Value (in this ELO) cannot be realized without *both* components.

Steve
http://www.chessprogramming.net - Maverick Chess Engine
jefk
Posts: 626
Joined: Sun Jul 25, 2010 10:07 pm
Location: the Netherlands
Full name: Jef Kaan

Re: Big new ideas in chess programming

Post by jefk »

[quote="gladius"]
I think there are definitely ideas out there remaining to be found that will be very impactful. Even within in the current alpha-beta framework, not talking about things like MC searches or something.
[quote]

well you mention M(onte)C(arlo) type of searches.
Maybe such search methods within the context of an engine
can be (further?) enhanced and being used in positions
for which it will be useful; and thus leading to better moves
and higher rating(s) again.

any ideas/opinions (about this) ?

jef
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Big new ideas in chess programming

Post by Don »

jefk wrote:
gladius wrote: I think there are definitely ideas out there remaining to be found that will be very impactful. Even within in the current alpha-beta framework, not talking about things like MC searches or something.


well you mention M(onte)C(arlo) type of searches.
Maybe such search methods within the context of an engine
can be (further?) enhanced and being used in positions
for which it will be useful; and thus leading to better moves
and higher rating(s) again.

any ideas/opinions (about this) ?

jef
Monte Carlo Tree Search is definitely not out of the question, amazing progress in Go has been a result of this. But it is far from obvious how to make it work in Chess. You cannot just copy how it's done in Go.

I tried a very basic and naive go-like approach a few years ago and it was awful. And the current approach is working so well that it's difficult to get motivated to spend a lot of energy on this.

I can tell you a little about the characteristics of such a program. It would play a very natural and intuitive game - appearing to be a very strong positional player and perhaps not so strong tactically. In closed positions it would be far stronger and in positions where the pawns are locked up but you are a queen up the program would not believe it was wining as modern programs do - it would just appear to have a much more intuitive feel for the game. If there were any hope for winning by giving up a piece to open up the pawn structure, it would naturally do what was required with no fuss.

Although I say it would not be particularly strong tactically it would have no hesitation for sacrificial play if this increased it's odds of winning. Such a program done well would be a joy to watch play. It would also be a fighter when it was losing - giving the opponent a great deal of resistance and picking moves which made the opponents win much more difficult. On the other hand when it was dead lost with no hope of winning, it would tend to play a bit aimlessly as these types of programs need to have something to work with. As long as there is some non-trivial chance of drawing or winning they will play great.

Larry and I want to experiment with this soon - we feel that even if it's not as strong as Komodo it might be very appealing to users.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Werewolf
Posts: 1796
Joined: Thu Sep 18, 2008 10:24 pm

Re: Big new ideas in chess programming

Post by Werewolf »

It's great to hear that you two are still full of ideas and have lots of things left to try.

I'm hoping to start some really big (100 cores) IDeA projects soon but it'd be completely pointless to start before Komodo 6 comes out.

What's the latest estimate on its release?
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Big new ideas in chess programming

Post by Don »

Werewolf wrote:It's great to hear that you two are still full of ideas and have lots of things left to try.

I'm hoping to start some really big (100 cores) IDeA projects soon but it'd be completely pointless to start before Komodo 6 comes out.

What's the latest estimate on its release?
We have some very interesting plans, but we have learned that it's counter-productive to reveal them too soon and in fact we don't have a release date.

What I can say is that in the future we do plan to move to a regular twice per year predictable release schedule combined with continuous updates to paying customers and other value-added features to make it a lot more work for pirates. There are a lot of details to work out on this however and we don't know if it will start with the next release or not.

I am also working on a GUI which I hope to be part of future distributions but not in time for Komodo 6. As far as I know it will be the only cross-platform GUI actually marketed with a commercial program. Support for Linux, Windows and Mac.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Werewolf
Posts: 1796
Joined: Thu Sep 18, 2008 10:24 pm

Re: Big new ideas in chess programming

Post by Werewolf »

Don wrote:
What I can say is that in the future we do plan to move to a regular twice per year predictable release schedule combined with continuous updates to paying customers and other value-added features to make it a lot more work for pirates.
Twice per year is a very good move, you've really got that one right.

I'm not sure about the 'continuous updates' bit just because unless they're bug fixes that's a lot of testing to do on rating lists.

But twice a year is a far, far better model than once a year. Even three times a year is possible, if you make progress.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Big new ideas in chess programming

Post by Don »

Werewolf wrote:
Don wrote:
What I can say is that in the future we do plan to move to a regular twice per year predictable release schedule combined with continuous updates to paying customers and other value-added features to make it a lot more work for pirates.
Twice per year is a very good move, you've really got that one right.

I'm not sure about the 'continuous updates' bit just because unless they're bug fixes that's a lot of testing to do on rating lists.
I see Stockfish on all the rating lists even though it's updated every day and this does not seem to be a problem for them. How would this be any different?

The rating lists are not expected to rate a new version every day - so they simply go with the last official release. On the lists you see Stockfish 4, not stockfish-latest-dev.

Probably the way it would work for us is that if you purchase Komodo you get releases up to the official next release and then it would stop unless you choose to purchase again. We would not make EVERY version available, a lot of versions are test versions that do not prove to be actual improvements - so each version we would release would be better in some way according to our belief. (On occasion regressions sneak in.)

But twice a year is a far, far better model than once a year. Even three times a year is possible, if you make progress.
Another option we are considering is a subscription service to Komodo which means any time you purchase you get 6 months or 12 months of updates starting from the day they subscribe. But we still need official release versions for the testers and if we did it this way we would probably have official releases every 4 months.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Werewolf
Posts: 1796
Joined: Thu Sep 18, 2008 10:24 pm

Re: Big new ideas in chess programming

Post by Werewolf »

Now that you've explained it, this makes a lot of sense. Yes I get what you're saying about the rating lists.

This new model reminds me of the heyday of chess in the first decade of the 21st century...Rybka 2.3.2a

Happy times.
rtitle
Posts: 33
Joined: Wed Aug 14, 2013 7:23 pm

Re: Big new ideas in chess programming

Post by rtitle »

As to why Stockfish has risen to near the top and stayed there, while previously top programs (e.g. Rybka) have fallen behind, I think the answer is simply that the open-source development model is superior to the closed-source commercial programming model. In the software development world we refer to this as the "many eyeballs effect" - if you have a lot of people looking at / thinking about / contributing to a given piece of software, you ultimately win against a private company trying to compete with closed source and just a few people funded from its own limited revenues. Even once-mighty Microsoft is losing in the Windows vs Linux war, for a similar reason.

Reading this forum over the years, it's often occurred to me that most of the chess engines being developed are so similar to one another that they're basically variants of the same program. A forum like this simply would not be possible in other softeware fields. E.g., imagine if an IBM DB2 engineer posted "I'm struggling with an issue in DB2" and an Oracle engineer replied "Oh, here's how I solved the same problem in Oracle". Even if company rules allowed such a conversation to occur, it wouldn't be possible because the designs of those 2 programs are so different that the Oracle guy wouldn't know what the IBM guy is talking about. But I *do* see that sort of discussion happening all the time in the "Technical" section of this forum. So I claim you're all really developing variations of the same program. And I've often thought as a follow-up thought, "why not form an open-source project and collaborate instead?". The Stockfish team has finally done that, and I predict as a result that program will prevail vs its competitors. Open source wins in the long run.

Rich Title
IanO
Posts: 496
Joined: Wed Mar 08, 2006 9:45 pm
Location: Portland, OR

Re: Big new ideas in chess programming

Post by IanO »

rtitle wrote:As to why Stockfish has risen to near the top and stayed there, while previously top programs (e.g. Rybka) have fallen behind, I think the answer is simply that the open-source development model is superior to the closed-source commercial programming model. In the software development world we refer to this as the "many eyeballs effect" - if you have a lot of people looking at / thinking about / contributing to a given piece of software, you ultimately win against a private company trying to compete with closed source and just a few people funded from its own limited revenues. Even once-mighty Microsoft is losing in the Windows vs Linux war, for a similar reason.

Reading this forum over the years, it's often occurred to me that most of the chess engines being developed are so similar to one another that they're basically variants of the same program. A forum like this simply would not be possible in other softeware fields. E.g., imagine if an IBM DB2 engineer posted "I'm struggling with an issue in DB2" and an Oracle engineer replied "Oh, here's how I solved the same problem in Oracle". Even if company rules allowed such a conversation to occur, it wouldn't be possible because the designs of those 2 programs are so different that the Oracle guy wouldn't know what the IBM guy is talking about. But I *do* see that sort of discussion happening all the time in the "Technical" section of this forum. So I claim you're all really developing variations of the same program. And I've often thought as a follow-up thought, "why not form an open-source project and collaborate instead?". The Stockfish team has finally done that, and I predict as a result that program will prevail vs its competitors. Open source wins in the long run.

Rich Title
There is obviously more to it than just Open Source. The GnuChess program has been open source for much longer but was merely mediocre. I think development and test methodology are much more important for a top level chess program. The recent distributed Stockfish test framework is a far more likely key to its success, along with the Stockfish team core principles of keeping the codebase clean and actively paring search and evaluation down to essentials.