Komodo 4 MP

[MM]

The MP work is basically a one-time thing. Once it's done, all future versions of SP will be automatically also be available as MP. Of course it's always possible to improve MP without working on SP, but any gains from this should be pretty small on normal hardware.

Hi Larry, the real problem of Komodo is that when it finally is released other engines will be much stronger than now, i refer to Rybka, Houdini, Critter and perhaps Shredder (i doubt anyway).

Wasn'it better to release one or more intermediate versions, SP, instead of trying to get the best goal in only one shot?

Regards

Hi Maurizio,

That's not what is happening here. We basically get interested in making releases when we perceive there has been a large enough improvement to make it exciting for everyone. We don't have a fixed release schedule.
Don

Hi Don,

sorry if i cut your post but i saved what i consider the most important thing that you said so the question is pretty natural: isn't Komodo already significantly stronger than Houdini even in blitz? I think yes.

Best Regards

[Don]

The MP work is basically a one-time thing. Once it's done, all future versions of SP will be automatically also be available as MP. Of course it's always possible to improve MP without working on SP, but any gains from this should be pretty small on normal hardware.

Hi Larry, the real problem of Komodo is that when it finally is released other engines will be much stronger than now, i refer to Rybka, Houdini, Critter and perhaps Shredder (i doubt anyway).

Wasn'it better to release one or more intermediate versions, SP, instead of trying to get the best goal in only one shot?

Regards

Hi Maurizio,

That's not what is happening here. We basically get interested in making releases when we perceive there has been a large enough improvement to make it exciting for everyone. We don't have a fixed release schedule.
Don

Hi Don,

sorry if i cut your post but i saved what i consider the most important thing that you said so the question is pretty natural: isn't Komodo already significantly stronger than Houdini even in blitz? I think yes.

Best Regards

It's hard to compete with either Critter or Houdini at anything faster than 5 minutes on good hardware as both these program are amazingly good at these time controls. But with 5 minutes and an increment such as 3 to 5 seconds they are beatable. I think 5+5 is about where Komodo equalizes and beyond that Komodo is stronger than Houdini 1.5. There is almost no difference in Houdini 2.0 and 1.5 except at these type of fast time controls but we need about 10 minutes plus decent increment to equalize Houdini 2.0 which is amazingly good at fast chess.

Our testers use a version of Komodo that is about 20% crippled in speed due to compiler issues and even the crippled versions are well above Komodo 4 from THEIR tests so we have no complaints about the performance. Of course we always wish it was more ...

We expect that when we release, our superiority on most of the lists will be very temporary (if we release first) but it's pretty obvious our progress has been steady so we are not going to change anything we are doing and if we can keep up the same pace we might be able to stay on top and hold it for a year or two - at least until I get too sick to continue or Larry gets too old We think we are doing pretty well for being two scrappy old dogs.

If Critter releases first, and I think a release is imminent, my sense is that it will be at least as good as Houdini 2.0 and perhaps even better than Houdini 1.5. We could see a situation where there would be 3 programs that are virtually tied at the top with no substantial strength difference between them and on many lists Houdini could even be in third place!

Houdart says he has made incredible progress and has something that is going to blow everyone away so we will see how that works out - but his program will never dominate again, already 2 programs have virtually caught up after a couple of years of eating dust so really he has no choice but to respond with something that is substantial to maintain any sort of credible lead. If he does, then we will just keep dogging him until we wear him out. Maybe someone else will come along and pass all of us?

Somehow I'm not too worried about Rybka. But don't count Vas out - he could definitely be a contender to keep an eye on. The Stockfish team is very potent too but they are not active right now - I would love to see them pick back up and harass us (but from behind please.)

I believe Komodo will be the best of the 3 programs if Critter and Komodo were to release today, but it remains to be proved. If it is, this next version will not have an undisputed superiority because you have to actually be at least 40 ELO ahead in reality to overcome the error margins and different testing conditions and different hardware on the various lists. Every program is going to "like" some hardware better than others, some time controls better than others and some testing conditions and books better than others. Some tests subject you to opponents hundreds of ELO weaker and some don't and some programs probably do a little better playing down or up - not to mention the margins of error that every tester has to deal with.

Don

[MM]

The MP work is basically a one-time thing. Once it's done, all future versions of SP will be automatically also be available as MP. Of course it's always possible to improve MP without working on SP, but any gains from this should be pretty small on normal hardware.

Hi Larry, the real problem of Komodo is that when it finally is released other engines will be much stronger than now, i refer to Rybka, Houdini, Critter and perhaps Shredder (i doubt anyway).

Wasn'it better to release one or more intermediate versions, SP, instead of trying to get the best goal in only one shot?

Regards

Hi Maurizio,

That's not what is happening here. We basically get interested in making releases when we perceive there has been a large enough improvement to make it exciting for everyone. We don't have a fixed release schedule.
Don

Hi Don,

sorry if i cut your post but i saved what i consider the most important thing that you said so the question is pretty natural: isn't Komodo already significantly stronger than Houdini even in blitz? I think yes.

Best Regards

It's hard to compete with either Critter or Houdini at anything faster than 5 minutes on good hardware as both these program are amazingly good at these time controls. But with 5 minutes and an increment such as 3 to 5 seconds they are beatable. I think 5+5 is about where Komodo equalizes and beyond that Komodo is stronger than Houdini 1.5. There is almost no difference in Houdini 2.0 and 1.5 except at these type of fast time controls but we need about 10 minutes plus decent increment to equalize Houdini 2.0 which is amazingly good at fast chess.

Our testers use a version of Komodo that is about 20% crippled in speed due to compiler issues and even the crippled versions are well above Komodo 4 from THEIR tests so we have no complaints about the performance. Of course we always wish it was more ...

We expect that when we release, our superiority on most of the lists will be very temporary (if we release first) but it's pretty obvious our progress has been steady so we are not going to change anything we are doing and if we can keep up the same pace we might be able to stay on top and hold it for a year or two - at least until I get too sick to continue or Larry gets too old We think we are doing pretty well for being two scrappy old dogs.

If Critter releases first, and I think a release is imminent, my sense is that it will be at least as good as Houdini 2.0 and perhaps even better than Houdini 1.5. We could see a situation where there would be 3 programs that are virtually tied at the top with no substantial strength difference between them and on many lists Houdini could even be in third place!

Houdart says he has made incredible progress and has something that is going to blow everyone away so we will see how that works out - but his program will never dominate again, already 2 programs have virtually caught up after a couple of years of eating dust so really he has no choice but to respond with something that is substantial to maintain any sort of credible lead. If he does, then we will just keep dogging him until we wear him out. Maybe someone else will come along and pass all of us?

Somehow I'm not too worried about Rybka. But don't count Vas out - he could definitely be a contender to keep an eye on. The Stockfish team is very potent too but they are not active right now - I would love to see them pick back up and harass us (but from behind please.)

I believe Komodo will be the best of the 3 programs if Critter and Komodo were to release today, but it remains to be proved. If it is, this next version will not have an undisputed superiority because you have to actually be at least 40 ELO ahead in reality to overcome the error margins and different testing conditions and different hardware on the various lists. Every program is going to "like" some hardware better than others, some time controls better than others and some testing conditions and books better than others. Some tests subject you to opponents hundreds of ELO weaker and some don't and some programs probably do a little better playing down or up - not to mention the margins of error that every tester has to deal with.

Don

Thank you Don for the very exhaustive post. I think you have the ''control'' of the situation now, you just need to decide when to release a next Komodo and i believe you will do it when it is strong enough to be anyway better than an improved version of Houdini or Critter. But don't forget the tactics. I played a huge number of times Komodo and i keep on thinking that if Komodo was at Houdini level in tactics it would be at least 50 elo stronger than it at any time control (that means that your engines is clearly better in positional play).

Best Regards

[Werewolf]

Yes, for analysis tactics matter. Even though Komodo 4 was quite a bit stronger than Komodo 3, its tactical strength dropped a bit.

Still a very fine program though.

[lkaufman]

How did you determine that its tactical strength dropped? I don't recall any report of a decline in performance on tactical test suites, but I may have missed one.

[Werewolf]

Hi Larry,

I've been running tactical tests for years and lately it's getting challenging to find positions which are still hard for computers! Every time there is a new engine out I test it and report the findings on the Rybka forum. My test suite is 100 positions, so obviously if you have data which contradicts it from another suite that needs to be considered.

You can find the latest list here:

http://rybkaforum.net/cgi-bin/rybkaforu ... ;hl=barren

Your program has only been tested on the quad simply because it's not MP yet.

[Don]

How did you determine that its tactical strength dropped? I don't recall any report of a decline in performance on tactical test suites, but I may have missed one.

I ran a couple of tactical tests and Komodo 4 actually showed an increase in tactical strength, but tactics are somewhat subjective because there are difference kinds and different sets. A check extension change can increase the tactical power of a chess program in positions where checks matter, but could hurt the performance a little when they don't.

[Werewolf]

How did you determine that its tactical strength dropped? I don't recall any report of a decline in performance on tactical test suites, but I may have missed one.

I ran a couple of tactical tests and Komodo 4 actually showed an increase in tactical strength, but tactics are somewhat subjective because there are difference kinds and different sets. A check extension change can increase the tactical power of a chess program in positions where checks matter, but could hurt the performance a little when they don't.

I've tried to combine positions which are "checky" with those which aren't for that very reason. However, I'd say at least half of my positions are of the checking type, which may hurt Komodo's results a little.

[Don]

How did you determine that its tactical strength dropped? I don't recall any report of a decline in performance on tactical test suites, but I may have missed one.

I ran a couple of tactical tests and Komodo 4 actually showed an increase in tactical strength, but tactics are somewhat subjective because there are difference kinds and different sets. A check extension change can increase the tactical power of a chess program in positions where checks matter, but could hurt the performance a little when they don't.

I've tried to combine positions which are "checky" with those which aren't for that very reason. However, I'd say at least half of my positions are of the checking type, which may hurt Komodo's results a little.

Komodo 5 will be a bit more aggressive with checks so perhaps it will do better on the test. Is your suite publicly available? I can check it for you if you want.

Don

[Werewolf]

Don I would love to help you in any way I can. However, some people on the Rybka forum urged me not to make the suite publicly available last year, so I haven't.

However, I do have some older test suites which are of a similar nature and I'll email you something in the next few days.