yanquis1972 wrote:i mention that HT is enabled because some seem to think it makes a difference in performance, & because i am multi-tasking while running the tournament. both engines are set to below-normal priority & while i am not running anything terribly intensive, the results should obviously be taken with a lump of salt. both engines are custom set to use 4 threads, which is my number of physical cores.
It is possible that some engines will respond differently to hyperthreading.
I have not checked it out with Komodo - but I suspect that it will not help.
Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
yanquis1972 wrote:i mention that HT is enabled because some seem to think it makes a difference in performance, & because i am multi-tasking while running the tournament. both engines are set to below-normal priority & while i am not running anything terribly intensive, the results should obviously be taken with a lump of salt. both engines are custom set to use 4 threads, which is my number of physical cores.
It is possible that some engines will respond differently to hyperthreading.
I have not checked it out with Komodo - but I suspect that it will not help.
John only means that he did not disable HT in the BIOS. This should be completely fine unless running on some really old OS, which I am sure is not the case.
I suppose Don is commenting on setting the number of threads to the number of hyperthreads.
correct ronald, both engines are set to use 4 threads, which is = to physical cores on my system, but hyperthreading is on. i figured some may find this relevant.
a link to a tactical test suite, which includes results from all recent engines:
komodo performs relatively awful! (it is 14th of 25 engines on the list; the engines above it are hardly slouches, but komodo is likely stronger than all but 2 of them, rybka cluster & houdini 3 [possibly 2 as well?])
i actually think this is fantastic news, at it seems to suggest 2 things: 1)its evaluation/knowledge is in fact top-notch, making it an excellent complementary engine for quiet positions, & 2)there's tremendous room for improvement. rybka was certainly not the top tactical engine upon release, but it was easily the strongest. by the time rybka 3 rolled around, nothing could touch it in any regard.
yanquis1972 wrote:correct ronald, both engines are set to use 4 threads, which is = to physical cores on my system, but hyperthreading is on. i figured some may find this relevant.
a link to a tactical test suite, which includes results from all recent engines:
komodo performs relatively awful! (it is 14th of 25 engines on the list; the engines above it are hardly slouches, but komodo is likely stronger than all but 2 of them, rybka cluster & houdini 3 [possibly 2 as well?])
i actually think this is fantastic news, at it seems to suggest 2 things: 1)its evaluation/knowledge is in fact top-notch, making it an excellent complementary engine for quiet positions, & 2)there's tremendous room for improvement. rybka was certainly not the top tactical engine upon release, but it was easily the strongest. by the time rybka 3 rolled around, nothing could touch it in any regard.
I don't trust those tactical tests because recently we experimented a bit with root move ordering inspired by a discussion we had here. Random initial root move ordering gave Komodo a huge tactical boost over ordering them by static evaluation. But static evaluation was the best for ELO.
If it was that important I am wondering what else might make a big tactical difference without having much impact on strength?
Anyway, it's not a surprise, Komodo has for a long time been a little behind the other program in raw tactics, even the programs that Komodo is far stronger than.
It's hard to understand why it's not more important.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
don, can't/haven't you experimented with a 'komodo tactical' to, at very least, figure out what parts of the code are causing the poor results? as i said, from my perspective, this seems like a very good thing, as it suggests a lot of elo points are to be gained if tactics can be improved w/out sacrificing 'intelligence', or at least, too much of it. while i know being the #1 engine is your goal, part of komodo's appeal to me has always been its knowledge, or what comes across as such.
it seems to me like, almost all of the time, the top engine in the world is either the best, or very close to the best, tactically. again, rybka was a huge exception, but vas found a way to not only keep but add knowledge, & -massively- improve rybka tactically at the same time.
perhaps with faster hardware (the above test was run on a 12 core monster), this is becoming less relevant, but i also recall people saying the same some 15 years ago (all CS TAL II needs to be competitve is an ultrafast machine, etc), so i tend to disregard this angle.
yanquis1972 wrote:don, can't/haven't you experimented with a 'komodo tactical' to, at very least, figure out what parts of the code are causing the poor results? as i said, from my perspective, this seems like a very good thing, as it suggests a lot of elo points are to be gained if tactics can be improved w/out sacrificing 'intelligence', or at least, too much of it. while i know being the #1 engine is your goal, part of komodo's appeal to me has always been its knowledge, or what comes across as such.
it seems to me like, almost all of the time, the top engine in the world is either the best, or very close to the best, tactically. again, rybka was a huge exception, but vas found a way to not only keep but add knowledge, & -massively- improve rybka tactically at the same time.
perhaps with faster hardware (the above test was run on a 12 core monster), this is becoming less relevant, but i also recall people saying the same some 15 years ago (all CS TAL II needs to be competitve is an ultrafast machine, etc), so i tend to disregard this angle.
We have no desire for Komodo to have a reputation for being weak in tactics, so we have tried to improve this - but we don't obsess about it because it's abundantly clear that solving problem sets does not correlate very well with actual strength of play.
We have tried more aggressive tactical extensions, less pruning, more pruning and other things. The easiest way to get a big tactical boost is to randomize the root move ordering. Does that seem like a good thing to you? Actually, it had only a very minor impact on the ELO, but it was clearly measurable. But it seems the path to getting really good scores on tactical sets is to be willing to give up a few ELO. We could get fantastic scores if we were willing to give up just 5 or 10 ELO, but we are not willing to do that.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Don wrote:(...)We could get fantastic scores if we were willing to give up just 5 or 10 ELO, but we are not willing to do that.
I think tactical ability with chess engines is akin to driving long straight drives in golf. They don't guarantee a better handicap but they make the punters "feel" better. For the average player sacrificing 10 ELO is nothing - but having the engine announce mate in 17 is something which is *amazing*.
As an aside I've tested Komodo 5.1 and it seem to perform well on my (favorite) Big Book of Tactics tests. It seems better than previous version but I have to admit I've only give it a superficial test.
Don wrote:(...)We could get fantastic scores if we were willing to give up just 5 or 10 ELO, but we are not willing to do that.
I think tactical ability with chess engines is akin to driving long straight drives in golf. They don't guarantee a better handicap but they make the punters "feel" better. For the average player sacrificing 10 ELO is nothing - but having the engine announce mate in 17 is something which is *amazing*.
As an aside I've tested Komodo 5.1 and it seem to perform well on my (favorite) Big Book of Tactics tests. It seems better than previous version but I have to admit I've only give it a superficial test.
Best regards,
Steve
We may put in a tactical mode in a future version of Komodo. It's fun working on the tactics of Komodo as a diversion.
Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
Getting so close to challenging H3 for top spot ought to make anyone sensitive towards sacrificing any rating points. However, a tactical mode, separate from the main engine, should cost you no Elo.
shrapnel wrote:Hmm... have a look at the IPON list.... both Houdini 3 AND Stockfish 3 beating Komodo 5.1 MP , even if not by much.
Sad and a little disappointing.
Must say that I'm really impressed with the way Stockfish is becoming stronger and stronger, with each Update (barring a few hiccups ).
Some chess bloggers say that Stockfish is beating Houdini 3 convincingly and quite consistently, in their Tests !
Don't be disappointed, Komodo MP probably scales well both in time and with number of cores, it will be a tough opponent on strong machines and LTC, like the TCEC competition. We have three super-competitors in different branches of development, Houdini, Komodo, SF, and that's good.
OK, but the million dollar question is (for me, at least) does it scale better than Houdini 3 ??
For instance, even though using 6 cores with Houdini 3, I find it difficult to beat guys using 4 cores.
, even if not by much.