I never said we have more evaluation terms than Stockfish. I have not tried to count them. But Larry and I try to keep the number of terms low, since it is hard to tune a huge number or terms. We rarely do much tuning. Automated tuning has not helped the last few times we try. So we try a few values for a new term and keep whatever tests best.Lyudmil Tsvetkov wrote:you mentioned also selective Komodo evaluation, it is true different terms will only kick in specific positions, but you still have to tune all of them within a single framework, and I have difficulties to imagine how one can tune couple of times more terms, but a factor of 1.6 is still manageable.
again, I can not explain to myself the practically almost identical nps...
anyone having a clue?
FGRL 60 sec + 0.6 sec Rating list - Komodo 11.2
Moderators: hgm, Rebel, chrisw
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
Komodo does scale opposite color bishop endings and ppxxx vs pxxx endings. I have not compared how much we scale by compared with Stockfish, but I assure you it is in there.
I did not include the bit board code in Komodo or Stockfish in my line counts since it is not part of the regular eval. Including the Komodo equivalent would greatly increase the K linecount.. And only a small part of types.h is eval related, although I see Stockfish puts piece values in there. You are right endgame file should have been included. Counting the same the ratio would be more than 2 to 1.
I did not include the bit board code in Komodo or Stockfish in my line counts since it is not part of the regular eval. Including the Komodo equivalent would greatly increase the K linecount.. And only a small part of types.h is eval related, although I see Stockfish puts piece values in there. You are right endgame file should have been included. Counting the same the ratio would be more than 2 to 1.
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
that makes more sense.mjlef wrote:I never said we have more evaluation terms than Stockfish. I have not tried to count them. But Larry and I try to keep the number of terms low, since it is hard to tune a huge number or terms. We rarely do much tuning. Automated tuning has not helped the last few times we try. So we try a few values for a new term and keep whatever tests best.Lyudmil Tsvetkov wrote:you mentioned also selective Komodo evaluation, it is true different terms will only kick in specific positions, but you still have to tune all of them within a single framework, and I have difficulties to imagine how one can tune couple of times more terms, but a factor of 1.6 is still manageable.
again, I can not explain to myself the practically almost identical nps...
anyone having a clue?
my impression has always been SF and Komodo evals are comparable.
Komodo might have a bit larger eval, but not substantially larger.
thanks for the very straightforward answer, Mark!
please, don't get angry at me, I am watching SF and Komodo play games all too frequently, and the impression of almost complete cluelessness of both SF and Komodo in more complicated positions, requiring more refined eval, never leaves me.
thus, I conclude both engines don't have very sophisticated evaluation functions.
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
"Lyudmil Tsvetkov" actually appears in the Komodo evaluation! It is crediting one of your suggestions you made here (I think, it was in the code when I started working on it). Although we do not always agree with all your ideas, they often lead to discussions and improvement in Komodo. So keep posting! It is very helpful. If you want, I can privately message you what is was about.Lyudmil Tsvetkov wrote:that makes more sense.mjlef wrote:I never said we have more evaluation terms than Stockfish. I have not tried to count them. But Larry and I try to keep the number of terms low, since it is hard to tune a huge number or terms. We rarely do much tuning. Automated tuning has not helped the last few times we try. So we try a few values for a new term and keep whatever tests best.Lyudmil Tsvetkov wrote:you mentioned also selective Komodo evaluation, it is true different terms will only kick in specific positions, but you still have to tune all of them within a single framework, and I have difficulties to imagine how one can tune couple of times more terms, but a factor of 1.6 is still manageable.
again, I can not explain to myself the practically almost identical nps...
anyone having a clue?
my impression has always been SF and Komodo evals are comparable.
Komodo might have a bit larger eval, but not substantially larger.
thanks for the very straightforward answer, Mark!
please, don't get angry at me, I am watching SF and Komodo play games all too frequently, and the impression of almost complete cluelessness of both SF and Komodo in more complicated positions, requiring more refined eval, never leaves me.
thus, I conclude both engines don't have very sophisticated evaluation functions.
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
I did not include bitboard.cpp, which is very large in SF too, but bitbase.cpp, which is another thing.mjlef wrote:Komodo does scale opposite color bishop endings and ppxxx vs pxxx endings. I have not compared how much we scale by compared with Stockfish, but I assure you it is in there.
I did not include the bit board code in Komodo or Stockfish in my line counts since it is not part of the regular eval. Including the Komodo equivalent would greatly increase the K linecount.. And only a small part of types.h is eval related, although I see Stockfish puts piece values in there. You are right endgame file should have been included. Counting the same the ratio would be more than 2 to 1.
well, somehow I don't quite unerstand that: one the one hand you would not claim Komodo has more terms than SF, on the other you still insists line count is more than 2/1.
really difficult for me to make precisely sense of what is going on, but maybe it is better to leave things like that now, anyway Komodo is not public domain and you reserve the right to reveal or not sensitive information.
[d]8/5kp1/4r3/2R5/5PP1/6K1/8/8 w - - 0 1
Komodo: +100cps, looks like not scaling this
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
Komodo has a kind of KPK database in it, which is like the rules Stockfish uses to generate the same information that is in bitbase.cpp. So I left them both out. Komodo's version had both data and code, which would unfairly increase the line count for Komodo. So I left it off of both of them. I could recount with them if you want but the ratio would increase.Lyudmil Tsvetkov wrote:I did not include bitboard.cpp, which is very large in SF too, but bitbase.cpp, which is another thing.mjlef wrote:Komodo does scale opposite color bishop endings and ppxxx vs pxxx endings. I have not compared how much we scale by compared with Stockfish, but I assure you it is in there.
I did not include the bit board code in Komodo or Stockfish in my line counts since it is not part of the regular eval. Including the Komodo equivalent would greatly increase the K linecount.. And only a small part of types.h is eval related, although I see Stockfish puts piece values in there. You are right endgame file should have been included. Counting the same the ratio would be more than 2 to 1.
well, somehow I don't quite unerstand that: one the one hand you would not claim Komodo has more terms than SF, on the other you still insists line count is more than 2/1.
really difficult for me to make precisely sense of what is going on, but maybe it is better to leave things like that now, anyway Komodo is not public domain and you reserve the right to reveal or not sensitive information.
[d]8/5kp1/4r3/2R5/5PP1/6K1/8/8 w - - 0 1
Komodo: +100cps, looks like not scaling this
types.h has almost no eval code in it (just the piece values for opening and endgame, whihc is just a couple of lines). The rest is general function prototypes and definitions used more in the move generator and search than in the eval. So that is why I did not include it.
Mark
-
- Posts: 684
- Joined: Mon Apr 19, 2010 7:07 pm
- Location: Sweden
- Full name: Peter Osterlund
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
mjlef wrote:Komodo uses a form of Lazy Eval that helps speedup its node rate. I think the development versions of Stockfish have Lazy Eval, but the version I saw them using when I checked a month or so ago was less aggressive than what we do in Komodo. I cannot say exactly how much "bigger" Komodo's eval is, but the code is much larger, and it has a lot more terms. I am not claiming bigger is better though.
I'm not sure how to combine the two statements you made (bolded by me).mjlef wrote:I never said we have more evaluation terms than Stockfish. I have not tried to count them. But Larry and I try to keep the number of terms low, since it is hard to tune a huge number or terms. We rarely do much tuning. Automated tuning has not helped the last few times we try. So we try a few values for a new term and keep whatever tests best.
-
- Posts: 1476
- Joined: Mon Jan 28, 2013 2:51 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo 11.2
Hi Andreas,
I am finding, preliminarily, that McBrain 2.6 is the strongest at LTC (120'+30" in my case), ahead of Komodo 11.01 and AsmFish 2017-07-09 (this test has been running for a while). May I humbly suggest that you schedule the latest McBrain for one of your longer time control tests, say, after Komodo 11.2.1? Michael Byrne is doing an outstanding job!
Best,
Dmitri
I am finding, preliminarily, that McBrain 2.6 is the strongest at LTC (120'+30" in my case), ahead of Komodo 11.01 and AsmFish 2017-07-09 (this test has been running for a while). May I humbly suggest that you schedule the latest McBrain for one of your longer time control tests, say, after Komodo 11.2.1? Michael Byrne is doing an outstanding job!
Best,
Dmitri
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
I feel flattered.mjlef wrote:"Lyudmil Tsvetkov" actually appears in the Komodo evaluation! It is crediting one of your suggestions you made here (I think, it was in the code when I started working on it). Although we do not always agree with all your ideas, they often lead to discussions and improvement in Komodo. So keep posting! It is very helpful. If you want, I can privately message you what is was about.Lyudmil Tsvetkov wrote:that makes more sense.mjlef wrote:I never said we have more evaluation terms than Stockfish. I have not tried to count them. But Larry and I try to keep the number of terms low, since it is hard to tune a huge number or terms. We rarely do much tuning. Automated tuning has not helped the last few times we try. So we try a few values for a new term and keep whatever tests best.Lyudmil Tsvetkov wrote:you mentioned also selective Komodo evaluation, it is true different terms will only kick in specific positions, but you still have to tune all of them within a single framework, and I have difficulties to imagine how one can tune couple of times more terms, but a factor of 1.6 is still manageable.
again, I can not explain to myself the practically almost identical nps...
anyone having a clue?
my impression has always been SF and Komodo evals are comparable.
Komodo might have a bit larger eval, but not substantially larger.
thanks for the very straightforward answer, Mark!
please, don't get angry at me, I am watching SF and Komodo play games all too frequently, and the impression of almost complete cluelessness of both SF and Komodo in more complicated positions, requiring more refined eval, never leaves me.
thus, I conclude both engines don't have very sophisticated evaluation functions.
typycal of Don style.
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
I see types.h including more eval-related stuff than otherwise.mjlef wrote:Komodo has a kind of KPK database in it, which is like the rules Stockfish uses to generate the same information that is in bitbase.cpp. So I left them both out. Komodo's version had both data and code, which would unfairly increase the line count for Komodo. So I left it off of both of them. I could recount with them if you want but the ratio would increase.Lyudmil Tsvetkov wrote:I did not include bitboard.cpp, which is very large in SF too, but bitbase.cpp, which is another thing.mjlef wrote:Komodo does scale opposite color bishop endings and ppxxx vs pxxx endings. I have not compared how much we scale by compared with Stockfish, but I assure you it is in there.
I did not include the bit board code in Komodo or Stockfish in my line counts since it is not part of the regular eval. Including the Komodo equivalent would greatly increase the K linecount.. And only a small part of types.h is eval related, although I see Stockfish puts piece values in there. You are right endgame file should have been included. Counting the same the ratio would be more than 2 to 1.
well, somehow I don't quite unerstand that: one the one hand you would not claim Komodo has more terms than SF, on the other you still insists line count is more than 2/1.
really difficult for me to make precisely sense of what is going on, but maybe it is better to leave things like that now, anyway Komodo is not public domain and you reserve the right to reveal or not sensitive information.
[d]8/5kp1/4r3/2R5/5PP1/6K1/8/8 w - - 0 1
Komodo: +100cps, looks like not scaling this
types.h has almost no eval code in it (just the piece values for opening and endgame, whihc is just a couple of lines). The rest is general function prototypes and definitions used more in the move generator and search than in the eval. So that is why I did not include it.
Mark
enum rank, enum file, enum scale_factor, enum phase, enum piece, enum score, enum castling, enum castling right...
also, most inline functions, inline fileof(square s), inline colourof(piece pc), relative rank(colour c, piece pc)...
so, basically, should be included.
I don't know why you want to make it a factor of more than 2, when this would contradict a couple of related conditions.