I am pleased to announce a revised version of Komodo-TCEC, which we call Komodo-TCECr. It fixes the reported error in fen diagrams followed by castling moves. It also fixes a minor issue of an error message when uses run Komodo in a Windows cmd window and close it using the title bar X instead of using the “quit” command.
Komodo TCEC owners can log into account on the Komodochess.com site (http://komodochess.com/store/secure/log ... /index.php) to download the revised version.
Note that users running matches using “Little Blitzer” will want to get the download, since Little Blitzer apparently converts PGNs to fens during match play. Since the error disallowed castling after many fen positions, Komodo might not find the castling move after a fen, which could hurt play on testers using fens. This is fixed in the TCECr version.
Thanks again for reporting the bug. We want Komodo to be rock solid and really appreciate it when you let us know about problems so we can fix them quickly.
I wish to thank Jeremy Bernstein for making the Windows X close patch.
Mark
Komodo TCEC: Serious castling bug !
Moderators: hgm, Rebel, chrisw
-
- Posts: 5228
- Joined: Thu Mar 09, 2006 9:40 am
- Full name: Vincent Lejeune
Re: Komodo TCEC: Serious castling bug !
May be Komodo-TCEC-r1 is a better (more clear) name.mjlef wrote:I am pleased to announce a revised version of Komodo-TCEC, which we call Komodo-TCECr.
...
Mark
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo TCEC: Serious castling bug !
I thought there was no need to make the name longer with the "1" because we do not anticipate another bugfix version, and if there is one we can still put a "2" after it.Vinvin wrote:May be Komodo-TCEC-r1 is a better (more clear) name.mjlef wrote:I am pleased to announce a revised version of Komodo-TCEC, which we call Komodo-TCECr.
...
Mark
-
- Posts: 454
- Joined: Tue Jan 15, 2013 4:33 pm
- Location: Ritz-Carlton, NYC
- Full name: Bobby Johnson
Re: Komodo TCEC: Serious castling bug !
doesn't work for me; is it working for anyone else?
SIM, PhD, MBA, PE
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo TCEC: Serious castling bug !
Testing shows that using the LittleBlitzer GUI and the same book as used by LS list, the bugfix tested as +2.5 elo over the initial release after 4000 bullet games. Using our own huge private book it was just +1 elo after about 18,000 games. Neither result is near the margin of error, but they seem about right to me for the fix. More important, no more time forfeits!mjlef wrote:I am pleased to announce a revised version of Komodo-TCEC, which we call Komodo-TCECr. It fixes the reported error in fen diagrams followed by castling moves. It also fixes a minor issue of an error message when uses run Komodo in a Windows cmd window and close it using the title bar X instead of using the “quit” command.
Komodo TCEC owners can log into account on the Komodochess.com site (http://komodochess.com/store/secure/log ... /index.php) to download the revised version.
Note that users running matches using “Little Blitzer” will want to get the download, since Little Blitzer apparently converts PGNs to fens during match play. Since the error disallowed castling after many fen positions, Komodo might not find the castling move after a fen, which could hurt play on testers using fens. This is fixed in the TCECr version.
Thanks again for reporting the bug. We want Komodo to be rock solid and really appreciate it when you let us know about problems so we can fix them quickly.
I wish to thank Jeremy Bernstein for making the Windows X close patch.
Mark
If anyone knows of any GUI other than LittleBlitzer in which the initial release was time-forfeiting in engine matches (I'm not talking about server play, that's a whole different problem) please let us know here, so we can post a warning for those who use those GUIs to get the bugfix.
Testers who were using LittleBlitzer or any GUI that was showing time forfeits should retest; for others thaere is no need to do so.
-
- Posts: 2435
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: Komodo TCEC: Serious castling bug !
lkaufman wrote:Testing shows that using the LittleBlitzer GUI and the same book as used by LS list, the bugfix tested as +2.5 elo over the initial release after 4000 bullet games. Using our own huge private book it was just +1 elo after about 18,000 games. Neither result is near the margin of error, but they seem about right to me for the fix. More important, no more time forfeits!mjlef wrote:I am pleased to announce a revised version of Komodo-TCEC, which we call Komodo-TCECr. It fixes the reported error in fen diagrams followed by castling moves. It also fixes a minor issue of an error message when uses run Komodo in a Windows cmd window and close it using the title bar X instead of using the “quit” command.
Komodo TCEC owners can log into account on the Komodochess.com site (http://komodochess.com/store/secure/log ... /index.php) to download the revised version.
Note that users running matches using “Little Blitzer” will want to get the download, since Little Blitzer apparently converts PGNs to fens during match play. Since the error disallowed castling after many fen positions, Komodo might not find the castling move after a fen, which could hurt play on testers using fens. This is fixed in the TCECr version.
Thanks again for reporting the bug. We want Komodo to be rock solid and really appreciate it when you let us know about problems so we can fix them quickly.
I wish to thank Jeremy Bernstein for making the Windows X close patch.
Mark
If anyone knows of any GUI other than LittleBlitzer in which the initial release was time-forfeiting in engine matches (I'm not talking about server play, that's a whole different problem) please let us know here, so we can post a warning for those who use those GUIs to get the bugfix.
Testers who were using LittleBlitzer or any GUI that was showing time forfeits should retest; for others thaere is no need to do so.
Thanx for the quick bugfix. LS-testrun will follow after the testrun of Stockfish 131210.
Stefan
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo TCEC: Serious castling bug !
I look forward to it. But it seems to me that you should limit the testing of any engine family to one version per month. Testing new stockfish versions every couple weeks introduces "survivor bias" in that one version will just get lucky and make Stockfish look artificially strong as you prune the weaker versions. It would be okay if you tested every engine with the same frequency, but you don't get new versions of most engines that often. Basically there has to be good reason to believe that a version has gained ten elo or so since the last version to overcome the margin of error for the difference between two versions to avoid survivor bias.pohl4711 wrote:lkaufman wrote:Testing shows that using the LittleBlitzer GUI and the same book as used by LS list, the bugfix tested as +2.5 elo over the initial release after 4000 bullet games. Using our own huge private book it was just +1 elo after about 18,000 games. Neither result is near the margin of error, but they seem about right to me for the fix. More important, no more time forfeits!mjlef wrote:I am pleased to announce a revised version of Komodo-TCEC, which we call Komodo-TCECr. It fixes the reported error in fen diagrams followed by castling moves. It also fixes a minor issue of an error message when uses run Komodo in a Windows cmd window and close it using the title bar X instead of using the “quit” command.
Komodo TCEC owners can log into account on the Komodochess.com site (http://komodochess.com/store/secure/log ... /index.php) to download the revised version.
Note that users running matches using “Little Blitzer” will want to get the download, since Little Blitzer apparently converts PGNs to fens during match play. Since the error disallowed castling after many fen positions, Komodo might not find the castling move after a fen, which could hurt play on testers using fens. This is fixed in the TCECr version.
Thanks again for reporting the bug. We want Komodo to be rock solid and really appreciate it when you let us know about problems so we can fix them quickly.
I wish to thank Jeremy Bernstein for making the Windows X close patch.
Mark
If anyone knows of any GUI other than LittleBlitzer in which the initial release was time-forfeiting in engine matches (I'm not talking about server play, that's a whole different problem) please let us know here, so we can post a warning for those who use those GUIs to get the bugfix.
Testers who were using LittleBlitzer or any GUI that was showing time forfeits should retest; for others thaere is no need to do so.
Thanx for the quick bugfix. LS-testrun will follow after the testrun of Stockfish 131210.
Stefan
Anyway it's of course your call; I just want everyone to be aware of survivor bias when too many versions of one engine are tested.
Best regards,
Larry
-
- Posts: 2435
- Joined: Sat Sep 03, 2011 7:25 am
- Location: Berlin, Germany
- Full name: Stefan Pohl
Re: Komodo TCEC: Serious castling bug !
First point: To test every engine with the same frequency is impossible (Houdini is updated only once in a year!).lkaufman wrote:
I look forward to it. But it seems to me that you should limit the testing of any engine family to one version per month. Testing new stockfish versions every couple weeks introduces "survivor bias" in that one version will just get lucky and make Stockfish look artificially strong as you prune the weaker versions. It would be okay if you tested every engine with the same frequency, but you don't get new versions of most engines that often. Basically there has to be good reason to believe that a version has gained ten elo or so since the last version to overcome the margin of error for the difference between two versions to avoid survivor bias.
Anyway it's of course your call; I just want everyone to be aware of survivor bias when too many versions of one engine are tested.
Best regards,
Larry
Second point: No more than 3 development-versions of Stockfish (or other engines) are part of the LS-ratinglist and the LS-gamebase, so there is no distortion of the ratinglist - all older, outdated versions and their games are deleted out of the database. So they have no influence on the bayeselo-calculation of the LS-ratinglist.
Third point: "Basically there has to be good reason to believe that a version has gained ten elo or so". Thats what I am doing. LS-testruns of a new development-Stockfish are mostly done, when there were some patches included, which seem to bring some Elo-gain. Stockfish 131210, which is tested at the moment, contains 3 new patches with some Elo-gain in the Stockfish-Framework-selftests. So I think, a test makes sense, although that version is not much newer than Stockfish DD.
Here the 3 patches of Stockfish 131210 (since Stockfish DD):
Update History and Counter move on TT hit
LLR: 2.96 (-2.94,2.94) [0.00,6.00]
Total: 5555 W: 935 L: 808 D: 3812
Research at intermediate depth if LMR is very high
LLR: 2.95 (-2.94,2.94) [0.00,6.00]
Total: 7292 W: 1195 L: 1061 D: 5036
Broader condition for dangerous pawn moves
LLR: 2.95 (-2.94,2.94) [0.00,6.00]
Total: 21459 W: 3486 L: 3286 D: 14687
Between Stockfish 131111 and Stockfish DD there was only one Elo-gain-patch (with a very, very, small gain), but Stockfish DD is an official release, so I had to test it (otherwise I wouldnt have tested it). And the result was nearly the same rating like Stockfish 131111 (-1 Elo)(as I expected).
And Stockfish DD adapC is an experimental Stockfish-mod, which I wanted to test...It will be handled like a Stockfish-development-version and will be deleted, when 3 newer development-versions of Stockfish are part of the LS-ratinglist.
Best regards - Stefan
-
- Posts: 157
- Joined: Tue Apr 30, 2013 1:29 am
Re: Komodo TCEC: Serious castling bug !
Thanks, this version works correctly for me in Tarrasch GUI.
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Komodo TCEC: Serious castling bug !
Just to be clear, does this mean that you were getting the time forfeits/hangups with the older version? If so we should add Tarrasch GUI to the list of ones that need the fixed version.OneTrickPony wrote:Thanks, this version works correctly for me in Tarrasch GUI.