Stockfish wins CCC 3: Stage 1 only after last 60th game

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Stockfish wins CCC 3: Stage 1 only after last 60th game

Post by Dann Corbit »

Interesting among those positions are the following imbalanced wood counts:

Code: Select all

>> ** IMBALANCED ** << [+111]:( r1bq1rk1/4bppp/p1p5/1p1nR3/8/1BP5/PP1P1PPP/RNBQ2K1 w - - )
>> ** IMBALANCED ** << [+111]:( r1bq1rk1/pp1nbppp/4pn2/2pP2B1/3P4/2N1PN2/PPQ2PPP/R3KB1R b KQ - )
>> ** IMBALANCED ** << [+111]:( r1bqkb1r/p4pp1/2p2n1p/n3N3/4p3/8/PPPPBPPP/RNBQK2R b KQkq - )
>> ** IMBALANCED ** << [+111]:( r1bqkb1r/ppp2ppp/5n2/n2Pp1N1/2B5/8/PPPP1PPP/RNBQK2R w KQkq - )
>> ** IMBALANCED ** << [+111]:( rnbqk2r/pp2ppbp/5np1/3P4/3P4/2N5/PP3PPP/R1BQKBNR w KQkq - )
>> ** IMBALANCED ** << [+111]:( rnbqkb1r/3ppppp/pP3n2/2pP4/8/8/PP2PPPP/RNBQKBNR b KQkq - )
>> ** IMBALANCED ** << [-111]:( r1bqk1nr/pppp1ppp/2n5/4p3/1bB1P3/2P2N2/P2P1PPP/RNBQK2R b KQkq - )
>> ** IMBALANCED ** << [-111]:( r1bqkb1r/1ppn1ppp/p3pn2/8/Q1pP4/5NP1/PP2PPBP/RNB1K2R w KQkq - )
>> ** IMBALANCED ** << [-111]:( r1bqkb1r/1ppp1ppp/p1n5/4p3/B3n3/5N2/PPPP1PPP/RNBQ1RK1 w kq - )
>> ** IMBALANCED ** << [-111]:( r1bqkb1r/pp2pppp/n1p2n2/8/P1pP4/2N2N2/1P2PPPP/R1BQKB1R w KQkq - )
>> ** IMBALANCED ** << [-111]:( r1bqkb1r/pppp1ppp/2n5/1B2p3/4n3/5N2/PPPP1PPP/RNBQ1RK1 w kq - )
>> ** IMBALANCED ** << [-111]:( r2qkb1r/1ppb1ppp/p1n1pn2/8/2pP4/4PNP1/PP2QPBP/RNB2RK1 b kq - )
>> ** IMBALANCED ** << [-111]:( r2qkb1r/pb1n1p2/2p1pn1p/1p2N1p1/2pPP3/2N3B1/PP2BPPP/R2QK2R w KQkq - )
>> ** IMBALANCED ** << [-111]:( rn1qk2r/pb1pbppp/1p2pn2/8/2Pp4/2NBPN2/PP3PPP/R1BQ1RK1 w kq - )
>> ** IMBALANCED ** << [-111]:( rn1qkb1r/ppp1pppp/5n2/8/3P2b1/2N2N2/PPP3PP/R1BQKB1R w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/pp2pppp/2p2n2/8/2pP4/2N2N2/PP2PPPP/R1BQKB1R w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/pp2pppp/2p2n2/8/P1pP4/2N2N2/1P2PPPP/R1BQKB1R b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/ppp1pppp/5n2/8/2pP4/4PN2/PP3PPP/RNBQKB1R b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/ppp1pppp/5n2/8/3P4/2N2N2/PPP3PP/R1BQKB1R b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/ppp2ppp/4pn2/6B1/3Pp3/2N5/PPP2PPP/R2QKBNR w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/ppp2ppp/4pn2/8/2pP4/5NP1/PP2PPBP/RNBQK2R b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/ppp2ppp/4pn2/8/3P4/2N2N2/PPP3PP/R1BQKB1R w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkb1r/pppp1ppp/3n4/4p2Q/8/1BN5/PPPP1PPP/R1B1K1NR b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/1pp1pppp/p7/8/2pP4/4PN2/PP3PPP/RNBQKB1R b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/pp1p1ppp/4p3/8/3pP3/5N2/PPP2PPP/RNBQKB1R w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/pp2pppp/3p4/8/3pP3/5N2/PPP2PPP/RNBQKB1R w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/ppp1pppp/8/8/2pP4/4P3/PP3PPP/RNBQKBNR b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/ppp1pppp/8/8/2pP4/5N2/PP2PPPP/RNBQKB1R b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/ppp1pppp/8/8/3Pp3/2N5/PPP2PPP/R1BQKBNR b KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/ppp1pppp/8/8/3Pp3/8/PPP2PPP/RNBQKBNR w KQkq - )
>> ** IMBALANCED ** << [-111]:( rnbqkbnr/pppp1ppp/8/8/2B1Pp2/8/PPPP2PP/RNBQK1NR b KQkq - )
>> ** IMBALANCED ** << [+222]:( r1bqkb1r/pp3ppp/2P2n2/nB2p1N1/8/8/PPPP1PPP/RNBQK2R b KQkq - )
>> ** IMBALANCED ** << [+222]:( rnbqkb1r/3ppp1p/P4np1/2pP4/8/8/PP2PPPP/RNBQKBNR w KQkq - )
>> ** IMBALANCED ** << [-333]:( r1bqk1nr/pppp1ppp/2n5/b7/2B1P3/2p2N2/P4PPP/RNBQ1RK1 w kq - )
>> ** IMBALANCED ** << [-526]:( r1bqk2r/pppp1ppp/2n5/3P4/2B1n3/2b2N2/PP3PPP/R1BQ1RK1 b kq - )
I guess that the bottom four might give people a start, when the book exited, but they are fair.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: Stockfish wins CCC 3: Stage 1 only after last 60th game

Post by carldaman »

Dann Corbit wrote: Wed Jan 23, 2019 12:43 am I have seen opening books with checkmate end points used in Elo measuring events.

So, is "down a queen" acceptable? If not, isn't it an arbitrary value just like any other?

Down a pawn is "pawn odds". If you were an engine author, would you like to fall out of book down a pawn?

Komodo, Houdini, Stockfish, Fire, Ethereal...
If they say you are down a pawn after a 40 ply search (or better) then you have a very serious disadvantage and you are probably going to lose.
Now, one engine might say -103 centipawns and the other might say -115 centipawns etc, but you will find after a deep search that they usually agree pretty nearly.

Now, anyone who runs a contest can set it up anyway that they like. But if they do not make an effort to make it fair, then I would not want my engine in the contest if I were an author.

There are gambits that are simply unsound. While it might be fun to watch games with those gambits, it is unfair if used in a contest to measure strength.

Now, my opinion is not any better than yours. But I think a lot of engine writers would agree with me.
"As long as the opening line is not proven to lose by force", as I wrote earlier, so checkmate right out of book would get ruled out. :)

I was also assuming the use of the same opening twice, with White and Black, to be fair. Any opening book created for testing should require some due diligence to eliminate the bad lines, either leading to book losses or forced repetitions.
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Stockfish wins CCC 3: Stage 1 only after last 60th game

Post by Dann Corbit »

carldaman wrote: I was also assuming the use of the same opening twice, with White and Black, to be fair. Any opening book created for testing should require some due diligence to eliminate the bad lines, either leading to book losses or forced repetitions.
Reversal does not make it fair. If Stockfish loses to an inferior engine then wins the reversed opening, it was probably still cheated out of a point because in a fair opening it would have (likely) won both games.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: Stockfish wins CCC 3: Stage 1 only after last 60th game

Post by carldaman »

Dann Corbit wrote: Wed Jan 23, 2019 8:04 am
carldaman wrote: I was also assuming the use of the same opening twice, with White and Black, to be fair. Any opening book created for testing should require some due diligence to eliminate the bad lines, either leading to book losses or forced repetitions.
Reversal does not make it fair. If Stockfish loses to an inferior engine then wins the reversed opening, it was probably still cheated out of a point because in a fair opening it would have (likely) won both games.

If SF loses from a weak opening to a clearly inferior engine then the line is busted and does not belong in the suite. Reversing proven busted lines does not make it fair, I would agree.
arunsoorya1309
Posts: 214
Joined: Tue Nov 03, 2015 4:55 pm

Re: Stockfish wins CCC 3: Stage 1 only after last 60th game

Post by arunsoorya1309 »

User avatar
Rebel
Posts: 6993
Joined: Thu Aug 18, 2011 12:04 pm

Re: Stockfish wins CCC 3: Stage 1 only after last 60th game

Post by Rebel »

Image
90% of coding is debugging, the other 10% is writing bugs.