Stockfish Handicap Matches

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
JJJ
Posts: 1316
Joined: Sat Apr 19, 2014 11:47 am

Re: Stockfish Handicap Matches

Post by JJJ » Thu Jul 16, 2020 12:31 am

lkaufman wrote:
Wed Jul 15, 2020 11:58 pm
JJJ wrote:
Wed Jul 15, 2020 8:16 pm
lkaufman wrote:
Wed Jul 15, 2020 5:39 pm
JJJ wrote:
Wed Jul 15, 2020 4:48 pm
lkaufman wrote:
Wed Jul 15, 2020 4:28 pm
Chessqueen wrote:
Wed Jul 15, 2020 4:32 am
JJJ wrote:
Sun Jul 12, 2020 11:42 am
I think it's a good idea Larry. I m not ready to play handicap match myself yet, but my suggestion was for others to play against Komodo at handicap and get an accurate rating for Komodo :)
I bought this Quad Intel i7 https://www.atmarktrade.com/products/re ... 4358391872 and decided to play versus Komodo three games 1st Knight plus b2 pawn odds, 2nd game with Knight and c2 pawn odds and last game by removing the Rook on a1 from Komodo and I removed my b7 pawn, and I am too ashamed to even post these three games here since I lost all the three games using this cheap refurbished compunter, and NOT to mention it is not even close to your 32 cores :roll: :mrgreen:

PS: Since I lost those three games using a much weaker system than what you have, I will NOT even try to play, komodo is simply too strong or it is the best program to give odds versus humans and even Jorge Sammour told me that, after he played versus Stockfish and Komodo . :shock:
I've been playing handicap games myself vs. the top engines and also pairing them in handicap games with "Benjamin" (near 2700 CCRL). In general Komodo does best, with Stockfish NNUE a close second. Regular SF maybe in third place, Lc0 70xxx in fourth place. Since I'm a bit too strong for knight odds in Rapid (maybe knight odds Armageddon where Black must win is fair though) I've been trying to determine the most challenging and interesting handicap for me, and it seems to be "pawn and three moves" (no f7, e4,d4 played, WTM). This is not that far from knight odds, as long as White prepares a bit and plays a few accurate initial moves. I have won from SF and got draws from others, but still lose more often than not. Similarly "Benjamin" (one thread) in bullet (1' + 1") comes out ahead of all the other engines on 7 threads at knight odds and behind at pawn and three moves, but not by big margins in either case. In the 1800s, pawn and 3 moves was not nearly as often played as pawn and 2 moves, probably because players weren't sure that it was less than knight odds so many just skipped that step. With engine analysis I can say that it is in between, but it is closer to knight odds than to pawn and two moves.
Don't you think Lc0 10XXX is far better than 70xxx for knight handicap ? Did you test that ? I was thinking Lc0 10xxx best net like 11248 or 11258 would be better than even Komodo at knight handicap.
Well, 11248/58 are trickier nets, they play more wild moves like g4 and such, so perhaps better vs. humans. But the 70xxx nets play handicap chess correctly, that is they show sensible evals and PVs and don't blunder, unlike the 20x thru 60x nets. Maybe I'll try comparative runs vs. Benjamin at knight odds to see. There is a difference between computers and humans though; it seems that Komodo mcts is better than standard Komodo at giving knight odds to humans, but worse at giving it to engines.
Let me know about your test. Knight handicap matters only against human to me, not against computer. That's why I m surprised you said Komodo mcts is better than lc0 at knight handicap.
Knight handicap is only important vs. humans, but the only way to test it and get decent samples under identical conditions is against weaker engines rated somewhere near the humans. My results (20 game runs at 1' + 1", 7 threads (or 2080 GPU) for White giving knight odds, 1 thread for Benjamin with the extra knight) were Lc0-11248 lost by 268 elo, Lc0-703747 lost by 210 elo, Stockfish (July 6, max Contempt) lost by 70 elo, StockfishNN2 (NNUE) lost by 53 elo, Komodo (latest dev, Standard, Contempt 100) lost by 35 elo, and same Komodo with Contempt 75 actually won by 35 elo, the first match victory over Benjamin at knight odds.
So Komodo is the best against a good "weak" engine at this handicap, but this test might be not accurate at all against human. From your own experience, I think you said Lc0 is the hardest to beat at knight handicap, then Komodo, then Stockfish right ?

lkaufman
Posts: 4324
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Stockfish Handicap Matches

Post by lkaufman » Thu Jul 16, 2020 1:02 am

JJJ wrote:
Thu Jul 16, 2020 12:31 am
lkaufman wrote:
Wed Jul 15, 2020 11:58 pm
JJJ wrote:
Wed Jul 15, 2020 8:16 pm
lkaufman wrote:
Wed Jul 15, 2020 5:39 pm
JJJ wrote:
Wed Jul 15, 2020 4:48 pm
lkaufman wrote:
Wed Jul 15, 2020 4:28 pm
Chessqueen wrote:
Wed Jul 15, 2020 4:32 am
JJJ wrote:
Sun Jul 12, 2020 11:42 am
I think it's a good idea Larry. I m not ready to play handicap match myself yet, but my suggestion was for others to play against Komodo at handicap and get an accurate rating for Komodo :)
I bought this Quad Intel i7 https://www.atmarktrade.com/products/re ... 4358391872 and decided to play versus Komodo three games 1st Knight plus b2 pawn odds, 2nd game with Knight and c2 pawn odds and last game by removing the Rook on a1 from Komodo and I removed my b7 pawn, and I am too ashamed to even post these three games here since I lost all the three games using this cheap refurbished compunter, and NOT to mention it is not even close to your 32 cores :roll: :mrgreen:

PS: Since I lost those three games using a much weaker system than what you have, I will NOT even try to play, komodo is simply too strong or it is the best program to give odds versus humans and even Jorge Sammour told me that, after he played versus Stockfish and Komodo . :shock:
I've been playing handicap games myself vs. the top engines and also pairing them in handicap games with "Benjamin" (near 2700 CCRL). In general Komodo does best, with Stockfish NNUE a close second. Regular SF maybe in third place, Lc0 70xxx in fourth place. Since I'm a bit too strong for knight odds in Rapid (maybe knight odds Armageddon where Black must win is fair though) I've been trying to determine the most challenging and interesting handicap for me, and it seems to be "pawn and three moves" (no f7, e4,d4 played, WTM). This is not that far from knight odds, as long as White prepares a bit and plays a few accurate initial moves. I have won from SF and got draws from others, but still lose more often than not. Similarly "Benjamin" (one thread) in bullet (1' + 1") comes out ahead of all the other engines on 7 threads at knight odds and behind at pawn and three moves, but not by big margins in either case. In the 1800s, pawn and 3 moves was not nearly as often played as pawn and 2 moves, probably because players weren't sure that it was less than knight odds so many just skipped that step. With engine analysis I can say that it is in between, but it is closer to knight odds than to pawn and two moves.
Don't you think Lc0 10XXX is far better than 70xxx for knight handicap ? Did you test that ? I was thinking Lc0 10xxx best net like 11248 or 11258 would be better than even Komodo at knight handicap.
Well, 11248/58 are trickier nets, they play more wild moves like g4 and such, so perhaps better vs. humans. But the 70xxx nets play handicap chess correctly, that is they show sensible evals and PVs and don't blunder, unlike the 20x thru 60x nets. Maybe I'll try comparative runs vs. Benjamin at knight odds to see. There is a difference between computers and humans though; it seems that Komodo mcts is better than standard Komodo at giving knight odds to humans, but worse at giving it to engines.
Let me know about your test. Knight handicap matters only against human to me, not against computer. That's why I m surprised you said Komodo mcts is better than lc0 at knight handicap.
Knight handicap is only important vs. humans, but the only way to test it and get decent samples under identical conditions is against weaker engines rated somewhere near the humans. My results (20 game runs at 1' + 1", 7 threads (or 2080 GPU) for White giving knight odds, 1 thread for Benjamin with the extra knight) were Lc0-11248 lost by 268 elo, Lc0-703747 lost by 210 elo, Stockfish (July 6, max Contempt) lost by 70 elo, StockfishNN2 (NNUE) lost by 53 elo, Komodo (latest dev, Standard, Contempt 100) lost by 35 elo, and same Komodo with Contempt 75 actually won by 35 elo, the first match victory over Benjamin at knight odds.
So Komodo is the best against a good "weak" engine at this handicap, but this test might be not accurate at all against human. From your own experience, I think you said Lc0 is the hardest to beat at knight handicap, then Komodo, then Stockfish right ?
I did have more trouble with Lc0 11248 at knight odds than with Komodo quite a few months ago, but now that I have played a lot of games with many top engines at knight odds I should try it again, maybe I just had to learn how to deal with the novel situation. More recently I rarely lose to any engine at knight odds in Rapid, though I do give up a lot of draws.
Komodo rules!

lkaufman
Posts: 4324
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Stockfish Handicap Matches

Post by lkaufman » Thu Jul 16, 2020 8:36 pm

JJJ wrote:
Thu Jul 16, 2020 12:31 am
lkaufman wrote:
Wed Jul 15, 2020 11:58 pm
JJJ wrote:
Wed Jul 15, 2020 8:16 pm
lkaufman wrote:
Wed Jul 15, 2020 5:39 pm
JJJ wrote:
Wed Jul 15, 2020 4:48 pm
lkaufman wrote:
Wed Jul 15, 2020 4:28 pm
Chessqueen wrote:
Wed Jul 15, 2020 4:32 am
JJJ wrote:
Sun Jul 12, 2020 11:42 am
I think it's a good idea Larry. I m not ready to play handicap match myself yet, but my suggestion was for others to play against Komodo at handicap and get an accurate rating for Komodo :)
I bought this Quad Intel i7 https://www.atmarktrade.com/products/re ... 4358391872 and decided to play versus Komodo three games 1st Knight plus b2 pawn odds, 2nd game with Knight and c2 pawn odds and last game by removing the Rook on a1 from Komodo and I removed my b7 pawn, and I am too ashamed to even post these three games here since I lost all the three games using this cheap refurbished compunter, and NOT to mention it is not even close to your 32 cores :roll: :mrgreen:

PS: Since I lost those three games using a much weaker system than what you have, I will NOT even try to play, komodo is simply too strong or it is the best program to give odds versus humans and even Jorge Sammour told me that, after he played versus Stockfish and Komodo . :shock:
I've been playing handicap games myself vs. the top engines and also pairing them in handicap games with "Benjamin" (near 2700 CCRL). In general Komodo does best, with Stockfish NNUE a close second. Regular SF maybe in third place, Lc0 70xxx in fourth place. Since I'm a bit too strong for knight odds in Rapid (maybe knight odds Armageddon where Black must win is fair though) I've been trying to determine the most challenging and interesting handicap for me, and it seems to be "pawn and three moves" (no f7, e4,d4 played, WTM). This is not that far from knight odds, as long as White prepares a bit and plays a few accurate initial moves. I have won from SF and got draws from others, but still lose more often than not. Similarly "Benjamin" (one thread) in bullet (1' + 1") comes out ahead of all the other engines on 7 threads at knight odds and behind at pawn and three moves, but not by big margins in either case. In the 1800s, pawn and 3 moves was not nearly as often played as pawn and 2 moves, probably because players weren't sure that it was less than knight odds so many just skipped that step. With engine analysis I can say that it is in between, but it is closer to knight odds than to pawn and two moves.
Don't you think Lc0 10XXX is far better than 70xxx for knight handicap ? Did you test that ? I was thinking Lc0 10xxx best net like 11248 or 11258 would be better than even Komodo at knight handicap.
Well, 11248/58 are trickier nets, they play more wild moves like g4 and such, so perhaps better vs. humans. But the 70xxx nets play handicap chess correctly, that is they show sensible evals and PVs and don't blunder, unlike the 20x thru 60x nets. Maybe I'll try comparative runs vs. Benjamin at knight odds to see. There is a difference between computers and humans though; it seems that Komodo mcts is better than standard Komodo at giving knight odds to humans, but worse at giving it to engines.
Let me know about your test. Knight handicap matters only against human to me, not against computer. That's why I m surprised you said Komodo mcts is better than lc0 at knight handicap.
Knight handicap is only important vs. humans, but the only way to test it and get decent samples under identical conditions is against weaker engines rated somewhere near the humans. My results (20 game runs at 1' + 1", 7 threads (or 2080 GPU) for White giving knight odds, 1 thread for Benjamin with the extra knight) were Lc0-11248 lost by 268 elo, Lc0-703747 lost by 210 elo, Stockfish (July 6, max Contempt) lost by 70 elo, StockfishNN2 (NNUE) lost by 53 elo, Komodo (latest dev, Standard, Contempt 100) lost by 35 elo, and same Komodo with Contempt 75 actually won by 35 elo, the first match victory over Benjamin at knight odds.
So Komodo is the best against a good "weak" engine at this handicap, but this test might be not accurate at all against human. From your own experience, I think you said Lc0 is the hardest to beat at knight handicap, then Komodo, then Stockfish right ?
I played a game with Lc011248 (GPU 2080) at 5' + 5", which is still considered blitz, at knight odds last night, and won very easily. It's only one game, but I think that Komodo's main rival now for giving handicaps is Stockfish NNUE.
Komodo rules!

JJJ
Posts: 1316
Joined: Sat Apr 19, 2014 11:47 am

Re: Stockfish Handicap Matches

Post by JJJ » Thu Jul 16, 2020 11:57 pm

What rating can achieve at blitz 3+2 or 5+5 the best Komodo at handicap knight ?

lkaufman
Posts: 4324
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Stockfish Handicap Matches

Post by lkaufman » Fri Jul 17, 2020 12:56 am

JJJ wrote:
Thu Jul 16, 2020 11:57 pm
What rating can achieve at blitz 3+2 or 5+5 the best Komodo at handicap knight ?
Time limit makes a big difference, and of course it's important to specify whether you mean FIDE blitz ratings or online blitz ratings, which are much higher in the GM range usually. Assuming you mean FIDE blitz ratings, I would say something like 2550 for 3' + 2", maybe 2450 for 5' + 5". At 15' + 10" performance has been around 2300 FIDE.
Komodo rules!

Post Reply