McBrain v2.0 Release

MikeB · Post by **MikeB** » Sat Mar 25, 2017 11:20 pm

leavenfish wrote:Michael -

When you download McBrain...should the defaults be tweeked for engine tournaments? I ask because, after purchasing Komodo I ran a quick test using some lines in the Silver Suite and the engine fared worse than I had expected:

Engine Score St St As Ko Mc S-B
1: Stockfish_17031823_x64_modern 13.5/24 ······ ==1=== ====== =====1 ==1=== 156.25
2: Stockfish_8_x64_popcnt 13.5/24 ==0=== ······ ====== 1=0=1= =1=1=1 152.25
3: AsmFishW_2016-12-26_popcnt 13.0/24 ====== ====== ······ 11=1== =0==== 152.00
4: Komodo-10.4-64bit 10.5/24 =====0 0=1=0= 00=0== ······ =1=1== 125.00
5: McBrain_2017_v20_x64_modern 9.5/24 ==0=== =0=0=0 =1==== =0=0== ······ 120.50

60 games played / Tournament finished
Name of the tournament: 1min 1sec
Site/ Country: DESKTOP-8HGTVN3, United States
Level: Blitz 1:01/1
Hardware: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz with 16.0 GB Memory
Operating system: Windows 10 Home Home Edition (Build 9200) 64 bit
PGN-File: C:\Program Files (x86)\Arena\Tournaments\G90c.pgn
Website:
E-Mail Address:

v2.1 will be much,much better at longer time control - I will look for the silver suite and see what I get with 2.1. I'm currently testing using tc 2 (min) 2 (sec). And of course 24 games are not much of a sample size, but all of the McBrains have been weaker than expected at more than game per minute - planning to fix that very soon.

Code: Select all

time control&#58; 2 min 2 sec
openings &#58; tcec 9 super final
Rank Name                    Rating   &#916;     +    -     #     &#931;    &#931;%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 McBrain 2.1f 64 POPCNT   3129   0.0   14   14   400  222.5  55.6   84   39  277  21.0  69.2  3093 
   2 McBrain 2.1d 64 POPCNT   3114  14.6   14   14   400  211.0  52.8   73   51  276  18.2  69.0  3097 
   3 McBrain 2.1e 64 POPCNT   3111   3.1   14   14   400  207.5  51.9   68   53  279  17.0  69.8  3097 
   4 Stockfish 8 64 POPCNT    3100  10.8   14   14   400  199.5  49.9   71   72  257  17.8  64.2  3100 
   5 Komodo 10.4 64-bit       3046  53.7   14   14   400  159.5  39.9   49  130  221  12.2  55.2  3113 
---------------------------------------------------------------------------------------------------------

McBrain 2.1f so far is leading candidate for version 2.1. McBrain-v2.0 would finish below stockfish 8 here. So we're looking at possible ~25 point ELo gain with 2.1. Of course a thousand games is not nearly enough either - but I do not have resources to test at more games at 2 2 nor do I have I have enough days remaining here on earth - so its has to do. But the odds are in my favor - here's the likelihood of superiority ( aka "los")

Code: Select all

ResultSet-EloRating>los
                        Mc Mc Mc St Ko
McBrain 2.1f 64 POPCNT     90 94 99 99
McBrain 2.1d 64 POPCNT   9    61 89 99
McBrain 2.1e 64 POPCNT   5 38    83 99
Stockfish 8 64 POPCNT    0 10 16    99
Komodo 10.4 64-bit       0  0  0  0

this was a nice run - versions 2.1a, 2.1b, 2.c , 2.1d were all successively better. e was regressive and since f versions g,h,i,j were all regressive - that's actually the norm , getting 4 straight better than the previous is very unusual , and so is the ~25 point ELo gain in a very short time. I'm now paying attention to longer tc - using 2 2 as my default tc for long tc. For my quick testing I use tc 15 sec .25 - I have found using tc any shorter than 15 .25 ( say 10 .1 or even 12 .2 to be very frustrating - many ideas work at these shorts tc, but do not scale well with longer tc at all. Some ideas do miserable with short tc and do very well at longer tc. I rather have the strength at longer tc. And of course having limited resources , I have to go with what I see.

Appreciate your feedback - thanks.

MikeB · Post by **MikeB** » Sun Mar 26, 2017 1:02 am

I kicked it off , using Dann's Silver.epd file:

Code: Select all

rnbqkbnr/pppp1ppp/8/4P3/8/8/PPPPP1PP/RNBQKBNR b KQkq - id "Silver-Gambit.01"; 
rnbqkbnr/ppppp1pp/8/5p2/4P3/5N2/PPPP1PPP/RNBQKB1R b KQkq - id "Silver-Gambit.02"; 
r1b1kbnr/ppppqppp/2n5/4P3/8/5N2/PPP1PPPP/RNBQKB1R w KQkq - id "Silver-Gambit.03"; 
rnbqkb1r/ppppp1pp/5n2/8/3Pp3/2N5/PPP2PPP/R1BQKBNR w KQkq - id "Silver-Gambit.04"; 
rnbqkbnr/pp1ppppp/8/8/1p2P3/8/P1PP1PPP/RNBQKBNR w KQkq - id "Silver-Gambit.05"; 
rnbqkbnr/pp1ppppp/8/8/4P3/2N5/PP3PPP/R1BQKBNR b KQkq - id "Silver-Gambit.06"; 
rnbqkbnr/pppp1ppp/8/8/4P3/2N5/PP3PPP/R1BQKBNR b KQkq - id "Silver-Gambit.07"; 
rnbqkb1r/pppp1ppp/5n2/8/2BpP3/5N2/PPP2PPP/RNBQK2R b KQkq - id "Silver-Gambit.08"; 
r1bqkb1r/pppp3p/2nn2p1/1N2pp2/8/1B3Q2/PPPP1PPP/R1B1K1NR w KQkq - id "Silver-Gambit.09"; 
rnbqkb1r/pppp1ppp/5n2/4p3/4PP2/2N5/PPPP2PP/R1BQKBNR b KQkq - id "Silver-Gambit.10"; 
rnbqkbnr/ppp2ppp/8/3Pp3/5P2/8/PPPP2PP/RNBQKBNR b KQkq - id "Silver-Gambit.11"; 
rnbqkbnr/pppp1p1p/8/6p1/2B1Pp2/5N2/PPPP2PP/RNBQK2R b KQkq - id "Silver-Gambit.12"; 
rnbqkbnr/pppp1p1p/8/8/2B1Pp2/5Q2/PPPP2PP/RNB2RK1 b kq - id "Silver-Gambit.13"; 
rnbqkbnr/pppp2pp/8/4pp2/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - id "Silver-Gambit.14"; 
rnbqkbnr/ppp3pp/3p4/4pp2/3PP3/5N2/PPP2PPP/RNBQKB1R w KQkq - id "Silver-Gambit.15"; 
rnbq1b1r/ppp2kpp/3p1n2/8/4P3/8/PPPP1PPP/RNBQKB1R w KQ - id "Silver-Gambit.16"; 
r1bqkbnr/pppp1ppp/2n5/8/4P3/2N2N2/PP3PPP/R1BQKB1R b KQkq - id "Silver-Gambit.17"; 
r1bqkb1r/pppp1ppp/2n2n2/4N3/4P3/2N5/PPPP1PPP/R1BQKB1R b KQkq - id "Silver-Gambit.18"; 
r1b1kbnr/pppp1Bpp/8/4N1q1/3nP3/8/PPPP1PPP/RNBQK2R b KQkq - id "Silver-Gambit.19"; 
r1bqk1nr/pppp1ppp/2n5/4p3/1bB1P3/2P2N2/P2P1PPP/RNBQK2R b KQkq - id "Silver-Gambit.20"; 
r1bqkbnr/pppp2pp/2n5/1B2pp2/4P3/5N2/PPPP1PPP/RNBQK2R w KQkq - id "Silver-Gambit.21"; 
r1bq1rk1/4bppp/p1p5/1p1nR3/8/1BP5/PP1P1PPP/RNBQ2K1 w - - id "Silver-Gambit.22"; 
rnbqkb1r/ppp1pppp/5n2/8/3Pp3/2N2P2/PPP3PP/R1BQKBNR b KQkq - id "Silver-Gambit.23"; 
rnbqkbnr/ppp2ppp/8/4P3/2Pp4/8/PP2PPPP/RNBQKBNR w KQkq - id "Silver-Gambit.24"; 
rnbqkb1r/p2p1ppp/4pn2/1ppP4/2P5/5N2/PP2PPPP/RNBQKB1R w KQkq - id "Silver-Gambit.25";

tc 2 min + 2 sec, results updated after every 30 games ( I'm using 10 of the 12 real core on my MacPro - 30 games takes about 30 minutes )

you can follow along here:

results: https://www.dropbox.com/s/et9gtoldp5mjd ... a.txt?dl=1

the pgn file is updated more frequently than the results - if you can't wait ;>)

https://www.dropbox.com/s/q661rrf61e4aq ... s.pgn?dl=1

so far:

Code: Select all

30 of 150 games completed...
time control&#58; 12000+200
Date&#58; 03/25/17 &#58; 19&#58;40&#58;40
30 game&#40;s&#41; loaded
Rank Name                    Rating   &#916;     +    -     #     &#931;    &#931;%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 McBrain 2.1f 64 POPCNT   3144   0.0   45   45    21   13.0  61.9    6    1   14  28.6  66.7  3078 
   2 Stockfish 8 64 POPCNT    3090  53.9   48   48    19    9.0  47.4    3    4   12  15.8  63.2  3107 
   3 Komodo 10.4 64-bit       3066  23.4   47   47    20    8.0  40.0    2    6   12  10.0  60.0  3119 
---------------------------------------------------------------------------------------------------------
30 game&#40;s&#41; loaded

note -early results do not often hold , and two I have never used Silver's epd suite before so I have no clue what to epect - there are only 25 positions - so this entire run will only be 150 games. Not nearly enough - even for my highly relaxed standards...

next update s/b around 8:10

MikeB · Post by **MikeB** » Tue Mar 28, 2017 6:38 am

for those that waiting for the next version of McBrain , stay tune.

After a quick start with v2.1 , making good progress, fairly rapidly to v2.1f, sort of hit a wall and all the versions from 2.1g to 2.1m were regressive and did not take. But version 2.1o, which was a take on 2.1n which was a very modest improvement - or perhaps equal - to 2.1f has seemed to hit paydirt in early testing at 2 m with 2 sec (making an concentrated attempt to have McB play better at tc 2 2 - since initial results were very disappointing to say the least)

Code: Select all

80 of 2000 games completed...
time control&#58; 12000+200
Date&#58; 03/28/17 &#58; 00&#58;11&#58;37
Rank Name                    Rating   &#916;     +    -     #     &#931;    &#931;%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 McBrain 2.1o 64 POPCNT   3130   0.0   23   23    33   19.5  59.1    6    0   27  18.2  81.8  3091 
   2 McBrain 2.1f 64 POPCNT   3112  17.5   25   25    31   16.5  53.2    3    1   27   9.7  87.1  3098 
   3 McBrain 2.1n 64 POPCNT   3111   1.6   25   25    32   17.0  53.1    3    1   28   9.4  87.5  3097 
   4 Stockfish 8 64 POPCNT    3090  20.7   24   24    32   15.0  46.9    3    5   24   9.4  75.0  3102 
   5 Komodo 10.4 64-bit       3057  33.2   23   23    32   12.0  37.5    3   11   18   9.4  56.2  3111 
---------------------------------------------------------------------------------------------------------

often these early results are exaggerated , and I expect it to be much closer at the end...

Follow the testing live :
https://www.dropbox.com/s/et9gtoldp5mjd ... a.txt?dl=1

pgn files:
https://www.dropbox.com/s/87ikp9q05yd3v18/all.pgn?dl=1

updates every 80 games, using 10 of the 12 real cores , so figure about every 40 minutes or so ( all of the McB engines are set to play fast when mate is found - no adjudications, 6 man syzygy, 64M hash, ponder off, opening book is a combined tcec superfinal 8 & 9 book with alternate colors of course

so far, 21.o has played 48 games against 2 stockfish based engines, stockfish 8 and komodo 10.4 at tc of 2 min /2 sec - without a loss, that's not shabby...

(quick update)

Code: Select all

120 game&#40;s&#41; loaded
Rank Name                    Rating   &#916;     +    -     #     &#931;    &#931;%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 McBrain 2.1o 64 POPCNT   3125   0.0   19   19    48   27.5  57.3    7    0   41  14.6  85.4  3094 
   2 McBrain 2.1f 64 POPCNT   3115  10.7   19   19    47   25.5  54.3    5    1   41  10.6  87.2  3097 
   3 McBrain 2.1n 64 POPCNT   3113   1.8   19   19    49   26.5  54.1    5    1   43  10.2  87.8  3096 
   4 Stockfish 8 64 POPCNT    3089  23.8   19   19    47   22.0  46.8    5    8   34  10.6  72.3  3103 
   5 Komodo 10.4 64-bit       3059  30.2   17   17    49   18.5  37.8    4   16   29   8.2  59.2  3110 
---------------------------------------------------------------------------------------------------------

if this bears fruit, look for an early release of version 2.1 very soon...

Guenther · Post by **Guenther** » Tue Mar 28, 2017 10:30 am

MikeB wrote:
note -early results do not often hold , and two I have never used Silver's epd suite before so I have no clue what to epect - there are only 25 positions...

next update s/b around 8:10

This was only the first release of it. Later it was extended to 50 positions.
Moreover a Silver gambit suite was also introduced later.
All of this can be easily found here at Talkchess with the search function.

Guenther

Vinvin · Post by **Vinvin** » Tue Mar 28, 2017 11:34 am

Guenther wrote:
MikeB wrote:
note -early results do not often hold , and two I have never used Silver's epd suite before so I have no clue what to epect - there are only 25 positions...

next update s/b around 8:10
This was only the first release of it. Later it was extended to 50 positions.
Moreover a Silver gambit suite was also introduced later.
All of this can be easily found here at Talkchess with the search function.

Guenther

And this list on WCP : http://chessprogramming.wikispaces.com/ ... g%20Suites

MikeB · Post by **MikeB** » Tue Mar 28, 2017 10:56 pm

I am going to kill this run right here as it looks like version 2.1f will prevail once again...

Code: Select all

1540 game&#40;s&#41; loaded
Rank Name                    Rating   &#916;     +    -     #     &#931;    &#931;%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 McBrain 2.1f 64 POPCNT   3116   0.0    9    9   616  328.0  53.2   98   58  460  15.9  74.7  3096 
   2 McBrain 2.1o 64 POPCNT   3113   3.4    9    9   616  324.5  52.7   88   55  473  14.3  76.8  3097 
   3 McBrain 2.1n 64 POPCNT   3108   5.3    9    9   615  316.5  51.5   75   57  483  12.2  78.5  3098 
   4 Stockfish 8 64 POPCNT    3101   6.5    9    9   616  309.5  50.2   93   90  433  15.1  70.3  3100 
   5 Komodo 10.4 64-bit       3062  39.4    9    9   617  261.5  42.4   57  151  409   9.2  66.3  3110 
---------------------------------------------------------------------------------------------------------

What has caught my interest, is that I can apply the same patch t0 2.1f which I applied to 2.1n which resulted in version 2.1o and see if the elo gains hold.

Gusev · Post by **Gusev** » Wed Mar 29, 2017 12:18 am

Where can I find version 2.1f? Meanhwhile, you're welcome to use my opening test suites as well, 90 variations and 1023 8-ply variations: http://www.foscec.org/FOSCECTestSuites_1-0.zip

MikeB wrote:I am going to kill this run right here as it looks like version 2.1f will prevail once again...

Code: Select all

1540 game&#40;s&#41; loaded
Rank Name                    Rating   &#916;     +    -     #     &#931;    &#931;%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------
   1 McBrain 2.1f 64 POPCNT   3116   0.0    9    9   616  328.0  53.2   98   58  460  15.9  74.7  3096 
   2 McBrain 2.1o 64 POPCNT   3113   3.4    9    9   616  324.5  52.7   88   55  473  14.3  76.8  3097 
   3 McBrain 2.1n 64 POPCNT   3108   5.3    9    9   615  316.5  51.5   75   57  483  12.2  78.5  3098 
   4 Stockfish 8 64 POPCNT    3101   6.5    9    9   616  309.5  50.2   93   90  433  15.1  70.3  3100 
   5 Komodo 10.4 64-bit       3062  39.4    9    9   617  261.5  42.4   57  151  409   9.2  66.3  3110 
---------------------------------------------------------------------------------------------------------

What has caught my interest, is that I can apply the same patch t0 2.1f which I applied to 2.1n which resulted in version 2.1o and see if the elo gains hold.

MikeB · Post by **MikeB** » Wed Mar 29, 2017 3:57 pm

Gusev wrote:Where can I find version 2.1f? Meanhwhile, you're welcome to use my opening test suites as well, 90 variations and 1023 8-ply variations: http://www.foscec.org/FOSCECTestSuites_1-0.zip

....soon , most likely - on or about April 1...

Damir · Post by **Damir** » Wed Mar 29, 2017 4:44 pm

April 1st, is not that an April fools joke ?

Scacchista1977 · Post by **Scacchista1977** » Fri Mar 31, 2017 6:54 pm

MikeB wrote: One of the highlights of this release came from SugaR (a SF based engine by Marco Zerbinati) - you can now toggle search functions "off" and "on" real time ( Futility, LMR, NullMove, ProbCut, Pruning, Razoring).
Michael

My humble opinion about it is that it was not a good choice to include these capabilities in the engine. In fact I have done tests with all possible configurations and the results have always been lower than expected. I think that to continue on the old road would be best, but I repeat again that's just my personal impression.

McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release

Re: McBrain v2.0 Release