H4 or S5 !?

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Vinvin
Posts: 5287
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: H4 or S5 !?

Post by Vinvin »

IWB wrote:
Vinvin wrote: I think 4pc Syzygy will change nothing in the SF strength. So for me, it's a second test with SF at the exact same level, which it is interesting too ;-)

BTW, it should be interesting to find one (yes, only one ! ) game where the 4pc Syzygy change the final outcome.
I am not sure either, but I doubt a significant win with 5pc as well.

Again, I hve no doubt that for a serious analysis of games/engamepostions the use a Tablebases is VERY usefull.

Time will tell, right now, after 50%+ it looks like a small 1 digit elo gain (3-5)

BYe
Ingo
The second run is finished, the 3 biggest differences up and down by opponent :

Code: Select all

Stockfish 5 SYZ4 - Equinox 2.02 (2975)   +4,1%
Stockfish 5 SYZ4 - Hannibal 1.4b (2870)  +4,1%
Stockfish 5 SYZ4 - Komodo 7a (3088)      +3,6%

Stockfish 5 SYZ4 - Texel 1.04 (2838)     -1,1%
Stockfish 5 SYZ4 - Jonny 6.00 (2798)     -1,1%
Stockfish 5 SYZ4 - Gull 3 (3057)         -1,6%
Complete comparison 1st run and 2nd run :

Code: Select all

Houdini 4          57,95%    57,05%
Komodo 7a          55,23%    58,86%
Gull 3             60,91%    59,32%
Critter 1,4a       68,41%    69,09%
Equinox 2,02       67,73%    71,82%
Deep Rybka 4,1     72,50%    73,41%
Chiron 2           77,50%    79,09%
Protector 1,6,0    82,27%    82,95%
Hannibal 1,4b      76,36%    80,45%
Naum 4,2           83,18%    84,09%
Texel 1,04         85,00%    83,86%
Senpai 1,0         85,00%    84,32%
HIARCS 14 WCSC 32b 85,45%    86,82%
Jonny 6,00         86,59%    85,45%
User avatar
Ajedrecista
Posts: 2104
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: H4 or SF5!?

Post by Ajedrecista »

Hello Miguel:
michiguel wrote:So, the reason for the discrepancy in the ranking order is exactly that: BE counts the draws twice, Ordo once.

Miguel
The following thread is probably related to that:

1 draw=1 win + 1 loss (always!)

It is a little long but worthwhile.

Regards from Spain.

Ajedrecista.
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: H4 or S5 !?

Post by IWB »

Vinvin wrote: The second run is finished, the 3 biggest differences up and down by opponent :

Code: Select all

Stockfish 5 SYZ4 - Equinox 2.02 (2975)   +4,1%
Stockfish 5 SYZ4 - Hannibal 1.4b (2870)  +4,1%
Stockfish 5 SYZ4 - Komodo 7a (3088)      +3,6%

Stockfish 5 SYZ4 - Texel 1.04 (2838)     -1,1%
Stockfish 5 SYZ4 - Jonny 6.00 (2798)     -1,1%
Stockfish 5 SYZ4 - Gull 3 (3057)         -1,6%
Complete comparison 1st run and 2nd run :

Code: Select all

Houdini 4          57,95%    57,05%
Komodo 7a          55,23%    58,86%
Gull 3             60,91%    59,32%
Critter 1,4a       68,41%    69,09%
Equinox 2,02       67,73%    71,82%
Deep Rybka 4,1     72,50%    73,41%
Chiron 2           77,50%    79,09%
Protector 1,6,0    82,27%    82,95%
Hannibal 1,4b      76,36%    80,45%
Naum 4,2           83,18%    84,09%
Texel 1,04         85,00%    83,86%
Senpai 1,0         85,00%    84,32%
HIARCS 14 WCSC 32b 85,45%    86,82%
Jonny 6,00         86,59%    85,45%
I think an individual comparision is missleading. You compare two sets of games with just 220 games each. The possible error is huge and even an identical run might be that much different.

Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.

Anyhow, I will play the missing games vs DF14 and will check if I replace SF5 standard with this one ...

BYe
Ingo
Vinvin
Posts: 5287
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: H4 or S5 !?

Post by Vinvin »

IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error. The other mean is the 4 pcs EGTB add not even a fraction of rating point to SF ...
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: H4 or S5 !?

Post by IWB »

Vinvin wrote:
IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error.
Do you know a method to answer this? I don't! Personaly I think it might be just statistical noise but even if it is due to the bases it is so small it can be neglected.

But for me the main point is the oportunity to come out of the "statisticcal dilema". I do not have (the VERY VERY RARE) missmatch between Bayes/Elostat/and Ordo anymore

:-)
Ingo
lkaufman
Posts: 6231
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: H4 or S5 !?

Post by lkaufman »

Vinvin wrote:
IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error. The other mean is the 4 pcs EGTB add not even a fraction of rating point to SF ...
4 pc EGTB should at least lead to playing queen vs. rook optimally. I don't know how often that occurs. My best guess is that it's worth about 2 elo. There may be some rare cases of other endgames, maybe rook vs knight or rook vs bishop, where it could make a difference. It helps to know which cases of those endings are won and which are drawn.
Vinvin
Posts: 5287
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: H4 or S5 !?

Post by Vinvin »

lkaufman wrote:
Vinvin wrote:
IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error. The other mean is the 4 pcs EGTB add not even a fraction of rating point to SF ...
4 pc EGTB should at least lead to playing queen vs. rook optimally. I don't know how often that occurs. My best guess is that it's worth about 2 elo. There may be some rare cases of other endgames, maybe rook vs knight or rook vs bishop, where it could make a difference. It helps to know which cases of those endings are won and which are drawn.
Do you "know" or do you "suppose" that SF without EGBT makes inferior results in the QvsR endgames ?
Vinvin
Posts: 5287
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: H4 or S5 !?

Post by Vinvin »

I just checked, 3 games SF without EGTB win the longest checkmate 3 times in a row against SF with EGTB.
In a very fast time control : 10sec+0.1sec !!

Vinvin wrote:
lkaufman wrote:
Vinvin wrote:
IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error. The other mean is the 4 pcs EGTB add not even a fraction of rating point to SF ...
4 pc EGTB should at least lead to playing queen vs. rook optimally. I don't know how often that occurs. My best guess is that it's worth about 2 elo. There may be some rare cases of other endgames, maybe rook vs knight or rook vs bishop, where it could make a difference. It helps to know which cases of those endings are won and which are drawn.
Do you "know" or do you "suppose" that SF without EGBT makes inferior results in the QvsR endgames ?
Uri Blass
Posts: 10805
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: H4 or S5 !?

Post by Uri Blass »

Vinvin wrote:
IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error. The other mean is the 4 pcs EGTB add not even a fraction of rating point to SF ...
I do not understand how do you know?

People talk only about advantage of playing the endgame perfectly
but it is not the only advantage and there are at least 2 different advantage by tablebases.

1)Knowing positions not to go.

Stockfish's static evaluation is wrong for some tablebases position and stockfish may fail by going into them when the problem is not playing the position perfectly but not to go to them.

Stockfish without tablebases may go to the following position with white only to discover too late that it is a draw

8 
7 
6 
5 
4 
3 
2 
1 
abcdefgh

k7/2Q5/8/8/8/8/7K/6r1 b - - 5 1

The fact that it can see at depth above 20 that it is a draw is not going to help it if the remaining depth in this position is only 10 plies.

2)Playing faster.
Stockfish can save time by not searching some tablebase positions so the advantage can be simply that it is going one ply deeper in the relevant lines that do not lead to 4 tablebases piece positions.
lkaufman
Posts: 6231
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: H4 or S5 !?

Post by lkaufman »

Vinvin wrote:
lkaufman wrote:
Vinvin wrote:
IWB wrote:Overall S5 with 4pc SYZ was better by 0.89% - but even that is within the error bar.
...
BYe
Ingo
Here's the main question : is the 0.89% difference due to the settings or due to statistical error ? For me, it's due 100% to statistical error. The other mean is the 4 pcs EGTB add not even a fraction of rating point to SF ...
4 pc EGTB should at least lead to playing queen vs. rook optimally. I don't know how often that occurs. My best guess is that it's worth about 2 elo. There may be some rare cases of other endgames, maybe rook vs knight or rook vs bishop, where it could make a difference. It helps to know which cases of those endings are won and which are drawn.
Do you "know" or do you "suppose" that SF without EGBT makes inferior results in the QvsR endgames ?
I suppose. But even if SF wins the queen side without TBs, it may defend the losing side better with them, which may pay off against opponents without TBs. Also, as Uri points out, it saves time whenever this (or other) endgames arise during the search.