jix wrote:Ponder was off, but for some reason when the child process was set to "Normal" Houdini would not search as deeply as it should.
Im running a 10 minute tournament right now, and thus far the results again are in Rybka's favor. I will post the result after 25/100 games, but thus far its +3/2/0 after 5 games.
FWIW, Rybka 4, with best time management settings, probably won't score more than ~55-56% in the long run against Houdini, which is roughly 40-45 Elo more. At least that has been the result of my testing so far at relatively fast TCs. Even if you did find an improved combination of piece values, it wouldn't yield 100 Elo, which is what 70% would equate to.
Bear in mind, my testing is done with single-cpu with ponder off. I know there are reports that Rybka gains enormously with uber-machines, but that is not covered with my results.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
I think you will get much higher than +45 ELO on big hardware like 8 core and more ... and from my testing on that platform the results are easily double that. I am still not sure what the problem is with smaller hardware but I really don't think it is simply about R4 having better scaling. According to Vas Rybka 4 uses processes and not threads, and in many systems the engine will more readily provide resources to threads than processes. Personally I think there is something in engine synchronization of processes that causes "lag" in normal play and this "lag" is more of a problem with less cores than it is when you have more cores. I also see this "lag" when Rybka is in a time rush and it will lose many games by simply taking too long to start a move. I calculate that in the endgame rush it takes about .2ms to .3ms to play a move while threaded engines can play 10X that fast. Maybe threaded engines are simply inherently more efficient at fast time controls and slower hardware.
M ANSARI wrote:I think you will get much higher than +45 ELO on big hardware like 8 core and more ... and from my testing on that platform the results are easily double that. I am still not sure what the problem is with smaller hardware but I really don't think it is simply about R4 having better scaling.
Scaling simply can not give R4 more than 50 elo per core doubling (when going from 4 to 8 cores). You have in your testing that R4 gains additional 50 elo when going from 4 to 8 cores. Since "Litos" already gain at least 40 elo when going from 4 to 8, this would mean R4 gains more than 90 elo, which is simply impossible. So something must be really wrong with your setup, which either strongly favors R4 or strongly disfavors "Litos" or both on 8 cores.
Personally I think there is something in engine synchronization of processes that causes "lag" in normal play and this "lag" is more of a problem with less cores than it is when you have more cores. I also see this "lag" when Rybka is in a time rush and it will lose many games by simply taking too long to start a move. I calculate that in the endgame rush it takes about .2ms to .3ms to play a move while threaded engines can play 10X that fast. Maybe threaded engines are simply inherently more efficient at fast time controls and slower hardware.
No need to rediscover the wheel. On windows processes are slower than threads and this is a well known fact. You have couple of posts on this topic in CCC, Bob's one for example is pretty convincing. However, with more processes/threads, this difference just increases, never reduces.
M ANSARI wrote:I think you will get much higher than +45 ELO on big hardware like 8 core and more ... and from my testing on that platform the results are easily double that. I am still not sure what the problem is with smaller hardware but I really don't think it is simply about R4 having better scaling.
Scaling simply can not give R4 more than 50 elo per core doubling (when going from 4 to 8 cores). You have in your testing that R4 gains additional 50 elo when going from 4 to 8 cores. Since "Litos" already gain at least 40 elo when going from 4 to 8, this would mean R4 gains more than 90 elo, which is simply impossible. So something must be really wrong with your setup, which either strongly favors R4 or strongly disfavors "Litos" or both on 8 cores.
Personally I think there is something in engine synchronization of processes that causes "lag" in normal play and this "lag" is more of a problem with less cores than it is when you have more cores. I also see this "lag" when Rybka is in a time rush and it will lose many games by simply taking too long to start a move. I calculate that in the endgame rush it takes about .2ms to .3ms to play a move while threaded engines can play 10X that fast. Maybe threaded engines are simply inherently more efficient at fast time controls and slower hardware.
No need to rediscover the wheel. On windows processes are slower than threads and this is a well known fact. You have couple of posts on this topic in CCC, Bob's one for example is pretty convincing. However, with more processes/threads, this difference just increases, never reduces.
Hi Milos,
Majd is an experienced tester and a hardware specialist in our community....I am sure that you've misunderstood him at a certain point....
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
It is possible that something is wrong with my setup, but I think I eliminated that possibility when I did the same tests on another 8 core machine with a clean OS install. Also the increase in scaling from 4 core to 8 core is dramatically better for the 8 core machine using other engines also running 8 cores (confirmed via task manager). I have tested on the same 8 core machine using 4 cores only and the results have been identical to the native 4 core machines. So hardware setup is definitely not the reason. The disproportional jump in scaling is also confirmed by two independent people I know using 8 cores, so I will tend to believe it is real, even though I know that theoretically that should not happen.
With relation to processes and threads, Vas seems to think that processes are better long term, but it sure as hell seems that many of Rybka's problems and bugs are directly related to "misbehaving" child processes. The famous EGTB bug was directly related to that, and in beta testing R4 another process initialization problem was eliminated that was also in R3. It is possible that if there is a disadvantage of using processes on 2 cores and 4 cores, this somehow disappears on 8 cores or more. It is also possible that there is some performance bug somewhere that is more relevant with 4 cores or less. Either way I am just speculating, but I do know that the testing results cannot be denied. It would have been very interesting to have a threaded R4 and compare scaling results against the Default R4.
Last edited by M ANSARI on Mon Jun 14, 2010 9:40 am, edited 1 time in total.
M ANSARI wrote:It is possible that something is wrong with my setup, but I think I eliminated that possibility when I did the same tests on another 8 core machine with a clean OS install. Also the increase in scaling from 4 core to 8 core is dramatically better for the 8 core machine using other engines also running 8 cores (confirmed via task manager). I have tested on the same 8 core machine using 4 cores only and the results have been identical to the native 4 core machines. So hardware setup is definitely not the reason. The disproportional jump in scaling is also confirmed by two independent people I know using 8 cores, so I will tend to believe it is real, even though I know that theoretically that should not happen.
What I would suggest for you to try is the following. Play on the very same 8 core machine, Rybka 4 on 8 cores vs. Rybka 4 on 4 cores to determine the exact change. If change is 50 elo or smaller, then it is not extreme R4 scaling but rather poor Houdini/Fire scaling on 8 cores (and here I'm talking about real scaling not just nps).
Oh I am pretty sure that Rybka 4's advantage (and actually R3) is much better scaling on 8 cores. Vas was pretty motivated to bring R3 scaling to the level of ZMII. At that time what most people never realized is that R2.3.2a was not actually stronger than ZMII on big hardware. I remember when I first built my Skulltrail and used a custom built dual vapor cooling system to get it to 5 Ghz, I was shocked to see that even at 16_0, ZMII was scoring even against R2.3.2a. At longer time controls ZMII was even stronger than R2.3.2a and I told quite a few people that Rybka would have a tough time in Mexico (including Dagh who was running Rybka) since the hardware there was big. There was a linear progression for ZMII results vs R2.3.2a as you went up in hardware. This all changed with R3 and scoring was pretty constant vs. ZMII even with hardware change. That would make me think that there was a very good improvement in MP scaling of R3 and this has carried on with R4. By the way R3 on 8 cores was also much stronger than any of the Ippolit derivatives. I haven't tested R3 much lately so not sure how it would do against Houdini or FB 1.31, but R4 on 8 cores is at least 90 ELO stronger at the time controls I tested.
but you did not get similar results (correct me if i'm wrong, i may well be) with R3 on 8 vs 4 or less cores, did you? this isn't just against Ip@$212 but against, say, zappa/naum/etc. (i think i've read shredder has bad scaling). i mean, if you did, why is this 'news' with everyone on rybkaforum all of a sudden?
i just know vas himself has said R4 scales only very slightly better than R3 on 8 cores, so i'm still just confused as to what the hubbub is about. i personally do not care, i think those with more than 4 cores should just appreciate the fact rybka scales much better on those machines than other engines & not worry about a 'fix' for engines with less cores...that's up to vas to figure out if he wants to or can do it.
jix wrote:TC Buffer: 3
TC Normal Move Time: 72
TC Max Move Time: 115
White Pawn cp: 10
Black Pawn cp: 10
White Knight cp: 40
Black Knight cp: 40
White King Bishop cp: 50
Black King Bishop cp: 50
White Queen Bishop cp: 45
Black Queen Bishop cp: 45
White Rook cp: 65
Black Rook cp: 65
White Queen cp: 110
Black Queen cp: 110
Rook Endgame Scaling: 100
Thanks, I'll check it out. BTW, just so you know: In increment games the TC Buffer is ignored.
Have you done anymore testing with these settings Albert.
I have found them very useful.
