Komodo and SF: small android tourney

[Gurcan Uckardes]

Yes i'm that crazy man. 5 sec experiment is stopped at 16000 games. Now i'm working on Rapidroid using 15 secs/move and two devices. 5000 games so far, about 200 games per engine.
The day i come to sufficient number of samples, the list will be some kind of SSDF because i will not stay limited to Android engines
Check out the current Android only list at: http://chesstroid.blogspot.com/2014/12/ ... gines.html

[Laskos]

Yes i'm that crazy man. 5 sec experiment is stopped at 16000 games. Now i'm working on Rapidroid using 15 secs/move and two devices. 5000 games so far, about 200 games per engine.
The day i come to sufficient number of samples, the list will be some kind of SSDF because i will not stay limited to Android engines
Check out the current Android only list at: http://chesstroid.blogspot.com/2014/12/ ... gines.html

Absolutely crazy. 15 sec/move is really painful to play many games. Thanks for the link and the other links to engines and such inside the page. You are sacrificing some four mobile devices to build this excellent rating list

I was curious about the issue SF 121014 vs. SF 5, and left overnight my tablet playing games to see a bit what happens. Engines on 4 cores, 50 openings from 8moves.pgn of SF testing framework, repeated both sides. Time control 1 sec/move (shorter than your TC), round-robin with a total of 300 games:

Code: Select all

                TABLE           POINTS      ELO   +   -

SF 121014    *   53.5  73.5    127.0/200    +65   39  38
SF 5        46.5   *   66.5    113.0/200    +32   36  36
Komodo 8    26.5  33.5   *      60.0/200    -97   41  41

Again, at 1 sec/move SF 121014 seems some 30 Elo points stronger here than SF 5. Maybe at your TC of 15 sec/move the things are different, as this TC is not really short. Also, the error bars are still large.

[Gurcan Uckardes]

Mmm... Quite interesting. My games were played on Intel using i686 binaries. 1 sec might have an effect, i'm not sure.
I think i'm gonna play a duel on Exynos with all 454 openings of TCEC6.
_________________
My blog for Android users: http://chesstroid.blogspot.com

[stavros]

Mmm... Quite interesting. My games were played on Intel using i686 binaries. 1 sec might have an effect, i'm not sure.
I think i'm gonna play a duel on Exynos with all 454 openings of TCEC6.
_________________
My blog for Android users: http://chesstroid.blogspot.com

hi. i think is a good idea to solve that mystery of droidfish versions
i will wait for your results, maybe at longer TC than 1sec/move?

[M ANSARI]

Wow ... SF on Android is a beast !!!

[Laskos]

Wow ... SF on Android is a beast !!!

Komodo 8 got understated in my tests, as I was using the version from the full package of Komodo 8 on different platforms. Mark Lefler told me that the Google Play latest version Komodo 8 v1.6.1 (last update on 29th of December 2014) is significantly stronger.

I got and tested it, on 50 positions from 8moves.pgn of SF testing framework, both sides of the opening. 1 sec/move with 4 cores each engine, ponder off, in Chess for Android GUI.

SF 121014 vs Generic Komodo 8 in the full package: 73.5:26.5 +177 Elo points.
SF 121014 vs Google Play Komodo 8 v1.6.1: 60.5:39.5 +74 Elo points.

So, Google Play version of Komodo is not that far off Stockfish, and probably even closer at longer TC.

[Gurcan Uckardes]

I couldn't achieve this even with 15 secs Kai. I think compiling matters and 32bit matters too, in favor of Stockfish. Another finding to confirm the reason is the nps perfo which favors Stockfish. Komodo has less nodes on Android while on PC, it overtakes SF, according to TCEC at least. We must check whether it's related to 16 cores efficiency of Komodo. Maybe 4 cores in PC would show the same. I don't have K8 on PC.
After all, Mark also would confirm the port of SF is perfect and there's still room for improvement regarding the port of Android.
You are right about the huge gap between bundled version of Komodo and zGplay version. As for the Gplay version, accumulated elo increase is about 10 elos. Not too significant yet. I have used all of them.
The Rapidroid ranking i'm releasing includes Komodo v1.6 and mostly 1.6.1.

[Gurcan Uckardes]

Additional remark: We know Komodo 8 has better time management compared to SF. By using fixed time per move, we are cutting out some elos. I think next Chess for Android version with fischer clock option will let us evaluate Komodo more accurately.

[Laskos]

Additional remark: We know Komodo 8 has better time management compared to SF. By using fixed time per move, we are cutting out some elos. I think next Chess for Android version with fischer clock option will let us evaluate Komodo more accurately.

Yes, I am waiting eagerly for clocks in "Chess for Android", as fixed time per move is quite inefficient, with clocks engines take less on often decided endgames and more on important moves.

You are right about the larger hit Komodo takes on 32 bit compared to SF, but my GPlay is significantly better than the full package android K8. And still, NPS of Komodo GPlay is significantly lower than that of SF android version, while on PC they are similar.

I played another tournament with Komodo 8 GPlay version 1.6.1, this time 4 best engines on 4 cores, 1 sec/move, and Komodo 8 is closer to Stockfish thant to Critter 1.6a, which for a time was the leader of android chess engines:

Code: Select all

    Program                            Score     %      Elo    +   -    Draws

  1 Stockfish 121014               :  98.5/150  65.7     90   45  44   38.0 %
  2 Stockfish 5                    :  94.0/150  62.7     73   44  43   40.0 %
  3 Komodo 8 32-bit                :  78.5/150  52.3     18   46  46   31.3 %
  4 Critter 1.6a 32-bit            :  29.0/150  19.3   -181   57  58   21.3 %

SF 121014 seems again better than SF 5 in these conditions, and K8 not far behind the two Stockfishes.

[Gurcan Uckardes]

Ok. If you can find time to repeat the same with 5 secs, i'm curious to see what it will deliver.