Komodo and SF: small android tourney

Discussion of computer chess matches and engine tournaments.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Gurcan Uckardes
Posts: 196
Joined: Tue Oct 28, 2014 11:42 pm
Contact:

Re: Komodo and SF: small android tourney

Post by Gurcan Uckardes » Sun Jan 04, 2015 3:34 pm

Yes i'm that crazy man. 5 sec experiment is stopped at 16000 games. Now i'm working on Rapidroid using 15 secs/move and two devices. 5000 games so far, about 200 games per engine.
The day i come to sufficient number of samples, the list will be some kind of SSDF because i will not stay limited to Android engines :-)
Check out the current Android only list at: http://chesstroid.blogspot.com/2014/12/ ... gines.html
My blog for Android users: http://chesstroid.blogspot.com

User avatar
Laskos
Posts: 10781
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Komodo and SF: small android tourney

Post by Laskos » Mon Jan 05, 2015 10:10 am

Gurcan Uckardes wrote:Yes i'm that crazy man. 5 sec experiment is stopped at 16000 games. Now i'm working on Rapidroid using 15 secs/move and two devices. 5000 games so far, about 200 games per engine.
The day i come to sufficient number of samples, the list will be some kind of SSDF because i will not stay limited to Android engines :-)
Check out the current Android only list at: http://chesstroid.blogspot.com/2014/12/ ... gines.html
Absolutely crazy. 15 sec/move is really painful to play many games. Thanks for the link and the other links to engines and such inside the page. You are sacrificing some four mobile devices to build this excellent rating list :)

I was curious about the issue SF 121014 vs. SF 5, and left overnight my tablet playing games to see a bit what happens. Engines on 4 cores, 50 openings from 8moves.pgn of SF testing framework, repeated both sides. Time control 1 sec/move (shorter than your TC), round-robin with a total of 300 games:

Code: Select all

                TABLE           POINTS      ELO   +   -

SF 121014    *   53.5  73.5    127.0/200    +65   39  38
SF 5        46.5   *   66.5    113.0/200    +32   36  36
Komodo 8    26.5  33.5   *      60.0/200    -97   41  41
Again, at 1 sec/move SF 121014 seems some 30 Elo points stronger here than SF 5. Maybe at your TC of 15 sec/move the things are different, as this TC is not really short. Also, the error bars are still large.

Gurcan Uckardes
Posts: 196
Joined: Tue Oct 28, 2014 11:42 pm
Contact:

Re: Komodo and SF: small android tourney

Post by Gurcan Uckardes » Mon Jan 05, 2015 7:05 pm

Mmm... Quite interesting. My games were played on Intel using i686 binaries. 1 sec might have an effect, i'm not sure.
I think i'm gonna play a duel on Exynos with all 454 openings of TCEC6.
_________________
My blog for Android users: http://chesstroid.blogspot.com
My blog for Android users: http://chesstroid.blogspot.com

stavros
Posts: 160
Joined: Tue Dec 02, 2014 12:29 am

Re: Komodo and SF: small android tourney

Post by stavros » Tue Jan 06, 2015 12:18 am

Gurcan Uckardes wrote:Mmm... Quite interesting. My games were played on Intel using i686 binaries. 1 sec might have an effect, i'm not sure.
I think i'm gonna play a duel on Exynos with all 454 openings of TCEC6.
_________________
My blog for Android users: http://chesstroid.blogspot.com
hi. i think is a good idea to solve that mystery of droidfish versions :)
i will wait for your results, maybe at longer TC than 1sec/move?

User avatar
M ANSARI
Posts: 3467
Joined: Thu Mar 16, 2006 6:10 pm

Re: Komodo and SF: small android tourney

Post by M ANSARI » Tue Jan 06, 2015 5:49 am

Wow ... SF on Android is a beast !!!

User avatar
Laskos
Posts: 10781
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Komodo and SF: small android tourney

Post by Laskos » Tue Jan 06, 2015 8:44 am

M ANSARI wrote:Wow ... SF on Android is a beast !!!
Komodo 8 got understated in my tests, as I was using the version from the full package of Komodo 8 on different platforms. Mark Lefler told me that the Google Play latest version Komodo 8 v1.6.1 (last update on 29th of December 2014) is significantly stronger.

I got and tested it, on 50 positions from 8moves.pgn of SF testing framework, both sides of the opening. 1 sec/move with 4 cores each engine, ponder off, in Chess for Android GUI.

SF 121014 vs Generic Komodo 8 in the full package: 73.5:26.5 +177 Elo points.
SF 121014 vs Google Play Komodo 8 v1.6.1: 60.5:39.5 +74 Elo points.

So, Google Play version of Komodo is not that far off Stockfish, and probably even closer at longer TC.

Gurcan Uckardes
Posts: 196
Joined: Tue Oct 28, 2014 11:42 pm
Contact:

Re: Komodo and SF: small android tourney

Post by Gurcan Uckardes » Tue Jan 06, 2015 9:10 pm

I couldn't achieve this even with 15 secs Kai. I think compiling matters and 32bit matters too, in favor of Stockfish. Another finding to confirm the reason is the nps perfo which favors Stockfish. Komodo has less nodes on Android while on PC, it overtakes SF, according to TCEC at least. We must check whether it's related to 16 cores efficiency of Komodo. Maybe 4 cores in PC would show the same. I don't have K8 on PC.
After all, Mark also would confirm the port of SF is perfect and there's still room for improvement regarding the port of Android.
You are right about the huge gap between bundled version of Komodo and zGplay version. As for the Gplay version, accumulated elo increase is about 10 elos. Not too significant yet. I have used all of them.
The Rapidroid ranking i'm releasing includes Komodo v1.6 and mostly 1.6.1.
My blog for Android users: http://chesstroid.blogspot.com

Gurcan Uckardes
Posts: 196
Joined: Tue Oct 28, 2014 11:42 pm
Contact:

Re: Komodo and SF: small android tourney

Post by Gurcan Uckardes » Tue Jan 06, 2015 9:16 pm

Additional remark: We know Komodo 8 has better time management compared to SF. By using fixed time per move, we are cutting out some elos. I think next Chess for Android version with fischer clock option will let us evaluate Komodo more accurately.
My blog for Android users: http://chesstroid.blogspot.com

User avatar
Laskos
Posts: 10781
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Komodo and SF: small android tourney

Post by Laskos » Wed Jan 07, 2015 4:48 pm

Gurcan Uckardes wrote:Additional remark: We know Komodo 8 has better time management compared to SF. By using fixed time per move, we are cutting out some elos. I think next Chess for Android version with fischer clock option will let us evaluate Komodo more accurately.
Yes, I am waiting eagerly for clocks in "Chess for Android", as fixed time per move is quite inefficient, with clocks engines take less on often decided endgames and more on important moves.

You are right about the larger hit Komodo takes on 32 bit compared to SF, but my GPlay is significantly better than the full package android K8. And still, NPS of Komodo GPlay is significantly lower than that of SF android version, while on PC they are similar.

I played another tournament with Komodo 8 GPlay version 1.6.1, this time 4 best engines on 4 cores, 1 sec/move, and Komodo 8 is closer to Stockfish thant to Critter 1.6a, which for a time was the leader of android chess engines:

Code: Select all

    Program                            Score     %      Elo    +   -    Draws

  1 Stockfish 121014               :  98.5/150  65.7     90   45  44   38.0 %
  2 Stockfish 5                    :  94.0/150  62.7     73   44  43   40.0 %
  3 Komodo 8 32-bit                :  78.5/150  52.3     18   46  46   31.3 %
  4 Critter 1.6a 32-bit            :  29.0/150  19.3   -181   57  58   21.3 %
SF 121014 seems again better than SF 5 in these conditions, and K8 not far behind the two Stockfishes.

Gurcan Uckardes
Posts: 196
Joined: Tue Oct 28, 2014 11:42 pm
Contact:

Re: Komodo and SF: small android tourney

Post by Gurcan Uckardes » Wed Jan 07, 2015 6:56 pm

Ok. If you can find time to repeat the same with 5 secs, i'm curious to see what it will deliver.
My blog for Android users: http://chesstroid.blogspot.com

Post Reply