Komodo 13.3 personalities strength ranking

[Leto]

Komodo 13.3 introduced 7 new personalities so I thought it'd be interesting to rank them according to their playing strength. For this experiment I set them all to skill level 24 and 12 cpus on my 12 core machine, and 2 minute blitz although time doesn't matter much as skill level 24 has them all moving almost instantly.

For the first round I had them play two games against each other:

1 2 3 4 5 6 7
1 Komodo 13.3 64-bit 24 12Human ** 11 0½ ½½ 11 11 11 9.5/12
2 Komodo 13.3 64-bit 24,12Active 00 ** 10 ½1 10 11 ½½ 6.5/12
3 Komodo 13.3 64-bit 24 12Defensive 1½ 01 ** ½½ ½½ ½0 ½½ 6.0/12
4 Komodo 13.3 64-bit 24 12Endgame ½½ ½0 ½½ ** 00 10 11 5.5/12 31.75
5 Komodo 13.3 64-bit 24 12Positional 00 01 ½½ 11 ** 00 ½1 5.5/12 29.50
6 Komodo 13.3 64-bit 24 12Aggressive 00 00 ½1 01 11 ** ½0 5.0/12
7 Komodo 13.3 64-bit 24 12Beginner 00 ½½ ½½ 00 ½0 ½1 ** 4.0/12

So in the first round the human personality came out on top but because there's a wide range of strength in this round it is not necessarily the case that the human personality is the strongest. The beginner personality came in last as expected.

For the second round I had the top three scorers face off against each other in 12 game matches:

Human vs Active match:

123456789012
1 Komodo 13.3 64-bit 24,12Active +154 0101111½½½11 8.5/12
2 Komodo 13.3 64-bit 24 12Human -154 1010000½½½00 3.5/12

This result makes me think the Human personality has a high contempt setting which would perhaps explain why it did so well in the wide tournament but not so well against the Active personality.


Human vs Defensive match:

123456789012
1 Komodo 13.3 64-bit 24 12Defensive +58 011½01½01110 7.0/12
2 Komodo 13.3 64-bit 24 12Human -58 100½10½10001 5.0/12

The Human personality again lost a match to one of the top scorers from the wide tournament.


Now the final match to determine the champion of the personalities:

Active vs Defensive match:

123456789012
1 Komodo 13.3 64-bit 24 12Defensive +89 11½0½½1½1½01 7.5/12
2 Komodo 13.3 64-bit 24,12Active -89 00½1½½0½0½10 4.5/12

Seems to me that out of the personalities the Defensive personality is probably the strongest.


Out of curiosity I decided to make a match between the two lowest scorers from the wide tournament:

Match of the weakest personalities:

123456789012
1 Komodo 13.3 64-bit 24 12Aggresive +154 ½01½111½½1½1 8.5/12
2 Komodo 13.3 64-bit 24 12Beginner -154 ½10½000½½0½0 3.5/12

Yes the beginner personality definitely seems the weakest.


So I figured why not put the champion of the personalities, Defensive, up against skill level 24 default Komodo 13.3 with 12 threads? :



123456789012
1 Komodo 13.3 64-bit 24 12 +120 011½½01½111½ 8.0/12
2 Komodo 13.3 64-bit 24 12Defensive -120 100½½10½000½ 4.0/12

Default Komodo is much stronger.


I thought it'd be interesting to see what happens when you set the weakest personality, Beginner, to the maximum skill level of 25 and set it to just one thread, would that be enough to defeat the surprisingly strong default Komodo at skill level 24 set to 12 threads? Here goes:


123456789012
1 Komodo 13.3 64-bit 25 1Beginner +89 1½1½½1½1½0½½ 7.5/12
2 Komodo 13.3 64-bit 24 12 -89 0½0½½0½0½1½½ 4.5/12

Incredibly skill level 25 Beginner with just one thread demolishes skill level 24 default Komodo with 12 threads! That is one mighty strong beginner, perhaps strong enough to defeat the human world champion?!

[lkaufman]

Komodo 13.3 introduced 7 new personalities so I thought it'd be interesting to rank them according to their playing strength. For this experiment I set them all to skill level 24 and 12 cpus on my 12 core machine, and 2 minute blitz although time doesn't matter much as skill level 24 has them all moving almost instantly.

For the first round I had them play two games against each other:

1 2 3 4 5 6 7
1 Komodo 13.3 64-bit 24 12Human ** 11 0½ ½½ 11 11 11 9.5/12
2 Komodo 13.3 64-bit 24,12Active 00 ** 10 ½1 10 11 ½½ 6.5/12
3 Komodo 13.3 64-bit 24 12Defensive 1½ 01 ** ½½ ½½ ½0 ½½ 6.0/12
4 Komodo 13.3 64-bit 24 12Endgame ½½ ½0 ½½ ** 00 10 11 5.5/12 31.75
5 Komodo 13.3 64-bit 24 12Positional 00 01 ½½ 11 ** 00 ½1 5.5/12 29.50
6 Komodo 13.3 64-bit 24 12Aggressive 00 00 ½1 01 11 ** ½0 5.0/12
7 Komodo 13.3 64-bit 24 12Beginner 00 ½½ ½½ 00 ½0 ½1 ** 4.0/12

So in the first round the human personality came out on top but because there's a wide range of strength in this round it is not necessarily the case that the human personality is the strongest. The beginner personality came in last as expected.

For the second round I had the top three scorers face off against each other in 12 game matches:

Human vs Active match:

123456789012
1 Komodo 13.3 64-bit 24,12Active +154 0101111½½½11 8.5/12
2 Komodo 13.3 64-bit 24 12Human -154 1010000½½½00 3.5/12

This result makes me think the Human personality has a high contempt setting which would perhaps explain why it did so well in the wide tournament but not so well against the Active personality.


Human vs Defensive match:

123456789012
1 Komodo 13.3 64-bit 24 12Defensive +58 011½01½01110 7.0/12
2 Komodo 13.3 64-bit 24 12Human -58 100½10½10001 5.0/12

The Human personality again lost a match to one of the top scorers from the wide tournament.


Now the final match to determine the champion of the personalities:

Active vs Defensive match:

123456789012
1 Komodo 13.3 64-bit 24 12Defensive +89 11½0½½1½1½01 7.5/12
2 Komodo 13.3 64-bit 24,12Active -89 00½1½½0½0½10 4.5/12

Seems to me that out of the personalities the Defensive personality is probably the strongest.


Out of curiosity I decided to make a match between the two lowest scorers from the wide tournament:

Match of the weakest personalities:

123456789012
1 Komodo 13.3 64-bit 24 12Aggresive +154 ½01½111½½1½1 8.5/12
2 Komodo 13.3 64-bit 24 12Beginner -154 ½10½000½½0½0 3.5/12

Yes the beginner personality definitely seems the weakest.


So I figured why not put the champion of the personalities, Defensive, up against skill level 24 default Komodo 13.3 with 12 threads? :



123456789012
1 Komodo 13.3 64-bit 24 12 +120 011½½01½111½ 8.0/12
2 Komodo 13.3 64-bit 24 12Defensive -120 100½½10½000½ 4.0/12

Default Komodo is much stronger.


I thought it'd be interesting to see what happens when you set the weakest personality, Beginner, to the maximum skill level of 25 and set it to just one thread, would that be enough to defeat the surprisingly strong default Komodo at skill level 24 set to 12 threads? Here goes:


123456789012
1 Komodo 13.3 64-bit 25 1Beginner +89 1½1½½1½1½0½½ 7.5/12
2 Komodo 13.3 64-bit 24 12 -89 0½0½½0½0½1½½ 4.5/12

Incredibly skill level 25 Beginner with just one thread demolishes skill level 24 default Komodo with 12 threads! That is one mighty strong beginner, perhaps strong enough to defeat the human world champion?!

Yes, your findings and conclusions make sense; human does have high Contempt and other changes that I expect would help against strong human players but not against engines; beginner personality should be the weakest, but in general the strength differences of the personalities are not intended to exceed one level, as that makes the level setting lose its meaning. The beginner personality on level 25 one thread should beat even default level 24 on 12 threads, since level 25 has unlimited search while level 24 will only search 12 plies, and even a rather crude 18 ply search (for example) with minimal chess understanding will beat a very high quality smart 12 ply search more often than not. The personality describes chess understanding, not tactical ability.