LS-ratinglist (news & comments)

[lkaufman]

The result of Komodo 6 is now online.

http://ls-ratinglist.beepworld.de


Stefan

(Perhaps you have to clear your browsercache or reload the website)

A gain of 27 elo over the previous MP (5.1) version is right in line with other tests and with my expectations (I estimated +25). Your list is really great at measuring improvement from one version to another within one family. But your time limit is way too fast for comparing unrelated engines; specifically Houdini and all other Ippo-derived engines are just incredibly strong at bullet speed relative to Stockfish and Komodo, but once you get out to normal blitz levels and beyond this superiority vanishes. Work continues and we keep gaining elo, but even after Komodo and Stockfish are clearly superior to Houdini 3 at normal blitz levels they will still lag beyond at your time control. Despite this, I very much appreciate your list as it has the samples needed for accuracy to measure small improvements from version to version. If I ever made my own list I would do things pretty much as you do but since I have much better hardware I would use a much longer time control.

Thanx!
The really interesting question is: Why are the Ippo-derivatives so incredible strong at fast time-controls and why Komodo and Stockfish cant do better at fast time-controls? The code of Ippolit and Firenzina is OpenSource, why nobody of Komodo or Stockfish-Team looks at it to find the answer? On much slower Mobile Devices, engines should play strong, too. And slow hardware is the same as very fast time-controls (on a modern PC)...If you want to publish Komodo on Android or iOS, the weakness of Komodo on slow mobile hardware could be a problem...

Best - Stefan

Well, weakness on slow hardware at fast time controls is only relative to Houdini, but we would very much like to improve this. We have studied the Ippo code in the past but never found anything that we could do to specifically help bullet-level play. One theory is just that our eval is slower but more sophisticated, which pays off more with increasing depth. But Stockfish has the same relative weakness at bullet levels and I don't think their eval is slow, so that's probably not the whole story. Any informed opinions are welcome.

Hi Larry.
I do not have an informed opinion and I am a layman, but my theory, just by looking at the playing style and weaknesses of the 3 engines, would be that Houdini is simply designed in a way (I do not know if eval, search or the mix) that it is tactically stronger than both Komodo and Stockfish.

Tactical strength matters more at shorter time controls, especially very short, and gradually fades with increase of time. Houdini sees at very short time control tactical continuations that other engines would miss at this TC, but would not at longer one.

I think you are probably right, but it is not clear to us why this is so. We don't know how to make Komodo tactically stronger without on balance reducing its elo.

[pohl4711]

Well I think the whole issue is blown up or kind of non existing
When you look close enough to all the data in the different ratinglists at different TC's you can observe that differences in the LS list between the engines diminishes at the longer TC ratinglists
And this counts for all engines. Also for engines out of same family class !
Look for instance at the ELO diff between Komodo 5 and Komodo CCT
At LS the diff = 35 while at 40/20 CEGT it is 17 not to speak about 40/20pb where there is just 10 ELO diff left
Than as we take the diff between Komodo 5 and Houdini3 at LS this is 95 ELO at CEGT 40/20 it is 55 so 40% less, while between Komodo CCT and Komodo 5 the ELO diff has become less by 50%

Even when we look further at the CEGT 40/120 list the ELO diff bewteen Houdini 3 and Komodo 5 no longer diminishes, but stabilizes as not to say even becomes bigger again in favour of Houdini 3

So whats the fuzz

Code: Select all

	LS	CEGT4/40	CEGT40/20	CEGT40/20pb	CEGT 40/120
H3			3152	3075		3049		3054		3046
KomCCT		3092	3031		3013		3009	
Kom5			3057	2997		2996		2999		2987

H3-Kom5		95	78		53		55		59
H3-KomCCT		35	34		17		10	

I think, thats the point!
The longer the thinking time (or faster the hardware), the closer all results and Elos will get (because the draw rate of all played games raises with a higher chess-quality-level). So it seems so, that Houdini 3 (worlds number 1) plays weaker and the weaker engines play stronger (with longer thinking-time). But this is a statistical illusion. Because the results are getting closer together, because of the higher draw-rate. But in no list or test (with enough games!) Komodo or Stockfish are stronger than Houdini 3 (only closer behind! Thats a big difference!).

Stefan

[pohl4711]

It is a really pity for me to see these illegal engines and it is an even bigger pity to see people simply don't care and give these engine visibility.....I think it is a way of life and blaming this can only, at best, delay, but not win. It is like tilting at windmills.

Sorry, but this is not true, Marco. An engine is illegal when a court judgment anywhere on the planet says so. Until such a court jugdment, it is only your private opinion.
You cannot demand to remove an engine out of my testwork, because of your private opinion. When I would do so, each engine-programmer could demand to boycott any competitor. Then all testwork would come to an end.
If there will be a court judgment against Amitis, then please tell me and I will remove Amitis from the LS-ratinglist immediately.

(Sorry for my bad english)

Stefan

[velmarin]

It is a really pity for me to see these illegal engines and it is an even bigger pity to see people simply don't care and give these engine visibility.....I think it is a way of life and blaming this can only, at best, delay, but not win. It is like tilting at windmills.

Sorry, but this is not true, Marco. An engine is illegal when a court judgment anywhere on the planet says so. Until such a court jugdment, it is only your private opinion.
You cannot demand to remove an engine out of my testwork, because of your private opinion. When I would do so, each engine-programmer could demand to boycott any competitor. Then all testwork would come to an end.
If there will be a court judgment against Amitis, then please tell me and I will remove Amitis from the LS-ratinglist immediately.

(Sorry for my bad english)

Stefan

Stefan,
You know this is not a matter of judgment,
open source project stuck in a judicial proceeding?.
That would cost money, at a minimum.

You have a list, and logical, wants to be valued.
Think if someone copies your data, puts it into a nice web,
and gives you advertising that you can not do, would you like?

Over time you will see the futility of it have had Amitis on your list, when you see for yourself that there is not a single change of the authors, only compilations directly from Github Stockfish.



"No hay mayor ciego que aquel que no quiere ver"

[pohl4711]

You have a list, and logical, wants to be valued.
Think if someone copies your data, puts it into a nice web..

That happens on chess2U Forum. A copy of my LS-ratinglist in my LS-thread. Not from me, because I prefer that people visit my website.
Not so nice, but thats Internet...So what...If you put open data in the internet, someone will use or copy it. If you dont want that to happen, dont publish data for free and open in the internet.

Best - Stefan.

[mcostalba]

It is a really pity for me to see these illegal engines and it is an even bigger pity to see people simply don't care and give these engine visibility.....I think it is a way of life and blaming this can only, at best, delay, but not win. It is like tilting at windmills.

Sorry, but this is not true, Marco. An engine is illegal when a court judgment anywhere on the planet says so. Until such a court jugdment, it is only your private opinion.
You cannot demand to remove an engine out of my testwork, because of your private opinion. When I would do so, each engine-programmer could demand to boycott any competitor. Then all testwork would come to an end.
If there will be a court judgment against Amitis, then please tell me and I will remove Amitis from the LS-ratinglist immediately.

(Sorry for my bad english)

Stefan

I don't think I have asked you anything.

I have stated that a program derived from a GPL one, released without source is illegal. I don't need any court to state this: it is written in the GPL license.

Eventually the court is needed to verify if a given engine, like Amitis in this case, it is actually derived from SF. But this is a different thing.

I have no prof that Amitis is a SF clone, so I asked you nothing. I just say that a SF clone released without sources is illegal.

Actually I wrote a more wider opinion. I wrote that even if an engine (not a particular one) is found derived from SF and closed sources and so illegal, I sadly believe that this will not be of interest for many people here that will continue to use that engine anyway.

[Red Hood]

It is a really pity for me to see these illegal engines and it is an even bigger pity to see people simply don't care and give these engine visibility.....I think it is a way of life and blaming this can only, at best, delay, but not win. It is like tilting at windmills.

Sorry, but this is not true, Marco. An engine is illegal when a court judgment anywhere on the planet says so. Until such a court jugdment, it is only your private opinion.
You cannot demand to remove an engine out of my testwork, because of your private opinion. When I would do so, each engine-programmer could demand to boycott any competitor. Then all testwork would come to an end.
If there will be a court judgment against Amitis, then please tell me and I will remove Amitis from the LS-ratinglist immediately.

(Sorry for my bad english)

Stefan

I don't think I have asked you anything.

I have stated that a program derived from a GPL one, released without source is illegal. I don't need any court to state this: it is written in the GPL license.

Eventually the court is needed to verify if a given engine, like Amitis in this case, it is actually derived from SF. But this is a different thing.

I have no prof that Amitis is a SF clone, so I asked you nothing. I just say that a SF clone released without sources is illegal.

Actually I wrote a more wider opinion. I wrote that even if an engine (not a particular one) is found derived from SF and closed sources and so illegal, I sadly believe that this will not be of interest for many people here that will continue to use that engine anyway.

I know this might open Pandora's box but i have to say it. I think Marco has to make SF license more stricter when it come's to it's source code. Marco and the rest of the developers from now on designated as "team". Team has already gave this engine to the world for free and i don't think is fair that some people use that to gain money. It's not ethical but sadly people today don't give a damn about ethics. SF is a leap of faith. Something that show's what people can do when they work together for the greater good. Bible says that God separated people in to nations and gave them their language, because people wanted to build a structure to reach God! That's right, something as tall as the sky! Not a small feat to do!

That show's what happens when people work together!

[pohl4711]

I don't think I have asked you anything.

I have stated that a program derived from a GPL one, released without source is illegal. I don't need any court to state this: it is written in the GPL license.

Eventually the court is needed to verify if a given engine, like Amitis in this case, it is actually derived from SF. But this is a different thing.

I have no prof that Amitis is a SF clone, so I asked you nothing. I just say that a SF clone released without sources is illegal.

Actually I wrote a more wider opinion. I wrote that even if an engine (not a particular one) is found derived from SF and closed sources and so illegal, I sadly believe that this will not be of interest for many people here that will continue to use that engine anyway.

Okay. But your post was in the thread of my ratinglist, so another comment from me (sorry):
I really hope, you and everybody else here in this thread, try to understand, that it is not so easy for a tester to find a proper and fair solution for testing engines, especially after the OpenSource Ippolit-code and the derivatives of this code (Ivanhoe, PanChess (closed source), Bouquet (closed source), Strelka (closed source), Saros (closed source), Virtuvius (closed source and commercial), Houdini (closed source and commercial)).
Nobody ever asked me, to ban Houdini or Vitruvius...
And what about the endless Rybka/Fruit-discussion? Ban Rybka, too? No ratinglist in the Internet banned Rybka and Houdini.
Where is the line between good and evil here?
I cant say it and because of that, I decided to test all engines, which are strong enough and downloadable/buyable for all.
Sure, it is no perfect solution, but which one would be better? Only test Fruit, Fritz, Shredder, Hiarcs, Junior, Komodo and Stockfish? I doubt that.

Best regards from a huge Stockfish-fan - Stefan

P.S: Result of Stockfish 131008 (hopefully) tomorrow on my website. At the moment it looks like there will be a nice progress again.

[velmarin]

Ippolit is Public Domain, Vitruvius or Houdini, the debate be commercial from public domain code is more ethical question of question legality, in my view.

You do a great service to the community,
only that the issue comes from other controversies Amitis in forum Immortal, causing splits in important members, but that's another topic.

As you say Stockfish is a flagship project right now, so it must have support from around the world.
Other projects how Bouquet, Panchess, Critter, "all projects" are also necessary, give a point of diversity, rivalry, are needed in the lists, tournaments, ect.

Talk a little does not mean you have all the support.

[Lyudmil Tsvetkov]

I don't think I have asked you anything.

I have stated that a program derived from a GPL one, released without source is illegal. I don't need any court to state this: it is written in the GPL license.

Eventually the court is needed to verify if a given engine, like Amitis in this case, it is actually derived from SF. But this is a different thing.

I have no prof that Amitis is a SF clone, so I asked you nothing. I just say that a SF clone released without sources is illegal.

Actually I wrote a more wider opinion. I wrote that even if an engine (not a particular one) is found derived from SF and closed sources and so illegal, I sadly believe that this will not be of interest for many people here that will continue to use that engine anyway.

Okay. But your post was in the thread of my ratinglist, so another comment from me (sorry):
I really hope, you and everybody else here in this thread, try to understand, that it is not so easy for a tester to find a proper and fair solution for testing engines, especially after the OpenSource Ippolit-code and the derivatives of this code (Ivanhoe, PanChess (closed source), Bouquet (closed source), Strelka (closed source), Saros (closed source), Virtuvius (closed source and commercial), Houdini (closed source and commercial)).
Nobody ever asked me, to ban Houdini or Vitruvius...
And what about the endless Rybka/Fruit-discussion? Ban Rybka, too? No ratinglist in the Internet banned Rybka and Houdini.
Where is the line between good and evil here?
I cant say it and because of that, I decided to test all engines, which are strong enough and downloadable/buyable for all.
Sure, it is no perfect solution, but which one would be better? Only test Fruit, Fritz, Shredder, Hiarcs, Junior, Komodo and Stockfish? I doubt that.

Best regards from a huge Stockfish-fan - Stefan

P.S: Result of Stockfish 131008 (hopefully) tomorrow on my website. At the moment it looks like there will be a nice progress again.

Hi Stefan.
Your site is the best for checking which engine is a significant contribution and which one a remake.
You only have to look at the numbers.
And of course, that is very useful.