recent article on alphazero ... 12/11/2017 ...

[jhellis3]

What color is the sky in your world?

[hgm]

Opening book is a part of the engine, even more for Cerebellum/SF.

Doesn't really sound like you were talking about the TCEC book here...

[Milos]

Opening book is a part of the engine, even more for Cerebellum/SF.

Doesn't really sound like you were talking about the TCEC book here...

You talk about no-book, there is no no-book tournament, so as usual you are just making hot air. Bla, bla.
Try coming with a reasonable argument for the fair comparison theory you are trying to present. So far, you didn't manage to make a single one. Just empty excuses that even 10 year-old kid would find ridiculous.

[shrapnel]

What color is the sky in your world?

Just ignore him, Ellis.
Looks like he is off his Meds.
And he accuses others of Trolling.

[hgm]

You talk about no-book, there is no no-book tournament, so as usual you are just making hot air. Bla, bla.

No, that is apparently just what you understood. But there is a large gap between what you understand, and what is actually the case.

I was just pointing out that according to your statement that books should be considered an integral part of engines, and in particular the Cerebellum book a integral part of Stockfish, competitions that would separate engines from their OWN books would no longer qualify as 'engine competitions'.

How you came to think that this could mean that tournaments like TCEC could be played entirely without book is, well, let's call that 'strange'. It is common knowledge that hardly any engine nowadays intentionally randomizes its moves, so that you typically get very similar and certainly not independent games when you let them play their own (searched) moves from the same starting position. One would expect you to know that. But perhaps your innate meanness got the better of you, and compelled you to nevertheless make such a silly assumption to provide you a opportunity to display it.

Of course when dealing with engines that randomize, opening books are not needed for creating game diversity.

[peter]

Hi!

Not to go on argueing about the match already done by Google, for which of course we all have to be grateful anyhow, for sure it was better than no match at all, and it will have costed some money and manpower still, even if the biggest efforts maybe will have been done for Go already.

Wouldn't you also just be interested in seeing another match against an engine with book and own time management at least?

Not to talk about some Freestyle- event maybe?

Or just some single quick corr.- games, TC could be limited to a few weeks e.g., just to see some more great moves of A0 but maybe some great ones of human supported high end chess too?

What we saw till now, were about 5 beautiful moves out of 10 arbitrarily chosen games (suspect that these might have been already the best ones at all to show is understandable, isn't it? Why not give at least one .pgn to download of all of the 100?) being made possible by a blundering SF.

As much as there has been written and YouTubed about these few great moves on and on:
would high class corr.- players have made them, nobody would have even showed them in any forum at all.
Any other one good corr.- player at once would have said: yes, well done, but the bigger thanks have to go to the opponent, letting these moves arise and not finding the best answers.

Coming from A0 they were talk of the year, called being from an other planet and things like that.

All of these moves were to be "found" and evaluated correctly by any good engine we have with good hardware- time, eventually faster with some clever settings or some Forward- Backward.
Some of them were called "human"- like, that's the old wish of human beings to find human traits in any clever move, isnt't it?
So what?

No matter how fair or unfair all of these other match- settings could be called again too of course and not willing to make you discuss that point on and on,

but wouldn't you just be interested in seeing play A0 against human opening theory once in a while too?

What I fear is just, Google doesn't have any reason to try this (publicly), if there isn't any demand for it from any side of relevance for them.

Just maybe, but I'd rather say probably, they wouldn't get any better success then they already had, would they

[hgm]

I would not see any scientific merit in such a match. Chess, or the way that humans play it, doesn't have any specific scientific interest. The interesting thing is that state-of-the-art performance could be reached providing the system no other information than the rules. With a manageable amount of analysis (i.e. number of training games).

The conditions of the test that is done were well described. (Fixed 1 min/move, 1GB hash, no book, 64 threads.) The resulting measured Elo difference was published. If you want to know how much the Elo difference would have been if Stockfish had been playing with some sort of Book, on 32 cores, with 256GB hash, at 60 moves/hr, or using a newer version.... You don't need AlphaZero for that. You can just run a number of Stockfish gauntlets, using one Stockfish setting or another, and determine exactly how many Elo stronger it would have been under those altered coditions, and how that compared to the published Elo of AlphaZero. The conditions of the match had no other relevance than defining a reproducible commonly available yardstick to measure the performance.

Scientifically, it doesn't make the slightest difference whether AlphaZero would end up 50 Elo stronger or 50 Elo weaker as the opponent in the final test match. No one claims the opponent or AlphaZero play perfect Chess, so it is always understood that stronger opponents would be possible, given more time (odds), faster hardware, more hardware (large cluster). The point is not whether AlphaZero finally saturated at 3450 or 3550. The point is that it went from (sub?)zero to >2500 just by itself.

[peter]

The point is not whether AlphaZero finally saturated at 3450 or 3550. The point is that it went from (sub?)zero to >2500 just by itself.

The point is, that A0 had a match- performance against an engine without its own time- management (which cripples its ability in varying the opening moves even more) playing maybe only 3 different opening lines.
Not more then these we have seen in the 10 most beautiful games, at least I'd like to know, if there was much more in the 100 at all.

The point is, that the question remains, if the big "learning success" has happened in anything else then this, to succeed against the crippled opponent learning to beat it in its crippled openings only.
I don't mind, if the 3500 Celo would be 3300 only in a rematch with reasonable TC and book, I doubt if the performance would be nearly as high as it was in the first one biased trial.

And as for good scientific practice, publishers should ask such questions in their summary by themselves, not let the audience find out, what kind of questions remained and should be answered by further tests.
Period.

As long as their haven't at least all 100 games been published, that's not a scientific paper, not even a prepriint to me at all.
No journal of any impact factor in mathematics, IT, not even in AI would publish it

[hgm]

Well, it turns out there certainly is one error of judgement the AlphaZero team made: They considered the TCEC a serious Chess championship, and assumed the winner would be a worthy opponent. They did't realize it was just a crippled wreck, because TCEC is just a joke and no test at all for the Chess quality of a potential search program. If we shoudl believe you, that is.

But from a scientific point of view that is totally immaterial: it is not a secret what they played against and by how much they beat it, and that is all that matters.

Most of the arguments are pure nonsense, of course. As was already remarked, AlphaZero probably suffered more from the fixed TC as Stockfish, because training for superior time management is orders of magnitude simpler than training for good Chess, and would have certainly been in the capabilities of the NN. Likewise, if Books had been used, AlphaZero would probably have had a much better book than Stockfish. These are all just wishful thoughts, by people that are trying to exploit the total absence of evidence one way or another by making totally unsubstantiated claims that the issue would make a huge difference in the direction of the conclusion that they happen to favor.

And as for good scientific practice, publishers should ask such questions in their summary by themselves, not let the audience find out, what kind of questions remained and should be answered by further tests.
Period.

As long as their haven't at least all 100 games been published, that's not a scientific paper, not even a prepriint to me at all.
No journal of any impact factor in mathematics, IT, not even in AI would publish it

Are you a scientist? I would guess not. You don't seem to be aware of the peer-review system. How many papers did you publish in Phys. Rev. Letters, Sciece or Nature?

Your claim about the games is prepostrous; they are totally immaterial and of no scientific value at all.

But we will see what evetually gets published about AlphaZero in Nature.

[syzygy]

The point is not whether AlphaZero finally saturated at 3450 or 3550. The point is that it went from (sub?)zero to >2500 just by itself.

The point is, that A0 had a match- performance against an engine without its own time- management (which cripples its ability in varying the opening moves even more) maybe only 3 different opening lines.

Only if you care whether this particular version of AlphaZero is or is not stronger than the "best" version of SF running on the "best" hardware with the "best" settings.

The supposed "crippling" is totally irrelevant if you care about AlphaZero going from nothing to >2500 all by itself. And this is where the scientific breakthrough lies (well, perhaps not a "breakthrough" after doing the same in Go).