Lee Sedol vs. AlphaGo [link to live feed]

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by diep »

George Tsavdaris wrote:
matthewlai wrote: Yes, that's exactly what I meant. Handwritten evaluation functions in chess are good enough that switching to a neural network (slightly better outputs but much slower) is a net negative.

I really hope someone skilled in ML steps in and take it as far as it can go and see what happens.

Even if the AlphaGo/Giraffe approach doesn't work, there are still many possible machine learning approaches that can be explored.

The situation is easy as i see it:

In GO the huge branching factor and number of moves for the average game is so huge, that no computer or human can base or improve his play, improving that. I.e improving search in GO does not do anything(compared to evaluation function).
So evaluation is THE MOST crucial factor by far!
oh comeon. Assuming you have a reasonable evaluation search is everything in Go of course.

From the 361 moves you can HARD FORWARD PRUNE kind of 300 moves already.

A few moves later in the game you can almost hard forward prune 330 moves out of the 340 legal moves or so. In chess this is totally impossible.

So search is EVERYTHING, however under 2 conditions.
Condition 1 is a reasonable evaluation function yet condition 2 might surprise most.

You *do* need to cover a specific search space and that means simply a lot of nodes. Now as most go programmers hardly know how to program compared to the chessprogrammers their programs are ugly slow. Usually using neural network for example that's pathetic slow.

So the easy solution then is to throw big hardware at it.

Once you do have big hardware - something most can afford now - say a 48 core AMD machine gets you far already and its peanuts to pick it up.

Yet in that case search is everything.

There is however little money to make (from an absolute viewpoint) with computer-go so no really good programmers ever got into that job of writing software for it let alone brilliant game tree search guys.

That's why it took so long for something to develop there.

If as much money and especially brilliant game tree searchers would've thrown into the game of go quite possibly the strongest go program now wouldn't be using a neural net and it would not be using monte carlo yet a clever selective search.

The advantage of that you basically see in game 4 very well. AlphaGo suffers from major horizon effect playing moves that just lost point after point and contributing to its loss.

Note that i wonder what they offered Lee Sedol, who isn't the strongest go player right now - he didn't even win a single big tournament past 4 years - that he played so bad first 3 games.

He made huge blunders there that lost him obviously game 2.

Such blunders are equal to me blundering away a piece.

Now i'm approaching IM level (having 2 IM norms) yet only if i play really little and under big time trouble i start blundering.

So alphago in chess rating would be roughly 2250 now or so.

Very few go players are professional go players and basically at age 11 you get chosen as a go player to get a professional player (usually).

With so few pro's there, how high are the odds that a talented kid of 11 years old also becomes a very strong professional player - or does he just get another IM elo 2360 who blunders a lot?

Draw your own conclusions on what Lee Sedol did do - what Bob described years ago as the computer shock (if i remember well) - namely the first time you play a very strong chessprogram - then pro's suddenly get faced with how tactical bad they play.

In chess what we see is that basically all old world top players who blunder lost like 100 elopoints or more and that a new generation got there which plays tactical far superior over any previous top player (Kasparov and Karpov and Kramnik maybe excepted). Anand managed to improve himself tactical quite a lot past 20 years.
He's actually around 2850 elo - just cuz there is nothing at stake at tournaments, candidates tournament excepted, he takes it too easy too many games and draws those.

That's why his elo is a lot lower.

Yet he'll win this candidates tournament quite possibly as i predicted months ago.

If we compare that with the state of the art in go - then there is just a 100 pro players or so, majority whom got picked to get a pro player at age of 11.

The strongest go player is an 18 year old Chinese player.
In 2015 at the age of 17 he won many important tournaments.

Would that be possible in chess?

I am no big expert in go (probably to most posters here i am as of course semi-professional chess versus semi-professional go you need the same skills to do well in those games) i can't rate the chinese yet i'm pretty convinced he can't be bribed to lose 3 games in a row.

Not saying that this is what Lee Sedol did do - yet let's face it - he didn't win a single big tournament past 4 years that Lee Sedol if i look at his wikipedia.

I guess Google (Alphabet) didn't have enough cash to play worlds strongest go player.

In 2007 i predicted exactly this scenario when we were standing in the bunker of Amsterdam.

The chinese Go delegation, one of the professional go players nearly wanted to kill me for that. Fire came out of her eyes.

"THAT WILL NEVER HAPPEN" she said. "A professional go player will never lose from a computer".

Where i answerred her that the real weakness of humankind is money.

What we saw from search viewpoint from alphago in game 2 and game 4 wasn't very impressive simply.

Big progress maybe for those who didn't follow computer go lately. Let's see whether Google still has the guts to challenge worlds number 1.

We know what IBM did do when kasparov played a few elo 2000 games against it. Or how Salov called it: "just a mediocre GM playing Deep Blue as kasparov without his openings isn't a very good player".

Where i disagree with that statement from Salov - it says enough about the quality of those games.

Yet it's really interesting to see what Google will do now...
matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 4:48 am
Location: London, UK

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by matthewlai »

diep wrote: oh comeon. Assuming you have a reasonable evaluation search is everything in Go of course.

From the 361 moves you can HARD FORWARD PRUNE kind of 300 moves already.
There is just a tiny little detail missing from this grand plan of yours I think... how exactly do you decide which 300 moves to HARD FORWARD PRUNE?

Randomly? LMR? How are you ordering them?

People use MCTS BECAUSE there is no reasonable evaluation function. AlphaGo changed that by actually using machine learning to build an evaluation function.

Yes, if you can hand-write a magical unicorn evaluation function that works just as well, you can probably beat the neural network.

If you can write a magical unicorn oracle function that can give you the optimal move for Go for any given position quickly, you can beat AlphaGo (and any other engine that may come up in the future) easily, too.

If you can write an oracle for the stock market, you can become incredibly rich overnight, too!

When are you going to start writing them?

I personally would not call a program crap just because, you know, if a few function that no one knows how to write existed, they would be able to beat it easily. But that's just me.
The strongest go player is an 18 year old Chinese player.
In 2015 at the age of 17 he won many important tournaments.

Would that be possible in chess?
Someone like Magnus Carlsen? According to Wikipedia he got to #1 on FIDE list when he was 19. Why is that so hard to believe?
Last edited by matthewlai on Mon Mar 14, 2016 3:01 am, edited 1 time in total.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
chessico
Posts: 58
Joined: Mon Jan 28, 2013 5:27 pm
Location: Germany

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by chessico »

It is amazing how full of yourself you are. Who are you to denigrate people like Lee Sedol, Kasparov, other programmers, everybody else? Stupid conspiracies in every single post of yours, go see a doctor, finally. please.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by diep »

matthewlai wrote:
diep wrote: oh comeon. Assuming you have a reasonable evaluation search is everything in Go of course.

From the 361 moves you can HARD FORWARD PRUNE kind of 300 moves already.
There is just a tiny little detail missing from this grand plan of yours I think... how exactly do you decide which 300 moves to HARD FORWARD PRUNE?

Randomly? LMR? How are you ordering them?
Until recently all of the top go programs hard forward pruned already in the root.

It's really true.

They reached dan level with that.

Yet this is why i say search is everything. You CAN make a very great selective search in go - yet it's a LOT of work from some genius guys in search. And they didn't have those in computer-go simply. Period.

Try that with a chessprogram!

They do not have world titles in go for professional players (only for amateurs if i understand well). So you need to compare world number 1 in go with the guy winning the world title by playing a match in chess - and not some fide knockout tournament like start this century when a faster time control just had been introduced - even the candidates is a considerable slower time control now.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by diep »

chessico wrote:It is amazing how full of yourself you are. Who are you to denigrate people like Lee Sedol, Kasparov, other programmers, everybody else? Stupid conspiracies in every single post of yours, go see a doctor, finally. please.
Who are you?

I really believe Lee Seedol at his CURRENT level is comparable to about an elo 2590 GM.

He blundered bigtime game 2. Commentator 9-dan pro (just like Lee Sedol 9 dan) said about that: "he (Lee Sedol) played 2 slack moves in the middlegame that lost him the game".

You can look that comments up. Around minute 35 here: https://www.youtube.com/watch?v=yCALyQRN3hw

You do believe a 9-dan pro calling Lee Sedol playing slack moves?

Yes i do believe the chess world top has WAY STRONGER players than the go world top. Selecting a few kids age 11 to become pro players...

And not too many either as there is a limited number of spots there...

Even the British old-schoolboys network is less outdated than that...
mar
Posts: 2555
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by mar »

chessico wrote:It is amazing how full of yourself you are. Who are you to denigrate people like Lee Sedol, Kasparov, other programmers, everybody else? Stupid conspiracies in every single post of yours, go see a doctor, finally. please.
While I disagree with most of the points, let's talk DB rematch for a while:
the games simply weren't impressive, a world champion wouldn't lose to a buggy program that blunders...
Have you analyzed the games? Have you seen the logs?
Do you know that DB prototype lost to Fritz running on single core P90 in WCCC?
Someone desperately needed success back in 97...
What's ridiculous is people who believe that DB would beat even average engines today, those should actually visit a doctor :roll:
K or SF on single core would destroy DB 100-0, I would bet my fortune on that.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by diep »

as for search please note i already dropped 10 years ago a proposal for that publicly.

You can do some work to make a function that assigns a reduction to each move. or rather than reduction i'd call it number of units that a move costs.

let's fictive use fractions. So the best few moves have fraction 1 unit.
Then the worst moves have fraction 20 units (cost is 20 plies). That's the moves you would want to hard forward prune.

In case you need to solve a local search most likely you adjust to that and so on.

When a move fails high that has a lot of units worth you first research it at say n / 2.
So a move with fractional plies that costs say 20 units (plies) you first research with cost 20/2 = 10 units and so on until you really fail high for it.

As horizon effects is a big problem in such searches.

A big reason why they couldn't make this some years ago is because the nps they got was just too little - whereas a kind of forced sequence in go is easily 30 plies (and that's not counting ladders yet) at top level.

So you really need quite a higher nps than they are getting at a PC with handful of cores.

One way to solve that is by programming that code better in C/C++ and speeding it up factor 3+ or so and adding more cores.

For searching with fractional plies, you really need to get quite a huge search depth for it to work well.

Then there is all kinds of tricks required to speedup the search a lot as the function to select the moves will be very slow of course.

The whole system is really a lot of work to get to work.

Getting rid of the neural network is not so easy. The thing they got "for free" kind of is that it got automatically tuned of course - whereas if you replace it with something else you need to also automatic parameter tune THOSE parameters in evaluation.

My go program didn't do that for example.

BIg big disadvantage...

So the most importnt thing is that in the above type of fractional search - you HAVE to get a search depth of 30 plies. If you aren't getting that then the fractional search depth idea of NOT forward pruning anything just reducing bigtime what you previously hard forward pruned - that whole concept will not work at all.

When you do implement it correctly AND get that 30 plies it'll work genius.

Note i got from root already easily 15 plies in 1999 or so and that was without fractional search depth, just with nullmove - so it's not really a problem.

with such huge fractional search depths of course nullmove gives a lot less...

So there is really a lot to tune and toy with to get it to work very well.

All together it's a big work - yet i'm sure no one ever tried it as only todays hardware and a good programmer would be able to do it. Yet they blindfolded follow what others use there of course :)

p.s. the hard forward pruning in root also might explain why most go programming authors are relative strong playing go players. if you forward prune a good move in root in a hard manner - you directly lose of course because of a 1 ply blunder...
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by diep »

mar wrote:
chessico wrote:It is amazing how full of yourself you are. Who are you to denigrate people like Lee Sedol, Kasparov, other programmers, everybody else? Stupid conspiracies in every single post of yours, go see a doctor, finally. please.
While I disagree with most of the points, let's talk DB rematch for a while:
the games simply weren't impressive, a world champion wouldn't lose to a buggy program that blunders...
Have you analyzed the games? Have you seen the logs?
Do you know that DB prototype lost to Fritz running on single core P90 in WCCC?
Someone desperately needed success back in 97...
What's ridiculous is people who believe that DB would beat even average engines today, those should actually visit a doctor :roll:
K or SF on single core would destroy DB 100-0, I would bet my fortune on that.
Well you can also point everyone to the analysis of Yasser Seirawan in ICGA journal june 1997.

The number of question marks and dubious signs behind Kasparov's moves, if you add them up and then calculate the average over 6 games, then compare it to the entire CAREER of kasparov as a chessplayer you can directly prove that this is like 400-600 elo below his other games.

In 1 game in theory it would be possible yet over a serie of 6 games that's entirely impossible.

Also if we then see that directly after losing the match, Kasparov is praising deep blue and complimenting IBM. Then backstage he hears that he doesn't get a rematch -something back then very normal in chess that the world champion always gets a rematch - he runs back to the press and camera's and smoke and fire gets out of his eyes as he starts to realize what he has done...

Let's hope alphago doesn't have the same faith like deep blue there. namely directly getting canned.

We've seen 12+ years of work now into the go program that now is called alphago. Would be a big waste of time to stop it now just because that's convenient to google - whereas we all know that ONE DAY computer is gonna be superior anyway - so why not play a serious match against a guy who won a series of tournaments - the 18 year old number 1 chinese player.

This Korean guy is someone from the history of the game. Someone who was strong 4 years ago. But most important: Korean.

And let's face it what he did do in game 1 and 3. He first gives alphago a strategically totally won position and then by means of tactics tries to advance. Of course if alphago just tactically keeps it dry - then alphago wins that game. Which is what happened.

to quote Michael Redmond, he called Lee Sedol playing 2 slack moves.
Also within a few seconds Michael recognized in game 3 that Sedol 'blew the chances against the center'.

Move 5 of white (alphago) Michael basically calls something you play at amateur level and which at professional level you directly refute by attacking it. As that would only be possible with black to play not with white.

Sedol didn't attack it. he just simly gave first a 70 point lead to alphago in the game and after that started playing his game. That might work against me as total amateur. Hah even giving me 9 moves ahead might win for Sedol against me as i don't know very well the forced sequence that is there you can play to win the game when 9 moves ahead.

Yet factual spoken the impression is not very positive about Sedol playing Alphago.

That's like having Bobby Fischer in 1997 play Deep Blue rather than Kasparov, who back then valued the dollar above the ruble.
User avatar
George Tsavdaris
Posts: 1627
Joined: Thu Mar 09, 2006 12:35 pm

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by George Tsavdaris »

diep wrote:
George Tsavdaris wrote: The situation is easy as i see it:

In GO the huge branching factor and number of moves for the average game is so huge, that no computer or human can base or improve his play, improving that. I.e improving search in GO does not do anything(compared to evaluation function).
So evaluation is THE MOST crucial factor by far!
oh comeon. Assuming you have a reasonable evaluation search is everything in Go of course.

From the 361 moves you can HARD FORWARD PRUNE kind of 300 moves already.
Well since you decided to talk not for this world but only for an imaginary one, why not take it further?
Let's assume having an evaluation function that prunes all moves every time and always choses the best move. So search is irrelevant.



**Oh and wait. If you are able in the first place to prune 300 out of 361 moves then your evaluation is not reasonably good, it's extra super good! So your super play will still be mainly because of evaluation function and not search. :lol:
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
User avatar
George Tsavdaris
Posts: 1627
Joined: Thu Mar 09, 2006 12:35 pm

Re: Lee Sedol vs. AlphaGo [link to live feed]

Post by George Tsavdaris »

diep wrote:
There is however little money to make (from an absolute viewpoint) with computer-go so no really good programmers ever got into that job of writing software for it let alone brilliant game tree search guys.
Yes this is logical as eastern Asia and especially China(where GO is something more than a national game, it's a culture for them), are low populated. :D

Watching the games i noted a comment from one(9d pro) of the professional GO players. He was asked whether he would buy AlphaGO if it will become available for some hundered dollars. He said he would do it easily(and he looked anxious about it, i.e looking forward for this to happen) for some thousands of dollars!
In Chinese and Japan GO forums of common hobbyist players as also of several amateurs with dan players, there was also an anxiety of whether AlphaGO will be available for obtaining it(for home PCs).

So taking in mind the high population and interest for GO in eastern Asia, it's very logical to assume someone that will create a PC program on par with strength with a >1 dan pro for example, he would make some very good money.
I tend to think that AlphaGO and its Nature's paper showed the way and we are very close from hearing somethng like that.

If as much money and especially brilliant game tree searchers would've thrown into the game of go quite possibly the strongest go program now wouldn't be using a neural net and it would not be using monte carlo yet a clever selective search.
Problem is that all these are not just COMPLETE speculations, they are wrong speculations. :D
The advantage of that you basically see in game 4 very well. AlphaGo suffers from major horizon effect playing moves that just lost point after point and contributing to its loss.
Of course good search is important, but this is completely irrelevant from the fact that evaluaiton is the most crucial part of GO.

Note that i wonder what they offered Lee Sedol, who isn't the strongest go player right now - he didn't even win a single big tournament past 4 years - that he played so bad first 3 games.
He won in 2012 the Samsung Cup defeating in the final one of the strongest GO players Gu Li. He won also 2 major events Myeongin in 2016 and 2012 also.

Also the statement that he played so bad in first 3 games is yours as most professional GO players seem to think COMPLETELY OTHERWISE!
They say he played maybe a little conservatively in the 1st game and making 2-3 small mistakes but that's all. In the second game they said he played a great solid game with only a minor inaccuracy but not something so big. And in the 3rd he played in his classic style but was just not able to outplay the computer.

He made huge blunders there that lost him obviously game 2.

Such blunders are equal to me blundering away a piece.

What huge blunders specifically are you talking about?
Because i have seen the game commented from 3 different pros(the 2 9 dan) and didn't see they said anything like this.

If we compare that with the state of the art in go - then there is just a 100 pro players or so, majority whom got picked to get a pro player at age of 11.

The strongest go player is an 18 year old Chinese player.
In 2015 at the age of 17 he won many important tournaments.
You seem to degrade GO world in such a bad manner. And this is of course ridiculous.
I guess this small age wonder, in GO players is because GO for "them", the Asians, is a culture, national matter and ther are taught Go in a very serious way from a very small age and they play it all the time and not just for fun.
Would that be possible in chess?
Carlsen. :D
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....