Unanswered question from Engine Origins forum

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
Eelco de Groot
Posts: 4565
Joined: Sun Mar 12, 2006 2:40 am
Full name:   

Unanswered question from Engine Origins forum

Post by Eelco de Groot »

http://www.talkchess.com/forum/viewtopi ... 79&t=44864

Harm-Geert Muller asked this question on the Engine Origins subforum but so far no one has been able or willing to answer it. I thought it was an interesting question, I don't recall myself that it has been answered in any depth before, on this forum. It is not too controversial I think that it should not be here, unless you want to argue that there may be material propietary to Vas Rajlich that should not be discussed here. I don't think that it would be a problem with Vas discussing Ippolit right now but he could always pass us a message about that. As far as I know he is also still a member here.

Maybe, and it is likely, that there is already some material from Vas himself directly related to the question in the hidden Rybka forum, but I did not have access at the time it could have been discussed there. Possibly, as was the case with the question of the contribution to computer chess techniques from Fruit, it is hard to answer HGM's question.

Eelco
hgm wrote:Perhaps a very ignorant question, but what exactly is the contribution of Ippolit to computer Chess in terms of new ideas? (I understand of course that it is a cloner's heaven to have an instant 3000+ engine to start from, but the question is more what you would find that could be helpful if you already had a 2800-Elo engine, if you wanted to boost its Elo, rather than just throw away your own code and use Ippolit in stead.)

Some time ago I had a glance at the Ippolit code (mainly because I wanted to see how a top engine exploits the extra possibilities bitboards offer over mailbox), and what I did notice about its search is that it counts in half-plies, and that its QS (i.e. the possibility to stand pat) already starts at d = 4 ply, using some (marginally) depth-dependent window tuning for deciding which moves to search in the first 4 ply of this QS. And that it used a table to remember the maximum gain effected by each move, used to decide if such a move should be searched in early QS.

Are these Ippolit-specific tricks, and do they contribute a lot to its playing strength?
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: Unanswered question from Engine Origins forum

Post by kranium »

hgm wrote:Perhaps a very ignorant question, but what exactly is the contribution of Ippolit to computer Chess in terms of new ideas?
Hi Harm-

Not at all ignorant!

I strongly suggest you contact the authors at http://ippolit.wikispaces.com/...
they are quite knowledgeable and friendly as well.

I'm sure they would be willing to explain in detail the many advanced techniques responsible for Ippolit's great strength..

(if not, just contact me via pm, I'll be happy to help as best I can...)

I'm a big fan of Micro-Max, so after they help you in this regard, I do really hope to see a see a new improved/stronger Micro-Max in the near future!

Best of luck-
Norm

PS - here's a related thread:
http://www.talkchess.com/forum/viewtopi ... ee57ccecdc

only problem is: nobody participating in the discussion has familiarity with the source code...
(except maybe Larry Kaufman, who recognizes PSTs that seem similar to those in Rybka 3)
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Unanswered question from Engine Origins forum

Post by Don »

Eelco de Groot wrote:http://www.talkchess.com/forum/viewtopi ... 79&t=44864

Harm-Geert Muller asked this question on the Engine Origins subforum but so far no one has been able or willing to answer it. I thought it was an interesting question, I don't recall myself that it has been answered in any depth before, on this forum. It is not too controversial I think that it should not be here, unless you want to argue that there may be material propietary to Vas Rajlich that should not be discussed here. I don't think that it would be a problem with Vas discussing Ippolit right now but he could always pass us a message about that. As far as I know he is also still a member here.

Maybe, and it is likely, that there is already some material from Vas himself directly related to the question in the hidden Rybka forum, but I did not have access at the time it could have been discussed there. Possibly, as was the case with the question of the contribution to computer chess techniques from Fruit, it is hard to answer HGM's question.

Eelco
hgm wrote:Perhaps a very ignorant question, but what exactly is the contribution of Ippolit to computer Chess in terms of new ideas? (I understand of course that it is a cloner's heaven to have an instant 3000+ engine to start from, but the question is more what you would find that could be helpful if you already had a 2800-Elo engine, if you wanted to boost its Elo, rather than just throw away your own code and use Ippolit in stead.)

Some time ago I had a glance at the Ippolit code (mainly because I wanted to see how a top engine exploits the extra possibilities bitboards offer over mailbox), and what I did notice about its search is that it counts in half-plies, and that its QS (i.e. the possibility to stand pat) already starts at d = 4 ply, using some (marginally) depth-dependent window tuning for deciding which moves to search in the first 4 ply of this QS. And that it used a table to remember the maximum gain effected by each move, used to decide if such a move should be searched in early QS.

Are these Ippolit-specific tricks, and do they contribute a lot to its playing strength?
It's my feeling that Fruit was the biggest contributor to modern chess by far. Ippolit and every strong program since then has just been minor refinements of the basic principles that we saw in Fruit. Fruit was a work of art in my opinion. I did not always feel that way as I was fooled by it's simplicity but many new ideas were introduced (or re-invented) in Fruit.

The big advocates of Ippolit have a false sense of the impact of Ippolit on computer chess. They believe it revolutionized computer chess. For example as soon as Ippolit came out and was proved to be very strong, Larry had this idea that everyone would simply copy all the ideas and that in a few days every program would be 200 ELO stronger. I'm not exaggerating here. Several months went by and Larry wondered where all these programs were. I explained to him that there was nothing special in Ippolit other than excellent engineering. Larry dedicated himself to "unlocking" the secrets of Ippolit and I resisted. He nagged me to look at the code and he worked with other people who could read code to find all the "magic."

We went though a period where Larry felt there was just 1 or 2 big "secrets" and he was going to find it. "We must be missing something" he would often say to me. And my response to him was that we were making progress faster than anyone else so why are we being distracted with this nonsense?

I think there was one excellent idea that was in Ippolit or Robbo or perhaps both. I cannot say they were first but I think they were. I was how they did singular extensions and it was nothing like the classic formulation written up by the Deep Blue team. It proved to be something like 20 ELO, about as much as any single change has been worth for Komodo - at least in recent years.

Ippolit had formula's for forward pruning based on depth, move number, the evaluation score and beta. A lot of fussing with forward pruning and coming up with the right margins for them. But that wasn't a new thing. I am at a loss to identify what "new thing" Ippolit introduced other than SE. I'm not saying there are none, but I don't know what they would be.

And I see that nobody has responded so I suspect that despite the prevailing belief that their contribution has been substantial, that is not the case.

It's possible that Ippolit revealed some things about Rybka. Their piece square table is almost a perfect match of Rybka's and the comparison article proved that virtually EVERYTHING in Ippolit was in Rybka - but in many cases reworked.

In several cases Larry would find out that Ippolit does something different that Komodo and he would insist that we try doing the same. There were probably a dozen times Larry (who can be incredibly optimistic) believed that some minor difference was the big "secret" that we had "missed" but almost in every case those changes just made Komodo weaker. Larry is much wiser and more realistic now. When one release of Komodo gained well over 100 ELO we could not explain it, it happened literally without us realizing it and we couldn't remember doing a single remarkable thing - it was just hard work, tons of testing and slow but sure progress. I think that opened his eyes. Computer chess is not a bag full of tricks where you just piece a bunch of them together but it's more like building a fast car, the parts must be carefully matched and tuned to work with each other or else you just have crap.

So in my opinion, Ippolit was evolutionary, not revolutionary and was not a particularly remarkable contribution to computer chess - except perhaps to show what was possible. Ippolit broke the 4 minute mile so to speak. The REAL innovation in computer chess in recent years, which is probably something the Ippolit authors were well aware of, it massive automated testing. They basically took Rybka and re-engineered it to gain 30-50 ELO.

So to answer HG's question, I don't hink there is much that Ippo contributed and if I'm wrong nobody here is refuting me.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
h1a8
Posts: 508
Joined: Fri Jun 04, 2010 7:23 am

Re: Unanswered question from Engine Origins forum

Post by h1a8 »

Don wrote:
Eelco de Groot wrote:http://www.talkchess.com/forum/viewtopi ... 79&t=44864

Harm-Geert Muller asked this question on the Engine Origins subforum but so far no one has been able or willing to answer it. I thought it was an interesting question, I don't recall myself that it has been answered in any depth before, on this forum. It is not too controversial I think that it should not be here, unless you want to argue that there may be material propietary to Vas Rajlich that should not be discussed here. I don't think that it would be a problem with Vas discussing Ippolit right now but he could always pass us a message about that. As far as I know he is also still a member here.

Maybe, and it is likely, that there is already some material from Vas himself directly related to the question in the hidden Rybka forum, but I did not have access at the time it could have been discussed there. Possibly, as was the case with the question of the contribution to computer chess techniques from Fruit, it is hard to answer HGM's question.

Eelco
hgm wrote:Perhaps a very ignorant question, but what exactly is the contribution of Ippolit to computer Chess in terms of new ideas? (I understand of course that it is a cloner's heaven to have an instant 3000+ engine to start from, but the question is more what you would find that could be helpful if you already had a 2800-Elo engine, if you wanted to boost its Elo, rather than just throw away your own code and use Ippolit in stead.)

Some time ago I had a glance at the Ippolit code (mainly because I wanted to see how a top engine exploits the extra possibilities bitboards offer over mailbox), and what I did notice about its search is that it counts in half-plies, and that its QS (i.e. the possibility to stand pat) already starts at d = 4 ply, using some (marginally) depth-dependent window tuning for deciding which moves to search in the first 4 ply of this QS. And that it used a table to remember the maximum gain effected by each move, used to decide if such a move should be searched in early QS.

Are these Ippolit-specific tricks, and do they contribute a lot to its playing strength?
It's my feeling that Fruit was the biggest contributor to modern chess by far. Ippolit and every strong program since then has just been minor refinements of the basic principles that we saw in Fruit. Fruit was a work of art in my opinion. I did not always feel that way as I was fooled by it's simplicity but many new ideas were introduced (or re-invented) in Fruit.

The big advocates of Ippolit have a false sense of the impact of Ippolit on computer chess. They believe it revolutionized computer chess. For example as soon as Ippolit came out and was proved to be very strong, Larry had this idea that everyone would simply copy all the ideas and that in a few days every program would be 200 ELO stronger. I'm not exaggerating here. Several months went by and Larry wondered where all these programs were. I explained to him that there was nothing special in Ippolit other than excellent engineering. Larry dedicated himself to "unlocking" the secrets of Ippolit and I resisted. He nagged me to look at the code and he worked with other people who could read code to find all the "magic."

We went though a period where Larry felt there was just 1 or 2 big "secrets" and he was going to find it. "We must be missing something" he would often say to me. And my response to him was that we were making progress faster than anyone else so why are we being distracted with this nonsense?

I think there was one excellent idea that was in Ippolit or Robbo or perhaps both. I cannot say they were first but I think they were. I was how they did singular extensions and it was nothing like the classic formulation written up by the Deep Blue team. It proved to be something like 20 ELO, about as much as any single change has been worth for Komodo - at least in recent years.

Ippolit had formula's for forward pruning based on depth, move number, the evaluation score and beta. A lot of fussing with forward pruning and coming up with the right margins for them. But that wasn't a new thing. I am at a loss to identify what "new thing" Ippolit introduced other than SE. I'm not saying there are none, but I don't know what they would be.

And I see that nobody has responded so I suspect that despite the prevailing belief that their contribution has been substantial, that is not the case.

It's possible that Ippolit revealed some things about Rybka. Their piece square table is almost a perfect match of Rybka's and the comparison article proved that virtually EVERYTHING in Ippolit was in Rybka - but in many cases reworked.

In several cases Larry would find out that Ippolit does something different that Komodo and he would insist that we try doing the same. There were probably a dozen times Larry (who can be incredibly optimistic) believed that some minor difference was the big "secret" that we had "missed" but almost in every case those changes just made Komodo weaker. Larry is much wiser and more realistic now. When one release of Komodo gained well over 100 ELO we could not explain it, it happened literally without us realizing it and we couldn't remember doing a single remarkable thing - it was just hard work, tons of testing and slow but sure progress. I think that opened his eyes. Computer chess is not a bag full of tricks where you just piece a bunch of them together but it's more like building a fast car, the parts must be carefully matched and tuned to work with each other or else you just have crap.

So in my opinion, Ippolit was evolutionary, not revolutionary and was not a particularly remarkable contribution to computer chess - except perhaps to show what was possible. Ippolit broke the 4 minute mile so to speak. The REAL innovation in computer chess in recent years, which is probably something the Ippolit authors were well aware of, it massive automated testing. They basically took Rybka and re-engineered it to gain 30-50 ELO.

So to answer HG's question, I don't hink there is much that Ippo contributed and if I'm wrong nobody here is refuting me.
I don't know much about computer chess programming but I disagree with your conclusion. There is significant evidence of multiple chess engines improving greatly from the release of ippolit. Also it is a 3000elo engine! IT HAS TO HAVE KEY IDEAS to make it that strong. Unless you think evaluation is the majority of it's strength.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Unanswered question from Engine Origins forum

Post by Don »

h1a8 wrote:
Don wrote:
Eelco de Groot wrote:http://www.talkchess.com/forum/viewtopi ... 79&t=44864

Harm-Geert Muller asked this question on the Engine Origins subforum but so far no one has been able or willing to answer it. I thought it was an interesting question, I don't recall myself that it has been answered in any depth before, on this forum. It is not too controversial I think that it should not be here, unless you want to argue that there may be material propietary to Vas Rajlich that should not be discussed here. I don't think that it would be a problem with Vas discussing Ippolit right now but he could always pass us a message about that. As far as I know he is also still a member here.

Maybe, and it is likely, that there is already some material from Vas himself directly related to the question in the hidden Rybka forum, but I did not have access at the time it could have been discussed there. Possibly, as was the case with the question of the contribution to computer chess techniques from Fruit, it is hard to answer HGM's question.

Eelco
hgm wrote:Perhaps a very ignorant question, but what exactly is the contribution of Ippolit to computer Chess in terms of new ideas? (I understand of course that it is a cloner's heaven to have an instant 3000+ engine to start from, but the question is more what you would find that could be helpful if you already had a 2800-Elo engine, if you wanted to boost its Elo, rather than just throw away your own code and use Ippolit in stead.)

Some time ago I had a glance at the Ippolit code (mainly because I wanted to see how a top engine exploits the extra possibilities bitboards offer over mailbox), and what I did notice about its search is that it counts in half-plies, and that its QS (i.e. the possibility to stand pat) already starts at d = 4 ply, using some (marginally) depth-dependent window tuning for deciding which moves to search in the first 4 ply of this QS. And that it used a table to remember the maximum gain effected by each move, used to decide if such a move should be searched in early QS.

Are these Ippolit-specific tricks, and do they contribute a lot to its playing strength?
It's my feeling that Fruit was the biggest contributor to modern chess by far. Ippolit and every strong program since then has just been minor refinements of the basic principles that we saw in Fruit. Fruit was a work of art in my opinion. I did not always feel that way as I was fooled by it's simplicity but many new ideas were introduced (or re-invented) in Fruit.

The big advocates of Ippolit have a false sense of the impact of Ippolit on computer chess. They believe it revolutionized computer chess. For example as soon as Ippolit came out and was proved to be very strong, Larry had this idea that everyone would simply copy all the ideas and that in a few days every program would be 200 ELO stronger. I'm not exaggerating here. Several months went by and Larry wondered where all these programs were. I explained to him that there was nothing special in Ippolit other than excellent engineering. Larry dedicated himself to "unlocking" the secrets of Ippolit and I resisted. He nagged me to look at the code and he worked with other people who could read code to find all the "magic."

We went though a period where Larry felt there was just 1 or 2 big "secrets" and he was going to find it. "We must be missing something" he would often say to me. And my response to him was that we were making progress faster than anyone else so why are we being distracted with this nonsense?

I think there was one excellent idea that was in Ippolit or Robbo or perhaps both. I cannot say they were first but I think they were. I was how they did singular extensions and it was nothing like the classic formulation written up by the Deep Blue team. It proved to be something like 20 ELO, about as much as any single change has been worth for Komodo - at least in recent years.

Ippolit had formula's for forward pruning based on depth, move number, the evaluation score and beta. A lot of fussing with forward pruning and coming up with the right margins for them. But that wasn't a new thing. I am at a loss to identify what "new thing" Ippolit introduced other than SE. I'm not saying there are none, but I don't know what they would be.

And I see that nobody has responded so I suspect that despite the prevailing belief that their contribution has been substantial, that is not the case.

It's possible that Ippolit revealed some things about Rybka. Their piece square table is almost a perfect match of Rybka's and the comparison article proved that virtually EVERYTHING in Ippolit was in Rybka - but in many cases reworked.

In several cases Larry would find out that Ippolit does something different that Komodo and he would insist that we try doing the same. There were probably a dozen times Larry (who can be incredibly optimistic) believed that some minor difference was the big "secret" that we had "missed" but almost in every case those changes just made Komodo weaker. Larry is much wiser and more realistic now. When one release of Komodo gained well over 100 ELO we could not explain it, it happened literally without us realizing it and we couldn't remember doing a single remarkable thing - it was just hard work, tons of testing and slow but sure progress. I think that opened his eyes. Computer chess is not a bag full of tricks where you just piece a bunch of them together but it's more like building a fast car, the parts must be carefully matched and tuned to work with each other or else you just have crap.

So in my opinion, Ippolit was evolutionary, not revolutionary and was not a particularly remarkable contribution to computer chess - except perhaps to show what was possible. Ippolit broke the 4 minute mile so to speak. The REAL innovation in computer chess in recent years, which is probably something the Ippolit authors were well aware of, it massive automated testing. They basically took Rybka and re-engineered it to gain 30-50 ELO.

So to answer HG's question, I don't hink there is much that Ippo contributed and if I'm wrong nobody here is refuting me.
I don't know much about computer chess programming but I disagree with your conclusion. There is significant evidence of multiple chess engines improving greatly from the release of ippolit. Also it is a 3000elo engine! IT HAS TO HAVE KEY IDEAS to make it that strong. Unless you think evaluation is the majority of it's strength.
And yet nobody is saying what those key idea are. So far I am the only one who has mentioned any at all.

And it does not "HAVE TO HAVE KEY IDEAS", that is an assertion based on fallible intuition and you are not even a chess programmer. Since I do not even trust my own intuition I don't trust yours.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: Unanswered question from Engine Origins forum

Post by kranium »

Don wrote: The REAL innovation in computer chess in recent years, which is probably something the Ippolit authors were well aware of, it massive automated testing. They basically took Rybka and re-engineered it to gain 30-50 ELO.
This viewpoint is completely unsubstantiated...
and it's sad to see you posting such claims without any proof

Please download and read report.pdf available here:
http://open-chess.org/viewtopic.php?f=3&t=119

According to BB's detailed and expert analysis,
Ippolit is far removed from Rybka...
i.e. the # of differences are enormous, and not just cosmetic

They involve critical program functionality, everything from hash routines and memory management to search and eval.
Rybka and Ippolit have little in common.
Don wrote: So in my opinion, Ippolit was evolutionary, not revolutionary and was not a particularly remarkable contribution to computer chess - except perhaps to show what was possible. Ippolit broke the 4 minute mile so to speak. So to answer HG's question, I don't think there is much that Ippo contributed and if I'm wrong nobody here is refuting me.[/b]
Hmmm...?
for years Ippolit has been publishing state-of-the-art open-source C code for:

a sophisticated SMP implementation (something many programs still lack today)
a complete innovative and original full-featured tablebase solution that easily surpasses Nalimov EGTB
RAM resident endgame bitbases
Montecarlo regression analysis
sophisticated hashing routines and options
compile tracing
Magic Bitboards
ZugZwang detection
Large pages memory
Slab memory implementation and management
Positional gain and history tables for each thread/cpu
Eval and Material explanation modes
Java GUI game explorer (similar to what Hiarcs now offers)
Java GUI endgame explorer
Chess960
and much more...

All of which are innovative and original implementations!
and it's in the top 5 on most/all ranking lists!

Yet you would have us believe that it's nothing special?
i.e. ..Ippolit is "not a particularly remarkable contribution to computer chess"?
LOL

The fact that Ippolit's role/contributions are systematically sabotaged and downplayed by members of this forum is a travesty...
It was (and still is) summarily condemned and executed without a shred of evidence of any wrongdoing...
It should be celebrated, not raped and shot

Your statements above seem to be part and parcel to the continuous and successful effort by you and others here to delegitimize a fine free and innovative open source program
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Unanswered question from Engine Origins forum

Post by mcostalba »

Don wrote: So to answer HG's question, I don't hink there is much that Ippo contributed and if I'm wrong nobody here is refuting me.
Biggest revolutionary (although not the first to do this) contribution of Ippolit has been to release the sources of a top rated engine from where all the top engines of today (including Komodo) were heavily influenced, directly or indirectly. Also all the commercial engines of course, but here the progress has been much slower, I guess due to the burden of legacy very old existing code that is not always easy to 'adapt' to new stuff without deep re-engineering.

It is a pity that you pretend Komodo appeared out of the blue, of course testing is everything and of course you can look at all the sources you want but without hard work on your side you go nowhere, but looking you trying to minimize the impact of Ippo leaves me a bad taste, I find Richard's approach based on clearness and transparency much more balanced and fair and I don't think Critter used other engines for its development more than Komodo: Larry stated many times that you have checked and tested ideas in open source engines one by one in a very metodhological and 'brute force' approach (please concede that the fact that the ideas were found useful for Komodo or not does not change a dime what we are discussing here and is not a sensible point).


P.S: You state that your hero is fruit. What are the new ideas brought on the table by fruit ? I mean, real novelties, stuff never heard of before?
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Unanswered question from Engine Origins forum

Post by Don »

kranium wrote:
Don wrote: The REAL innovation in computer chess in recent years, which is probably something the Ippolit authors were well aware of, it massive automated testing. They basically took Rybka and re-engineered it to gain 30-50 ELO.
This viewpoint is completely unsubstantiated...
and it's sad to see you posting such claims without any proof

Please download and read report.pdf available here:
http://open-chess.org/viewtopic.php?f=3&t=119

According to BB's detailed and expert analysis,
Ippolit is far removed from Rybka...
i.e. the # of differences are enormous, and not just cosmetic
I read the report. The difference are all cosmetic. Rybka will do something on the last 3 ply, Ippolit will do it on the last 4. Tons of things like that. Yes, an enormous number of differences and Ippolit is not a direct clone, but the report showed the pattern very clearly. Almost everything in Rybka was in Ippolit just done slightly differently, in some cases exactly the same and in a few cases even dropped in Ivanhoe. I did not see a single innovation there if you consider Rybka the "gold standard." There was not a single case of where I thought, "oh yes, the way Ippo did this was far better." In fact all of those "enormous" differences you mentioned added up to only 30-50 ELO and it appears that much of this came from dropped features (mostly evaluation) that probably didn't help Rybka - at least at short searches where speed rules.

They involve critical program functionality, everything from hash routines and memory management to search and eval.
Rybka and Ippolit have little in common.
Don wrote: So in my opinion, Ippolit was evolutionary, not revolutionary and was not a particularly remarkable contribution to computer chess - except perhaps to show what was possible. Ippolit broke the 4 minute mile so to speak. So to answer HG's question, I don't think there is much that Ippo contributed and if I'm wrong nobody here is refuting me.[/b]
Hmmm...?
for years Ippolit has been publishing state-of-the-art open-source C code for:

a sophisticated SMP implementation (something many programs still lack today)
a complete innovative and original full-featured tablebase solution that easily surpasses Nalimov EGTB
RAM resident endgame bitbases
Montecarlo regression analysis
sophisticated hashing routines and options
compile tracing
Magic Bitboards
ZugZwang detection
Large pages memory
Slab memory implementation and management
Positional gain and history tables for each thread/cpu
Eval and Material explanation modes
Java GUI game explorer (similar to what Hiarcs now offers)
Java GUI endgame explorer
Chess960
and much more...
All you have done here is put down a list of features, none of which is truly innovative. And some of this has nothing to do with the chess program, stuff like Java GUI game explorer were added to make your list seem big. Montecarlo regression analysis is a feature that is in Rybka and it's not used in matches - has nothing to do with chess program strength and Magic Bitboards are nothing new. Chess960 ? What a great innovation!

And of course Nalimov EGTB, I thought those were created by Eugene Nalimov not the Ippo team. What sense is there in enumerating the feature list like you just did? Every program could produce a silly list like that.

It's obvious that you do not think much of Ippolit and it's contribution to modern computer chess if you consider "Java GUI endgame explorer" a major advancement to computer chess.

Here is the original question HG asked that we are focused on here:
Perhaps a very ignorant question, but what exactly is the contribution of Ippolit to computer Chess in terms of new ideas? (I understand of course that it is a cloner's heaven to have an instant 3000+ engine to start from, but the question is more what you would find that could be helpful if you already had a 2800-Elo engine, if you wanted to boost its Elo, rather than just throw away your own code and use Ippolit in stead.)
I don't think Java GUI explorer or chess960 is the answer he expected.

Generally when there is a major innovation you see a sudden increase in program strength from everyone. Where are all these programs? Presumably all we have to do is take a few minutes to cut and paste all the good ideas into our chess programs and suddenly every program should be close to the strength of the clones. There are literally hundreds of programs out there and other than directly copied programs I don't see this. Do you? Which programs not based directly on Ippolit have made major advancement due to idea in Ippolit?

The only ones that you could possibly name are Stockfish and Komodo. You could try to claim that these 2 programs reaped major benefit from Ippolit - but something innovative should be evidenced by a major revolution - not two programs which had been on a very fast improvement path ANYWAY. Both Komodo as well as Stockfish had been releasing major upgrades well before Ippolit - but suddenly Ippolit comes along and fanboys like you want to claim credit for Stockfish and Komodo? Also, whether Ippo came along or not you would expect every program to advance some but in fact advancement in general has not been any more rapid than before. Trying to take credit for other peoples advancement is vanity and ego.

So what what is your answer to HG's question? What exactly is the contribution of Ippolit to computer Chess in terms of new ideas (other than chess960 and JavaGUI.)

You cannot answer that one because despite the existence of Ippolit now for years there has been no major innovations introduced.

Major innovations are things like null move pruning, check extension (that was innovative in it's day), futility pruning, and LMR. LMR has been used for a very long time but the innovation was the modern implementation of it. When programs starting using hash tables that was a major innovation allowing programs to play a passable endgame for the first time. Ipppolit boasts nothing like any of those things.

All of which are innovative and original implementations!
and it's in the top 5 on most/all ranking lists!

Yet you would have us believe that it's nothing special?
i.e. ..Ippolit is "not a particularly remarkable contribution to computer chess"?
LOL
The question has nothing to do with whether Ippolit is special. I concede that point. It's an incredibly strong program. But the question was "what exactly is the contribution of Ippolit to computer Chess in terms of new ideas." You might say it's contribution to computer chess was to provide a very strong program for free - and I would agree - but that is not a "new idea."

If there is a contribution in terms of new ideas then it should be witnesses by a revolution in stronger programs. And no, Houdini and Critter do not count as it's trivial to prove that Houdini was built on top of Ippolit.

You essentially proved my point for me. You are quick to note that Ippolit tops most lists and yet after years all those "wonderful innovations" have not made it into the other programs? Isn't that pretty strange?

The answer is that writing a strong program like Ippolit is just plain hard work. You obviously don't understand the concept and just want to believe that it is packed full of major innovations - innovations that for some odd reason nobody has chosen to benefit from despite the fact that it has now been around for years. And you still cannot name what those innovation are.


The fact that Ippolit's role/contributions are systematically sabotaged and downplayed by members of this forum is a travesty...
It was (and still is) summarily condemned and executed without a shred of evidence of any wrongdoing...
It should be celebrated, not raped and shot

Your statements above seem to be part and parcel to the continuous and successful effort by you and others here to delegitimize a fine free and innovative open source program
I do not deny that it is a fine program. But it's rather annoying when people like you try to make more of it that it really is. Anyone with 2 eyes can see that that there has been no serious idea contribution from Ippolit and you can easily witness that from the rating lists. You are the one who called attention to the fact that these programs top the rating lists which make my point for me. We have all these wonderful ideas but nobody has figured out what they are despite having the source code around for years?

I think the major innovation Ippo introduced was the concept of starting your own chess program from a copy of a strong program. That's why we have Houdini.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Unanswered question from Engine Origins forum

Post by hgm »

kranium wrote:Hmmm...?
for years Ippolit has been publishing state-of-the-art open-source C code for:

a sophisticated SMP implementation (something many programs still lack today)
a complete innovative and original full-featured tablebase solution that easily surpasses Nalimov EGTB
RAM resident endgame bitbases
Montecarlo regression analysis
sophisticated hashing routines and options
compile tracing
Magic Bitboards
ZugZwang detection
Large pages memory
Slab memory implementation and management
Positional gain and history tables for each thread/cpu
Eval and Material explanation modes
Java GUI game explorer (similar to what Hiarcs now offers)
Java GUI endgame explorer
Chess960
and much more...
Thanks for the list! I get the feeling that you and Don are not really on the same 'wavelength', however, so there is the possibility that you are both right. Note my initial question was intended to refer to the public Ippolit source originally released by the 'bolchewist' group, not so much about later developments. That Ippolit was a single-CPU engine some 50 Elo stronger than Rybka, which itself was some 150 Elo above everyone else, and closed source. So I wondered if it contained some clearly identifiable ideas that were responsible for that 200 Elo it had over any other public code.

I don't want to downplay anything, and to me the possible answer that there wasn't really anything new per se, but that the strength increase is sort of an 'emergent property' due to synergy between the employed ideas, in no way would imply it is not a great development. We have seen the same in Fruit, of which Fabien himself has said it did not contain anything new. Yet Fruit is great, almost a work of art, especially because the ideas it uses are so simple and easy to understand.

I have the feeling that Don's answer more closely addresses the question I intended to ask, while you answer the more general question what the impact of the later developents it triggered were on computer Chess as a whole, rather than more specifically engine design. E.g. SMP and tablebaes were definitely not in the original Ippolit, and tablebases hardly provide any Elo anyway. Having new GUIs is always great, but has absolutely nothing to do with Elo-providing ideas in engine design, and Chess960 is just another variant, which doesn't help an engine play 200-Elo better orthodox Chess. (And none of that was in the original Ippolit too, IIRC.) I am also not sure why magic bitboards are on the list; these already existed before Rybka.

So if I delete the stuff from your list that is not about the original Ippolit, and stuff that is not really about development of algorithms, but on writing efficient code, I am left with
Montecarlo regression analysis
sophisticated hashing routines and options
ZugZwang detection
And I am not sure if the MC regression belongs in this list (because I have no idea what it means). As it would amaze me if totally mishadling Zugzwangs could cost you 200 Elo, I don't see much contradiction between your answer and Don's, given that you were answering different questions.
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Unanswered question from Engine Origins forum

Post by hgm »

And a call to everyone:

please don't let this degenerate into a discussion of who stole what from whom, and who you dislike because of what character flaw. This is completely irrelevant to my question, because even if innovating ideas were stolen from a closed-source engine, it would not make them less innovative, or have any less impact when they finally were published.

And getting losts in squibbles about that, will make it very more difficult to answer the original question.

Perhaps I should rephrase the question in a more specific way:

Now that knowledge of the Ippolit code has reach me, how would I go about changing Fruit to bring its strength on par with Ippolit? Would the major improvements come from (say)

More speed by better algorithms (e.g. bitboard, if you are not on a 32-bit machine)
More speed by reaping benefits from hardware improvement (e.g. large memory pages)
Count depth in fractional plies
Be more concise in hashing nodes for the same position that pruned differently
Search many more moves than just captures in the first 3-4 ply of QS
Keep account of how much you gain by non-captures, and extend high-gain moves in QS
Do reduced-depth pilot searches with shifted window to recognize 'easy-move' situations, and give those a singular extension.