Michael Sherwin wrote:Any idea I mention here has been cluster tested.bob wrote:Michael Sherwin wrote:Just one more example of something that I have posted about years ago, that has been in RomiChess for years and that has been migrated to Crafty and other engins. And everyone else gets the credit for it.jwes wrote:Not doing null-move when the depth remaining is 1.metax wrote:Which change?bob wrote:The change to null-move suggested by Cleveland was another 5-10.
I posted that history tables were useless for move ordering shortly after RomiChess was first released. Later they were dropped from Crafty.
History for _ordering_ has gone from Crafty since a 20.x version which is 3-4 years old at least. It was _not_ bad or useless. The effect was minimal and in cleaning up the code, we slightly modified killers and hashing and found we could do without it.
*****************************
RomiChess is older than 3-4 years.
*****************************
Again, it is not of "no benefit". It just adds some quirks to hashing that I wanted to get rid of, rather than working around them. There are circimstances where it is pretty good.
I posted that IID is of no benefit. Later it was removed from Crafty.
*************************************************************
There are circumstances when generating a random move to play may be pretty good. But, until you work around the problems of IID and the hash, and then cluster test it, there is no acceptable proof that it is of any use.
*************************************************************
I posted way before Bob that LMR as done in Fruit did not add to Fruits strength. I was pooh poohed.
There you are wrong. LMR in fruit _does_ work. But the history part of it is completely ineffective.
*************************************************************
I removed LMR from Fruit 2.1 and it did not loose any strength in my test.
*************************************************************
Then you didn't do it right, because I did the same thing last year and disabled it in Fruit, Crafty and Glaurung, and discovered it is worth about 30-40 Elo if you remove it without disabling null move. Without null-move, disabling LMR will cost about 80 in any of those. From testing on the cluster...
In which case you should be able to play 1,000 games against crafty 23.1, and then win the majority of the games after that. if it works "perfectly"???Please try again, and provide the _exact_ quote. I believe I commented about RomiChess, where you were going on and on about how it would _eventually_ use its learning to beat crafty. It never did, because that type of learning is simply flawed.
There were a few other things that made it into Crafty or was dropped from Crafty sometimes years after I had posted about them. If I had the time I would go back through all my post and catalog them.
I posted many times that various ideas were left in engines just because they did not seem to hurt. Much later Bob post the same conclusion.
Yet Bob posted once at the old winboard site that he did not believe that there was any possibility that there was anything of value that he could learn from me concerning chess programming.
*************************************************************
Well I am not going to bother, but we were not talking about learning at that time. My comment to you was something like, 'no wonder Crafty is falling out from among the top engines since you are not willing to explore
the ideas from new open source engines. So, there is not possibly anything you could learn from me or any other new guy on the block.
************************************************************
************************************************************
For what my learning was intended for--human/engine sparring--it is not flawed. It works perfectly!
***********************************************************
Going on vacation now.Somewhat "ego-centric" today, are we???
But that is just how the world works!
************************************************************
Not really. Just frustrated and wondering why I bother when all my ideas are crap.
***********************************************************
Futility pruning, Ext futility pruning and Limited Razoring.
Moderators: hgm, Rebel, chrisw
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Futility pruning, Ext futility pruning and Limited Razor
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
RomiChessX64P3k vs Crafty-23.1-win64 learning test underway
Well, I did say that the learning was perfect for what it was designed for--engine/human sparring. However, no one seems interested in that.
So, just to humor the masses!
RomiChessX64P3k - Crafty-23.1-win64 : 21.5/221 8-186-27 (00000=0000=000=0000000000000=00010000010000000=000000000000000000000000=0=01==00000000000010000000000000001000000=0000000000000000000=00000=00=0=000010000=0=0000000000=0000=00000000=00000==0=000000000=01000==00000=00100=0) 10% -382
I was a bit too generous and allowed Crafty to use two threads on a Q6600. Crafty being a lot stronger now and using two threads makes it hard for Romi to gain points. I am using the book from the ftp site. I assume that that is okay. Crafty is varying its openings and not repeating openings in which it lost. Or at least it seems that way so I assume that the Crafty default is set up that way. Romi's percentage is climbing but, with only 221 games in it may just be noise. We shall see.
So, just to humor the masses!
RomiChessX64P3k - Crafty-23.1-win64 : 21.5/221 8-186-27 (00000=0000=000=0000000000000=00010000010000000=000000000000000000000000=0=01==00000000000010000000000000001000000=0000000000000000000=00000=00=0=000010000=0=0000000000=0000=00000000=00000==0=000000000=01000==00000=00100=0) 10% -382
I was a bit too generous and allowed Crafty to use two threads on a Q6600. Crafty being a lot stronger now and using two threads makes it hard for Romi to gain points. I am using the book from the ftp site. I assume that that is okay. Crafty is varying its openings and not repeating openings in which it lost. Or at least it seems that way so I assume that the Crafty default is set up that way. Romi's percentage is climbing but, with only 221 games in it may just be noise. We shall see.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
I just happened to check the progress when Romi got to 11%. It seems that either Crafty is trying some inferior lines from its book or it is already running out of new lines to try. Which ever way it is Romi is taking Crafty's opening lines away from it one at a time as Romi learns to defeat each and every one of them. But, there is a long way to go and I may decide to run this match past the 2,000 games that Bob dictated.
RomiChessX64P3k - Crafty-23.1-win64 : 58.0/545 29-458-58 (00000=0000=000=0000000000000=00010000010000000=000000000000000000000000=0=01==00000000000010000000000000001000000=0000000000000000000=00000=00=0=000010000=0=0000000000=0000=00000000=00000==0=000000000=01000==00000=00100=0=1000=0000000000000=0100000000000000000000000=000000000=00000000==0000=0=000=0=000000=00=10000000100=00000000=0000=000001000000=001000000000000000000000=00000000000000100010=0000000=000000000000000000000000001000=1001000000=0000000=00=00=0=000=010000000100=00000000000001000000000000000000000000000000000100010=0=1=001010101) 11% -363
RomiChessX64P3k - Crafty-23.1-win64 : 58.0/545 29-458-58 (00000=0000=000=0000000000000=00010000010000000=000000000000000000000000=0=01==00000000000010000000000000001000000=0000000000000000000=00000=00=0=000010000=0=0000000000=0000=00000000=00000==0=000000000=01000==00000=00100=0=1000=0000000000000=0100000000000000000000000=000000000=00000000==0000=0=000=0=000000=00=10000000100=00000000=0000=000001000000=001000000000000000000000=00000000000000100010=0000000=000000000000000000000000001000=1001000000=0000000=00=00=0=000=010000000100=00000000000001000000000000000000000000000000000100010=0=1=001010101) 11% -363
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
The book from my ftp site is reasonably small to keep download times low. It is made from a fairly reasonable PGN collection, and then very restricted in how many times a move has to be played before it is included...
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100
points or less then in my opinion Romi would be ahead in the match. So,
even if the learning is flawed against Crafty's methodology that does not
mean that it is with out value. As it stands now Romi is of benefit to your
team (and others) because, out of about every thousand games played Romi can show
you ~50 losses that you could use to tune your eval and opening book to
eliminate weaknesses in both. Said another way--Crafty with two threads
is far out searching Romi and yet there are weaknesses in Crafty that
allow Romi to win some games after a few tries. These wins by Romi could
be useful in finding those weaknesses!
points or less then in my opinion Romi would be ahead in the match. So,
even if the learning is flawed against Crafty's methodology that does not
mean that it is with out value. As it stands now Romi is of benefit to your
team (and others) because, out of about every thousand games played Romi can show
you ~50 losses that you could use to tune your eval and opening book to
eliminate weaknesses in both. Said another way--Crafty with two threads
is far out searching Romi and yet there are weaknesses in Crafty that
allow Romi to win some games after a few tries. These wins by Romi could
be useful in finding those weaknesses!
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
Try Gaviota. They are of similar strength with Romichess and Gaviota comes with books of two different sizes (or you can make one). In addition, you can turn book and positional learning on and off in the ini file (or in Engine settings in Winboard )Michael Sherwin wrote:Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100
points or less then in my opinion Romi would be ahead in the match. So,
even if the learning is flawed against Crafty's methodology that does not
mean that it is with out value. As it stands now Romi is of benefit to your
team (and others) because, out of about every thousand games played Romi can show
you ~50 losses that you could use to tune your eval and opening book to
eliminate weaknesses in both. Said another way--Crafty with two threads
is far out searching Romi and yet there are weaknesses in Crafty that
allow Romi to win some games after a few tries. These wins by Romi could
be useful in finding those weaknesses!
Miguel
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
When this match is finished I will test against Gaviota with all its learning turned on. Unless someone were to beat me to it.michiguel wrote:Try Gaviota. They are of similar strength with Romichess and Gaviota comes with books of two different sizes (or you can make one). In addition, you can turn book and positional learning on and off in the ini file (or in Engine settings in Winboard )Michael Sherwin wrote:Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100
points or less then in my opinion Romi would be ahead in the match. So,
even if the learning is flawed against Crafty's methodology that does not
mean that it is with out value. As it stands now Romi is of benefit to your
team (and others) because, out of about every thousand games played Romi can show
you ~50 losses that you could use to tune your eval and opening book to
eliminate weaknesses in both. Said another way--Crafty with two threads
is far out searching Romi and yet there are weaknesses in Crafty that
allow Romi to win some games after a few tries. These wins by Romi could
be useful in finding those weaknesses!
Miguel
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
That kind of tuning is worthless. All it would do is to make crafty avoid playing a move that is actually not necessarily bad, without any understanding of why, other than a program played it so many times that it eventually learned a path to reach a victory. Several have asked me to add position learning back into crafty, which would likely just about eliminate this kind of learning result because Crafty would also be learning what to not repeat the next time around... But that's a different issue.Michael Sherwin wrote:Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100
points or less then in my opinion Romi would be ahead in the match. So,
even if the learning is flawed against Crafty's methodology that does not
mean that it is with out value. As it stands now Romi is of benefit to your
team (and others) because, out of about every thousand games played Romi can show
you ~50 losses that you could use to tune your eval and opening book to
eliminate weaknesses in both. Said another way--Crafty with two threads
is far out searching Romi and yet there are weaknesses in Crafty that
allow Romi to win some games after a few tries. These wins by Romi could
be useful in finding those weaknesses!
I don't believe Romi is finding "weaknesses". It is essentially doing random probing thru a large search space to find paths that are more promising, without having a clue about why they are promising.
There as another famous learning program, I think from the University of California (maybe Irvine campus, not sure). It suffered from the same flaw. And once the flaw is understood, a human will have no problems with the program, because he will not continue to play the same opening moves game after game waiting on the program to discover a way to bust it. And he isn't going to play it thousands (or tens of thousands) of games to give it that opportunity either...
That's why I said the idea is highly impractical. It is just as impractical as "position learning" unless you pair two programs that do this against each other. And then they are not learning how to play better chess. They are stumbling around until they fall into a win and then try to repeat that without any understanding of why. That's not what I call "learning".
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
The search for alternatives that win is not random at all. The bonus/penalty is
very small and will lead the engine to try the second best move and then
back to the first move again with a try of a second best at another ply before
going to a third best move elsewhere. It therefore does a very systematic
exploration of the search space using very high quality alternative moves
before exploring lesser moves.
By the way, Romi has just achieved 12%!
RomiChessX64P3k - Crafty-23.1-win64 : 87.0/743 40-609-94
(00000=0000=000=0000000000000=00010000010000000=000000000000
000000000000=0=01==00000000000010000000000000001000000=0000
000000000000000=00000=00=0=000010000=0=0000000000=0000=0000
0000=00000==0=000000000=01000==00000=00100=0=1000=00000000
00000=0100000000000000000000000=000000000=00000000==0000=0=
000=0=000000=00=10000000100=00000000=0000=000001000000=0010
00000000000000000000=00000000000000100010=0000000=0000000000
00000000000000001000=1001000000=0000000=00=00=0=000=0100000
00100=00000000000001000000000000000000000000000000000100010=
0=1=0010101010=0=0000=0000000=0000000=0=0000000=000==0=00
=0000=0000=0=0=0=00=00=0=0=000000000=1100000000001000=000
0000=0100000000000100=00=0000100100000000000000000=0===0000
=000=0=0=000000000000100=0=0001=000101) 12% -346
very small and will lead the engine to try the second best move and then
back to the first move again with a try of a second best at another ply before
going to a third best move elsewhere. It therefore does a very systematic
exploration of the search space using very high quality alternative moves
before exploring lesser moves.
By the way, Romi has just achieved 12%!
RomiChessX64P3k - Crafty-23.1-win64 : 87.0/743 40-609-94
(00000=0000=000=0000000000000=00010000010000000=000000000000
000000000000=0=01==00000000000010000000000000001000000=0000
000000000000000=00000=00=0=000010000=0=0000000000=0000=0000
0000=00000==0=000000000=01000==00000=00100=0=1000=00000000
00000=0100000000000000000000000=000000000=00000000==0000=0=
000=0=000000=00=10000000100=00000000=0000=000001000000=0010
00000000000000000000=00000000000000100010=0000000=0000000000
00000000000000001000=1001000000=0000000=00=00=0=000=0100000
00100=00000000000001000000000000000000000000000000000100010=
0=1=0010101010=0=0000=0000000=0000000=0=0000000=000==0=00
=0000=0000=0=0=0=00=00=0=0=000000000=1100000000001000=000
0000=0100000000000100=00=0000100100000000000000000=0===0000
=000=0=0=000000000000100=0=0001=000101) 12% -346
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: RomiChessX64P3k vs Crafty-23.1-win64 learning test under
"systematic" in this case is essentially "random". That was my point. You don't try to figure out why a move didn't work, and try to choose other moves that appear to be better, you actually try moves that initially appeared to be worse, only because the first one failed.Michael Sherwin wrote:The search for alternatives that win is not random at all. The bonus/penalty is
very small and will lead the engine to try the second best move and then
back to the first move again with a try of a second best at another ply before
going to a third best move elsewhere. It therefore does a very systematic
exploration of the search space using very high quality alternative moves
before exploring lesser moves.
To see the flaw in this, create a new book from Crafty from something like the "enormous.pgn" file that is available all over the place. Use this command:
book create enormous.pgn 60 1
Then you will see the flaw in this type of learning. If Crafty has a book of 20M + games to choose from, it is going to take tens of thousands of games to get back to something previously played...
By the way, Romi has just achieved 12%!
RomiChessX64P3k - Crafty-23.1-win64 : 87.0/743 40-609-94
(00000=0000=000=0000000000000=00010000010000000=000000000000
000000000000=0=01==00000000000010000000000000001000000=0000
000000000000000=00000=00=0=000010000=0=0000000000=0000=0000
0000=00000==0=000000000=01000==00000=00100=0=1000=00000000
00000=0100000000000000000000000=000000000=00000000==0000=0=
000=0=000000=00=10000000100=00000000=0000=000001000000=0010
00000000000000000000=00000000000000100010=0000000=0000000000
00000000000000001000=1001000000=0000000=00=00=0=000=0100000
00100=00000000000001000000000000000000000000000000000100010=
0=1=0010101010=0=0000=0000000=0000000=0=0000000=000==0=00
=0000=0000=0=0=0=00=00=0=0=000000000=1100000000001000=000
0000=0100000000000100=00=0000100100000000000000000=0===0000
=000=0=0=000000000000100=0=0001=000101) 12% -346