You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
Stockfish Save Hash version that loads EPD
Moderators: hgm, Rebel, chrisw
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Stockfish Save Hash version that loads EPD
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Stockfish Save Hash version that loads EPD
No problema at all!!! Thanks for working on itDann Corbit wrote:You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
Daniel José - http://www.andscacs.com
-
- Posts: 142
- Joined: Wed Jul 08, 2015 12:30 pm
Re: Stockfish Save Hash version that loads EPD
Dann Corbit wrote:You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
Hi, Dann. Thanks for the program. It seems that the savehash option of Stockfish is generatiing a flurry of new idea!
I am sure you already did, but could you briefly summarize how you would envision that this option could serve chessplayers and programs? Even better, it would be fantastic if you could provide a motivating example, like Rodolfo recently did for his correspondence games.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Stockfish Save Hash version that loads EPD
Consider things like CCRL and CEGT, TCEC, etc.giovanni wrote:Dann Corbit wrote:You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
Hi, Dann. Thanks for the program. It seems that the savehash option of Stockfish is generatiing a flurry of new idea!
I am sure you already did, but could you briefly summarize how you would envision that this option could serve chessplayers and programs? Even better, it would be fantastic if you could provide a motivating example, like Rodolfo recently did for his correspondence games.
These contests create thousands and thousands of compute hours generated by strong computers at various time controls.
Now, it is possible to mine this data to get computer analysis for more than a billion EPD positions.
Suppose that we make a huge hash file (e.g. 64 GB) and load it with this data. The file would contain an enormous number of pv nodes with perfect scores.
Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Stockfish Save Hash version that loads EPD
Dann Corbit wrote:Consider things like CCRL and CEGT, TCEC, etc.giovanni wrote:It might be interesting to have two hash tables.Dann Corbit wrote:You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
One is the normal hash table, seeded from a special pv hash table.
When the engine starts up, it could take the preloaded pv hash table and seed the main hash from it.
When the engine is done playing, it would scan the normal hash table looking for pv nodes. If it finds any new nodes or any improved nodes, it would store them into the pv hash.
This would be another kind of learning.
Hi, Dann. Thanks for the program. It seems that the savehash option of Stockfish is generatiing a flurry of new idea!
I am sure you already did, but could you briefly summarize how you would envision that this option could serve chessplayers and programs? Even better, it would be fantastic if you could provide a motivating example, like Rodolfo recently did for his correspondence games.
These contests create thousands and thousands of compute hours generated by strong computers at various time controls.
Now, it is possible to mine this data to get computer analysis for more than a billion EPD positions.
Suppose that we make a huge hash file (e.g. 64 GB) and load it with this data. The file would contain an enormous number of pv nodes with perfect scores.
Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Stockfish Save Hash version that loads EPD
Dann Corbit wrote:Consider things like CCRL and CEGT, TCEC, etc.giovanni wrote:It might be interesting to have two hash tables.Dann Corbit wrote:You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
One is the normal hash table, seeded from a special pv hash table.
When the engine starts up, it could take the preloaded pv hash table and seed the main hash from it.
When the engine is done playing, it would scan the normal hash table looking for pv nodes. If it finds any new nodes or any improved nodes, it would store them into the pv hash.
This would be another kind of learning.
Hi, Dann. Thanks for the program. It seems that the savehash option of Stockfish is generatiing a flurry of new idea!
I am sure you already did, but could you briefly summarize how you would envision that this option could serve chessplayers and programs? Even better, it would be fantastic if you could provide a motivating example, like Rodolfo recently did for his correspondence games.
These contests create thousands and thousands of compute hours generated by strong computers at various time controls.
Now, it is possible to mine this data to get computer analysis for more than a billion EPD positions.
Suppose that we make a huge hash file (e.g. 64 GB) and load it with this data. The file would contain an enormous number of pv nodes with perfect scores.
Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 142
- Joined: Wed Jul 08, 2015 12:30 pm
Re: Stockfish Save Hash version that loads EPD
Thanks again, Dann. Very exciting. I have just one technical question. Which one is the best way to get epd positions from these pgn files? I guess you need to preserve the evaluation and pheraps even the depth.Dann Corbit wrote:giovanni wrote:Dann Corbit wrote:You can get it here:
http://rybkaforum.net/cgi-bin/rybkaforu ... pid=574711
No attachments in this forum.
I had to make a lot of changes so that I could make a working version that I understood.
Something appeared to be wrong with the EPD parsing, so I made a simplified C like version. Hopefully, Daniel will not be insulted at my crude carpentry of his pretty code.
There is a binary in the src folder.
Hi, Dann. Thanks for the program. It seems that the savehash option of Stockfish is generatiing a flurry of new idea!
I am sure you already did, but could you briefly summarize how you would envision that this option could serve chessplayers and programs? Even better, it would be fantastic if you could provide a motivating example, like Rodolfo recently did for his correspondence games.
Consider things like CCRL and CEGT, TCEC, etc.
These contests create thousands and thousands of compute hours generated by strong computers at various time controls.
Now, it is possible to mine this data to get computer analysis for more than a billion EPD positions.
Suppose that we make a huge hash file (e.g. 64 GB) and load it with this data. The file would contain an enormous number of pv nodes with perfect scores.
Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
-
- Posts: 545
- Joined: Tue Jun 06, 2017 4:49 pm
- Location: Italy
Re: Stockfish Save Hash version that loads EPD
It looks great. The only inconvenience could be the fact stored evaluations come from engines which are very different each other. The improvement by your SF savehash engine will be absolutely relevant because that minimax needs to be "hammered".Dann Corbit wrote: Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
I think it's the first time someone performs such an experiment. It'll be interesting to see how it evolves.
F.S.I. Chess Teacher
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: Stockfish Save Hash version that loads EPD
were you planning on doing this.?and at this stage do you have an estimate for elo gain ?Dann Corbit wrote:
Suppose that we make a huge hash file (e.g. 64 GB) and load it with this data. The file would contain an enormous number of pv nodes with perfect scores.
what do you call a long period of time. ?wrote: Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Stockfish Save Hash version that loads EPD
I might do the experiment. I have a box with 64 cores and 128 GB RAM, and I have a database with a billion analyzed positions.duncan wrote:were you planning on doing this.?and at this stage do you have an estimate for elo gain ?Dann Corbit wrote:
Suppose that we make a huge hash file (e.g. 64 GB) and load it with this data. The file would contain an enormous number of pv nodes with perfect scores.
what do you call a long period of time. ?wrote: Now, take a really strong engine and have it play with (say) 64 cores at long time control for a long period of time. It is going to minimax this data naturally, vastly improving it.
Eventually, that file would contain nearly perfect information and the engines attached would play nearly perfect chess.
I can always do some mining to increase the data.
I think I will want to add other information to the hash, so I may want to customize it.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.