Page 3 of 22

Re: Stockfish version with hash saving capability

Posted: Fri Jul 28, 2017 4:49 pm
by Milos
cdani wrote:
Dann Corbit wrote:
retep1 wrote:I followed the steps you specified.
Unfortunately it doesn't work - neither in the Fritz 15 GUI nor in Aquarium 2017 - although the hash-file is created, but after loading the hashfile the analysis starts again at depth 0 - not 40 where I saved the analysis.
What could be wrong here? Any idea?
Nothing is wrong.
Look at how long it takes to analyze the same position compared to the first time.

The engine has the old hash values, but it still has to perform the search.
Thanks for explaining! This has given me an idea, to save also the whole state of the engine, so to allow it to recover the search from the point it was. I will try to see if is feasible in Andscacs.
That's pretty trivial, you just need to provide mechanisms for saving of your tree state and loading it.

Re: Stockfish version with hash saving capability

Posted: Fri Jul 28, 2017 5:55 pm
by cdani
Milos wrote:This has given me an idea, to save also the whole state of the engine, so to allow it to recover the search from the point it was. I will try to see if is feasible in Andscacs.
That's pretty trivial, you just need to provide mechanisms for saving of your tree state and loading it.[/quote]

And state of pawn hash, refutation tables, other tables, some globals... The triviality depend on how is done the engine.
Of course if I had known from the begin that I will want to do it, I had done some things differently.
Also think that probably, at least that I know, no engine has done this. Maybe just no programmer has thought about it.

Re: Stockfish version with hash saving capability

Posted: Fri Jul 28, 2017 8:33 pm
by Rodolfo Leoni
cdani wrote: And state of pawn hash, refutation tables, other tables, some globals... The triviality depend on how is done the engine.
Of course if I had known from the begin that I will want to do it, I had done some things differently.
Also think that probably, at least that I know, no engine has done this. Maybe just no programmer has thought about it.
Hi Daniel,

I got interested to PHs and learning systems since a lot of time. I guess the only systems I never looked at are those of commercial engines. As far as I know, nobody did it before. You are... going where no man has gone before. :)

And what about an option of using hashes as book (min depth X)? Just a flash I had one minute ago. One lets the engine thinting at a position, goes to work, gets home, kisses his wife and... let us see how it works as book! Wouldn't be nice? (Most of all would be to kiss the wife, of course..)

Re: Stockfish version with hash saving capability

Posted: Fri Jul 28, 2017 9:01 pm
by cdani
Rodolfo Leoni wrote: And what about an option of using hashes as book (min depth X)? Just a flash I had one minute ago. One lets the engine thinting at a position, goes to work, gets home, kisses his wife and... let us see how it works as book! Wouldn't be nice?
If I understand well, what you propose can be done simply by loading a hash/engine state, and let the engine play using it. So it's already doable with those versions.

Re: Stockfish version with hash saving capability

Posted: Fri Jul 28, 2017 9:10 pm
by Rodolfo Leoni
cdani wrote:
Rodolfo Leoni wrote: And what about an option of using hashes as book (min depth X)? Just a flash I had one minute ago. One lets the engine thinting at a position, goes to work, gets home, kisses his wife and... let us see how it works as book! Wouldn't be nice?
If I understand well, what you propose can be done simply by loading a hash/engine state, and let the engine play using it. So it's already doable with those versions.
Yes, with a tuneable hash min depth for using it as a book. Example, we have an hash file which reaches depth 40. We set the UCI option "Hash as a book depth 25" and engine will play best moves in the hash for 15 plies. Of course if opponent goes "out of hashes" then Andscacs should perform a normal search.

Your opinion?

Re: Stockfish version with hash saving capability

Posted: Sat Jul 29, 2017 12:33 am
by Zerbinati
cdani wrote:If someone wants to propose this change for official Stockfish, please go ahead. I don't plan to do it anytime soon. You can take this code or make your own version, as you wish.

Also I forgot to tell that in this code there is the small change proposed by Hgm about maintaining the age during infinite analysis session. As the other change, you can submit it to official Stockfish if you want if someone has not already done it.
this is pull requests
https://github.com/official-stockfish/S ... 6d9b02515b

Re: Stockfish version with hash saving capability

Posted: Sat Jul 29, 2017 1:51 am
by Dann Corbit
Rodolfo Leoni wrote:
cdani wrote:
Rodolfo Leoni wrote: And what about an option of using hashes as book (min depth X)? Just a flash I had one minute ago. One lets the engine thinting at a position, goes to work, gets home, kisses his wife and... let us see how it works as book! Wouldn't be nice?
If I understand well, what you propose can be done simply by loading a hash/engine state, and let the engine play using it. So it's already doable with those versions.
Yes, with a tuneable hash min depth for using it as a book. Example, we have an hash file which reaches depth 40. We set the UCI option "Hash as a book depth 25" and engine will play best moves in the hash for 15 plies. Of course if opponent goes "out of hashes" then Andscacs should perform a normal search.

Your opinion?
Engines already have code to play hash moves instantly.
Basically the requirement is the hash hit has to pass some simple tests to ensure it is the proper position, and then it also has to be deep enough.

Here is the code from Sting85 that probes the hash and returns the hash move if all the tests pass muster.

Code: Select all

    tte = TT.probe(posKey);
    if (tte)
    {
        rtte = *tte;
        tte = &rtte;
    }

    if (tte && (    (tte->move() && !pos.move_is_pseudo_legal(tte->move()))
                    || (!inCheck && (tte->static_value() == VALUE_NONE || tte->static_value_margin() == VALUE_NONE))
                    || ( inCheck && (tte->static_value() != VALUE_NONE || tte->static_value_margin() != VALUE_NONE)) ))
        tte = NULL;

    ttMove = Root ? Rml[0].pv[0] : tte ? tte->move() : MOVE_NONE;

    // At PV nodes we check for exact scores, while at non-PV nodes we check for
    // a fail high/low. Biggest advantage at probing at PV nodes is to have a
    // smooth experience in analysis mode.
    if (   !Root
            &&  tte
            &&  tte->value() != VALUE_NONE
            && &#40;PvNode ? tte->depth&#40;) >= depth && tte->type&#40;) == VALUE_TYPE_EXACT && tte->value&#40;) > alpha && tte->value&#40;) < beta
                &#58; ok_to_use_TT&#40;tte, depth, beta, ss->ply&#41;) )
    &#123;
        TT.store&#40;posKey, tte->value&#40;), tte->type&#40;), tte->depth&#40;), tte->move&#40;), tte->static_value&#40;), tte->static_value_margin&#40;));
        ss->currentMove = ttMove; // Can be MOVE_NONE
        if &#40;tte->value&#40;) >= VALUE_MATE_IN_PLY_MAX&#41;
            ss->mateKiller = ttMove;
        if (    tte->value&#40;) >= beta
                &&  ttMove
                && !pos.move_is_capture_or_promotion&#40;ttMove&#41;
                &&  ttMove != ss->killers&#91;0&#93;)
        &#123;
            ss->killers&#91;1&#93; = ss->killers&#91;0&#93;;
            ss->killers&#91;0&#93; = ttMove;
        &#125;
        return value_from_tt&#40;tte->value&#40;), ss->ply&#41;;
    &#125;

Re: Stockfish version with hash saving capability

Posted: Sat Jul 29, 2017 7:56 am
by Rodolfo Leoni
Dann Corbit wrote: Engines already have code to play hash moves instantly.
Basically the requirement is the hash hit has to pass some simple tests to ensure it is the proper position, and then it also has to be deep enough.

Here is the code from Sting85 that probes the hash and returns the hash move if all the tests pass muster.

Code: Select all

    tte = TT.probe&#40;posKey&#41;;
    if &#40;tte&#41;
    &#123;
        rtte = *tte;
        tte = &rtte;
    &#125;

    if &#40;tte && (    &#40;tte->move&#40;) && !pos.move_is_pseudo_legal&#40;tte->move&#40;)))
                    || (!inCheck && &#40;tte->static_value&#40;) == VALUE_NONE || tte->static_value_margin&#40;) == VALUE_NONE&#41;)
                    || ( inCheck && &#40;tte->static_value&#40;) != VALUE_NONE || tte->static_value_margin&#40;) != VALUE_NONE&#41;) ))
        tte = NULL;

    ttMove = Root ? Rml&#91;0&#93;.pv&#91;0&#93; &#58; tte ? tte->move&#40;) &#58; MOVE_NONE;

    // At PV nodes we check for exact scores, while at non-PV nodes we check for
    // a fail high/low. Biggest advantage at probing at PV nodes is to have a
    // smooth experience in analysis mode.
    if (   !Root
            &&  tte
            &&  tte->value&#40;) != VALUE_NONE
            && &#40;PvNode ? tte->depth&#40;) >= depth && tte->type&#40;) == VALUE_TYPE_EXACT && tte->value&#40;) > alpha && tte->value&#40;) < beta
                &#58; ok_to_use_TT&#40;tte, depth, beta, ss->ply&#41;) )
    &#123;
        TT.store&#40;posKey, tte->value&#40;), tte->type&#40;), tte->depth&#40;), tte->move&#40;), tte->static_value&#40;), tte->static_value_margin&#40;));
        ss->currentMove = ttMove; // Can be MOVE_NONE
        if &#40;tte->value&#40;) >= VALUE_MATE_IN_PLY_MAX&#41;
            ss->mateKiller = ttMove;
        if (    tte->value&#40;) >= beta
                &&  ttMove
                && !pos.move_is_capture_or_promotion&#40;ttMove&#41;
                &&  ttMove != ss->killers&#91;0&#93;)
        &#123;
            ss->killers&#91;1&#93; = ss->killers&#91;0&#93;;
            ss->killers&#91;0&#93; = ttMove;
        &#125;
        return value_from_tt&#40;tte->value&#40;), ss->ply&#41;;
    &#125;
All that remains is to have a tuneable UCI option with it, then.

Stockfish PA GTB has it with the PHs, but don't ask me to show you the code...

Re: Stockfish version with hash saving capability

Posted: Sun Jul 30, 2017 2:23 am
by duncan
cdani wrote:As I think many people will not see it as is buried in a long thread, I publish here this again.

I added to Stockfish the capability of saving the full hash to file, to allow the user to recover a previous analysis session and continue it. It has the same added uci options than in Andscacs.
The saved hash file will be of the same size of the hash memory, so if you defined 4 GB of hash, such will be the file size. Saving and loading such big files can take some time.

Source and executable:
www.andscacs.com/downloads/stockfish_x6 ... vehash.zip

To be able to do it I have added 4 new uci parameters:

option name NeverClearHash type check default false
option name HashFile type string default hash.hsh
option name SaveHashtoFile type button
option name LoadHashfromFile type button

You can set the NeverClearHash option to avoid that the hash could be cleared by a Clear Hash or ucinewgame command.
The HashFile parameter is the full file name with path information. If you don't set the path, it will be saved in the current folder. It defaults to hash.hsh.
To save the hash, stop the analysis and press the SaveHashtoFile button in the uci options screen of the GUI.
To load the hash file, load the game you are interested in, load the engine withouth starting it, and press the LoadHashfromFile button in the uci options screen of the GUI. Now you can start the analysis.

I have not tested it much. Anything just tell me.

The modified code is between
//dani170724
and
//enddani170724
is there a reason why LoadHashfromFile has to be clicked for it to work, as opposed to loading by default from current directory. ?as there is a danger if you forget to click LoadHashfromFile and then save it. you can overwrite days of analysis.

Re: Stockfish version with hash saving capability

Posted: Sun Jul 30, 2017 6:29 am
by cdani
duncan wrote: is there a reason why LoadHashfromFile has to be clicked for it to work, as opposed to loading by default from current directory. ?as there is a danger if you forget to click LoadHashfromFile and then save it. you can overwrite days of analysis.
I had not thougth at it, and I followed what Komodo and Houdini does. I suppose that not a few users will not understand that those options worked differently.
Also if setting the hash file name had the effect of loading it and the user was not expecting it, or simply if the user had put a wrong but existing file name, could also provoke the lost of its current analysis.
So if you put a button for loading the hash, in fact you make absolutely clear when will happen, and also you give the full responsibility to the user.