CLOP: when to stop?

[zenpawn]

Does CLOP automatically stop once it has settled on some values or does it require somehow interpreting the colors of the Hessian or Eigenvectors tabs of the GUI?

[brianr]

You have to decide. I used to turn off the plotting updates after a while as the overhead was significant. You can start and stop the run and then update the graph and decide when you think you are near a good result.

Then, you should test in regular games with the optimized values. I never found any CLOP results to be an actual improvement, despite many seemingly promising runs. Texel-type tuning can be more fruitful, but after a large initial jump it gets harder to wring more improvements (at least without the massive resources of something like Fishtest).

Good luck.

PS IIRC need many 10s of thousands of runs...

[zenpawn]

You have to decide. I used to turn off the plotting updates after a while as the overhead was significant. You can start and stop the run and then update the graph and decide when you think you are near a good result.

Hmmm... Any tips on what to look for as a clue? There are lots of numbers on the individual tabs, and even color coding, but I'm not sure what to make of it all.

Then, you should test in regular games with the optimized values. I never found any CLOP results to be an actual improvement, despite many seemingly promising runs. Texel-type tuning can be more fruitful, but after a large initial jump it gets harder to wring more improvements (at least without the massive resources of something like Fishtest).

Yeah, Texel tuning definitely intrigues me as well, but there doesn't seem to be a ready-made program to execute the algorithm on any 'ole engine like CLOP.

Good luck.

PS IIRC need many 10s of thousands of runs...

Thanks, and ouch on the 10xk runs, lol.

[ZirconiumX]

Then, you should test in regular games with the optimized values. I never found any CLOP results to be an actual improvement, despite many seemingly promising runs. Texel-type tuning can be more fruitful, but after a large initial jump it gets harder to wring more improvements (at least without the massive resources of something like Fishtest).

Yeah, Texel tuning definitely intrigues me as well, but there doesn't seem to be a ready-made program to execute the algorithm on any 'ole engine like CLOP.

It requires more effort, yes, but the 150 elo leap my engine made with it shows it can produce big rewards. The problem with such a universal tuner is the communication latencies are massive.

Taking the zurichess epd file as an example, we have a 765k position file. Let's say it takes N time to parse a FEN. You have to go through R rounds of tuning, where I've found the 10k rounds figure fairly accurate. So that's 765k positions x 10k rounds x N seconds. Assuming you can parse a FEN in a millisecond (possibly far less, I have not checked), that's 765k x 10k x 0.001 = 7.65 million seconds spent parsing FEN positions. That's 88.5 days.

And the obvious answer of "why don't we just parse the FENs once and store them in the engine?" ruins the universality of the universal tuner.

[jdart]

You have to go through R rounds of tuning, where I've found the 10k rounds figure fairly accurate.

This is CLOP you are talking about, or Texel? For the latter my tuning program typically converges in 50-60 iterations. It uses either Adagrad or ADAM (https://arxiv.org/pdf/1412.6980.pdf) to find the optimum.

--Jon

[ZirconiumX]

You have to go through R rounds of tuning, where I've found the 10k rounds figure fairly accurate.

This is CLOP you are talking about, or Texel? For the latter my tuning program typically converges in 50-60 iterations. It uses either Adagrad or ADAM (https://arxiv.org/pdf/1412.6980.pdf) to find the optimum.

--Jon

Texel. My tuning method is by no means fancy - pick a random variable, add a small random number to it, run the error calculation, if it's lower, it becomes the new best.

Also, my tuning uses random restarts, so it's probably far from optimal. The 150 elo gain speaks for itself, however.

[AlvaroBegue]

You have to go through R rounds of tuning, where I've found the 10k rounds figure fairly accurate.

This is CLOP you are talking about, or Texel? For the latter my tuning program typically converges in 50-60 iterations. It uses either Adagrad or ADAM (https://arxiv.org/pdf/1412.6980.pdf) to find the optimum.

--Jon

I tried some first-order gradient-descent algorithms (momentum, Adam) and the convergence was too slow. Perhaps I could play around with larger learning rates...

Anyway, yesterday I managed to get L-BFGS working and my first tests seem very promising.

[zenpawn]

Yeah, Texel tuning definitely intrigues me as well, but there doesn't seem to be a ready-made program to execute the algorithm on any 'ole engine like CLOP.

It requires more effort, yes, but the 150 elo leap my engine made with it shows it can produce big rewards. The problem with such a universal tuner is the communication latencies are massive.

I'd be happy to implement it myself, but the math is apparently over my head.

[cdani]

Yeah, Texel tuning definitely intrigues me as well, but there doesn't seem to be a ready-made program to execute the algorithm on any 'ole engine like CLOP.

It requires more effort, yes, but the 150 elo leap my engine made with it shows it can produce big rewards. The problem with such a universal tuner is the communication latencies are massive.

I'd be happy to implement it myself, but the math is apparently over my head.

Some code that maybe can help about Texel tuning:

http://www.talkchess.com/forum/viewtopi ... =&start=20

[zenpawn]

Some code that maybe can help about Texel tuning:

http://www.talkchess.com/forum/viewtopi ... =&start=20

Hi Daniel,

Is this some sort of pre-step to the Texel tuning itself?