Re: Texel tuning method question
Posted: Thu Jul 20, 2017 8:05 am
Sample basic Texel tuning code in python.
Tune.py
Calculation of E is done by random number for simulation purposes.
Sample output:
So you get your best param and best error
bestE : 0.14404
bestParam : [['pawn', 99], ['knight', 297], ['bishop', 299]]
In certain situation you can stop the tuning and record the best so far.
Remember the best error and param.
When you resume the tuning, you can now use the last best tuning info.
So your next input run for example would be:
At a point where there is successful reduction of error like,
Remember the param tuned:
[['pawn', 100], ['knight', 298], ['bishop', 299]]
You can use that to test your engine in actual matches, this is called sampling. It can happen that those values may actually perform in actual game test.
It is good to experiment varied delta array like,
Expand the param if you like.
Tune.py
Code: Select all
"""
Sample Texel tuning in Python
Tested on python 3.6
"""
import sys
import random
from copy import deepcopy
import time
MAXBESTE = 1000.0
def DoQSearchAndGetE(K):
""" Just return random for now """
time.sleep(0.5) # Sleep for 0.5s for simulation purposes
return random.randint(14400, 14600)/100000.0
def SendParamToEngine(param):
""" Set the param to the engine, pass for now """
pass
def Tune(paramToOptimize, initBestE, delta, K, iLog, cycleLimit):
""" paramToOptimize is a list of list
initBestE is float
delta is a list
"""
goodCnt = 1
goodCntOld = 0
lastBestE = initBestE
lastBestParam = []
for g in range(cycleLimit): # g = 0, 1, 2, ... cycleLimit-1
if iLog:
print('Tune >> param cycle: %d' %(g+1))
# Exit tuning if goodCnt is not increased
if goodCnt <= goodCntOld:
if iLog:
print('Tune >> bestE has not been improved, exiting the tuning now ...')
break
goodCntOld = goodCnt
if len(lastBestParam):
paramToOptimize = deepcopy(lastBestParam)
for p in paramToOptimize:
pName = p[0]
pValue = p[1]
for d in delta:
dValue = pValue + d
if iLog:
print('Tune >> param to optimize: %s' %(pName))
print('Tune >> delta: %+d' %(d))
# Create a new param set instead of paramToOptimize.
# This is the set that will be sent to the engine.
paramToBeTested = []
for a in paramToOptimize:
if a[0] == pName:
a[1] = dValue
paramToBeTested.append([a[0], a[1]])
if iLog:
print('Tune >> paramSet to try: %s' %(paramToBeTested))
print('Tune >> Send this set to engine')
# Send param values to engine
SendParamToEngine(paramToBeTested)
if iLog:
print('Tune >> lastBestE: %0.5f' %(lastBestE))
print('Tune >> Calculate E')
E = DoQSearchAndGetE(K)
if iLog:
print('Tune >> CalculatedE: %0.5f' %(E))
if E < lastBestE:
goodCnt += 1
lastBestE = E
if iLog:
print('Tune >> NewBestE: %0.5f' %(lastBestE))
print('Tune >> CalculatedE is good -------- !!\n')
lastBestParam = deepcopy(paramToBeTested)
# Get out of 'for delta' and try the next p
break
else:
if iLog:
print('Tune >> CalculatedE is not good ----- ?\n')
# Log if wer have reached cycle limit
if g == cycleLimit-1:
if iLog:
print('Tune >> param cycle limit has been reached, exiting tuning now ...')
return lastBestE, lastBestParam
def main(argv):
bestK = 1.13
bestESTart = MAXBESTE
enableLog = True
paramCycleLimit = 1000
paramToOptimize = [
['pawn', 100],
['knight', 300],
['bishop', 300]
]
delta = [+1, -1]
# Show init values
print('origBestE : %0.5f' %(bestESTart))
print('origParam : %s' %(paramToOptimize))
print('bestK : %0.3f' %(bestK))
print('delta : %s' %(delta))
print('paramCycleLimit : %d' %(paramCycleLimit))
print('EnableLogging : %s\n' %('On' if enableLog else 'Off'))
t1 = time.clock()
# Run the tuner
optiE, optiParam = Tune(paramToOptimize, bestESTart, delta, bestK, enableLog, paramCycleLimit)
t2 = time.clock()
print('\nbestE : %0.5f' %(optiE))
print('bestParam : %s' %(optiParam))
print('Elapsed : %ds' %(t2-t1))
if __name__ == "__main__":
main(sys.argv[1:])
Sample output:
Code: Select all
origBestE : 1000.00000
origParam : [['pawn', 100], ['knight', 300], ['bishop', 300]]
bestK : 1.130
delta : [1, -1]
paramCycleLimit : 1000
EnableLogging : On
Tune >> param cycle: 1
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 101], ['knight', 300], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 1000.00000
Tune >> Calculate E
Tune >> CalculatedE: 0.14514
Tune >> NewBestE: 0.14514
Tune >> CalculatedE is good -------- !!
Tune >> param to optimize: knight
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 101], ['knight', 301], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14514
Tune >> Calculate E
Tune >> CalculatedE: 0.14517
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 101], ['knight', 299], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14514
Tune >> Calculate E
Tune >> CalculatedE: 0.14462
Tune >> NewBestE: 0.14462
Tune >> CalculatedE is good -------- !!
Tune >> param to optimize: bishop
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 101], ['knight', 299], ['bishop', 301]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14462
Tune >> Calculate E
Tune >> CalculatedE: 0.14560
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 101], ['knight', 299], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14462
Tune >> Calculate E
Tune >> CalculatedE: 0.14596
Tune >> CalculatedE is not good ----- ?
Tune >> param cycle: 2
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 102], ['knight', 299], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14462
Tune >> Calculate E
Tune >> CalculatedE: 0.14543
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: pawn
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 100], ['knight', 299], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14462
Tune >> Calculate E
Tune >> CalculatedE: 0.14435
Tune >> NewBestE: 0.14435
Tune >> CalculatedE is good -------- !!
Tune >> param to optimize: knight
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 300], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14496
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 100], ['knight', 298], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14599
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 298], ['bishop', 301]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14575
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 100], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14457
Tune >> CalculatedE is not good ----- ?
Tune >> param cycle: 3
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 101], ['knight', 299], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14533
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: pawn
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 99], ['knight', 299], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14448
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 99], ['knight', 300], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14555
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 99], ['knight', 298], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14553
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 99], ['knight', 298], ['bishop', 301]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14553
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 99], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14435
Tune >> Calculate E
Tune >> CalculatedE: 0.14419
Tune >> NewBestE: 0.14419
Tune >> CalculatedE is good -------- !!
Tune >> param cycle: 4
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14419
Tune >> Calculate E
Tune >> CalculatedE: 0.14416
Tune >> NewBestE: 0.14416
Tune >> CalculatedE is good -------- !!
Tune >> param to optimize: knight
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 299], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14444
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 100], ['knight', 297], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14539
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 297], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14510
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 100], ['knight', 297], ['bishop', 298]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14425
Tune >> CalculatedE is not good ----- ?
Tune >> param cycle: 5
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 101], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14557
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: pawn
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 99], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14561
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 99], ['knight', 299], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14535
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 99], ['knight', 297], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14416
Tune >> Calculate E
Tune >> CalculatedE: 0.14404
Tune >> NewBestE: 0.14404
Tune >> CalculatedE is good -------- !!
Tune >> param to optimize: bishop
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 99], ['knight', 297], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14599
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 99], ['knight', 297], ['bishop', 298]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14518
Tune >> CalculatedE is not good ----- ?
Tune >> param cycle: 6
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 297], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14426
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: pawn
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 98], ['knight', 297], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14437
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 98], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14475
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: knight
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 98], ['knight', 296], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14519
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 98], ['knight', 296], ['bishop', 300]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14522
Tune >> CalculatedE is not good ----- ?
Tune >> param to optimize: bishop
Tune >> delta: -1
Tune >> paramSet to try: [['pawn', 98], ['knight', 296], ['bishop', 298]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14404
Tune >> Calculate E
Tune >> CalculatedE: 0.14477
Tune >> CalculatedE is not good ----- ?
Tune >> param cycle: 7
Tune >> bestE has not been improved, exiting the tuning now ...
bestE : 0.14404
bestParam : [['pawn', 99], ['knight', 297], ['bishop', 299]]
Elapsed : 18s
bestE : 0.14404
bestParam : [['pawn', 99], ['knight', 297], ['bishop', 299]]
In certain situation you can stop the tuning and record the best so far.
Remember the best error and param.
When you resume the tuning, you can now use the last best tuning info.
So your next input run for example would be:
Code: Select all
bestESTart =0.14404
paramToOptimize = [
['pawn', 99],
['knight', 297],
['bishop', 299]
]
Code: Select all
Tune >> param cycle: 4
Tune >> param to optimize: pawn
Tune >> delta: +1
Tune >> paramSet to try: [['pawn', 100], ['knight', 298], ['bishop', 299]]
Tune >> Send this set to engine
Tune >> lastBestE: 0.14419
Tune >> Calculate E
Tune >> CalculatedE: 0.14416
Tune >> NewBestE: 0.14416
Tune >> CalculatedE is good -------- !!
[['pawn', 100], ['knight', 298], ['bishop', 299]]
You can use that to test your engine in actual matches, this is called sampling. It can happen that those values may actually perform in actual game test.
It is good to experiment varied delta array like,
Code: Select all
delta = [+1, -1, +2, -2, +3, -3]
Code: Select all
paramToOptimize = [
['pawn', 100],
['knight', 300],
['bishop', 300],
['rook', 500],
['queen', 1000]
]