Whiskers wrote: ↑Fri Feb 24, 2023 10:46 pm
algerbrex wrote: ↑Fri Feb 24, 2023 6:57 pm
JoAnnP38 wrote: ↑Thu Feb 23, 2023 2:35 pm
This is much too large to be used directly, ...
Just a note. If you haven't already started optimizing your tuner, you'll be happy to know there is definitely room for improvement
I've managed to get Blunder's tuner to comfortably handle 1M+ positions when tuning, and it finishes even several thousand iterations in a reasonable amount of time (10-20 minutes).
The biggest improvement is to use a sparse array for the feature vector of a position. If you're not sure what this all means, I'll be happy to explain more because I'm not entirely sure about all your implementation details and how idiosyncratic they may be.
Forgive me if I'm getting this totally wrong, but does this basically mean
1. reformatting your dataset so that instead of FENs, it contains the weights that apply to the evaluation of that particular position
2. rewriting your evaluation function so that instead of using a board it just uses the weights to make an evaluation? something like (for i = 0; i < N; i++){evl += params{i} * weights{i} ; )}
Actually, you don't need to reformat your data set! Your dataset can stay exactly the same.
As you wrote for your second point, that approach is commonly used to model the evaluation just by using two vectors. The set of evaluation weights are treated as one long vector, and then the challenge is to find a feature vector for each position such that the
dot-product (pairwise element multiplication, and then adding each product together) between the weight vector and this feature vector give you the correct evaluation score. You don't have to do model the evaluation this way, but it makes computing the gradients fairly straightforward.
The feature isn't terribly hard to figure out how to compute, but it defintely takes some thought how to compute it. The way I've figured out how to compute mine for this current version of Blunder I'm writing is to basically sit down and break down the main expression for my evaluation (mgPhase * mgScore + egPhase * egScore), and then asking myself how I can rework this expression to be the dot product between a weight and feature vector. Let me know if you want to elaborate on this more.
Once you've figured out how to compute this feature vector, you'll likely realize that the majority of the entries in the vector are 0's. This wastes time when computing the dot product since that specfic index with contribute nothing to the final product. So what you can do instead is pre-process this features vector: Go through and pick out every index with a non-zero entry, and use a tuple like data structure to add the non-zero entry along with the index it occured at, to a new array.
To compute the dot product you can then just iterate over this feature array, which will be much smaller, and peform the dot product by multiplying the value of each feature with the weight at the index you saved that was paired with the feature's value. In my code this looks something like this:
Code: Select all
func computeDotProduct(weights []float64, features []Feature) (sum float64) {
for _, feature := range features {
sum += weights[feature.Index] * feature.Value
}
return sum
}
Where Feature looks like:
Code: Select all
type Feature struct {
Value float64
Index uint16
}
