Gradient Descent Introduction
Posted: Sun Dec 09, 2018 3:32 pm
Hello everybody,
i am interested in a simple gradient descent implementation. Unfortunately i am not able to put the puzzle pieces together.
Here is what i think that i understand and what i can do for now:
base model:
1. i have a sample set of positions including results
2. i have a parameter list with N elements.
3. i have a cost function (MSE)
a. to minimize the cost function, which is a squared function, i need the derivative which leads to a linear model y=mx+b.
b. solved this, i can tune the parameter the way, that y=mx+b gets close to 0.
example:
1. SAMPLESIZE 10000
2. PARAMETERLIST 5
3. MSE = (sum(result-computed_value)^2) / SAMPLESIZE
How do i have to iterate over my parameterlist and the samples to compute m,b ?
Do i have to compute m,b for each single sample ? m,b for the batch ? how do i get m,b for the batch ?
I red some articles on the web, but i am interested in the dialogue and the practice how to handle it in the context chess parameter tuning.
So, i think i got the idea but need to know how to do it.
Thanks a lot in advance...
i am interested in a simple gradient descent implementation. Unfortunately i am not able to put the puzzle pieces together.
Here is what i think that i understand and what i can do for now:
base model:
1. i have a sample set of positions including results
2. i have a parameter list with N elements.
3. i have a cost function (MSE)
a. to minimize the cost function, which is a squared function, i need the derivative which leads to a linear model y=mx+b.
b. solved this, i can tune the parameter the way, that y=mx+b gets close to 0.
example:
1. SAMPLESIZE 10000
2. PARAMETERLIST 5
3. MSE = (sum(result-computed_value)^2) / SAMPLESIZE
How do i have to iterate over my parameterlist and the samples to compute m,b ?
Do i have to compute m,b for each single sample ? m,b for the batch ? how do i get m,b for the batch ?
I red some articles on the web, but i am interested in the dialogue and the practice how to handle it in the context chess parameter tuning.
So, i think i got the idea but need to know how to do it.
Thanks a lot in advance...