|
SHOGUN
4.1.0
|
The class implements the stochastic gradient descend (SGD) minimizer.
A good introduction to SGD can be found at http://cs231n.github.io/neural-networks-3/#sgd
在文件 SGDMinimizer.h 第 45 行定义.
Public 成员函数 | |
| SGDMinimizer () | |
| SGDMinimizer (FirstOrderStochasticCostFunction *fun) | |
| virtual | ~SGDMinimizer () |
| virtual float64_t | minimize () |
| virtual bool | supports_batch_update () const |
| virtual void | set_gradient_updater (DescendUpdater *gradient_updater) |
| virtual void | set_number_passes (int32_t num_passes) |
| virtual void | load_from_context (CMinimizerContext *context) |
| virtual void | set_learning_rate (LearningRate *learning_rate) |
| virtual int32_t | get_iteration_counter () |
| virtual void | set_cost_function (FirstOrderCostFunction *fun) |
| virtual CMinimizerContext * | save_to_context () |
| virtual void | set_penalty_weight (float64_t penalty_weight) |
| virtual void | set_penalty_type (Penalty *penalty_type) |
Protected 成员函数 | |
| virtual void | init_minimization () |
| virtual void | do_proximal_operation (SGVector< float64_t >variable_reference) |
| virtual void | update_context (CMinimizerContext *context) |
| virtual float64_t | get_penalty (SGVector< float64_t > var) |
| virtual void | update_gradient (SGVector< float64_t > gradient, SGVector< float64_t > var) |
Protected 属性 | |
| DescendUpdater * | m_gradient_updater |
| int32_t | m_num_passes |
| int32_t | m_cur_passes |
| int32_t | m_iter_counter |
| LearningRate * | m_learning_rate |
| FirstOrderCostFunction * | m_fun |
| Penalty * | m_penalty_type |
| float64_t | m_penalty_weight |
| SGDMinimizer | ( | ) |
Default constructor
在文件 SGDMinimizer.cpp 第 36 行定义.
|
virtual |
Destructor
在文件 SGDMinimizer.cpp 第 42 行定义.
|
protectedvirtualinherited |
Do proximal update in place
| variable_reference | variable_reference to be updated |
在文件 FirstOrderStochasticMinimizer.h 第 167 行定义.
|
virtualinherited |
How many samples/mini-batch does the minimizer use?
在文件 FirstOrderStochasticMinimizer.h 第 160 行定义.
Get the penalty given target variables For L2 penalty, the target variable is \(w\) and the value of penalty is \(\lambda \frac{w^t w}{2}\), where \(\lambda\) is the weight of penalty
| var | the variable used in regularization |
在文件 FirstOrderMinimizer.h 第 164 行定义.
|
protectedvirtual |
|
virtualinherited |
Load the given context object to restores mutable variables Usually it is used in deserialization.
| context | a context object |
重载 FirstOrderMinimizer .
被 SMDMinimizer , 以及 SMIDASMinimizer 重载.
在文件 FirstOrderStochasticMinimizer.h 第 136 行定义.
|
virtual |
Do minimization and get the optimal value
实现了 FirstOrderStochasticMinimizer.
在文件 SGDMinimizer.cpp 第 52 行定义.
|
virtualinherited |
Return a context object which stores mutable variables Usually it is used in serialization.
被 LBFGSMinimizer , 以及 NLOPTMinimizer 重载.
在文件 FirstOrderMinimizer.h 第 103 行定义.
|
virtualinherited |
|
virtualinherited |
Set a gradient updater
| gradient_updater | the gradient_updater |
在文件 FirstOrderStochasticMinimizer.h 第 104 行定义.
|
virtualinherited |
Set the learning rate of a minimizer
| learning_rate | learn rate or step size |
在文件 FirstOrderStochasticMinimizer.h 第 151 行定义.
|
virtualinherited |
Set the number of times to go through all data points (samples) For example, num_passes=1 means go through all data points once.
Recall that a stochastic cost function \(f(w)\) can be written as \(\sum_i{ f_i(w) }\), where \(f_i(w)\) is the differentiable function for the i-th sample.
| num_passes | the number of times |
在文件 FirstOrderStochasticMinimizer.h 第 125 行定义.
|
virtualinherited |
Set the type of penalty For example, L2 penalty
| penalty_type | the type of penalty. If NULL is given, regularization is not enabled. |
在文件 FirstOrderMinimizer.h 第 137 行定义.
|
virtualinherited |
Set the weight of penalty
| penalty_weight | the weight of penalty, which is positive |
在文件 FirstOrderMinimizer.h 第 126 行定义.
|
virtualinherited |
Does minimizer support batch update
实现了 FirstOrderMinimizer.
在文件 FirstOrderStochasticMinimizer.h 第 98 行定义.
|
protectedvirtualinherited |
Update a context object to store mutable variables
| context | a context object |
重载 FirstOrderMinimizer .
被 SMDMinimizer , 以及 SMIDASMinimizer 重载.
在文件 FirstOrderStochasticMinimizer.h 第 187 行定义.
|
protectedvirtualinherited |
Add gradient of the penalty wrt target variables to unpenalized gradient For least sqaure with L2 penalty,
\[ L2f(w)=f(w) + L2(w) \]
where \( f(w)=\sum_i{(y_i-w^T x_i)^2}\) is the least sqaure cost function and \(L2(w)=\lambda \frac{w^t w}{2}\) is the L2 penalty
Target variables is \(w\) Unpenalized gradient is \(\frac{\partial f(w) }{\partial w}\) Gradient of the penalty wrt target variables is \(\frac{\partial L2(w) }{\partial w}\)
| gradient | unpenalized gradient wrt its target variable |
| var | the target variable |
在文件 FirstOrderMinimizer.h 第 190 行定义.
|
protectedinherited |
current iteration to go through data
在文件 FirstOrderStochasticMinimizer.h 第 215 行定义.
|
protectedinherited |
Cost function
在文件 FirstOrderMinimizer.h 第 205 行定义.
|
protectedinherited |
the gradient update step
在文件 FirstOrderStochasticMinimizer.h 第 200 行定义.
|
protectedinherited |
number of used samples/mini-batches
在文件 FirstOrderStochasticMinimizer.h 第 218 行定义.
|
protectedinherited |
learning_rate object
在文件 FirstOrderStochasticMinimizer.h 第 221 行定义.
|
protectedinherited |
iteration to go through data
在文件 FirstOrderStochasticMinimizer.h 第 212 行定义.
|
protectedinherited |
the type of penalty
在文件 FirstOrderMinimizer.h 第 208 行定义.
|
protectedinherited |
the weight of penalty
在文件 FirstOrderMinimizer.h 第 211 行定义.
