-Value函数拟合算法01-StateValue估算：TD函数逼近算法06【线性逼近器/TD-Linear算法总结与分析】)
三、Summary of the storySummary of the story首先从一个objective function出发J ( w ) = E [ ( v π ( S ) − v ^ ( S , w ) ) 2 ] \begin{aligned}J(w)=\mathbb{E}[(v_\pi(S)-\hat{v}(S,w))^2]\end{aligned}J(w)=E[(vπ(S)−v^(S,w))2]这个目标函数表明这是一个policy evaluation问题.然后对这个objective function进行优化，优化方法使用gradient-descent algorithm:w t + 1 = w t + α t ( v π ( s t ) − v ^ ( s t , w t ) ) ∇ w v ^ ( s t , w t ) w_{t+1}=w_t\:+\alpha_t(v_\pi(s_t)-\hat{v}(s_t,w_t))\nabla_w\hat{v}(s_t,w_t)wt+1=wt+αt(vπ(st)−v^(st,wt))∇wv^(st,wt)但是问题是里边有一个v π ( s t ) v_{\pi}\left(s_{t}\right)v
