Use more consistently p for iterates instead of x and fix a few widehat typos.

Author: kellertuer

src/plans/quasi_newton_plan.jl

@@ -31,7 +31,7 @@ abstract type AbstractQuasiNewtonUpdateRule end
 
 indicates in [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the Riemannian BFGS update is used in the Riemannian quasi-Newton method.
 
-Denote by ``$(_tex(:widetilde, "H"))_k^$(_tex(:rm, "BFGS"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "H"))_k^$(_tex(:rm, "BFGS"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -41,8 +41,8 @@ H^$(_tex(:rm, "BFGS"))_{k+1} = $(_tex(:widetilde, "H"))^$(_tex(:rm, "BFGS"))_k
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, " and "))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, " and "))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively.
@@ -54,7 +54,7 @@ struct BFGS <: AbstractQuasiNewtonUpdateRule end
 
 indicates in [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the inverse Riemannian BFGS update is used in the Riemannian quasi-Newton method.
 
-Denote by ``$(_tex(:widetilde, "B"))_k^$(_tex(:rm, "BFGS"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "B"))_k^$(_tex(:rm, "BFGS"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -70,8 +70,8 @@ $(_tex(:Bigr))) + $(_tex(:frac, "s_k s^{$(_tex(:rm, "T"))}_k", "s^{$(_tex(:rm, "
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively.
@@ -83,7 +83,7 @@ struct InverseBFGS <: AbstractQuasiNewtonUpdateRule end
 
 indicates in an [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the Riemannian DFP update is used in the Riemannian quasi-Newton method.
 
-Denote by ``$(_tex(:widetilde, "H"))_k^$(_tex(:rm, "DFP"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "H"))_k^$(_tex(:rm, "DFP"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -99,8 +99,8 @@ $(_tex(:Bigr))) + $(_tex(:frac, "y_k y^{$(_tex(:rm, "T"))}_k", "s^{$(_tex(:rm, "
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively.
@@ -112,7 +112,7 @@ struct DFP <: AbstractQuasiNewtonUpdateRule end
 
 indicates in [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the inverse Riemannian DFP update is used in the Riemannian quasi-Newton method.
 
-Denote by ``$(_tex(:widetilde, "B"))_k^$(_tex(:rm, "DFP"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "B"))_k^$(_tex(:rm, "DFP"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -123,8 +123,8 @@ B^$(_tex(:rm, "DFP"))_{k+1} = $(_tex(:widetilde, "B"))^$(_tex(:rm, "DFP"))_k + $
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively.
@@ -136,7 +136,7 @@ struct InverseDFP <: AbstractQuasiNewtonUpdateRule end
 
 indicates in [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the Riemannian SR1 update is used in the Riemannian quasi-Newton method.
 
-Denote by ``$(_tex(:widetilde, "H"))_k^{$(_tex(:rm, "SR1"))}`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "H"))_k^{$(_tex(:rm, "SR1"))}`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -153,8 +153,8 @@ H^{$(_tex(:rm, "SR1"))}_{k+1} = $(_tex(:widetilde, "H"))^{$(_tex(:rm, "SR1"))}_k
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively.
@@ -178,7 +178,7 @@ end
 
 indicates in [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the inverse Riemannian SR1 update is used in the Riemannian quasi-Newton method.
 
-Denote by ``$(_tex(:widetilde, "B"))_k^{$(_tex(:rm, "SR1"))}`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "B"))_k^{$(_tex(:rm, "SR1"))}`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -195,8 +195,8 @@ B^{$(_tex(:rm, "SR1"))}_{k+1} = $(_tex(:widetilde, "B"))^{$(_tex(:rm, "SR1"))}_k
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively.
@@ -220,7 +220,7 @@ end
 
 indicates in [`AbstractQuasiNewtonDirectionUpdate`](@ref) that the Riemannian Broyden update is used in the Riemannian quasi-Newton method, which is as a convex combination of [`BFGS`](@ref) and [`DFP`](@ref).
 
-Denote by ``$(_tex(:widetilde, "H"))_k^$(_tex(:rm, "Br"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+Denote by ``$(_tex(:widetilde, "H"))_k^$(_tex(:rm, "Br"))`` the operator concatenated with a vector transport and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -244,8 +244,8 @@ H^{$(_tex(:rm, "Br"))}_{k+1}
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively, and ``φ_k`` is the Broyden factor which is `:constant` by default but can also be set to `:Davidon`.
@@ -267,7 +267,7 @@ is used in the Riemannian quasi-Newton method, which is as a convex combination
 of [`InverseBFGS`](@ref) and [`InverseDFP`](@ref).
 
 Denote by ``$(_tex(:widetilde, "H"))_k^{$(_tex(:rm, "Br"))}`` the operator concatenated with a vector transport
-and its inverse before and after to act on ``x_{k+1} = R_{x_k}(α_k η_k)``.
+and its inverse before and after to act on ``x_{k+1} = R_{p_k}(α_k η_k)``.
 Then the update formula reads
 
 ```math
@@ -295,8 +295,8 @@ B^{$(_tex(:rm, "Br"))}_{k+1}
 where ``s_k`` and ``y_k`` are the coordinate vectors with respect to the current basis (from [`QuasiNewtonState`](@ref)) of
 
 ```math
-T^{S}_{x_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
-$(_tex(:grad))f(x_{k+1}) - T^{S}_{x_k, α_k η_k}($(_tex(:grad))f(x_k)) ∈ T_{x_{k+1}} $(_math(:M)),
+T^{S}_{p_k, α_k η_k}(α_k η_k) $(_tex(:quad))$(_tex(:text, "and"))$(_tex(:quad))
+$(_tex(:grad))f(x_{k+1}) - T^{S}_{p_k, α_k η_k}($(_tex(:grad))f(p_k)) ∈ T_{x_{k+1}} $(_math(:M)),
 ```
 
 respectively, and ``φ_k`` is the Broyden factor which is `:constant` by default but can also be set to `:Davidon`.
@@ -314,12 +314,12 @@ _doc_QN_H_update = "``H_k ↦ H_{k+1}``"
 _doc_QN_B_update = "``B_k ↦ B_{k+1}``"
 _doc_QN_H_full_system = """
 ```math
-$(_tex(:text, "Solve"))$(_tex(:quad))$(_tex(:hat, "η_k")) = - H_k $(_tex(:widehat, "$(_tex(:grad))f(x_k)")),
+$(_tex(:text, "Solve"))$(_tex(:quad))$(_tex(:hat, "η_k")) = - H_k $(_tex(:widehat, "$(_tex(:grad))f(p_k)")),
 ```
 """
 _doc_QN_B_full_system = """
 ```math
-$(_tex(:hat, "η_k")) = - B_k $(_tex(:widehat, "$(_tex(:grad))f(x_k)")),
+$(_tex(:hat, "η_k")) = - B_k $(_tex(:widehat, "$(_tex(:grad))f(p_k)")),
 ```
 """
 
@@ -390,17 +390,17 @@ a basis ``$(_math(:Sequence, "b", "i", "1", "n"))`` are determined by solving a
 $_doc_QN_H_full_system
 
 where ``H_k`` is the matrix representing the operator with respect to the basis ``$(_math(:Sequence, "b", "i", "1", "n"))``
-and ``\\widehat{$(_tex(:grad))} f(p_k)`` represents the coordinates of the gradient of
-the objective function ``f`` in ``x_k`` with respect to the basis ``$(_math(:Sequence, "b", "i", "1", "n"))``.
+and ``$(_tex(:widehat, "$(_tex(:grad)) f(p_k)"))`` represents the coordinates of the gradient of
+the objective function ``f`` in ``p_k`` with respect to the basis ``$(_math(:Sequence, "b", "i", "1", "n"))``.
 If a method is chosen where Hessian inverse is approximated, the coordinates of the search
 direction ``η_k`` with respect to a basis ``$(_math(:Sequence, "b", "i", "1", "n"))`` are obtained simply by
 matrix-vector multiplication
 
 $_doc_QN_B_full_system
 
 where ``B_k`` is the matrix representing the operator with respect to the basis ``$(_math(:Sequence, "b", "i", "1", "n"))``
-and ``\\widehat{$(_tex(:grad))} f(p_k)``. In the end, the search direction ``η_k`` is
-generated from the coordinates ``\\hat{eta_k}`` and the vectors of the basis ``$(_math(:Sequence, "b", "i", "1", "n"))``
+and ``$(_tex(:widehat, "$(_tex(:grad)) f(p_k)"))``. In the end, the search direction ``η_k`` is
+generated from the coordinates ``$(_tex(:hat, "η"))_k`` and the vectors of the basis ``$(_math(:Sequence, "b", "i", "1", "n"))``
 in both variants.
 The [`AbstractQuasiNewtonUpdateRule`](@ref) indicates which quasi-Newton update rule is used.
 In all of them, the Euclidean update formula is used to generate the matrix ``H_{k+1}``
@@ -536,14 +536,14 @@ $(_doc_QN_B)
 
 are used. The two-loop recursion can be understood as that the [`InverseBFGS`](@ref) update
 is executed ``m`` times in a row on ``$(_tex(:Cal, "B"))^{(0)}_k[⋅]`` using the tangent vectors ``$(_tex(:widehat, "s"))_i,$(_tex(:widehat, "y"))_i``,
-and in the same time the resulting operator ``$(_tex(:Cal, "B"))^{$(_tex(:rm, "LRBFGS"))}_k [⋅]`` is directly applied on ``$(_tex(:grad))f(x_k)``.
+and in the same time the resulting operator ``$(_tex(:Cal, "B"))^{$(_tex(:rm, "LRBFGS"))}_k [⋅]`` is directly applied on ``$(_tex(:grad))f(p_k)``.
 When updating there are two cases: if there is still free memory, ``k < m``, the previously
 stored vector pairs ``$(_tex(:widehat, "s"))_i,$(_tex(:widehat, "y"))_i`` have to be
 transported into the upcoming tangent space ``$(_math(:TpM; p = "p_{k+1}"))``.
 If there is no free memory, the oldest pair ``$(_tex(:widehat, "s"))_i,$(_tex(:widehat, "y"))_i``
 has to be discarded and then all the remaining vector pairs ``$(_tex(:widehat, "s"))_i,$(_tex(:widehat, "y"))_i``
 are transported into the tangent space ``$(_math(:TpM; p = "p_{k+1}"))``.
-After that the new values ``s_k = $(_tex(:widehat, "s"))_k = T^{S}_{x_k, α_k η_k}(α_k η_k)`` and ``y_k = $(_tex(:widehat, "y"))_k``
+After that the new values ``s_k = $(_tex(:widehat, "s"))_k = T^{S}_{p_k, α_k η_k}(α_k η_k)`` and ``y_k = $(_tex(:widehat, "y"))_k``
 are stored at the beginning. This process ensures that new information about the objective
 function is always included and the old, probably no longer relevant, information is discarded.
 
@@ -735,7 +735,7 @@ as given in [`QuasiNewtonMatrixDirectionUpdate`](@ref) or [`QuasiNewtonLimitedMe
 butut the update  then is only executed if
 
 ```math
-$(_tex(:frac, "g_{x_{k+1}}(y_k,s_k)", "$(_tex(:norm, "s_k"; index = "x_{k+1}"))^{2}")) ≥ θ $(_tex(:norm, "$(_tex(:grad))f(x_k)"; index = "x_k")),
+$(_tex(:frac, "g_{x_{k+1}}(y_k,s_k)", "$(_tex(:norm, "s_k"; index = "x_{k+1}"))^{2}")) ≥ θ $(_tex(:norm, "$(_tex(:grad))f(p_k)"; index = "p_k")),
 ```
 
 is satisfied, where ``θ`` is a monotone increasing function satisfying ``θ(0) = 0``