12.7. The SUNNonlinSol_Newton implementation

This section describes the SUNNonlinSol implementation of Newton’s method. To access the SUNNonlinSol_Newton module, include the header file sunnonlinsol/sunnonlinsol_newton.h. We note that the SUNNonlinSol_Newton module is accessible from SUNDIALS integrators without separately linking to the libsundials_sunnonlinsolnewton module library.

12.7.1. SUNNonlinSol_Newton description

To find the solution to

(12.9)\[F(y) = 0\]

given an initial guess \(y^{(0)}\), Newton’s method computes a series of approximate solutions

\[y^{(m+1)} = y^{(m)} + \delta^{(m+1)}\]

where \(m\) is the Newton iteration index, and the Newton update \(\delta^{(m+1)}\) is the solution of the linear system

(12.10)\[A(y^{(m)}) \delta^{(m+1)} = -F(y^{(m)}) \, ,\]

in which \(A\) is the Jacobian matrix

(12.11)\[A \equiv \partial F / \partial y \, .\]

Depending on the linear solver used, the SUNNonlinSol_Newton module will employ either a Modified Newton method or an Inexact Newton method [10, 15, 31, 33, 61]. When used with a direct linear solver, the Jacobian matrix \(A\) is held constant during the Newton iteration, resulting in a Modified Newton method. With a matrix-free iterative linear solver, the iteration is an Inexact Newton method.

In both cases, calls to the integrator-supplied SUNNonlinSolLSetupFn function are made infrequently to amortize the increased cost of matrix operations (updating \(A\) and its factorization within direct linear solvers, or updating the preconditioner within iterative linear solvers). Specifically, SUNNonlinSol_Newton will call the SUNNonlinSolLSetupFn function in two instances:

when requested by the integrator (the input callLSetSetup is SUNTRUE) before attempting the Newton iteration, or
when reattempting the nonlinear solve after a recoverable failure occurs in the Newton iteration with stale Jacobian information (jcur is SUNFALSE). In this case, SUNNonlinSol_Newton will set jbad to SUNTRUE before calling the SUNNonlinSolLSetupFn() function.

Whether the Jacobian matrix \(A\) is fully or partially updated depends on logic unique to each integrator-supplied SUNNonlinSolSetupFn routine. We refer to the discussion of nonlinear solver strategies provided in the package-specific Mathematics section of the documentation for details.

The default maximum number of iterations and the stopping criteria for the Newton iteration are supplied by the SUNDIALS integrator when SUNNonlinSol_Newton is attached to it. Both the maximum number of iterations and the convergence test function may be modified by the user by calling the SUNNonlinSolSetMaxIters() and/or SUNNonlinSolSetConvTestFn() functions after attaching the SUNNonlinSol_Newton object to the integrator.

12.7.2. SUNNonlinSol_Newton functions

The SUNNonlinSol_Newton module provides the following constructor for creating the SUNNonlinearSolver object.

SUNNonlinearSolver SUNNonlinSol_Newton(N_Vector y, SUNContext sunctx)

This creates a SUNNonlinearSolver object for use with SUNDIALS integrators to solve nonlinear systems of the form \(F(y) = 0\) using Newton’s method.

Arguments:

y – a template for cloning vectors needed within the solver.
sunctx – the SUNContext object (see §2.1)

Return value:

A SUNNonlinSol object if the constructor exits successfully, otherwise it will be NULL.

The SUNNonlinSol_Newton module implements all of the functions defined in §12.1.1–§12.1.3 except for SUNNonlinSolSetup(). The SUNNonlinSol_Newton functions have the same names as those defined by the generic SUNNonlinSol API with _Newton appended to the function name. Unless using the SUNNonlinSol_Newton module as a standalone nonlinear solver the generic functions defined in §12.1.1–§12.1.3 should be called in favor of the SUNNonlinSol_Newton-specific implementations.

The SUNNonlinSol_Newton module also defines the following user-callable function.

int SUNNonlinSolGetSysFn_Newton(SUNNonlinearSolver NLS, SUNNonlinSolSysFn *SysFn)

This returns the residual function that defines the nonlinear system.

Arguments:

NLS – a SUNNonlinSol object.
SysFn – the function defining the nonlinear system.

Return value:

The return value should be zero for a successful call, and a negative value for a failure.

Notes:

This function is intended for users that wish to evaluate the nonlinear residual in a custom convergence test function for the SUNNonlinSol_Newton module. We note that SUNNonlinSol_Newton will not leverage the results from any user calls to SysFn.

int SUNNonlinSolSetInfoFile_Newton(SUNNonlinearSolver NLS, FILE *info_file)

This sets the output file where all informative (non-error) messages should be directed.

Arguments:

NLS – a SUNNonlinSol object.
info_file – pointer to output file (stdout by default);
a NULL input will disable output.

Return value:

SUN_NLS_SUCCESS if successful.
SUN_NLS_MEM_NULL if the SUNNonlinSol memory was NULL.
SUN_NLS_ILL_INPUT if SUNDIALS was not built with monitoring enabled.

Notes:

This function is intended for users that wish to monitor the nonlinear solver progress. By default, the file pointer is set to stdout.

Warning

SUNDIALS must be built with the CMake option SUNDIALS_LOGGING_LEVEL >= 3 to utilize this function. See §14.1.2 for more information.

Deprecated since version 6.2.0: Use SUNLogger_SetInfoFilename() instead.

int SUNNonlinSolSetPrintLevel_Newton(SUNNonlinearSolver NLS, int print_level)

This specifies the level of verbosity of the output.

Arguments:

NLS – a SUNNonlinSol object.
print_level – flag indicating level of verbosity; must be one of:
- 0, no information is printed (default).
- 1, for each nonlinear iteration the residual norm is printed.

Return value:

SUN_NLS_SUCCESS if successful.
SUN_NLS_MEM_NULL if the SUNNonlinearSolver memory was NULL.
SUN_NLS_ILL_INPUT if SUNDIALS was not built with monitoring enabled, or the print level value was invalid.

Notes:

This function is intended for users that wish to monitor the nonlinear solver progress. By default, the print level is 0.

Warning

SUNDIALS must be built with the CMake option SUNDIALS_BUILD_WITH_MONITORING to utilize this function. See §14.1.2 for more information.

Deprecated since version 6.2.0: Use SUNLogger_SetInfoFilename() instead.

12.7.3. SUNNonlinSol_Newton content

The content field of the SUNNonlinSol_Newton module is the following structure.

struct _SUNNonlinearSolverContent_Newton {

  SUNNonlinSolSysFn      Sys;
  SUNNonlinSolLSetupFn   LSetup;
  SUNNonlinSolLSolveFn   LSolve;
  SUNNonlinSolConvTestFn CTest;

  N_Vector    delta;
  booleantype jcur;
  int         curiter;
  int         maxiters;
  long int    niters;
  long int    nconvfails;
  void*       ctest_data;

  int         print_level;
  FILE*       info_file;
};

These entries of the content field contain the following information:

Sys – the function for evaluating the nonlinear system,
LSetup – the package-supplied function for setting up the linear solver,
LSolve – the package-supplied function for performing a linear solve,
CTest – the function for checking convergence of the Newton iteration,
delta – the Newton iteration update vector,
jcur – the Jacobian status (SUNTRUE = current, SUNFALSE = stale),
curiter – the current number of iterations in the solve attempt,
maxiters – the maximum number of Newton iterations allowed in a solve,
niters – the total number of nonlinear iterations across all solves,
nconvfails – the total number of nonlinear convergence failures across all solves,
ctest_data – the data pointer passed to the convergence test function,
print_level - controls the amount of information to be printed to the info file,
info_file - the file where all informative (non-error) messages will be directed.

12.8. The SUNNonlinSol_FixedPoint implementation

This section describes the SUNNonlinSol implementation of a fixed point (functional) iteration with optional Anderson acceleration. To access the SUNNonlinSol_FixedPoint module, include the header file sunnonlinsol/sunnonlinsol_fixedpoint.h. We note that the SUNNonlinSol_FixedPoint module is accessible from SUNDIALS integrators without separately linking to the libsundials_sunnonlinsolfixedpoint module library.

12.8.1. SUNNonlinSol_FixedPoint description

To find the solution to

(12.12)\[G(y) = y \,\]

given an initial guess \(y^{(0)}\), the fixed point iteration computes a series of approximate solutions

(12.13)\[y^{(n+1)} = G(y^{(n)})\]

where \(n\) is the iteration index. The convergence of this iteration may be accelerated using Anderson’s method [2, 39, 70, 94]. With Anderson acceleration using subspace size \(m\), the series of approximate solutions can be formulated as the linear combination

(12.14)\[y^{(n+1)} = \beta \sum_{i=0}^{m_n} \alpha_i^{(n)} G(y^{(n-m_n+i)}) + (1 - \beta) \sum_{i=0}^{m_n} \alpha_i^{(n)} y_{n-m_n+i}\]

where \(m_n = \min{\{m,n\}}\) and the factors

\[\alpha^{(n)} =(\alpha_0^{(n)}, \ldots, \alpha_{m_n}^{(n)})\]

solve the minimization problem \(\min\limits_\alpha \| F_n \alpha^T \|_2\) under the constraint that \(\sum\limits_{i=0}^{m_n} \alpha_i = 1\) where

\[F_{n} = (f_{n-m_n}, \ldots, f_{n})\]

with \(f_i = G(y^{(i)}) - y^{(i)}\). Due to this constraint, in the limit of \(m=0\) the accelerated fixed point iteration formula (12.14) simplifies to the standard fixed point iteration (12.13).

Following the recommendations made in [94], the SUNNonlinSol_FixedPoint implementation computes the series of approximate solutions as

(12.15)\[y^{(n+1)} = G(y^{(n)})-\sum_{i=0}^{m_n-1} \gamma_i^{(n)} \Delta g_{n-m_n+i} - (1 - \beta) (f(y^{(n)}) - \sum_{i=0}^{m_n-1} \gamma_i^{(n)} \Delta f_{n-m_n+i})\]

with \(\Delta g_i = G(y^{(i+1)}) - G(y^{(i)})\) and where the factors

\[\gamma^{(n)} =(\gamma_0^{(n)}, \ldots, \gamma_{m_n-1}^{(n)})\]

solve the unconstrained minimization problem \(\min\limits_\gamma \| f_n - \Delta F_n \gamma^T \|_2\) where

\[\Delta F_{n} = (\Delta f_{n-m_n}, \ldots, \Delta f_{n-1}),\]

with \(\Delta f_i = f_{i+1} - f_i\). The least-squares problem is solved by applying a QR factorization to \(\Delta F_n = Q_n R_n\) and solving \(R_n \gamma = Q_n^T f_n\).

The acceleration subspace size \(m\) is required when constructing the SUNNonlinSol_FixedPoint object. The default maximum number of iterations and the stopping criteria for the fixed point iteration are supplied by the SUNDIALS integrator when SUNNonlinSol_FixedPoint is attached to it. Both the maximum number of iterations and the convergence test function may be modified by the user by calling SUNNonlinSolSetMaxIters() and SUNNonlinSolSetConvTestFn() after attaching the SUNNonlinSol_FixedPoint object to the integrator.

12.8.2. SUNNonlinSol_FixedPoint functions

The SUNNonlinSol_FixedPoint module provides the following constructor for creating the SUNNonlinearSolver object.

SUNNonlinearSolver SUNNonlinSol_FixedPoint(N_Vector y, int m, SUNContext sunctx)

This creates a SUNNonlinearSolver object for use with SUNDIALS integrators to solve nonlinear systems of the form \(G(y) = y\).

Arguments:

y – a template for cloning vectors needed within the solver.
m – the number of acceleration vectors to use.
sunctx – the SUNContext object (see §2.1)

Return value:

A SUNNonlinSol object if the constructor exits successfully, otherwise it will be NULL.

Since the accelerated fixed point iteration (12.13) does not require the setup or solution of any linear systems, the SUNNonlinSol_FixedPoint module implements all of the functions defined in §12.1.1–§12.1.3 except for the SUNNonlinSolSetup(), SUNNonlinSolSetLSetupFn(), and SUNNonlinSolSetLSolveFn() functions, that are set to NULL. The SUNNonlinSol_FixedPoint functions have the same names as those defined by the generic SUNNonlinSol API with _FixedPoint appended to the function name. Unless using the SUNNonlinSol_FixedPoint module as a standalone nonlinear solver the generic functions defined in §12.1.1–§12.1.3 should be called in favor of the SUNNonlinSol_FixedPoint-specific implementations.

The SUNNonlinSol_FixedPoint module also defines the following user-callable functions.

int SUNNonlinSolGetSysFn_FixedPoint(SUNNonlinearSolver NLS, SUNNonlinSolSysFn *SysFn)

This returns the fixed-point function that defines the nonlinear system.

Arguments:

NLS – a SUNNonlinSol object.
SysFn – the function defining the nonlinear system.

Return value:

The return value is zero for a successful call, and a negative value for a failure.

Notes:

This function is intended for users that wish to evaluate the fixed-point function in a custom convergence test function for the SUNNonlinSol_FixedPoint module. We note that SUNNonlinSol_FixedPoint will not leverage the results from any user calls to SysFn.

int SUNNonlinSolSetDamping_FixedPoint(SUNNonlinearSolver NLS, realtype beta)

This sets the damping parameter \(\beta\) to use with Anderson acceleration. By default damping is disabled i.e., \(\beta = 1.0\).

Arguments:

NLS – a SUNNonlinSol object.
beta – the damping parameter \(0 < \beta \leq 1\).

Return value:

SUN_NLS_SUCCESS if successful.
SUN_NLS_MEM_NULL if NLS was NULL.
SUN_NLS_ILL_INPUT if beta was negative.

Notes:

A beta value should satisfy \(0 < \beta < 1\) if damping is to be used. A value of one or more will disable damping.

int SUNNonlinSolSetInfoFile_FixedPoint(SUNNonlinearSolver NLS, FILE *info_file)

Thissets the output file where all informative (non-error) messages should be directed.

Arguments:

NLS – a SUNNonlinSol object.
info_file – pointer to output file (stdout by default);
a NULL input will disable output.

Return value:

SUN_NLS_SUCCESS if successful.
SUN_NLS_MEM_NULL if NLS was NULL.
SUN_NLS_ILL_INPUT if SUNDIALS was not built with monitoring enabled.

Notes:

This function is intended for users that wish to monitor the nonlinear solver progress. By default, the file pointer is set to stdout.

Warning

SUNDIALS must be built with the CMake option SUNDIALS_LOGGING_LEVEL >= 3 to utilize this function. See §14.1.2 for more information.

Deprecated since version 6.2.0: Use SUNLogger_SetInfoFilename() instead.

int SUNNonlinSolSetPrintLevel_FixedPoint(SUNNonlinearSolver NLS, int print_level)

This specifies the level of verbosity of the output.

Arguments:

NLS – a SUNNonlinSol object.
print_level – flag indicating level of verbosity; must be one of:
- 0, no information is printed (default).
- 1, for each nonlinear iteration the residual norm is printed.

Return value:

SUN_NLS_SUCCESS if successful.
SUN_NLS_MEM_NULL if NLS was NULL.
SUN_NLS_ILL_INPUT if SUNDIALS was not built with monitoring enabled, or the print level value was invalid.

Notes:

This function is intended for users that wish to monitor the nonlinear solver progress. By default, the print level is 0.

Warning

SUNDIALS must be built with the CMake option SUNDIALS_BUILD_WITH_MONITORING to utilize this function. See §14.1.2 for more information.

Deprecated since version 6.2.0: Use SUNLogger_SetInfoFilename() instead.

12.8.3. SUNNonlinSol_FixedPoint content

The content field of the SUNNonlinSol_FixedPoint module is the following structure.

struct _SUNNonlinearSolverContent_FixedPoint {

  SUNNonlinSolSysFn      Sys;
  SUNNonlinSolConvTestFn CTest;

  int          m;
  int         *imap;
  realtype    *R;
  booleantype  damping
  realtype     beta
  realtype    *gamma;
  realtype    *cvals;
  N_Vector    *df;
  N_Vector    *dg;
  N_Vector    *q;
  N_Vector    *Xvecs;
  N_Vector     yprev;
  N_Vector     gy;
  N_Vector     fold;
  N_Vector     gold;
  N_Vector     delta;
  int          curiter;
  int          maxiters;
  long int     niters;
  long int     nconvfails;
  void        *ctest_data;
  int          print_level;
  FILE*        info_file;
};

The following entries of the content field are always allocated:

Sys – function for evaluating the nonlinear system,
CTest – function for checking convergence of the fixed point iteration,
yprev – N_Vector used to store previous fixed-point iterate,
gy – N_Vector used to store \(G(y)\) in fixed-point algorithm,
delta – N_Vector used to store difference between successive fixed-point iterates,
curiter – the current number of iterations in the solve attempt,
maxiters – the maximum number of fixed-point iterations allowed in a solve,
niters – the total number of nonlinear iterations across all solves,
nconvfails – the total number of nonlinear convergence failures across all solves,
ctest_data – the data pointer passed to the convergence test function,
m – number of acceleration vectors,
print_level - controls the amount of information to be printed to the info file, and
info_file - the file where all informative (non-error) messages will be directed.

If Anderson acceleration is requested (i.e., \(m>0\) in the call to SUNNonlinSol_FixedPoint()), then the following items are also allocated within the content field:

imap – index array used in acceleration algorithm (length m),
damping – a flag indicating if damping is enabled,
beta – the damping parameter,
R – small matrix used in acceleration algorithm (length m*m),
gamma – small vector used in acceleration algorithm (length m),
cvals – small vector used in acceleration algorithm (length m+1),
df – array of N_Vectors used in acceleration algorithm (length m),
dg – array of N_Vectors used in acceleration algorithm (length m),
q – array of N_Vectors used in acceleration algorithm (length m),
Xvecs – N_Vector pointer array used in acceleration algorithm (length m+1),
fold – N_Vector used in acceleration algorithm, and
gold – N_Vector used in acceleration algorithm.

12.9. The SUNNonlinSol_PetscSNES implementation

This section describes the SUNNonlinSol interface to the PETSc SNES nonlinear solver(s). To enable the SUNonlinSol_PetscSNES module, SUNDIALS must be configured to use PETSc. Instructions on how to do this are given in §14.1.4.5. To access the SUNNonlinSol_PetscSNES module, include the header file sunnonlinsol/sunnonlinsol_petscsnes.h. The library to link to is libsundials_sunnonlinsolpetsc.lib where .lib is typically .so for shared libaries and .a for static libraries. Users of the SUNNonlinSol_PetscSNES module should also see §9.14 which discusses the NVECTOR interface to the PETSc Vec API.

12.9.1. SUNNonlinSol_PetscSNES description

The SUNNonlinSol_PetscSNES implementation allows users to utilize a PETSc SNES nonlinear solver to solve the nonlinear systems that arise in the SUNDIALS integrators. Since SNES uses the KSP linear solver interface underneath it, the SUNNonlinSol_PetscSNES implementation does not interface with SUNDIALS linear solvers. Instead, users should set nonlinear solver options, linear solver options, and preconditioner options through the PETSc SNES, KSP, and PC APIs.

Important usage notes for the SUNNonlinSol_PetscSNES implementation:

The SUNNonlinSol_PetscSNES implementation handles calling SNESSetFunction at construction. The actual residual function \(F(y)\) is set by the SUNDIALS integrator when the SUNNonlinSol_PetscSNES object is attached to it. Therefore, a user should not call SNESSetFunction on a SNES object that is being used with SUNNonlinSol_PetscSNES. For these reasons it is recommended, although not always necessary, that the user calls SUNNonlinSol_PetscSNES() with the new SNES object immediately after calling SNESCreate.
The number of nonlinear iterations is tracked by SUNDIALS separately from the count kept by SNES. As such, the function SUNNonlinSolGetNumIters() reports the cumulative number of iterations across the lifetime of the SUNNonlinearSolver object.
Some “converged” and “diverged” convergence reasons returned by SNES are treated as recoverable convergence failures by SUNDIALS. Therefore, the count of convergence failures returned by SUNNonlinSolGetNumConvFails() will reflect the number of recoverable convergence failures as determined by SUNDIALS, and may differ from the count returned by SNESGetNonlinearStepFailures.
The SUNNonlinSol_PetscSNES module is not currently compatible with the CVODES or IDAS staggered or simultaneous sensitivity strategies.

12.9.2. SUNNonlinearSolver_PetscSNES functions

The SUNNonlinSol_PetscSNES module provides the following constructor for creating a SUNNonlinearSolver object.

SUNNonlinearSolver SUNNonlinSol_PetscSNES(N_Vector y, SNES snes, SUNContext sunctx)

This creates a SUNNonlinSol object that wraps a PETSc SNES object for use with SUNDIALS. This will call SNESSetFunction on the provided SNES object.

Arguments:

snes – a PETSc SNES object.
y – a N_Vector object of type NVECTOR_PETSC that is used as a template for the residual vector.
sunctx – the SUNContext object (see §2.1)

Return value:

A SUNNonlinSol object if the constructor exits successfully, otherwise it will be NULL.

Warning

This function calls SNESSetFunction and will overwrite whatever function was previously set. Users should not call SNESSetFunction on the SNES object provided to the constructor.

The SUNNonlinSol_PetscSNES module implements all of the functions defined in §12.1.1–§12.1.3 except for SUNNonlinSolSetup(), SUNNonlinSolSetLSetupFn(), SUNNonlinSolSetLSolveFn(), SUNNonlinSolSetConvTestFn(), and SUNNonlinSolSetMaxIters().

The SUNNonlinSol_PetscSNES functions have the same names as those defined by the generic SUNNonlinSol API with _PetscSNES appended to the function name. Unless using the SUNNonlinSol_PetscSNES module as a standalone nonlinear solver the generic functions defined in §12.1.1–§12.1.3 should be called in favor of the SUNNonlinSol_PetscSNES specific implementations.

The SUNNonlinSol_PetscSNES module also defines the following user-callable functions.

int SUNNonlinSolGetSNES_PetscSNES(SUNNonlinearSolver NLS, SNES *snes)

This gets the SNES object that was wrapped.

Arguments:

NLS – a SUNNonlinSol object.
snes – a pointer to a PETSc SNES object that will be set upon return.

Return value:

The return value (of type int) should be zero for a successful call, and a negative value for a failure.

int SUNNonlinSolGetPetscError_PetscSNES(SUNNonlinearSolver NLS, PestcErrorCode *error)

This gets the last error code returned by the last internal call to a PETSc API function.

Arguments:

NLS – a SUNNonlinSol object.
error – a pointer to a PETSc error integer that will be set upon return.

Return value:

The return value (of type int) should be zero for a successful call, and a negative value for a failure.

int SUNNonlinSolGetSysFn_PetscSNES(SUNNonlinearSolver NLS, SUNNonlinSolSysFn *SysFn)

This returns the residual function that defines the nonlinear system.

Arguments:

NLS – a SUNNonlinSol object.
SysFn – the function defining the nonlinear system.

Return value:

The return value (of type int) should be zero for a successful call, and a negative value for a failure.

12.9.3. SUNNonlinearSolver_PetscSNES content

The content field of the SUNNonlinSol_PetscSNES module is the following structure.

struct _SUNNonlinearSolverContent_PetscSNES {
  int sysfn_last_err;
  PetscErrorCode petsc_last_err;
  long int nconvfails;
  long int nni;
  void *imem;
  SNES snes;
  Vec r;
  N_Vector y, f;
  SUNNonlinSolSysFn Sys;
};

These entries of the content field contain the following information:

sysfn_last_err – last error returned by the system defining function,
petsc_last_err – last error returned by PETSc,
nconvfails – number of nonlinear converge failures (recoverable or not),
nni – number of nonlinear iterations,
imem – SUNDIALS integrator memory,
snes – PETSc SNES object,
r – the nonlinear residual,
y – wrapper for PETSc vectors used in the system function,
f – wrapper for PETSc vectors used in the system function,
Sys – nonlinear system definining function.