13.1. The SUNMemoryHelper API

This API consists of three new SUNDIALS types: SUNMemoryType, SUNMemory, and SUNMemoryHelper:

typedef struct SUNMemory_ *SUNMemory

The SUNMemory type is a pointer a structure containing a pointer to actual data (ptr), the data memory type, and a flag indicating ownership of that data pointer. This structure is defined as

struct SUNMemory_
{
  void*         ptr;
  SUNMemoryType type;
  sunbooleantype   own;
  size_t        bytes;
};

SUNMemory SUNMemoryNewEmpty(SUNContext sunctx)

This function returns an empty SUNMemory object.

Arguments:

sunctx – the SUNContext object.

Returns:

an uninitialized SUNMemory object

Changed in version 7.0.0: The function signature was updated to add the SUNContext argument.

enum SUNMemoryType

The SUNMemoryType type is an enumeration that defines the supported memory types:

typedef enum
{
  SUNMEMTYPE_HOST,      /* pageable memory accessible on the host     */
  SUNMEMTYPE_PINNED,    /* page-locked memory accesible on the host   */
  SUNMEMTYPE_DEVICE,    /* memory accessible from the device          */
  SUNMEMTYPE_UVM        /* memory accessible from the host or device  */
} SUNMemoryType;

typedef struct SUNMemoryHelper_ *SUNMemoryHelper

The SUNMemoryHelper type is a pointer to a structure containing a pointer to the implementation-specific member data (content) and a virtual method table of member functions (ops). This strucutre is defined as

struct SUNMemoryHelper_
{
  void*               content;
  SUNMemoryHelper_Ops ops;
  SUNContext          sunctx;
};

typedef struct SUNMemoryHelper_Ops_ *SUNMemoryHelper_Ops

The SUNMemoryHelper_Ops type is defined as a pointer to the structure containing the function pointers to the member function implementations. This structure is define as

struct SUNMemoryHelper_Ops_
{
  /* operations that implementations are required to provide */
  SUNErrCode (*alloc)(SUNMemoryHelper, SUNMemory* memptr size_t mem_size,
               SUNMemoryType mem_type, void* queue);
  SUNErrCode (*dealloc)(SUNMemoryHelper, SUNMemory mem, void* queue);
  SUNErrCode (*copy)(SUNMemoryHelper, SUNMemory dst, SUNMemory src,
              size_t mem_size, void* queue);

  /* operations that provide default implementations */
  SUNErrCode (*copyasync)(SUNMemoryHelper, SUNMemory dst,
                          SUNMemory src, size_t mem_size, void* queue);
  SUNErrCode (*getallocstats)(SUNMemoryHelper, SUNMemoryType mem_type, unsigned long* num_allocations,
                              unsigned long* num_deallocations, size_t* bytes_allocated,
                              size_t* bytes_high_watermark);
  SUNMemoryHelper (*clone)(SUNMemoryHelper);
  SUNErrCode      (*destroy)(SUNMemoryHelper);
};

13.1.1. Implementation defined operations

The SUNMemory API defines the following operations that an implementation to must define:

SUNMemory SUNMemoryHelper_Alloc(SUNMemoryHelper helper, SUNMemory *memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

helper – the SUNMemoryHelper object.
memptr – pointer to the allocated SUNMemory.
mem_size – the size in bytes of the ptr.
mem_type – the SUNMemoryType of the ptr.
queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

A new SUNMemory object.

SUNErrCode SUNMemoryHelper_Dealloc(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

helper – the SUNMemoryHelper object.
mem – the SUNMemory object.
queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_Copy(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object should use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

A SUNErrCode indicating success or failure.

13.1.2. Utility Functions

The SUNMemoryHelper API defines the following functions which do not require a SUNMemoryHelper instance:

SUNMemory SUNMemoryHelper_Alias(SUNMemoryHelper helper, SUNMemory mem1)

Returns a SUNMemory object whose ptr field points to the same address as mem1. The new object will not have ownership of ptr, therefore, it will not free ptr when SUNMemoryHelper_Dealloc() is called.

Arguments:

helper – a SUNMemoryHelper object.
mem1 – a SUNMemory object.

Returns:

A SUNMemory object or NULL if an error occurs.

Changed in version 7.0.0: The SUNMemoryHelper argument was added to the function signature.

SUNMemory SUNMemoryHelper_Wrap(SUNMemoryHelper helper, void *ptr, SUNMemoryType mem_type)

Returns a SUNMemory object whose ptr field points to the ptr argument passed to the function. The new object will not have ownership of ptr, therefore, it will not free ptr when SUNMemoryHelper_Dealloc() is called.

Arguments:

helper – a SUNMemoryHelper object.
ptr – the data pointer to wrap in a SUNMemory object.
mem_type – the SUNMemoryType of the ptr.

Returns:

A SUNMemory object or NULL if an error occurs.

Changed in version 7.0.0: The SUNMemoryHelper argument was added to the function signature.

SUNMemoryHelper SUNMemoryHelper_NewEmpty(SUNContext sunctx)

Returns an empty SUNMemoryHelper. This is useful for building custom SUNMemoryHelper implementations.

Arguments:

helper – a SUNMemoryHelper object.

Returns:

A SUNMemoryHelper object or NULL if an error occurs.

Changed in version 7.0.0: The SUNMemoryHelper argument was added to the function signature.

SUNErrCode SUNMemoryHelper_CopyOps(SUNMemoryHelper src, SUNMemoryHelper dst)

Copies the ops field of src to the ops field of dst. This is useful for building custom SUNMemoryHelper implementations.

Arguments:

src – the object to copy from.
dst – the object to copy to.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_GetAllocStats(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about the allocations performed with the helper.

Arguments:

helper – the SUNMemoryHelper object.
mem_type – the SUNMemoryType to get stats for.
num_allocations – (output argument) number of allocations done through the helper.
num_deallocations – (output argument) number of deallocations done through the helper.
bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.
bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

A SUNErrCode indicating success or failure.

13.1.3. Implementation overridable operations with defaults

In addition, the SUNMemoryHelper API defines the following optionally overridable operations which an implementation may define:

SUNErrCode SUNMemoryHelper_CopyAsync(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object should use the memory types of dst and src to determine the appropriate transfer type necessary. The ctx argument is used when a different execution stream needs to be provided to perform the copy in, e.g. with CUDA this would be a cudaStream_t.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

An int flag indicating success (zero) or failure (non-zero).

Note

If this operation is not defined by the implementation, then SUNMemoryHelper_Copy() will be used.

SUNMemoryHelper SUNMemoryHelper_Clone(SUNMemoryHelper helper)

Clones the SUNMemoryHelper object itself.

Arguments:

helper – the SUNMemoryHelper object to clone.

Returns:

A SUNMemoryHelper object.

Note

If this operation is not defined by the implementation, then the default clone will only copy the SUNMemoryHelper_Ops structure stored in helper->ops, and not the helper->content field.

SUNErrCode SUNMemoryHelper_Destroy(SUNMemoryHelper helper)

Destroys (frees) the SUNMemoryHelper object itself.

Arguments:

helper – the SUNMemoryHelper object to destroy.

Returns:

A SUNErrCode indicating success or failure.

Note

If this operation is not defined by the implementation, then the default destroy will only free the helper->ops field and the helper itself. The helper->content field will not be freed.

13.1.4. Implementing a custom SUNMemoryHelper

A particular implementation of the SUNMemoryHelper API must:

Define and implement the required operations. Note that the names of these routines should be unique to that implementation in order to permit using more than one SUNMemoryHelper module in the same code.
Optionally, specify the content field of SUNMemoryHelper.
Optionally, define and implement additional user-callable routines acting on the newly defined SUNMemoryHelper.

An example of a custom SUNMemoryHelper is given in examples/utilities/custom_memory_helper.h.

13.2. The SUNMemoryHelper_Cuda Implementation

The SUNMemoryHelper_Cuda module is an implementation of the SUNMemoryHelper API that interfaces to the NVIDIA [5] library. The implementation defines the constructor

SUNMemoryHelper SUNMemoryHelper_Cuda(SUNContext sunctx): Allocates and returns a SUNMemoryHelper object for handling CUDA memory if successful. Otherwise it returns NULL.

13.2.1. SUNMemoryHelper_Cuda API Functions

The implementation provides the following operations defined by the SUNMemoryHelper API:

SUNMemory SUNMemoryHelper_Alloc_Cuda(SUNMemoryHelper helper, SUNMemory memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

helper – the SUNMemoryHelper object.
memptr – pointer to the allocated SUNMemory.
mem_size – the size in bytes of the ptr.
mem_type – the SUNMemoryType of the ptr. Supported values are:
- SUNMEMTYPE_HOST – memory is allocated with a call to malloc.
- SUNMEMTYPE_PINNED – memory is allocated with a call to cudaMallocHost.
- SUNMEMTYPE_DEVICE – memory is allocated with a call to cudaMalloc.
- SUNMEMTYPE_UVM – memory is allocated with a call to cudaMallocManaged.
queue – currently unused.

Returns:

A new SUNMemory object.

SUNErrCode SUNMemoryHelper_Dealloc_Cuda(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

helper – the SUNMemoryHelper object.
mem – the SUNMemory object.
queue – currently unused.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_Copy_Cuda(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – currently unused.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_CopyAsync_Cuda(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – the cudaStream_t handle for the stream that the copy will be performed on.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_GetAllocStats_Cuda(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about memory allocations performed with the helper.

Arguments:

helper – the SUNMemoryHelper object.
mem_type – the SUNMemoryType to get stats for.
num_allocations – (output argument) number of memory allocations done through the helper.
num_deallocations – (output argument) number of memory deallocations done through the helper.
bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.
bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

A SUNErrCode indicating success or failure.

13.3. The SUNMemoryHelper_Hip Implementation

The SUNMemoryHelper_Hip module is an implementation of the SUNMemoryHelper API that interfaces to the AMD ROCm HIP library [2]. The implementation defines the constructor

SUNMemoryHelper SUNMemoryHelper_Hip(SUNContext sunctx): Allocates and returns a SUNMemoryHelper object for handling HIP memory if successful. Otherwise it returns NULL.

13.3.1. SUNMemoryHelper_Hip API Functions

The implementation provides the following operations defined by the SUNMemoryHelper API:

SUNMemory SUNMemoryHelper_Alloc_Hip(SUNMemoryHelper helper, SUNMemory memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

helper – the SUNMemoryHelper object.
memptr – pointer to the allocated SUNMemory.
mem_size – the size in bytes of the ptr.
mem_type – the SUNMemoryType of the ptr. Supported values are:
- SUNMEMTYPE_HOST – memory is allocated with a call to malloc.
- SUNMEMTYPE_PINNED – memory is allocated with a call to hipMallocHost.
- SUNMEMTYPE_DEVICE – memory is allocated with a call to hipMalloc.
- SUNMEMTYPE_UVM – memory is allocated with a call to hipMallocManaged.
queue – currently unused.

Returns:

An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_Dealloc_Hip(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

helper – the SUNMemoryHelper object.
mem – the SUNMemory object.

Returns:

An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_Copy_Hip(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.

Returns:

An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_CopyAsync_Hip(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – the hipStream_t handle for the stream that the copy will be performed on.

Returns:

An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_GetAllocStats_Hip(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about memory allocations performed with the helper.

Arguments:

helper – the SUNMemoryHelper object.
mem_type – the SUNMemoryType to get stats for.
num_allocations – (output argument) number of memory allocations done through the helper.
num_deallocations – (output argument) number of memory deallocations done through the helper.
bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.
bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

An int flag indicating success (zero) or failure (non-zero).

13.4. The SUNMemoryHelper_Sycl Implementation

The SUNMemoryHelper_Sycl module is an implementation of the SUNMemoryHelper API that interfaces to the SYCL abstraction layer. The implementation defines the constructor

SUNMemoryHelper SUNMemoryHelper_Sycl(SUNContext sunctx): Allocates and returns a SUNMemoryHelper object for handling SYCL memory using the provided queue. Otherwise it returns NULL.

13.4.1. SUNMemoryHelper_Sycl API Functions

The implementation provides the following operations defined by the SUNMemoryHelper API:

SUNMemory SUNMemoryHelper_Alloc_Sycl(SUNMemoryHelper helper, SUNMemory memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

helper – the SUNMemoryHelper object.
memptr – pointer to the allocated SUNMemory.
mem_size – the size in bytes of the ptr.
mem_type – the SUNMemoryType of the ptr. Supported values are:
- SUNMEMTYPE_HOST – memory is allocated with a call to malloc.
- SUNMEMTYPE_PINNED – memory is allocated with a call to sycl::malloc_host.
- SUNMEMTYPE_DEVICE – memory is allocated with a call to sycl::malloc_device.
- SUNMEMTYPE_UVM – memory is allocated with a call to sycl::malloc_shared.
queue – the sycl::queue handle for the stream that the allocation will be performed on.

Returns:

A new SUNMemory object.

SUNErrCode SUNMemoryHelper_Dealloc_Sycl(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

helper – the SUNMemoryHelper object.
mem – the SUNMemory object.
queue – the sycl::queue handle for the queue that the deallocation will be performed on.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_Copy_Sycl(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – the sycl::queue handle for the queue that the copy will be performed on.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_CopyAsync_Sycl(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

helper – the SUNMemoryHelper object.
dst – the destination memory to copy to.
src – the source memory to copy from.
mem_size – the number of bytes to copy.
queue – the sycl::queue handle for the queue that the copy will be performed on.

Returns:

A SUNErrCode indicating success or failure.

SUNErrCode SUNMemoryHelper_GetAllocStats_Sycl(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about memory allocations performed with the helper.

Arguments:

helper – the SUNMemoryHelper object.
mem_type – the SUNMemoryType to get stats for.
num_allocations – (output argument) number of memory allocations done through the helper.
num_deallocations – (output argument) number of memory deallocations done through the helper.
bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.
bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

A SUNErrCode indicating success or failure.