14.1. The SUNMemoryHelper API

This API consists of three new SUNDIALS types: SUNMemoryType, SUNMemory, and SUNMemoryHelper:

typedef struct SUNMemory_ *SUNMemory

The SUNMemory type is a pointer the structure

struct SUNMemory_
void *ptr;

The actual data.

SUNMemoryType type;

The data memory type.

sunbooleantype own;

A flag indicating ownership.

size_t bytes;

The size of the data allocated.

SUNMemory SUNMemoryNewEmpty(SUNContext sunctx)

This function returns an empty SUNMemory object.

Arguments:

  • sunctx – the SUNContext object.

Returns:

  • an uninitialized SUNMemory object

Changed in version 7.0.0: The function signature was updated to add the SUNContext argument.

enum SUNMemoryType

The SUNMemoryType type is an enumeration that defines the supported memory types:

enumerator SUNMEMTYPE_HOST

Pageable memory accessible on the host

enumerator SUNMEMTYPE_PINNED

Page-locked memory accessible on the host

enumerator SUNMEMTYPE_DEVICE

Memory accessible from the device

enumerator SUNMEMTYPE_UVM

Memory accessible from the host or device

typedef struct SUNMemoryHelper_ *SUNMemoryHelper

The SUNMemoryHelper type is a pointer to the structure

struct SUNMemoryHelper_
void *content;

Pointer to the implementation-specific member data

SUNMemoryHelper_Ops ops;

A virtual method table of member functions

SUNContext sunctx;

The SUNDIALS simulation context

typedef struct SUNMemoryHelper_Ops_ *SUNMemoryHelper_Ops

The SUNMemoryHelper_Ops type is defined as a pointer to the structure containing the function pointers to the member function implementations. This structure is define as

struct SUNMemoryHelper_Ops_
SUNErrCode (*alloc)(SUNMemoryHelper, SUNMemory *memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

The function implementing SUNMemoryHelper_Alloc()

SUNErrCode (*dealloc)(SUNMemoryHelper, SUNMemory mem, void *queue)

The function implementing SUNMemoryHelper_Dealloc()

SUNErrCode (*copy)(SUNMemoryHelper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

The function implementing SUNMemoryHelper_Copy()

SUNErrCode (*copyasync)(SUNMemoryHelper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

The function implementing SUNMemoryHelper_CopyAsync()

SUNErrCode (*getallocstats)(SUNMemoryHelper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

The function implementing SUNMemoryHelper_GetAllocStats()

SUNMemoryHelper (*clone)(SUNMemoryHelper)

The function implementing SUNMemoryHelper_Clone()

SUNErrCode (*destroy)(SUNMemoryHelper)

The function implementing SUNMemoryHelper_Destroy()

14.1.1. Implementation defined operations

The SUNMemory API defines the following operations that an implementation to must define:

SUNMemory SUNMemoryHelper_Alloc(SUNMemoryHelper helper, SUNMemory *memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

  • helper – the SUNMemoryHelper object.

  • memptr – pointer to the allocated SUNMemory.

  • mem_size – the size in bytes of the ptr.

  • mem_type – the SUNMemoryType of the ptr.

  • queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

SUNErrCode SUNMemoryHelper_Dealloc(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem – the SUNMemory object.

  • queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

SUNErrCode SUNMemoryHelper_Copy(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object should use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

14.1.2. Utility Functions

The SUNMemoryHelper API defines the following functions which do not require a SUNMemoryHelper instance:

SUNMemory SUNMemoryHelper_Alias(SUNMemoryHelper helper, SUNMemory mem1)

Returns a SUNMemory object whose ptr field points to the same address as mem1. The new object will not have ownership of ptr, therefore, it will not free ptr when SUNMemoryHelper_Dealloc() is called.

Arguments:

  • helper – a SUNMemoryHelper object.

  • mem1 – a SUNMemory object.

Returns:

  • A SUNMemory object or NULL if an error occurs.

Changed in version 7.0.0: The SUNMemoryHelper argument was added to the function signature.

SUNMemory SUNMemoryHelper_Wrap(SUNMemoryHelper helper, void *ptr, SUNMemoryType mem_type)

Returns a SUNMemory object whose ptr field points to the ptr argument passed to the function. The new object will not have ownership of ptr, therefore, it will not free ptr when SUNMemoryHelper_Dealloc() is called.

Arguments:

  • helper – a SUNMemoryHelper object.

  • ptr – the data pointer to wrap in a SUNMemory object.

  • mem_type – the SUNMemoryType of the ptr.

Returns:

  • A SUNMemory object or NULL if an error occurs.

Changed in version 7.0.0: The SUNMemoryHelper argument was added to the function signature.

SUNMemoryHelper SUNMemoryHelper_NewEmpty(SUNContext sunctx)

Returns an empty SUNMemoryHelper. This is useful for building custom SUNMemoryHelper implementations.

Arguments:

  • helper – a SUNMemoryHelper object.

Returns:

  • A SUNMemoryHelper object or NULL if an error occurs.

Changed in version 7.0.0: The SUNMemoryHelper argument was added to the function signature.

SUNErrCode SUNMemoryHelper_CopyOps(SUNMemoryHelper src, SUNMemoryHelper dst)

Copies the ops field of src to the ops field of dst. This is useful for building custom SUNMemoryHelper implementations.

Arguments:

  • src – the object to copy from.

  • dst – the object to copy to.

Returns:

SUNErrCode SUNMemoryHelper_GetAllocStats(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about the allocations performed with the helper.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem_type – the SUNMemoryType to get stats for.

  • num_allocations – (output argument) number of allocations done through the helper.

  • num_deallocations – (output argument) number of deallocations done through the helper.

  • bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.

  • bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

14.1.3. Implementation overridable operations with defaults

In addition, the SUNMemoryHelper API defines the following optionally overridable operations which an implementation may define:

SUNErrCode SUNMemoryHelper_CopyAsync(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object should use the memory types of dst and src to determine the appropriate transfer type necessary. The ctx argument is used when a different execution stream needs to be provided to perform the copy in, e.g. with CUDA this would be a cudaStream_t.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – typically a handle for an object representing an alternate execution stream (e.g., a CUDA/HIP stream or SYCL queue), but it can also be any implementation specific data.

Returns:

An int flag indicating success (zero) or failure (non-zero).

Note

If this operation is not defined by the implementation, then SUNMemoryHelper_Copy() will be used.

SUNMemoryHelper SUNMemoryHelper_Clone(SUNMemoryHelper helper)

Clones the SUNMemoryHelper object itself.

Arguments:

  • helper – the SUNMemoryHelper object to clone.

Returns:

  • A SUNMemoryHelper object.

Note

If this operation is not defined by the implementation, then the default clone will only copy the SUNMemoryHelper_Ops structure stored in helper->ops, and not the helper->content field.

SUNErrCode SUNMemoryHelper_Destroy(SUNMemoryHelper helper)

Destroys (frees) the SUNMemoryHelper object itself.

Arguments:

  • helper – the SUNMemoryHelper object to destroy.

Returns:

Note

If this operation is not defined by the implementation, then the default destroy will only free the helper->ops field and the helper itself. The helper->content field will not be freed.

14.1.4. Implementing a custom SUNMemoryHelper

A particular implementation of the SUNMemoryHelper API must:

  • Define and implement the required operations. Note that the names of these routines should be unique to that implementation in order to permit using more than one SUNMemoryHelper module in the same code.

  • Optionally, specify the content field of SUNMemoryHelper.

  • Optionally, define and implement additional user-callable routines acting on the newly defined SUNMemoryHelper.

An example of a custom SUNMemoryHelper is given in examples/utilities/custom_memory_helper.h.

14.2. The SUNMemoryHelper_Cuda Implementation

The SUNMemoryHelper_Cuda module is an implementation of the SUNMemoryHelper API that interfaces to the NVIDIA [5] library. The implementation defines the constructor

SUNMemoryHelper SUNMemoryHelper_Cuda(SUNContext sunctx)

Allocates and returns a SUNMemoryHelper object for handling CUDA memory if successful. Otherwise it returns NULL.

14.2.1. SUNMemoryHelper_Cuda API Functions

The implementation provides the following operations defined by the SUNMemoryHelper API:

SUNMemory SUNMemoryHelper_Alloc_Cuda(SUNMemoryHelper helper, SUNMemory memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

  • helper – the SUNMemoryHelper object.

  • memptr – pointer to the allocated SUNMemory.

  • mem_size – the size in bytes of the ptr.

  • mem_type – the SUNMemoryType of the ptr. Supported values are:

    • SUNMEMTYPE_HOST – memory is allocated with a call to malloc.

    • SUNMEMTYPE_PINNED – memory is allocated with a call to cudaMallocHost.

    • SUNMEMTYPE_DEVICE – memory is allocated with a call to cudaMalloc.

    • SUNMEMTYPE_UVM – memory is allocated with a call to cudaMallocManaged.

  • queue – currently unused.

Returns:

SUNErrCode SUNMemoryHelper_Dealloc_Cuda(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem – the SUNMemory object.

  • queue – currently unused.

Returns:

SUNErrCode SUNMemoryHelper_Copy_Cuda(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – currently unused.

Returns:

SUNErrCode SUNMemoryHelper_CopyAsync_Cuda(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – the cudaStream_t handle for the stream that the copy will be performed on.

Returns:

SUNErrCode SUNMemoryHelper_GetAllocStats_Cuda(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about memory allocations performed with the helper.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem_type – the SUNMemoryType to get stats for.

  • num_allocations – (output argument) number of memory allocations done through the helper.

  • num_deallocations – (output argument) number of memory deallocations done through the helper.

  • bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.

  • bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

14.3. The SUNMemoryHelper_Hip Implementation

The SUNMemoryHelper_Hip module is an implementation of the SUNMemoryHelper API that interfaces to the AMD ROCm HIP library [2]. The implementation defines the constructor

SUNMemoryHelper SUNMemoryHelper_Hip(SUNContext sunctx)

Allocates and returns a SUNMemoryHelper object for handling HIP memory if successful. Otherwise it returns NULL.

14.3.1. SUNMemoryHelper_Hip API Functions

The implementation provides the following operations defined by the SUNMemoryHelper API:

SUNMemory SUNMemoryHelper_Alloc_Hip(SUNMemoryHelper helper, SUNMemory memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

  • helper – the SUNMemoryHelper object.

  • memptr – pointer to the allocated SUNMemory.

  • mem_size – the size in bytes of the ptr.

  • mem_type – the SUNMemoryType of the ptr. Supported values are:

    • SUNMEMTYPE_HOST – memory is allocated with a call to malloc.

    • SUNMEMTYPE_PINNED – memory is allocated with a call to hipMallocHost.

    • SUNMEMTYPE_DEVICE – memory is allocated with a call to hipMalloc.

    • SUNMEMTYPE_UVM – memory is allocated with a call to hipMallocManaged.

  • queue – currently unused.

Returns:

  • An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_Dealloc_Hip(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem – the SUNMemory object.

Returns:

  • An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_Copy_Hip(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

Returns:

  • An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_CopyAsync_Hip(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – the hipStream_t handle for the stream that the copy will be performed on.

Returns:

  • An int flag indicating success (zero) or failure (non-zero).

SUNErrCode SUNMemoryHelper_GetAllocStats_Hip(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about memory allocations performed with the helper.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem_type – the SUNMemoryType to get stats for.

  • num_allocations – (output argument) number of memory allocations done through the helper.

  • num_deallocations – (output argument) number of memory deallocations done through the helper.

  • bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.

  • bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns:

  • An int flag indicating success (zero) or failure (non-zero).

14.4. The SUNMemoryHelper_Sycl Implementation

The SUNMemoryHelper_Sycl module is an implementation of the SUNMemoryHelper API that interfaces to the SYCL abstraction layer. The implementation defines the constructor

SUNMemoryHelper SUNMemoryHelper_Sycl(SUNContext sunctx)

Allocates and returns a SUNMemoryHelper object for handling SYCL memory using the provided queue. Otherwise it returns NULL.

14.4.1. SUNMemoryHelper_Sycl API Functions

The implementation provides the following operations defined by the SUNMemoryHelper API:

SUNMemory SUNMemoryHelper_Alloc_Sycl(SUNMemoryHelper helper, SUNMemory memptr, size_t mem_size, SUNMemoryType mem_type, void *queue)

Allocates a SUNMemory object whose ptr field is allocated for mem_size bytes and is of type mem_type. The new object will have ownership of ptr and will be deallocated when SUNMemoryHelper_Dealloc() is called.

Arguments:

  • helper – the SUNMemoryHelper object.

  • memptr – pointer to the allocated SUNMemory.

  • mem_size – the size in bytes of the ptr.

  • mem_type – the SUNMemoryType of the ptr. Supported values are:

    • SUNMEMTYPE_HOST – memory is allocated with a call to malloc.

    • SUNMEMTYPE_PINNED – memory is allocated with a call to sycl::malloc_host.

    • SUNMEMTYPE_DEVICE – memory is allocated with a call to sycl::malloc_device.

    • SUNMEMTYPE_UVM – memory is allocated with a call to sycl::malloc_shared.

  • queue – the sycl::queue handle for the stream that the allocation will be performed on.

Returns:

SUNErrCode SUNMemoryHelper_Dealloc_Sycl(SUNMemoryHelper helper, SUNMemory mem, void *queue)

Deallocates the mem->ptr field if it is owned by mem, and then deallocates the mem object.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem – the SUNMemory object.

  • queue – the sycl::queue handle for the queue that the deallocation will be performed on.

Returns:

SUNErrCode SUNMemoryHelper_Copy_Sycl(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Synchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – the sycl::queue handle for the queue that the copy will be performed on.

Returns:

SUNErrCode SUNMemoryHelper_CopyAsync_Sycl(SUNMemoryHelper helper, SUNMemory dst, SUNMemory src, size_t mem_size, void *queue)

Asynchronously copies mem_size bytes from the the source memory to the destination memory. The copy can be across memory spaces, e.g. host to device, or within a memory space, e.g. host to host. The helper object will use the memory types of dst and src to determine the appropriate transfer type necessary.

Arguments:

  • helper – the SUNMemoryHelper object.

  • dst – the destination memory to copy to.

  • src – the source memory to copy from.

  • mem_size – the number of bytes to copy.

  • queue – the sycl::queue handle for the queue that the copy will be performed on.

Returns:

SUNErrCode SUNMemoryHelper_GetAllocStats_Sycl(SUNMemoryHelper helper, SUNMemoryType mem_type, unsigned long *num_allocations, unsigned long *num_deallocations, size_t *bytes_allocated, size_t *bytes_high_watermark)

Returns statistics about memory allocations performed with the helper.

Arguments:

  • helper – the SUNMemoryHelper object.

  • mem_type – the SUNMemoryType to get stats for.

  • num_allocations – (output argument) number of memory allocations done through the helper.

  • num_deallocations – (output argument) number of memory deallocations done through the helper.

  • bytes_allocated – (output argument) total number of bytes allocated through the helper at the moment this function is called.

  • bytes_high_watermark – (output argument) max number of bytes allocated through the helper at any moment in the lifetime of the helper.

Returns: