mfem::Device Class Reference

The MFEM Device class abstracts hardware devices such as GPUs, as well as programming models such as CUDA, OCCA, RAJA and OpenMP. More...

#include <device.hpp>

Public Member Functions
	Device ()
	Default constructor. Unless Configure() is called later, the default Backend::CPU will be used.
	Device (const std::string &device, const int device_id=0)
	Construct a Device and configure it based on the device string. See Configure() for more details.
	~Device ()
	Destructor.
void	Configure (const std::string &device, const int device_id=0)
	Configure the Device backends.
void	Print (std::ostream &os=mfem::out)
	Print the configuration of the MFEM virtual device object.

Static Public Member Functions
static void	SetMemoryTypes (MemoryType h_mt, MemoryType d_mt)
	Set the default host and device MemoryTypes, h_mt and d_mt.
static bool	IsConfigured ()
	Return true if Configure() has been called previously.
static bool	IsAvailable ()
	Return true if an actual device (e.g. GPU) has been configured.
static bool	IsEnabled ()
	Return true if any backend other than Backend::CPU is enabled.
static bool	IsDisabled ()
	The opposite of IsEnabled().
static int	GetId ()
	Get the device ID of the configured device.
static int	GetDeviceCount ()
	Get the number of available devices (may be called before configuration).
static bool	Allows (unsigned long b_mask)
	Return true if any of the backends in the backend mask, b_mask, are allowed.
static MemoryType	GetHostMemoryType ()
	Get the current Host MemoryType. This is the MemoryType used by most MFEM classes when allocating memory used on the host.
static MemoryClass	GetHostMemoryClass ()
	Get the current Host MemoryClass. This is the MemoryClass used by most MFEM host Memory objects.
static MemoryType	GetDeviceMemoryType ()
	Get the current Device MemoryType. This is the MemoryType used by most MFEM classes when allocating memory to be used with device kernels.
static MemoryType	GetMemoryType ()
	(DEPRECATED) Equivalent to GetDeviceMemoryType().
static MemoryClass	GetDeviceMemoryClass ()
	Get the current Device MemoryClass. This is the MemoryClass used by most MFEM device kernels to access Memory objects.
static MemoryClass	GetMemoryClass ()
	(DEPRECATED) Equivalent to GetDeviceMemoryClass().
static void	SetGPUAwareMPI (const bool force=true)
	Manually set the status of GPU-aware MPI flag for use in MPI communication routines which have optimized implementations for device buffers.
static bool	GetGPUAwareMPI ()
	Get the status of GPU-aware MPI flag.
static MemoryType	QueryMemoryType (const void *ptr)
static int	NumMultiprocessors (int device_id)
	The number of hardware compute units/streaming multiprocessors available on a given compute device device_id.
static int	NumMultiprocessors ()
	Same as NumMultiprocessors(int), for the currently active device.
static int	WarpSize (int device_id)
	The number of threads in a warp on a given compute device device_id.
static int	WarpSize ()
	Same as WarpSize(int), for the currently active device.
static void	DeviceMem (size_t *free, size_t *total)
	Gets the free and total memory on the device.

Friends
class	MemoryManager

Detailed Description

The MFEM Device class abstracts hardware devices such as GPUs, as well as programming models such as CUDA, OCCA, RAJA and OpenMP.

This class represents a "virtual device" with the following properties:

• At most one object of this class can be constructed and that object is controlled by its static methods.
• If no Device object is constructed, the static methods will use a default global object which is never configured and always uses Backend::CPU.
• Once configured, the object cannot be re-configured during the program lifetime.
• MFEM classes use this object to determine where (host or device) to perform an operation and which backend implementation to use.
• Multiple backends can be configured at the same time; currently, a fixed priority order is used to select a specific backend from the list of configured backends. See the Backend class and the Configure() method in this class for details.

Definition at line 123 of file device.hpp.

Constructor & Destructor Documentation

Device() [1/2]

mfem::Device::Device

(

)

Default constructor. Unless Configure() is called later, the default Backend::CPU will be used.

Note

At most one Device object can be constructed during the lifetime of a program.

This object should be destroyed after all other MFEM objects that use the Device are destroyed.

Definition at line 74 of file device.cpp.

Device() [2/2]

mfem::Device::Device ( const std::string & device, const int device_id = 0 )

inline

Construct a Device and configure it based on the device string. See Configure() for more details.

Note

At most one Device object can be constructed during the lifetime of a program.

This object should be destroyed after all other MFEM objects that use the Device are destroyed.

Definition at line 180 of file device.hpp.

~Device()

mfem::Device::~Device

(

)

Destructor.

Definition at line 156 of file device.cpp.

Member Function Documentation

Allows()

static bool mfem::Device::Allows ( unsigned long b_mask )

inlinestatic

Return true if any of the backends in the backend mask, b_mask, are allowed.

This method can be used with any of the Backend::Id constants, the Backend::*_MASK, or combinations of those.

Definition at line 262 of file device.hpp.

Configure()

void mfem::Device::Configure	(	const std::string &	device,
		const int	device_id = 0 )

Configure the Device backends.

The string parameter device must be a comma-separated list of backend string names (see below). The device_id argument specifies the ID of the actual devices (e.g. GPU) to use.

• The available backends are described by the Backend class.
• The string name of a backend is the lowercase version of the Backend::Id enumeration constant with '_' replaced by '-', e.g. the string name of 'RAJA_CPU' is 'raja-cpu'. The string name of the debug backend (Backend::Id 'DEBUG_DEVICE') is exceptionally set to 'debug'.
• The 'cpu' backend is always enabled with lowest priority.
• The current backend priority from highest to lowest is: 'ceed-cuda', 'occa-cuda', 'raja-cuda', 'cuda', 'ceed-hip', 'hip', 'debug', 'occa-omp', 'raja-omp', 'omp', 'ceed-cpu', 'occa-cpu', 'raja-cpu', 'cpu'.
• The following backend aliases are also available: 'ceed-gpu', 'occa-gpu', 'raja-gpu', and 'gpu' where they alias their respective '-cuda' or '-hip' backends depending on the MFEM build-time configuration.
• Multiple backends can be configured at the same time.
• Only one 'occa-*' backend can be configured at a time.
• The backend 'occa-cuda' enables the 'cuda' backend unless 'raja-cuda' is already enabled.
• The backend 'occa-omp' enables the 'omp' backend (if MFEM was built with MFEM_USE_OPENMP=YES) unless 'raja-omp' is already enabled.
• Only one 'ceed-*' backend can be configured at a time.
• The backend 'ceed-cpu' delegates to a libCEED CPU backend the setup and evaluation of the operator.
• The backend 'ceed-cuda' delegates to a libCEED CUDA backend the setup and evaluation of operators and enables the 'cuda' backend to avoid transfers between host and device.
• The backend 'ceed-hip' delegates to a libCEED HIP backend the setup and evaluation of operators and enables the 'hip' backend to avoid transfers between host and device.
• The 'debug' backend should not be combined with other device backends.

Note

If the device is actually enabled, this method will also update the current host/device MemoryType and MemoryClass.

Definition at line 189 of file device.cpp.

DeviceMem()

void mfem::Device::DeviceMem ( size_t * free, size_t * total )

static

Gets the free and total memory on the device.

Definition at line 700 of file device.cpp.