Chapter 9 discusses how to share a single calculation between multiple GPUs on a workstation. CUDA provides a number of tools to both manage individual devices and for memory management so that multiple devices can see a common shared memory pool. CUDA unified virtual addressing (UVA) is an example of this. Transfers of data between the host and GPU memory can also be automated or eliminated using unified memory (UM) or zero-copy memory. To scale beyond a single workstation the well-known message passing interface (MPI) library is often used and this is described with a simple example.

Chapter 9 discusses how to share a single calculation between multiple GPUs on a workstation. CUDA provides a number of tools to both manage individual devices and for memory management so that multiple devices can see a common shared memory pool. CUDA unified virtual addressing (UVA) is an example of this. Transfers of data between the host and GPU memory can also be automated or eliminated using unified memory (UM) or zero-copy memory. To scale beyond a single workstation the well-known message passing interface (MPI) library is often used and this is described with a simple example.