Introducing
GPU-accelerated Arrays

Charles Pooh, Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

Outline

Introduction to GPU Programming

Deep Dive into GPUArray

Future Directions

What’s a GPU?

A graphics processing unit (GPU) is an electronic circuit that can perform mathematical calculations at high speed.

GPU Programming

CUDALink

CUDALink allows the Wolfram Language to use the CUDA parallel computing architecture on Graphical Processing Units (GPUs).

transpose[matrix_] := 
Module[
   {dInputMatrix, dOutputMatrix, hOutputMatrix, blockDim, transposeFun},
   
   blockDim = {16, 16};
   
   dInputMatrix = CUDAMemoryLoad[matrix];
   dOutputMatrix = CUDAMemoryAllocate[Real, Reverse@ Dimensions[matrix]];
   
   transposeFun = CUDAFunctionLoad[{FileNameJoin[{$CUDALinkPath, "SupportFiles", 
        "transpose.cu"}]}, 
     "transpose_kernel", {{_Real, "Input"}, {_Real, "Output"}, 
      "Integer32", "Integer32"}, blockDim];
   
   transposeFun[dInputMatrix, dOutputMatrix, 
    Sequence @@ Reverse[Dimensions[matrix]]];
   
   hOutputMatrix = CUDAMemoryGet[dOutputMatrix];
   
   CUDAMemoryUnload[dInputMatrix];
   CUDAMemoryUnload[dOutputMatrix];
   
   Return[hOutputMatrix];
];

CUDACompiledTools

CUDACompiledTools is a set of tools that call NVIDIA libraries from the Wolfram Compiler.

∘ Set of  functions load/retrieve data on the GPU
∘ Hooked into the memory management
∘ Memory free automatically

transpose = FunctionCompile[
	Function[Typed[matrix, "NumericArray"["Real32", 2]], 
	  Module[{res, vec},
	    vec = CreateCUDAMatrix[matrix];
	    res = CUDATranspose[vec];
	    FromCUDAMatrix[vec]
	  ]
	]
];

GPU Toolkit

Wolfram Language

Easier to use and Integration

What’s GPUArray?

GPUArray[…] represents an array stored in memory accessible for GPU accelerated computation.

Features

GPUArray provides GPU-accelerated array framework fully integrated in the Wolfram Language.

Native GPU accelerations for core areas such as linear algebra, signal processing, image processing, graphs, geometry, …

Support of NVIDIA GPUs, Apple GPUs, AMD GPUs, Intel GPUs, ...

Multi-device GPU-CPU support

Code compilation and generation

Create a GPUArray

GPUArray[array] creates an array in GPU memory or unified memory.

The array can have the following forms:

({{      , {(Subscript[e, 1]),…},{{(Subscript[e, 1]),…},…},…, full array of numeric elements}, {      , NumericArray[…], numeric array of a specified type}, {      , SparseArray[…], sparse array with few nonzero elements}})

GPUArray[RandomReal[1,20]]

Get Data From GPU

Normal[GPUArray[…]] retrieve data from GPU.

Additional conversions include:

({{      , SparseArray[GPUArray[…]], sparse array with few nonzero elements}, {      , NumericArray[GPUArray[…],type], numeric array of a specified type}})

ga=GPUArray[RandomReal[1,20]]

Normal[ga]

Array Properties

GPUArrayQ — test whether an object is a valid GPU array

ArrayQ ▫ VectorQ ▫ MatrixQ ▫ ArrayDepth ▫ Dimensions ▫ Length ▫ Part ▫ First ▫ Last ▫ ByteCount

ga=GPUArray[RandomReal[1,20]]

GPUArrayQ[ga]

Dimensions[ga]

VectorQ[ga]

Linear Algebra

Dot — matrix operations

Transpose ▫ Det

LinearSolve — solve linear systems

Inverse ▫ PseudoInverse

LUDecomposition — matrix decompositions

CholeskyDecomposition ▫ QRDecomposition ▫ SingularValueDecomposition

Eigenvalues — eigenvalues and vectors

Eigenvectors ▫ EigenSystem

Discrete Fourier Analysis

Fourier — find the discrete Fourier transform

InverseFourier — find the discrete inverse Fourier transform

ga=GPUArray[RandomReal[1,20]]

Fourier[ga]

Statistics

Mean — descriptive statistics

GeometricMean ▫ Variance ▫ Skewness ▫ StandardDeviation ▫ RootMeanSquare ▫ Kurtosis

Sort — order statistics

Min ▫ Max ▫ MinMax ▫ Ordering

ga=GPUArray[RandomReal[1,20]]

Mean[ga]

Mathematical Functions

Plus, Times — arithmetic functions

Subtract ▫ Divide ▫ Total ▫ Mean ▫ Accumulate ▫ Differences ▫ Ratios

Sin, Cos — trigonometric functions

Tan ▫ Csc ▫ Sec ▫ Cot ▫ ArcSin ▫ ArcCos ▫ ArcTan ▫ ArcCsc ▫ ArcSec ▫ ArcCot ▫ Haversine ▫ InverseHaversine

Exp, Log — transcendental functions

Log2 ▫ Log10 ▫ Power ▫ Sqrt ▫ CubeRoot ▫ Surd

Sinh, Cosh — hyperbolic functions

Tanh ▫ Csch ▫ Sech ▫ Coth ▫ ArcSinh ▫ ArcCosh ▫ ArcTanh ▫ ArcCsch ▫ ArcSech ▫ ArcCoth ▫ Gudermannian ▫ InverseGudermannian

BesselJ, Gamma — special functions

Erf ▫ InverseErf ▫ Erfc ▫ InverseErfc

Ceiling, Floor — integer functions

Round ▫ IntegerPart ▫ FractionalPart ▫ Chop

Abs, Sign — absolute values and signs

RealAbs ▫ RealSign ▫ Re ▫ Im ▫ Arg ▫ AbsArg ▫ Conjugate

ga=GPUArray[RandomReal[1,20]]

Cos[ga]

Sin[ga]

Normal[%]

Code Compilation & Generation

"GPUArray" — GPUArray type

FunctionCompile ▫ FunctionCompileLibrary

FunctionCompile[
  Function[Typed[a,"GPUArray"::["CUDAVector","MachineReal"]],
    Cos[a] + 3 * Sin[a]
]

Summary

Built-in support of array computations on GPU.

Linear algebra, discrete fourier analysis, mathematical functions, graphs and more.

Support of Nvidia GPUs, Apple GPUs, AMD GPUs, Intel GPUs, ...

Multi-device with operations running on CPU if GPU is not available

Code compilation and generation

Q&A

Linear Algebra and GPU Computing — Today, 3:00PM

Office Hours — Friday, 3:00PM

R&D: Discrete Computation Booth

Subscribe to @WolframRD on YouTube for up to date content!