JAX on GB10: jit, grad and vmap in Practice
JAX is a library for numerical computing that looks like NumPy but has three extra skills: it compiles your functions (jit), it computes derivatives by itself (grad) and it stretches a function written for one example over a whole batch (vmap). Here we put it on a local AI server of the GB10 class and honestly check whether it runs on the graphics card.
pmap as a main tool, jax[cuda12] as the recommendation and the "EU project scoring" example. We added: installation in a separate virtual environment with jax[cuda13], a check that JAX really sees the graphics card, how to time things honestly (warm-up and block_until_ready), a small neural network on made-up data and a note about memory.
01What you'll learn
- What JAX is and how it differs from plain NumPy.
- How to install it in a separate environment and check whether it uses the graphics card.
- What
jit,gradandvmapdo and how they combine. - How to measure speed without fooling yourself.
- How to train a small neural network with JAX only.
- How JAX takes memory and how to stop it from taking more than needed.
02Before you start
- A machine of the NVIDIA GB10 class (for example ASUS Ascent GX10 or DGX Spark) with DGX OS and access to the terminal.
- Python 3 with
venvandpip. - An NVIDIA driver of version 580 or newer — a requirement of the JAX wheels for CUDA 13 on Linux (per the JAX documentation).
- The basics of NumPy.
- Internet to download the packages.
venv) they do not damage the system Python, and you can delete the environment with one command.03Steps
-
What JAX is
By its official description JAX is a Python library for accelerator-oriented array computation, designed for high-performance numerical computing and machine learning. It gives you three things:
- A NumPy-style API in
jax.numpy— familiar functions and arrays. - Function transformations:
jax.jit(compile),jax.grad(derivative),jax.vmap(automatic batching). - The same code on a CPU, a GPU and a TPU.
Compilation goes through XLA — a compiler for linear algebra that fuses many small operations into larger kernels.
- A NumPy-style API in
-
Environment and installation
The JAX documentation recommends the wheels with built-in CUDA libraries (
jax[cuda13]) and says that NVIDIA has released CUDA packages for x86_64 and aarch64. So on a GB10 (ARM64) the documented path is this:bash · on the machinepython3 -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install --upgrade "jax[cuda13]"Two rules from the documentation: the driver must be ≥ 580 for CUDA 13 on Linux, and the variable
LD_LIBRARY_PATHmust not be set, because it can override the libraries that come with the package. Wheels for CUDA 12 are offered too, but JAX recommends moving to CUDA 13.⚠️The command has not been run on GB10The JAX documentation lists "Linux, aarch64 + NVIDIA GPU" as a supported platform, but we have not checked whether the GB10 graphics card in particular is recognised. The Isaac Lab documentation has an older note saying JAX has CUDA wheels only for x86_64 — it disagrees with the current JAX documentation. So do not guess: run the check in the next step. -
Check whether JAX sees the graphics card
bashpython -c "import jax; print(jax.devices()); print(jax.__version__)"You expect a list with a device of the CUDA type (for example
CudaDevice(id=0)). If you see onlyCpuDevice, JAX is running on the CPU only — it is not broken, but it will not speed anything up on the graphics card, and all the measurements below are for the CPU. In that case check the driver version and look at NVIDIA's JAX-Toolbox containers (in the sources). -
jit— compile the functionjax.jitcompiles the function on the first call and then repeats the ready code. The first call is slow (it compiles), the following ones are fast. JAX sends the work to the graphics card asynchronously, so when timing you must wait for the result withblock_until_ready(), otherwise you only measure the time to send it.python · not runimport time import jax import jax.numpy as jnp def f(x): return jnp.sum(x ** 2 + jnp.sin(x)) f_jit = jax.jit(f) x = jnp.ones((10_000_000,)) f_jit(x).block_until_ready() # warm-up: the compilation happens here t = time.perf_counter() for _ in range(100): f_jit(x).block_until_ready() print("with jit:", time.perf_counter() - t) t = time.perf_counter() for _ in range(100): f(x).block_until_ready() print("without jit:", time.perf_counter() - t)The result depends on the machine — we promise no values. The important habit is: one warm-up, then measure with
block_until_ready(). -
grad— a derivative without manual workjax.grad(f)returns a new function that computes the gradient offwith respect to its first argument. This lets you minimise a function with gradient descent.python · not runimport jax import jax.numpy as jnp g = jax.grad(lambda x: x ** 3) print(g(2.0)) # 12.0, because the derivative is 3·x² def mse(params, x, y): return jnp.mean((params["w"] * x + params["b"] - y) ** 2) x = jnp.arange(5.0) y = 2.0 * x + 1.0 # made-up data: w=2, b=1 params = {"w": 0.0, "b": 0.0} grad_fn = jax.jit(jax.grad(mse)) for _ in range(500): grads = grad_fn(params, x, y) params = jax.tree.map(lambda p, d: p - 0.05 * d, params, grads) print(params) # close to w=2, b=1The parameters are a dictionary — JAX works with such "trees", and
jax.tree.mapapplies a function to every leaf. -
vmap— from one example to a whole batchYou write the function for one example, and
jax.vmapmakes it work over a batch, with no Python loop.python · not runimport jax import jax.numpy as jnp def dot(a, b): return jnp.dot(a, b) batched_dot = jax.vmap(dot) A = jnp.ones((1000, 512)) B = jnp.ones((1000, 512)) print(batched_dot(A, B).shape) # (1000,) fast = jax.jit(jax.vmap(dot)) # transformations combine print(fast(A, B).shape) -
A small neural network with JAX only
We build a network with one hidden layer and train it on made-up data: the class is 1 when the sum of the first two values is positive. This way no outside datasets are needed.
python · not runimport jax import jax.numpy as jnp def init_params(key, sizes): keys = jax.random.split(key, len(sizes) - 1) return [ (jax.random.normal(k, (m, n)) * jnp.sqrt(2.0 / m), jnp.zeros(n)) for k, m, n in zip(keys, sizes[:-1], sizes[1:]) ] def predict(params, x): for W, b in params[:-1]: x = jax.nn.relu(x @ W + b) W, b = params[-1] return x @ W + b def loss(params, x, y): logp = jax.nn.log_softmax(predict(params, x)) return -jnp.mean(jnp.take_along_axis(logp, y[:, None], axis=1)) @jax.jit def step(params, x, y): l, grads = jax.value_and_grad(loss)(params, x, y) params = jax.tree.map(lambda p, d: p - 0.1 * d, params, grads) return params, l key = jax.random.PRNGKey(0) x = jax.random.normal(key, (2000, 20)) y = (x[:, 0] + x[:, 1] > 0).astype(jnp.int32) params = init_params(jax.random.PRNGKey(1), [20, 64, 2]) for i in range(200): params, l = step(params, x, y) if i % 50 == 0: print(i, float(l)) acc = jnp.mean(jnp.argmax(predict(params, x), axis=1) == y) print("accuracy on the training data:", float(acc))You expect the loss to fall and the accuracy to rise. The exact numbers will differ. Note that
stepis the whole training step — computing the loss, the gradient and the update — andjitcompiles it in one go. -
The graphics card's memory
By the JAX documentation (the "GPU memory allocation" section) JAX by default preallocates a large share of the graphics card's memory. You can turn that off with an environment variable, before you import
jax:pythonimport os os.environ["XLA_PYTHON_CLIENT_PREALLOCATE"] = "false" import jaxOn GB10 the CPU and the GPU share the same 128 GB, so it makes sense for JAX not to grab a large share in advance. ⚠️ We have not measured exactly how JAX behaves on this shared memory; watch it with
nvidia-smiandfree -h.
04Check
- The virtual environment is created and
jax[cuda13]is installed. jax.devices()shows a CUDA device, not onlyCpuDevice(otherwise you are on the CPU).- You timed a
jitfunction against the plain one, with a warm-up. jax.gradreturned a derivative that you can check by hand.jax.vmapran a one-example function over a batch without a loop.- The network's loss falls and its accuracy rises on the made-up data.
- You know how to turn off memory preallocation.
Quiz
1. jax.devices() returns only CpuDevice. What does it mean?
2. Why do you call block_until_ready() when timing speed?
3. What does jax.grad(f) return?
4. What is jax.vmap for?
05What's next
06Sources
- JAX: installation 🔒 local — supported platforms, wheels for NVIDIA GPUs, driver ≥ 580 for CUDA 13,
LD_LIBRARY_PATH. - JAX 101:
gradandvmap· JAX 201:jit— what the transformations do. - JAX: GPU memory allocation — preallocation and the environment variables.
- Isaac Lab: installation ("DGX Spark" section) — the older note about JAX on aarch64.
- NVIDIA JAX-Toolbox — NVIDIA containers with JAX.
- NVIDIA DGX Spark: known issues — shared memory, "Memory-Usage: Not Supported".