AI Relighting on GX10: New Light on the People in a Video

01What you will learn

• What Relighting does and what you can and cannot expect from it.
• What an HDR environment map is and how you pick a ready-made one or your own.
• How the service is launched and checked per the documentation and how the sample client is run.
• Which settings change the strength of the light, the background and the encoding.
• Why consent and disclosure are needed when you change how people look.

02Before you start

• A machine with Docker and the NVIDIA Container Toolkit; driver 571.21 or later (per the support table).
• A personal NGC key with the "NGC Catalog" service included. The key is a secret.
• Python and pip for the sample client, ffmpeg and Git.
• A video in MP4 with H.264. The documentation gives no other formats.
• Only your own or invented recordings with consent for the trial.

What the documentation says (as of 03.10.2026)

Licences (per the official pages, 03.10.2026)

03Steps

1. How relighting works

   First the service separates the person from the background. Then it "lights" the person anew as if they were standing in the chosen environment. The environment is an HDR map — a 360-degree picture that keeps where the light comes from, what colour it is and how strong. Finally the person is placed on a background of your choice: the original, another picture or the HDR environment itself.

   Two things about expectations: the documentation describes video (not single photos), and the result depends on the input — we have not checked how it behaves in poor light, with several people in frame, or with hair and transparent objects. Try it with your own clips.

2. Prepare the video

   The video is MP4 with H.264. For best performance make it "streamable" — the metadata moves to the start, and the service begins before it has received the whole file:

   bashcopy

   ffmpeg -i input.mp4 -movflags +faststart -c copy output_streamable.mp4

   If the video is not streamable, the service switches to "transactional" mode: it waits for the whole file before it starts. Both modes work, but streaming is recommended. The command is from the documentation and has not been run by us.

3. Log in to the catalogue and start the service

   Create a personal NGC key with "NGC Catalog" and log in. The key is a password: pass it through an environment variable.

   bashcopy

   export NGC_API_KEY=<your-key>
   echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

   The user name $oauthtoken is literally so. The start command is from the documentation; we only bound the ports to the local address so they are not visible from outside (the documentation has -p 8000:8000 and -p 8001:8001, plus port 9002 for metrics):

   bashcopy

   docker run -it --rm --name=relighting-nim \
     --runtime=nvidia \
     --gpus all \
     --shm-size=8GB \
     -e NGC_API_KEY=$NGC_API_KEY \
     -e NIM_MAX_CONCURRENCY_PER_GPU=1 \
     -e NIM_HTTP_API_PORT=8000 \
     -e NIM_GRPC_API_PORT=8001 \
     -p 127.0.0.1:8000:8000 \
     -p 127.0.0.1:8001:8001 \
     nvcr.io/nim/nvidia/ai4m-relighting:1.1.0

   We have not run this command. The model profile (NIM_MANIFEST_PROFILE) is not mandatory: if you do not set it, the service chooses by itself according to the card. Set for a different card, it leads to an error at run time. If the service does not start for you, the machine itself may be the reason: the documentation does not list GB10 and does not say whether the image is built for ARM64. The image tag is the one in the documentation today — check the current one in the catalogue.

   • Parallel requests: 1 per card by default (NIM_MAX_CONCURRENCY_PER_GPU).
   • Protection: by default the connection is not encrypted; there are modes tls and mtls (NIM_SSL_MODE). Keep the ports on the local address.

4. Check that the service is ready

   The documentation has two ways. The first is an HTTP readiness address:

   bashcopy

   curl -s http://127.0.0.1:8000/v1/health/ready

   The second is over gRPC with grpcurl (the documentation shows a package for amd64; on ARM64 take the version for your architecture):

   bashcopy

   wget https://raw.githubusercontent.com/grpc/grpc/master/src/proto/grpc/health/v1/health.proto
   grpcurl --plaintext --proto health.proto localhost:8001 grpc.health.v1.Health/Check

   When ready, the answer of the second is { "status": "SERVING" }.

5. Run the sample client

   bashcopy

   git clone https://github.com/NVIDIA-Maxine/nim-clients.git
   cd nim-clients/relighting
   pip install -r requirements.txt
   cd scripts
   
   python relighting.py \
     --target 127.0.0.1:8001 \
     --video-input output_streamable.mp4 \
     --output result.mp4 \
     --hdri-id 3

   All parameters are optional; without them the client uses a sample file from the repository and the "Lounge" environment. The numbers of the ready-made environments are: 0 Lounge, 1 Cobblestone Street Night, 2 Glasshouse Interior, 3 Little Paris Eiffel Tower, 4 Wooden Studio. We have not run this command.

   💡

   Your own HDR map

   With --hdr <file.hdr> you pass your own map. Check the licence of the file: many free maps have terms of use. How good the quality is with your own map, the documentation does not say.

6. Settings for the effect, the background and the encoding

   Parameter	What it does (per the documentation)
   --hdri-id · --hdr	A ready-made environment (0–4) or your own .hdr file
   --pan · --vfov	Where the camera "looks" in the map (angle, default −90°) and the vertical field of view (default 60°)
   --autorotate · --rotation-rate	Rotates the environment at the chosen rate (degrees per second; default 20)
   --background-source	0 — the original video; 1 — your own picture (--background-image); 2 — HDR projection. There is also a solid colour (--background-color)
   --foreground-gain · --background-gain	Strength of the lighting on the person and on the background, from 0.0 to 2.0 (default 1.0)
   --blur	Background blur, from 0.0 to 1.0
   --specular	Highlights on skin and objects, from 0.0 to 2.0 (default 0)
   --bitrate · --idr-interval · --lossless	Output quality and size: bitrate (client default 10 Mbps), interval between key frames (8) or lossless video

   Examples from the documentation: a blurred background — --blur 0.5; your own picture as a background — --background-source 1 --background-image background.png. Try a weak effect first: stronger does not mean better, and artificial light is easy to notice on faces.

7. Many clips — one at a time

   By default the service processes one stream per card, so we go through the clips one after another. The script is only a sketch per the documentation and has not been run; the file names are invented:

   bash · batch.shcopy

   mkdir -p out
   for v in clips/*.mp4; do
     n=$(basename "$v" .mp4)
     [ -f "out/$n.mp4" ] && continue          # already done
     python relighting.py --target 127.0.0.1:8001 \
       --video-input "$v" --output "out/$n.mp4" --hdri-id 4
   done

8. Review and label the result

   Watch the result at normal and at slow speed: look at the edges around hair and body outline, skin colour, flicker between frames, artefacts in fast movement. Measuring speed: take your own clip, time it from launch to the finished file and note it down — the first run does not count.

   Before publishing, label the video as artificially altered: in the description or caption and, optionally, in the file's metadata (the command is standard ffmpeg and has not been run by us):

   bashcopy

   ffmpeg -i result.mp4 -c copy -metadata comment="AI-altered video: lighting changed" result_labelled.mp4

   Keep the record of the consents too. If the people in the video have not consented, do not publish it.

04Check

• You know the lesson was not run on GB10 and that support for GB10 and ARM64 is not confirmed.
• You have written consent from every person shown for this processing and for the place of publication.
• You have not uploaded recordings of people to the trial service in the cloud.
• The ports are bound to the local address; the NGC key is not in a file that is shared.
• The video is MP4 with H.264 and streamable; the readiness check answers; the client writes the output file.
• The result is labelled as artificially altered and reviewed by a person before publishing.
• The licences of the video, HDR files and backgrounds are checked.

Quiz

05What next

06Sources

1. Relighting NIM — documentation 🔒 local — overview, launch, usage, support table.
2. Relighting — NVIDIA catalogue page 🌐 global — description, model licence, trial-service terms.
3. NVIDIA-Maxine/nim-clients — sample clients.
4. EUR-Lex: Regulation (EU) 2016/679 (GDPR) — Art. 4.
5. EUR-Lex: Regulation (EU) 2024/1689 (AI Act) — Art. 50 and 113.