With recent advancements in machine learning, robotics, and autonomy, video is now an essential test artifact for modern hardware development teams.

Nominal has 1st class support for video ingestion, analysis, time sychronization across sensor channels, and automated checks that signal when a video feature is out-of-spec.

This guide demonstrates how to upload a video file to Nominal in Python. Other guides in the Video section demonstrate basic Python video analysis and computer vision for video pre-processing prior to Nominal upload.

Connect to Nominal

When using the Nominal client library, there are two primary ways of authenticating:

$
$ python -m nominal auth set-token
>
>
# Alternatively, use the globally installed CLI
>
$ nom auth set-token

1
import nominal
2
3
# Simply grab the "default" client using your stored credentials
4
client = nominal.get_default_client()
5
6
# Get details about the currently logged-in user to validate authentication
7
# Will display an object like: `User(display_name='your_email@your_company.com', ...)`
8
print(client.get_user())

1
import nominal
2
3
# Set login details for the user
4
nominal.set_token("<insert api key>")
5
6
# Get an instance of the client using provided credentials
7
client = nominal.get_default_client()
8
9
# Get details about the currently logged-in user to validate authentication
10
# Will display an object like: `User(display_name='your_email@your_company.com', ...)`
11
print(client.get_user())

Download sample video

1
dataset_repo_id = 'nominal-io/raptor-engine-fire'
2
dataset_filename = 'raptor_fire_first_5_seconds.mov'

For convenience, Nominal hosts sample test data on Hugging Face. To download the sample data for this guide, copy-paste the snippet below.

1
from huggingface_hub import hf_hub_download
2
3
dataset_path = hf_hub_download(
4
    repo_id=f"{dataset_repo_id}", 
5
    filename=f"{dataset_filename}", 
6
    repo_type='dataset'
7
)
8
9
print(f"File saved to: {dataset_path}")

(Make sure to first install huggingface_hub with pip3 install huggingface_hub).

Since our dataset is a video file, we’ll rename dataset_path:

1
video_path = dataset_path

Correct video format

For a video file to be successfully ingested into Nominal, it needs to satisfy the following conditions:

• be H264 or H265 encoded,
• use YUV420p color space, and
• contain an audio track.

It is also recommended that the video has reasonably-spaced key-frames (at most ~10s between key-frames) for optimal playback performance.

To help customers ingest a wide variety of video formats, the Python client provides the following utility function:

1
from nominal.experimental.video_processing import normalize_video
2
3
# Perform all conversions necessary to ingest this video into nominal, and output to the specified path
4
input_path = "path/to/video.mp4"
5
normalized_path = "path/to/re-encoded/video.mp4"
6
normalize_video(input_path, normalized_path)

Upload video to Nominal

Once uploaded to Nominal, video test artifacts can be analyzed collaboratively in no-code workflows and integrated with checks to signal off-nominal video features.

Nominal requires a video start time for video file upload. If the absolute time that the video was captured is not important, you can use an arbitrary datetime like datetime.now() or 2011-11-11 11:11:11.

1
import nominal as nm
2
from datetime import datetime
3
4
vid = nm.upload_video(
5
    file = video_path,
6
    name = 'Raptor Fire Test',
7
    start = datetime.strptime('2011-11-11 11:11:11', '%Y-%m-%d %H:%M:%S')
8
)

Add a Video to a Run

Once you’ve saved a Video to the Nominal platform, it’s easy to add the Video to a Run - Nominal’s container for multi-modal datasets that belong to the same test run and time domain.

Create an empty Run

The below code will create an empty Run container. Any type of data that Nominal supports (Video, CSV files, database connections, etc) and has an overlapping time domain can be attached to this Run container.

First, we’ll use OpenCV to get the video length in seconds. Nominal Runs expect both a start and end timestamp. In the example above, we’ve used 2011-11-11 11:11:11 as an arbitrary start time. To get the Run end time, we’ll simply add the video duration (in seconds) to this start time. If you haven’t make sure to install OpenCV for Python with pip install opencv-python.

1
import cv2
2
from datetime import datetime, timedelta
3
4
video = cv2.VideoCapture(video_path)
5
6
# Get the frames per second and total frame count
7
fps = video.get(cv2.CAP_PROP_FPS)
8
frame_count = video.get(cv2.CAP_PROP_FRAME_COUNT)
9
10
# Calculate the duration
11
duration_seconds = frame_count / fps
12
print(f"Duration: {duration_seconds} seconds")
13
14
video.release()
15
16
start_time = datetime.strptime('2011-11-11 11:11:11', '%Y-%m-%d %H:%M:%S')
17
end_time = start_time + timedelta(seconds=duration_seconds)

Now that we have the Run start and end times as variables, we create our empty Run container:

1
import nominal as nm
2
3
engine_fire_run = nm.create_run(
4
    name = 'Engine Fire Run',
5
    start = start_time,
6
    end = end_time,
7
    description = 'Run for Raptor engine fire.',
8
)
9
10
engine_fire_run

Attach Nominal Video to Run

We’ll use add_datset() to add the video file to the Run:

1
engine_fire_run.add_video(
2
    ref_name = "engine fire video",
3
    video = vid,
4
)

Retrieve a video

Retrieve a video with its RID (resource ID), which can be copy pasted for any video on the Videos page.

1
vid = nm.get_video('ri.video.cerulean-staging.video.d6c6e12c-05b3-4bb0-9f45-97609b7c9da2')

Archive a video

Archiving a video prevents it from displaying on the Videos page.

1
vid.archive()

To unarchive a video:

1
vid.unarchive()

Now the video will display again on the Videos page.

Appendix

Display video inline

If you’re working in Jupyter notebook, here’s a shortcut to display the video inline in your notebook.

1
from ipywidgets import Video
2
Video.from_file(video_path)

Inspect video metadata with OpenCV

The free OpenCV Python library is invaluable for video inspection and low-level video editing.

Below is a simple script that demonstrates how to capture basic video file properties such as video duration, frames per second, frame size, and total frame count.

You can install OpenCV for Python with pip install opencv-python.

1
import cv2
2
3
# Load the video
4
# video_path = 'raptor_fire_first_5_seconds.mp4'
5
# ☝️ Replace with your video_path or use the example video_path from above
6
cap = cv2.VideoCapture(video_path)
7
8
# Get video properties
9
frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))  # Width of the video frames
10
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))  # Height of the video frames
11
fps = cap.get(cv2.CAP_PROP_FPS)  # Frames per second (fps)
12
frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))  # Total number of frames
13
14
# Calculate the duration of the video in seconds
15
duration = frame_count / fps
16
17
# Print video properties
18
print(f"Video Size: {frame_width}x{frame_height} (width x height)")
19
print(f"FPS: {fps}")
20
print(f"Total Frames: {frame_count}")
21
print(f"Duration: {duration:.2f} seconds")
22
23
# Release the video capture object
24
cap.release()

Video Size: 640x360 (width x height)
FPS: 30.0
Total Frames: 150
Duration: 5.00 seconds

If interested in more advanced video processing in Python, check out the object identification guide.