Load a CSV file in Python

CSV (Comma Separated Values) is a widely used format for logging flight, sensor, and machine data. It is human-readable, editable with spreadsheets, and compatible with practically any data analysis tool.

Nominal is the leading platform for operationalizing test data.

This guide describes an automatable pipeline for uploading CSV files to Nominal.

Connect to Nominal

Get your Nominal API token from your User settings page.

See the Quickstart for more details on connecting to Nominal from Python.

1import nominal.nominal as nm
2
3nm.set_token(
4 base_url = 'https://api.gov.nominal.io/api',
5 token = '* * *' # Replace with your Access Token from
6 # https://app.gov.nominal.io/settings/user?tab=tokens
7)
If you’re not sure whether your company has a Nominal tenant, please reach out to us.

Download sample data

For convenience and educational purposes, Nominal hosts test sample data on the Nominal Hugging Face account.

Let’s download a 1000 rows of data generated in the X-Plane flight simulator.

We’ll use the popular Polars library to download and inspect the data. Polars is installed automatically as part of the nominal Python library.

1import polars as pl
2
3df = pl.read_csv('hf://datasets/nominal-io/frosty-flight/frosty_flight_1k_rows.csv')
4df.write_csv('frosty_flight_1k_rows.csv')
5
6df.head().select(df.columns[:6])
source_timef_act_secf_sim_secframe_timecpu_timegpu_time
strf64f64f64f64f64
”2024-06-08T05:58:42.000Z”0.2764519.93.617280.00220.00078
”2024-06-08T05:58:51.000Z”0.2315419.94.318881.633620.13793
”2024-06-08T05:58:52.000Z”0.2643532.300793.782893.772450.00089
”2024-06-08T05:58:52.000Z”0.2992919.93.341233.772450.00089
”2024-06-08T05:58:52.000Z”0.3415730.360992.92773.772450.00089

Upload your data to Nominal

You can upload this CSV file to Nominal with the upload_csv() function.

1import nominal.nominal as nm
2
3dataset = nm.upload_csv(
4 'frosty_flight_1k_rows.csv',
5 name='Frosty Flight',
6 timestamp_column='source_time',
7 timestamp_type='iso_8601',
8)
9
10print('Uploaded dataset:', dataset.rid)

Equivalently, since your CSV is already loaded in the Polars dataframe df, you can upload it with the upload_polars() method:

1import nominal.nominal as nm
2
3dataset = nm.upload_polars(
4 df,
5 name='Frosty Flight',
6 timestamp_column='source_time',
7 timestamp_type='iso_8601',
8)
9
10print('Uploaded dataset:', dataset.rid)

After upload, navigate to Nominal’s Datasets page (login required). You’ll see your CSV at the top!

Acceptable values for timestamp_type include:

iso_8601, epoch_days, epoch_hours, epoch_minutes, epoch_seconds, epoch_milliseconds, epoch_microseconds, epoch_nanoseconds, relative_days, relative_hours, relative_minutes, relative_seconds, relative_milliseconds, relative_microseconds, relative_nanoseconds

Timestamps in the form 2024-06-08T05:58:42.000Z will have a timestamp_type of iso_8601.

Timestamps in the form 1581933170999989 will most likely be epoch_microseconds.

epoch_ timestamps refers to timestamps in Unix format.

For more information about Nominal timestamps in Python, see the nominal.ts docs page.