Load a CSV file in Python
CSV (Comma Separated Values) is a widely used format for logging flight, sensor, and machine data. It is human-readable, editable with spreadsheets, and compatible with practically any data analysis tool.
Nominal is the leading platform for operationalizing test data.
This guide describes an automatable pipeline for uploading CSV files to Nominal.
Connect to Nominal
Concepts
- Base URL: The URL through which the Nominal API is accessed (typically
https://api.gov.nominal.io/api
). - Workspace: A mechanism by which to isolate datasets; each user has one or more workspace, and data in one cannot be seen from another. Note that one token may access multiple workspaces.
- Profile: A combination of base URL, API key, and workspace.
There are two primary ways of authenticating the Nominal Client. The first is to use a profile stored on disk, and the second is to use a token directly.
Storing credentials to disk
Run the following in a terminal and follow on-screen prompts to set up a connection profile:
Here, “default” can be any name chosen to represent this profile (reminder: a profile represents a base URL, API key, and workspace).
The profile will be stored in ~/.config/nominal/config.yml
, and can then be used to create a client:
If you have previously used nom
to store credentials, prior to the availability of profiles, you will need to migrate your old configuration file (~/.nominal.yml
) to the new format (~/.config/nominal/config.yml
).
You can do this with the following command:
Directly using credentials in your scripts
NOTE: you should never share your Nominal API key with anyone. We therefore recommend that you not save it in your code and/or scripts.
Download sample data
For convenience and educational purposes, Nominal hosts test sample data on the Nominal Hugging Face account.
Let’s download a 1000 rows of data generated in the X-Plane flight simulator.
We’ll use the popular Polars library to download and inspect the data. Polars is installed automatically as part of the nominal
Python library.
Upload your data to Nominal
Equivalently, since your CSV is already loaded in the Polars dataframe df
, you can upload it with the upload_dataframe()
method:
After upload, navigate to Nominal’s Datasets page (login required). You’ll see your CSV at the top!
Acceptable values for timestamp_type
include:
iso_8601
,
epoch_days
,
epoch_hours
,
epoch_minutes
,
epoch_seconds
,
epoch_milliseconds
,
epoch_microseconds
,
epoch_nanoseconds
,
relative_days
,
relative_hours
,
relative_minutes
,
relative_seconds
,
relative_milliseconds
,
relative_microseconds
,
relative_nanoseconds
Timestamps in the form 2024-06-08T05:58:42.000Z
will have a timestamp_type
of iso_8601
.
Timestamps in the form 1581933170999989
will most likely be epoch_microseconds
.
epoch_
timestamps refers to timestamps in Unix format.
For more information about Nominal timestamps in Python, see the
nominal.ts
docs page.