Trigger File Ingest

POST

Triggers an ingest job of a new file, allowing either creating a new dataset or uploading to an existing one.

This endpoint expects an object.

sourceobjectRequired

Source data for the ingest. Supported file types include:

CSV (*.csv)
Compressed CSV (*.csv.gz)
Parquet (*.parquet)
Parquet archives (_.parquet.tar, _.parquet.tar.gz, *.parquet.zip). Note that timestamp column must have the same name and format across files. Non parquet files will be ignored. Each file can contribute to the overall schema, but conflicting types will fail the ingest. Conflicting values (same timestamp, column) across files will be de-conflicted based on archive file ordering (first point taken)

sourceMetadataobjectRequired

destinationobjectRequired

This endpoint returns an object.

datasetRidstring

datasetFileIdUUIDOptional

asyncHandleobjectOptional

1	curl -X POST https://api.gov.nominal.io/api/ingest/v1/trigger-file-ingest \
2	-H "Authorization: Bearer <token>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"source": {
6	"type": "s3",
7	"s3": {
8	"path": "path"
9	}
10	},
11	"sourceMetadata": {},
12	"destination": {
13	"type": "newDataset",
14	"newDataset": {
15	"properties": {
16	"properties": "properties"
17	},
18	"labels": [
19	"labels"
20	]
21	}
22	}
23	}'

1	{
2	"datasetRid": "datasetRid",
3	"datasetFileId": "d5e9c84f-c2b2-4bf4-b4b0-7ffd7a9ffc32",
4	"asyncHandle": {
5	"rid": "rid"
6	}
7	}