Skip to navigation

Importing Data with the Bulk Import API

The Bulk Import API creates and updates Benchling objects at scale through an asynchronous, file-based task. Upload a JSONL file of input objects, start the task, then poll for results and errors.

The Bulk Import API creates and updates Benchling objects in bulk through an asynchronous task. It is part of the V3 REST API, and the relevant endpoints are the ones suffixed :bulk-create and :bulk-update. 68 resource types expose them; see the V3 API reference for the full list.

Batch or bulk?

Bulk is the file-based, asynchronous path, and it is the right choice only for large data sets. For most multi-object writes, use the inline :batch-create and :batch-update operations instead. They take JSON in the request body and return synchronously. See Batch and bulk writes.

Beta endpoints

Most bulk endpoints are beta. If a documented endpoint returns 404, retry the request with the EARLY-ACCESS: true header, which is how beta operations are reached on the unified /api/v3 base path. See the V3 Stability Guidelines.

Limits

A single bulk request accepts up to 60,000 input objects or a 100 MB file, whichever limit is reached first.

Only one bulk mutation per type can run at a time. A second :bulk-create for the same type has to wait for the first to finish.

Step 1: Define your input objects

Write your input objects to a JSONL file, one JSON object per line. Each line has two keys:

  • request_id: an identifier you choose. Results and errors are keyed by it, so it is how you map an outcome back to an input. It needs to be unique within the file.
  • input_object: the object to create or update. Its shape is the request body of the type’s single-object POST (for :bulk-create) or PATCH (for :bulk-update). Look the shape up in the API reference for that type.

For custom entities, a file named example_input_objects.jsonl might look like this:

example_input_objects.jsonl
{"request_id": "my_entity_1", "input_object": {"name": "Entity 1", "folderId": "lib_123", "schemaId": "ts_fOCoL4ogYi", "schema_fields": {"schema_field_1": {"value": "test"}}}}
{"request_id": "my_entity_2", "input_object": {"name": "Entity 2", "folderId": "lib_123", "schemaId": "ts_fOCoL4ogYi"}}

Each line is a complete JSON object, and the file has no enclosing array and no commas between lines.

Step 2: Upload the file

Bulk operations read their input from a scratch file, so upload the JSONL file through the ScratchFile endpoints before starting the task. This is the same three-step flow used to attach files to documents.

a. Create the scratch file. Only filename is required; name is an optional display name. The response contains the id you will need in step 3 and a presigned uploadUrl with a short expiration.

POST /api/v3/scratch-file
EARLY-ACCESS: true

b. Upload the file to the presigned URL. S3 expects a PUT with the file as the request body, and requires the server-side encryption header:

Upload
curl -H "x-amz-server-side-encryption: AES256" \
-X PUT -T example_input_objects.jsonl \
-L "<PRESIGNED_S3_URL>"

If the uploadUrl expires before you finish, GET /api/v3/scratch-file/{scratch_file_id} returns a fresh one.

c. Mark the upload as complete. Benchling does not observe the S3 upload, so the file stays unusable until you finalize it:

PATCH /api/v3/scratch-file/scrfile_jdf8BV24kLmN
EARLY-ACCESS: true

Step 3: Start the bulk import task

Start the task by referencing the scratch file’s ID. The request body is the same for every type and every bulk operation: a single fileId. Creates are POST; updates are PATCH.

POST /api/v3/custom-entity:bulk-create
PATCH /api/v3/custom-entity:bulk-update
EARLY-ACCESS: true

The scratch file’s upload must have completed successfully, or the request is rejected.

Step 4: Poll for results

Use the taskId from step 3 with the three task endpoints below. pollingUri in the response is the fully qualified form of the first one.

Task status

GET /api/v3/tasks/bulk/{task_id}
EARLY-ACCESS: true

status is NOT_STARTED, IN_PROGRESS, COMPLETED, or FAILED. Poll until it reaches COMPLETED or FAILED, then read result for the success and error counts.

A COMPLETED task is not necessarily a task in which every object succeeded. Check result.errorCount and fetch the errors file even on COMPLETED.

Successful results

GET /api/v3/tasks/bulk/{task_id}/successful-results
EARLY-ACCESS: true

items is a presigned URL for a file pairing each request_id with the id of the object that was created or updated, which is how you resolve your identifiers to Benchling API IDs. listingUri points at the listing endpoint for the imported type.

While the task is still running, this endpoint returns 202 with a message instead of a results URL.

Errors

GET /api/v3/tasks/bulk/{task_id}/errors
EARLY-ACCESS: true

errors is a presigned URL for a JSONL file with one line per failed input object, giving its request_id, an error message, and an error code. When there is nothing to download, errors is null and message explains why, for example "No errors found for this bulk task." As with successful results, a 202 means the task is still in progress.

Confirm the format of the successful-results file

The API reference describes the successful-results download as JSONL in the operation summary and as CSV with request_id,id headers in the field description. Inspect the first line of the file rather than assuming a format. Errors and input files are JSONL.