Direct Data Share: Outbound

Outbound Direct Data Share publishes your Benchling records as Apache Iceberg tables you query from your own platform: Snowflake, Databricks, or any Iceberg client. See Direct Data Share for how it fits with inbound sharing, supported platforms, and beta availability.

This page has three parts:

• Common setup applies to every platform: getting access, what’s shared, freshness, and your connection details.
• Connecting from Snowflake and Connecting from Databricks cover each platform’s setup.
• Writing queries, Performance notes, and Troubleshooting at the end apply to both.

Common setup

Getting access

Outbound Data Share is in beta. To enable it, contact your Benchling representative and tell them:

• Where you’ll read the data.
  • Snowflake: send your Snowflake account identifier (<org>.<account>) if you want the data shared into your own Snowflake account as a listing.
  • Databricks: say that you’ll read from Databricks. Benchling then turns on the Databricks export path for your tenant. You can use both platforms.
• Which object types to include, if you want less than the full catalog. Most customers start with everything.

Which objects are shared

Direct Data Share covers a fixed catalog of Benchling object types. The catalog spans the notebook, registry, inventory, molecular biology, results and runs, lab automation, workflows, studies, chemistry, and platform configuration. Not every Benchling object is currently shareable.

A table appears only once there is at least one record of that type to share, so object types you don’t use won’t show up. For the full list of shareable object types, including the ones your tenant has no data for yet, see Direct Data Share: Data Shape. To see what your tenant actually has, list the tables on your platform; there are examples in the Snowflake and Databricks sections below.

Data freshness

A change in Benchling is currently visible to your queries within about an hour.

The tables shared with you are current-state tables: one row per object, reflecting its latest state.

Your connection details

Once provisioning finishes, a Benchling tenant admin can see your connection profile under Profile Settings → Data Sharing Connection Details → Show Connection Details:

The read role can only read your tenant’s schemas, and it cannot write. A leaked token cannot reach another tenant’s data or modify anything, but it does grant read access to everything shared with you. See Access control.

Connecting from Snowflake

A Snowflake listing in your own account

If you already run Snowflake, Benchling can publish your namespace as a private listing targeted at your account. You mount it once, then query it as a database in your own account, using your own roles, warehouses, and governance. No Benchling credential sits in your pipelines, and there is no cross-account token to rotate.

Give your Benchling representative your Snowflake account identifier (<org>.<account>) to have the listing targeted at you. Once it appears under Data Products → Private Sharing in Snowsight, mount it and query:

SELECT "id", "name", "createdAt"
FROM <your_mounted_database>.BENCHLING_TEN_ABC123.ENTRY
WHERE NOT "archived";

Exploring the catalog

-- Every table shared with you
SHOW TABLES IN SCHEMA BENCHLING_DATA_SHARING_<DEPLOY>.BENCHLING_TEN_ABC123;
-- Columns for one object type
SELECT column_name, data_type
FROM BENCHLING_DATA_SHARING_<DEPLOY>.INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'BENCHLING_TEN_ABC123'
  AND table_name = 'ENTRY'
ORDER BY ordinal_position;

If you query through a mounted listing, replace BENCHLING_DATA_SHARING_<DEPLOY> with your mounted database name. Everything in Writing queries applies unchanged.

Other Iceberg clients

The tables are Iceberg, so any Iceberg REST catalog client can read them, such as PyIceberg, Spark, Trino, and Flink. Use the catalog URI, token endpoint, and PAT from your connection details.

from pyiceberg.catalog.rest import RestCatalog
catalog = RestCatalog(
    name="benchling",
    uri="<catalog_uri>",
    warehouse="<catalog_name>",
    credential="<programmatic_access_token>",
    **{"oauth2-server-uri": "<token_endpoint>"},
)
table = catalog.load_table(("BENCHLING_TEN_ABC123", "ENTRY"))
df = table.scan(selected_fields=("id", "name", "createdAt")).to_arrow()

Connecting from Databricks

In Databricks you read the current-state tables in your shared schema (BENCHLING_<YOUR_TENANT_ID>_shared) through a federated catalog:

• How it appears. Benchling’s catalog shows up in Unity Catalog as a normal foreign catalog, governed like your other catalogs.
• Where queries run. Queries run on your Databricks compute, which reads the table files directly from Benchling’s Amazon S3 bucket using an IAM role you own.
• What Snowflake does. Snowflake is only used to look up which tables exist.

Setup takes three Unity Catalog objects:

• A storage credential: your IAM role.
• A read-only external location on your data’s S3 prefix.
• A Snowflake connection with a foreign catalog.

There is one handshake with Benchling: you send the ARN of your IAM role, and Benchling grants it read access to your data.

Before you start

• Connection details. You need the host, reader user, PAT, warehouse, database, and shared schema from your connection details.
• S3 location. Your Benchling contact also sends the S3 bucket, region, and prefix, in the form s3://<BUCKET>/warehouse/shared/<NAMESPACE>/. Profile Settings doesn’t show them. S3 paths are case-sensitive.
• Permissions. You need CREATE STORAGE CREDENTIAL, CREATE EXTERNAL LOCATION, CREATE CONNECTION, and CREATE CATALOG on the metastore (usually a metastore admin). You also need to be able to create an IAM role in your AWS account.

Step 1. Create the IAM role and the storage credential

Databricks needs the role’s ARN to create the credential, and only shows the external ID once the credential exists. So you create the role with a placeholder trust policy first, and finish it in Step 2.

1. Create the IAM role in AWS, for example benchling-shared-uc-credential, with this placeholder trust policy. The principal is the Databricks Unity Catalog role for AWS commercial accounts. 0000 is a placeholder you replace in Step 2.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::414351767826:role/unity-catalog-prod-UCMasterRole-14S5ZJVKOTYTL"},
      "Action": "sts:AssumeRole",
      "Condition": {"StringEquals": {"sts:ExternalId": "0000"}}
    }
  ]
}

2. Attach this permissions policy to the role, using your exact bucket and prefix:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:GetObjectVersion"],
      "Resource": "arn:aws:s3:::<BUCKET>/warehouse/shared/<NAMESPACE>/*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::<BUCKET>"
    },
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::<YOUR_ACCOUNT>:role/<THIS_ROLE>"
    }
  ]
}

3. Create the storage credential in Databricks. Go to Catalog → External Data → Credentials → Create credential, choose Storage credential with type AWS IAM role, and enter the role ARN. Name it, for example, benchling_shared_cred. After you save it, Databricks shows the credential’s external ID. If it isn’t shown right away, open the credential’s details.

Step 2. Finish the trust policy and send the role ARN to Benchling

1. Update the role’s trust policy in AWS. Replace 0000 with the external ID from Step 1, and add the role itself as a principal: Databricks requires storage-credential roles to be able to assume themselves.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::414351767826:role/unity-catalog-prod-UCMasterRole-14S5ZJVKOTYTL",
          "arn:aws:iam::<YOUR_ACCOUNT>:role/<THIS_ROLE>"
        ]
      },
      "Action": "sts:AssumeRole",
      "Condition": {"StringEquals": {"sts:ExternalId": "<EXTERNAL_ID_FROM_DATABRICKS>"}}
    }
  ]
}

2. Validate the credential in Databricks. Assume Role, Self Assume Role, and External ID Condition should show Success. A failed Read check is expected: your access is limited to your prefix, and the Databricks check probes the bucket root.
3. Send the role ARN to your Benchling contact. Benchling grants it read access to your prefix only.

Step 3. Create a read-only external location

Go to Catalog → External Data → External Locations → Create location:

Read and File Events validation may fail for the same reason as in Step 2. Choose Force create. With the Databricks CLI, pass "read_only": true and --skip-validation; the SQL CREATE EXTERNAL LOCATION statement has no read-only option.

Then let your users read through it:

GRANT READ FILES ON EXTERNAL LOCATION benchling_shared TO `<your-group>`;

Step 4. Create the Snowflake connection and foreign catalog

Go to Catalog → External Data → Connections → Create connection:

The wizard then creates the foreign catalog:

Grant your users access to the catalog as you would for any other catalog, for example:

GRANT USE CATALOG, USE SCHEMA, SELECT ON CATALOG benchling TO `<your-group>`;

Step 5. Query

SHOW TABLES IN benchling.`BENCHLING_TEN_ABC123_shared`;
SELECT * FROM benchling.`BENCHLING_TEN_ABC123_shared`.`ENTRY` LIMIT 100;

df = spark.table("benchling.`BENCHLING_TEN_ABC123_shared`.`ENTRY`")

• Query by name through the catalog. Reading the S3 paths directly doesn’t work: table folders are named <TABLE>.<hash>, and Databricks splits a path on the dot.
• Check where it ran. Open the query profile. You should see a file scan through your external location, and no Snowflake query.

Databricks troubleshooting

Writing queries

See Outbound Data Shape for table and column naming, schema field value tables, archived records, and timestamp formatting. Three conventions from there account for most mistakes:

1. Quote column names, not table names. SELECT "createdAt" FROM ENTRY, never SELECT createdAt. In Databricks, quote with backticks instead of double quotes.
2. Filter archived records with WHERE NOT "archived" unless you want them.
3. Parse timestamps with TRY_TO_TIMESTAMP("createdAt") before any date arithmetic.

The examples below are Snowflake SQL and use BENCHLING_TEN_ABC123 as the schema. Substitute your own, and set it with USE SCHEMA so you can write bare table names. In Databricks, use your shared schema (for example USE benchling.`BENCHLING_TEN_ABC123_shared` ) and backticks for quoted identifiers.

Simple joins

Reference fields are IDs, so joining is how you get names. This lists recent notebook entries with their author, folder, project, and schema:

SELECT
    e."displayId",
    e."name"                                AS entry_name,
    TRY_TO_TIMESTAMP(e."createdAt")         AS created_at,
    u."name"                                AS creator_name,
    f."name"                                AS folder_name,
    p."name"                                AS project_name,
    s."name"                                AS schema_name
FROM ENTRY AS e
LEFT JOIN USER        AS u ON u."id" = e."creator_id"
LEFT JOIN FOLDER      AS f ON f."id" = e."folder_id"
LEFT JOIN PROJECT     AS p ON p."id" = f."project_id"
LEFT JOIN ENTRYSCHEMA AS s ON s."id" = e."schema_id"
WHERE NOT e."archived"
ORDER BY created_at DESC
LIMIT 100;

Use LEFT JOIN for reference fields that are optional: an unfiled entry has no "folder_id", and an entry with no schema has no "schema_id". An inner join silently drops those rows.

For a registry census, an inventory-with-storage-hierarchy example, and schema field joins (including pivoting fields into columns), see Outbound Data Shape.

Performance notes

• Project the columns you need. These are columnar Iceberg tables, so SELECT "id", "name" reads far less than SELECT *. Object tables can be wide, and a few columns hold large payloads, including DNASEQUENCE."bases" and ENTRY."sections".
• Filter before joining schema field values. Narrow the parent table first in a CTE, then join to the value tables. The value tables hold a row per field per object and are usually the largest tables in the schema.
• Prefer "systemName" over "name" in field-definition predicates. System names are stable; display names change when someone renames a field.
• Cache pivots you reuse. If a pivoted view backs a dashboard, materialize it on your side on a schedule that matches the hourly refresh. Avoid re-pivoting on every query.

Troubleshooting

For Databricks-specific errors, see Databricks troubleshooting. For identifier-casing errors, missing tables, and schema-field pivot issues, see Troubleshooting on the Data Shape page.