Skip to content

Databricks

Databricks connections

Save a Databricks workspace connection with a personal access token, a service principal OAuth secret, or a keyless service principal that trusts Sim through OpenID Connect

A Databricks connection holds a workspace host and how Sim authenticates to it. The Databricks block signs in with a saved connection, so the host and secret live in one place instead of in each block.

Choose how Sim signs in

OptionWhat Sim storesWhen to use it
Service principal (keyless, via Sim OIDC)Nothing secret: the host and the service principal's application IDRecommended. No secret to rotate or leak.
Service principal OAuth secretThe application ID and an OAuth secretMachine-to-machine access where federation isn't set up. Sim exchanges the secret for a short-lived token on each run.
Personal access tokenA personal access tokenThe same behavior as a token typed into the block, shared across blocks.

Every option resolves on Sim's servers at run time; tokens never appear in block outputs, logs, or anything a model sees.

Set up a keyless service principal

  1. In Sim, add a Databricks connection, choose Service principal (keyless, via Sim OIDC), and enter the workspace host and the service principal's application ID.

  2. A Databricks account admin runs the Federation policy command from the dialog with the service principal's numeric ID. First sign in to the account with databricks auth login --host <account console URL> --account-id <account ID> (not a workspace profile or the workspace web terminal). Find the numeric ID with databricks account service-principals list --filter 'applicationId eq "<application ID>"', or on the service principal's page in the account console. The policy trusts Sim's issuer, the Sim audience, and this connection's subject only.

  3. Make sure the service principal can access the workspace and the resources your workflows use, then save. Sim verifies the connection by exchanging a token and reading the service principal's identity, and refuses a policy that would also trust a different subject, since every Sim workspace shares the issuer and audience.

Reconnecting a keyless connection keeps its subject by default, so the federation policy keeps working.

From the API or CLI

Run sim credentials keyless-subject databricks-service-account, trust the printed subject as above, then add host and clientId to its keylessConfig and pass it to sim credentials create databricks-service-account. See the CLI credentials reference for the full two-step example; the proof expires after 24 hours and works only for the user who requested it.

Self-hosted deployments

Keyless connections need WORKLOAD_IDENTITY_SIGNING_KEY set and an https issuer Databricks can reach over the public internet. See AWS credentials for the same settings and for rotating the signing key.

Troubleshooting

  • "Databricks rejected the token request": the federation policy's issuer, audience, or subject doesn't match the dialog, or the policy is on a different service principal than the application ID you entered.
  • "This service principal's federation policy also trusts other Sim credentials": the policy's subject_claim isn't sub, so it matches more than this connection. Recreate it with the command from the dialog.
Was this helpful?
Suggest an edit