Skip to main content

Create a Databricks Workspace

Fractal Cloud can provision and manage a Databricks workspace on Azure, AWS, and GCP. The setup story is different per cloud because Databricks itself exposes a different bootstrap contract on each one. This page explains what is required up-front so the agent can provision the workspace and everything that depends on it (clusters, jobs, MLflow experiments, mounts).

Azure — no manual setup

Azure Databricks workspaces are first-class Azure resources, provisioned through Azure ARM under the customer's AAD tenant. There is no separate "Databricks account" to enroll, and the agent discovers the tenant context from the LiveSystem's Azure subscription automatically.

The workspace component output field accountId is published as the AAD tenant id — for contract uniformity with AWS and GCP — but is not a required input.

// Azure SDK — minimum required input
fractal.databricks({
id: 'my-workspace',
azureResourceGroup: { name: 'rg-analytics', region: 'westeurope' },
pricingTier: 'premium',
});

AWS and GCP — one-time customer setup needed

Databricks on AWS and GCP is sold through the cloud marketplace. After subscribing, Databricks issues a customer-specific accountId that is mandatory for every account-level API call the agent makes (workspace create / update / delete all go through accounts.cloud.databricks.com/api/2.0/accounts/{accountId}/...).

Why Fractal cannot derive accountId for you

We looked at automating this end-to-end. It is currently not possible:

  • AWS Marketplace Agreements API (released May 2026) lets you programmatically subscribe to a SaaS listing, but the response only contains AWS-side identifiers (CustomerIdentifier, CustomerAWSAccountId, LicenseArn, ProductCode). None of these is the Databricks accountId.
  • Databricks fulfillment — the handoff that exchanges the AWS Marketplace token for a Databricks accountId happens inside Databricks's interactive signup form. There is no public Databricks API that exchanges a Marketplace token for an accountId unattended.
  • Databricks SDKAccountClient constructor requires accountId. There is no accounts.list() / whoami operation that works without it.

The same constraints apply on GCP Marketplace.

Until Databricks publishes a programmatic fulfillment endpoint, the accountId must be passed to the workspace component as a parameter the first time you deploy it.

One-time setup

  1. Subscribe via the cloud marketplace:
    • AWS: search "Databricks" in AWS Marketplace and follow the SaaS Quick Launch flow.
    • GCP: search "Databricks" in GCP Marketplace and follow the procurement flow.
  2. Finish Databricks signup in the redirect form. After signup, Databricks shows your accountId (a UUID, e.g. aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee) in the Account console → Settings.
  3. Supply that id to the Databricks workspace component the first time you declare it.

AWS SDK example

fractal.databricks({
id: 'my-workspace',
awsRegion: 'eu-west-1',
accountId: 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee', // from Databricks account console
pricingTier: 'premium',
credentialsId: 'creds-xxxxxxxx', // from Databricks console
storageConfigurationId: 'storage-xxxxxxxx', // from Databricks console
});

GCP SDK example

fractal.databricks({
id: 'my-workspace',
region: 'europe-west1',
accountId: 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee', // from Databricks account console
pricingTier: 'premium',
});

After the first reconcile

Once the workspace is Active, the agent stamps accountId into the component's output fields. The output field becomes the source of truth for every subsequent reconcile, so:

  • You can remove accountId from the component parameters on later deploys (the agent still reads it from the sticky output field).
  • Dependent components — DatabricksCluster, DatabricksJob, DatabricksMlflowExperiment, DatabricksMount — never need accountId directly. They talk to the workspace via the workspace URL using WorkspaceClient, and only the workspace component itself uses AccountClient.

Required IAM / permissions

CloudPermission
AzureFractal Cloud agent's existing AAD principal — no extra grants for Databricks
AWSAWSMarketplaceFullAccess or AWSMarketplaceManageSubscriptions at signup time (customer, one-off). Agent uses standard Databricks IAM-based auth thereafter — no extra grants.
GCProles/billing.user at signup time (customer, one-off). Agent uses GCP service-account OAuth thereafter — no extra grants.

Error you will see if accountId is missing

Databricks account id is required to provision an AWS Databricks workspace.
Sign up via AWS Marketplace once, copy the account id from the Databricks account console,
and supply it as the 'accountId' component parameter on the Databricks workspace component.
After the first successful reconcile the id is also published to this component's 'accountId'
output field. See fractal-docs how-to 'create-databricks-workspace'.
[componentId: '<your-component-id>']

(GCP message is the same, swap AWS Marketplace for GCP Marketplace.)

Output fields

All three providers publish accountId as an output field, so the surface is uniform regardless of where the workspace runs.