Databricks

Jobs Commands

Orchestrate data pipelines and ML workflows with Databricks Jobs. Learn to create, schedule, monitor, and manage job runs from the command line.

7 commands

Pro Tips

Use 'databricks jobs run-now' for immediate execution and 'databricks jobs create' for scheduled workflows.

Define jobs as JSON/YAML files for version control and reproducibility across environments.

Use job clusters instead of all-purpose clusters for production jobs to optimize costs.

Common Mistakes

Job runs can fail silently if cluster permissions are misconfigured. Always verify job permissions before scheduling.

Deleting a job does not cancel running instances. Cancel active runs before deletion.

Commands

List all jobs

$ databricks jobs list

List all jobs in the workspace.

Get job details

$ databricks jobs get --job-id <job-id>

Get detailed information about a specific job.

Run job now

$ databricks jobs run-now --job-id <job-id>

Trigger an immediate run of a job.

Create job from JSON

$ databricks jobs create --json-file job-config.json

Create a new job from a JSON configuration.

Delete job

$ databricks jobs delete --job-id <job-id>

Delete a job from the workspace.

List job runs

$ databricks runs list --job-id <job-id>

List all runs for a specific job.

Cancel run

$ databricks runs cancel --run-id <run-id>

Cancel a running job execution.