JIT, UARs, and Time-Bound Access Enhance Engineering Productivity and Security at Databricks

JIT, UARs, and Time-Bound Access Enhance Engineering Productivity and Security at Databricks

Featured

B2B Saas

Databricks chose Opal to deliver just-in-time access, automate UARs, and lay the foundations for evolving persona-based access.

Jump to section

Jump to section

  • Company: Databricks is the Data and AI company. More than 20,000 organizations worldwide rely on the unified Databricks Data Intelligence Platform to build and scale data and AI apps, analytics, and agents. Founded in 2013, the company recently surpassed $100B in valuation and has over $4B in revenue run rate.

  • Challenge: Manual access reviews were run through Python scripts, and frequently required brittle changes and tweaks when they crashed mid-run. With more than 8,000 employees, many identity security platforms couldn’t handle the sheer scale and speed of requests.

  • Solution: Databricks adopted Opal to automate UARs, set up time-bound access to sensitive resources, and develop dynamic persona-based entitlement matching. Tracking changes in entitlements requests across cohorts and personas allowed entitlement sets to flexibly adjust over time.

Operating Environment

  • Identity provider: Okta

  • Core systems: Databricks, Azure, AWS, GCP

  • Workforce: Over 8,000 employees distributed globally, across over 30 offices in over 20 countries

  • Deployment: SaaS and Self-Hosted (FedRAMP High) 

Challenge 

As Databricks won more customers in security-sensitive industries, the Identity team knew they needed to eliminate standing access and fragile RBAC. The expanding list of customers required compliance certifications like SOC 2 Type 2, ISO 27001, FedRAMP, and more. 

For compliance purposes, the Security team began to use Python scripts and user lists in Databricks to track access reviews, but running the scripts was a fragile process that required attention and numerous manual updates. Additionally, kicking off quarterly UARs was a matter of scheduling the event and then running those scripts manually. 

Goal 

The Databricks security team wanted a low-friction, scalable, and performant system for access governance that would: 

  • Eliminate standing access to sensitive resources, including customer data 

  • Enforce least privilege through just-in-time (JIT) access and session-based access to AWS 

  • Reduce the amount of toil required during quarterly Access Reviews 

  • Run self-hosted and meet FedRAMP High requirements (for select customers’ deployments) 

The Opal Solution 

Databricks adopted Opal to manage all time-bound access to sensitive resources within the company, and is the default provider for just-in-time access. 

Opal now governs Databricks’ most critical environments, including AWS resources and Okta groups, while a small number of external systems continue to be reviewed manually. 

Databricks implemented configurable approval chains in Opal to ensure that the right approvers were notified of a request in the event of an out-of-office or a manager on leave. 

“At Databricks, our workforce has grown exponentially over the last few years. By utilizing Opal's extensive APIs, our IT and Infrastructure teams have been able to stay on top of the needs of our end users and empower them to gain Just-in-Time (JIT) Access, Just Enough Access (JEA), and enforce Time Bound Access Controls (TBAC) for security and compliance. We now have tens of thousands of integration points with Opal, administering access to thousands of internal production and engineering systems.” 

—Jack Zaldivar, Jr., Staff Systems Engineer, Databricks

Today, Access Reviews are largely automatic, scheduled events. Most human involvement comes in the form of remedying overly permissive or standing privileges wherever they crop up, rather than maintaining revisions of spreadsheets. 

Key Results 

  • Improves the persona-based entitlement process for new hires: The security team no longer has to ask managers if their toolset/needs have changed; using Databricks, the team can apply “fuzzy matching” to new request sets, and automatically get notified when needs change 

  • Accelerated deployment velocity: Engineers now receive production access in minutes instead of hours or days 

  • Self-hosted deployment compliant with FedRAMP High: Databricks now runs two instances, one as SaaS (Opal-hosted) and the other self-hosted for FedRAMP compliance. 

  • Friction-free and largely automatic access reviews: GRC teams require quarterly access reviews, and Opal kicks them off automatically, allowing request and telemetry reviews in-app, right next to the ability to revoke access that persists where it shouldn’t. 

  • Growing usage for access requests: 710% in requests from 2023 to 2025

  • Productivity gains through time saved: a nearly 97% decrease in median time taken to either approve or deny those requests 

Impact and Scope 

Opal became a key control for compliance at Databricks. Opal’s UARs automatically generate compliance reports for auditor review that specify sync frequency, as well as timestamps and telemetry for all access requests across every entitlement. Every time-bound and session-based access to sensitive systems at Databricks is routed through Opal, helping Databricks maintain a least-privilege security posture. Approval chains are created to ensure that engineering work isn’t gated on an approver who is on PTO or parental leave. 

By integrating Opal’s APIs with Databricks notebooks, the team is able to generate custom reports and analyze access and authorization data by Okta group and see how it changes over time. The Databricks Security team is experimenting with Opal’s MCP server to automatically adjust groups, remediate access anomalies, and generate reports on the changing entitlement needs of various teams. 

As Databricks’ enterprise platform offerings now extend to LLM agents and MCP-compatible products, the Security Team plans to secure employee usage of agents and bearer tokens alike in Opal, and leverage Opal’s MCP server to keep up with the ephemeral nature and access patterns of these agents.

“Before implementing Opal, most of our access provisioning that wasn’t automated through our SSO platform was handled manually. Approvals had to be requested and documented manually,

which became increasingly difficult as our company scaled rapidly. Tracking ownership and ensuring compliance turned into a major operational challenge, and manual processes left more room for human error. Implementing Just-in-Time (JIT) access was nearly impossible without adding significant overhead. 

With Opal, we were able to transition quickly to a self-service access model that supports both JIT and Just-Enough-Access (JEA). This shift not only streamlined our provisioning workflows but also empowered teams with the right level of access, exactly when needed. Once our access data was centralized in Opal, generating user access reviews became effortless. 

Opal has allowed us to scale securely and efficiently — transforming what used to be a time-consuming, error-prone process into an automated, auditable, and compliant system.”

See why the best security teams manage access with Opal

See why the best security teams manage access with Opal

See why the best security teams manage access with Opal

See why the best security teams manage access with Opal