In Phase 1 we decided who gets in. In Phase 2 we separated the environments. Phase 3 is about what happens inside them: who sees which data, and why.

In Databricks, that is the job of Unity Catalog. It is the governance layer for all your data and AI assets, from tables and files to models. Permissions, ownership, lineage and audit logs all live in one place. Set it up well, and every access decision can be explained. Let it grow by itself, and nobody can.

The question for leadership

For any sensitive dataset, can you show who can see it, who owns it and who used it last month?

In a healthy setup, that is a query. In many organisations it is a project: weeks of exports, spreadsheets and interviews, ending in an answer nobody fully trusts. That matters for privacy regulation, for audits, and increasingly for AI. Models and agents read the same data your people do.

Where it goes wrong

Access is rarely designed. It grows, one request at a time.

  • Grants per person, per table. Every request becomes a direct grant to one user on one table. After a year there are thousands of them, and nobody can explain why a given person sees a given table.
  • Owned by whoever built it. The engineer who created a table owns it. When that person changes role or leaves, nobody can change its permissions without an admin stepping in.
  • Copies instead of policies. To hide sensitive columns, teams create “anonymised” copies for each audience. The copies drift from the original, and the original stays open for anyone who finds it.

Five building blocks

Databricks’ own Unity Catalog guidance points to the same foundation we implement for clients.

1. A clear catalog structure

Catalogs are the main unit of isolation. Organise them by environment, by domain, or both, for example finance_prod and finance_dev. Keep the number small, a handful per metastore, and give teams and projects their own schemas rather than their own catalogs. One metastore per region covers all workspaces in that region.

2. Groups own, groups get access

Production catalogs and schemas are owned by a group, never by an individual. Permissions are granted to groups that follow real roles, as set up in Phase 1. Write access to production tables is reserved for service principals, so no person can accidentally overwrite production data.

3. Findable, not readable

Everyone in the organisation can browse what data exists, without seeing its contents. When people find something they need, they request access through the catalog, and the owning group decides. Discovery stops depending on who you know.

4. Sensitive data protected by policy

Instead of copies, sensitive data is protected where it lives. Columns are tagged, for example as personal data, and attribute-based access control (ABAC) policies on a catalog or schema apply column masks and row filters to every table carrying that tag. A new table with a tagged column is protected the moment it appears. Data Classification can help find sensitive columns that nobody tagged yet.

5. All access through the catalog

Data is stored as Unity Catalog managed tables and volumes. Nobody reads storage directly, through mounts or bucket permissions. External tables are the exception, used only when another system needs direct access. Because every read goes through the catalog, lineage shows where data comes from and where it goes, and the audit log shows who used what.

Four checks for your leadership team

You don’t need to open the platform to know whether this phase is in order. Ask these four questions:

  • Is every permission granted to a group, and is every production catalog owned by a group?
  • Are sensitive columns protected by a policy, or by a copy of the table?
  • Can anyone read data directly from storage, around Unity Catalog?
  • Can we show, within an hour, who used a given dataset last month?

If the answers are yes, a policy, no and yes, you know who sees what, and why.

Next up: Phase 4, Network

With access organised, the next question is the route data takes: between your cloud, Databricks and the internet, and who could listen in along the way. That is Phase 4.

This article is part of our series Databricks in Production. Want to know where your platform stands? Get in touch with Ferdinand on LinkedIn.