The first three phases decided who gets in, how environments are separated and who sees which data. Phase 4 is about the roads between them: which routes your data can take into and out of the platform.

Network design is the least visible part of a data platform. Nobody notices it when it works. But it decides whether a stolen password is enough to get in, and whether a single line of code can send data to any place on the internet.

The question for leadership

Could your data leave the platform without anyone noticing?

If nobody in your organisation can name every route in and out, assume the answer is yes. That matters for security, for compliance, and for every customer or regulator who asks where their data can go.

Where it goes wrong

Most networks are not designed badly. They are not designed at all, because the pilot worked without it.

  • Open to the whole internet. The workspace can be reached from anywhere. A stolen password works just as well from a café abroad as from the office.
  • Outbound to anywhere. Clusters and serverless compute can call any website or API. One line of code in a notebook is enough to send a table somewhere nobody approved.
  • Built for the pilot. The network was set up quickly for the first use case: subnets too small to grow, no plan for private connections or DNS. Growing means rebuilding.

Five building blocks

Databricks’ own network guidance and reference architectures point to the same pattern we implement for clients.

1. Your own network, sized to grow

Classic compute runs in a network you control: a customer-managed VPC on AWS or VNet on Azure. You decide the address ranges, routing and security rules. Size the subnets for where you are going, not where you are. Every compute node uses two IP addresses, so a subnet that fits the pilot will not fit production.

2. No public IPs on compute

With secure cluster connectivity, which is the default, compute nodes have no public IP addresses and no open inbound ports. Clusters open a connection to the Databricks control plane themselves; nothing on the internet can open a connection to them.

3. Known entry points only

Users and tools can only reach the platform from networks you know, such as the office or the corporate VPN. IP access lists at account level enforce this for every workspace at once. For sensitive environments, Private Link takes traffic off the public internet entirely.

4. Outbound only to approved places

Data may only leave for destinations you have approved: your storage, internal systems, specific services. For classic compute, a firewall or egress filtering enforces this. For serverless compute, egress control policies and network connectivity configurations do the same, with stable IP addresses that your storage and databases can allow. Use separate configurations per environment.

5. A security level that fits the data

Not every environment needs the same level. Databricks describes three reference architectures: managed security as the baseline, hardened connectivity with private links and egress control, and a fully isolated environment for regulated industries. Choose a level per environment on purpose, and write the choice down.

Four checks for your leadership team

You don’t need to read a network diagram to know whether this phase is in order. Ask these four questions:

  • Can someone log in from a network we don’t know?
  • Do any of our clusters have a public IP address?
  • Can a notebook send data to any website it likes?
  • Have we chosen a security level per environment, deliberately and on paper?

If the answers are no, no, no and yes, you know how your data travels.

Next up: Phase 5, Storage

With the routes in place, the next question is the destination: where your data actually lives, who can reach that storage, and how it is protected. That is Phase 5.

This article is part of our series Databricks in Production. Want to know where your platform stands? Get in touch with Ferdinand on LinkedIn.