Databricks Interactive Runtime — Architecture from Process Signals An architecture diagram generated by Archify. Notebook / Client · JDBC · DB Connect · Architecture component Notebook / Client JDBC · DB Connect Control Plane · webapp API · command routing · Control plane — outside the cluster Control Plane webapp API · command routing ChauffeurDaemon · own JVM · supervisor · Driver node — one machine, two network layers ChauffeurDaemon own JVM · supervisor VM-layer Daemons · DBFS · IAM token · mTLS relay · Driver node — one machine, two network layers VM-layer Daemons DBFS · IAM token · mTLS relay Unity Catalog · ManagedCatalogClient · Control plane — outside the cluster Unity Catalog ManagedCatalogClient DriverDaemon · one driver JVM · one OS process · Driver node — one machine, two network layers › One driver JVM — one OS process DriverDaemon one driver JVM · one OS process DriverCorral · command/session/status dispatcher · Driver node — one machine, two network layers › One driver JVM — one OS process DriverCorral command/session/status dispatcher DriverILoop · Scala REPL · in-process · Driver node — one machine, two network layers › One driver JVM — one OS process DriverILoop Scala REPL · in-process REPL Process · ipykernel · Almond · non-root · Driver node — one machine, two network layers REPL Process ipykernel · Almond · non-root SparkConnectService · server · UDS grpc.sock · Driver node — one machine, two network layers › One driver JVM — one OS process SparkConnectService server · UDS grpc.sock CheckPermissions · analyzer rule — every path · Driver node — one machine, two network layers › One driver JVM — one OS process CheckPermissions analyzer rule — every path SparkContext · one shared Spark application · Driver node — one machine, two network layers › One driver JVM — one OS process SparkContext one shared Spark application Spark Master · own JVM · standalone · Architecture component Spark Master own JVM · standalone Worker → Executor · where Spark tasks run · Worker container Worker → Executor where Spark tasks run SafeSpark · gVisor sandbox · UDF · Sandbox container SafeSpark gVisor sandbox · UDF LokiFileSystem · driver and executor library · Architecture component LokiFileSystem driver and executor library Object Storage · S3 · ADLS · GCS · Architecture component Object Storage S3 · ADLS · GCS submit cell IAM / DBFS Scala Dedicated isolate user code logical plan in-heap py4j resolve permissions request executors I/O path temporary credentials Control plane — outside the cluster Driver node — one machine, two network layers One driver JVM — one OS process Worker container Sandbox container Legend Client Runtime process Catalog Cloud / managed service Isolation / enforcement Command dispatcher VM-layer service