Spark Reads Files.com Over s3a
Treat your Files.com site as an S3-compatible store, and Spark reads and writes your folders directly through the S3-compatible endpoint. No staging bucket, no copy step, no new library.
Ingest Files Into Databricks Straight From Files.com
Databricks can read from any S3-compatible endpoint — and Files.com is one. Spark’s s3a connector reads and writes your Files.com folders directly, with no staging bucket in between. Any source filesystem you mount behind those folders comes along for the ride.
No credit card required · 7-day free trial · Live in minutes
Every protocol included
No credit card to start
Live in minutes
4,000 organizations rely on Files.com every day
Real companies. Real file flows. Real results.
How Files.com Fits With Databricks
Databricks reads object storage. The data a lakehouse actually needs starts everywhere else: counterparties on SFTP, business systems exporting on schedules, teams with files on shares the cluster cannot reach. The standard answer is a landing bucket plus the glue you write around it: sync scripts, a decryption step, hand-carried loads for the awkward sources, and no monitoring on any of it.
Files.com is the landing zone Spark reads directly, with the file layer already built. The S3-compatible endpoint makes your site the store the cluster reads; mounts and the Agent bring in the filesystems Databricks can’t reach; and Automations, PGP, and the audit trail govern everything around the job.
The Databricks File Flows Teams Actually Run
Direct s3a reads, on-prem sources reached, and client intake landed, all running on Files.com in production.
Any Source Filesystem Behind The Endpoint
Mount S3, Azure, SharePoint, or an on-premises share behind the same folders and Spark reads through them in real time. That pass-through gets a filesystem Databricks can’t reach into a job it can run.
The Share Databricks Cannot Read
Databricks can’t read an on-premises SMB share. The outbound-only Agent carries those files out through the firewall without an inbound rule, and they land where the cluster already reads. The analytics feed moves without rebuilding the estate behind it.
Client Data Landed For The Lakehouse
Clients deliver files however they can: SFTP, browser, whatever their side supports. Each lands in its own scoped folder, validates on arrival, and waits where the data science team’s jobs read. No engineer copies files by hand in the middle.
Output Delivered Back Down To Earth
Jobs write results back through the same endpoint, and Automations deliver them onward: to the analysts who work on local machines, to a counterparty’s server, or to the archive that keeps the run’s record.
The Feed That Went Quiet
A scheduled job over an empty folder succeeds at processing nothing. Delivery monitoring knows when each upstream feed is due and raises an alert when the window closes without a file, hours before anyone asks why the numbers look stale.
What The Databricks Connection Needs
Pointing a cluster at a new endpoint raises the same questions every time: credentials, permissions, governance, and formats.
The s3a Configuration
Point Spark’s S3A connector at the Files.com endpoint with an S3-compatible key pair. Set it at the cluster level to cover every notebook, or at the session level to test before committing a cluster change.
Keys That Inherit Real Permissions
The S3-compatible API key belongs to a Files.com user and inherits that user’s folder access exactly. Read permission loads data; write permission lets jobs put results back. Sign in as that user first and confirm it sees what the cluster should.
Where Unity Catalog Stands
This connection is cluster-scoped S3A access, not a Unity Catalog external location. Unity Catalog’s cloud-IAM role assumption isn’t part of the S3-compatible credential model. Governance over the exchanged files lives on the Files.com side: permissions, audit, retention.
The Formats Spark Expects
Whatever the job reads (CSV, JSON, Parquet, Avro, ORC), Automations and TransformScript can rename, convert, and route inbound files into that shape before the cluster ever lists the folder.
Customers Running Files.com With Databricks
Banking & Finance
Calamos Investments Eliminates Copy Jobs by Landing Vendor Files Where Databricks Reads Them Directly
Files.com preserves the SMB paths behind Calamos’s batch estate, lets vendors cut over one at a time, and sends migrated analytics feeds to Azure without a second copy job.
Read story →
Health & Life Sciences
The Public Interest Company Routes Client Claims Data Through Files.com Into Databricks for Processing
Files.com gave TPIC one governed way to meet client-mandated transfer requirements while keeping regulated claims data segregated and auditable.
Read story →
Services
Bates White Moves Databricks Output From S3 to the Economists
A one-way S3 sync through the Files.com Agent now carries new and updated cloud output onto on-premises storage without changing how analysts work.
Read story →
Marketing & Advertising
Marketbridge Lands Client Data for Its Databricks Team Without an Engineer Copying Files
Remote Server Mounts let Marketbridge close a TLS audit finding, automate client-data intake, and govern access without breaking its analytics pipeline.
Read story →
“Files.com's strengths are simplicity, ease of use, and the cloud connectors. We don't have to invent custom infrastructure for every partner.”
Tommy Chapley
Senior Software Engineer, Equifax
“Files.com is versatile — it can manage many different situations from a single platform. We've consolidated multiple tools onto it.”
Regis Litre
Chief Information Officer, Rag & Bone
“Files.com is robust and scales to a large enterprise. We get multiple files per minute, per second — and we're a 24/7 organization, so everything has to always be up.”
Nelson Miranda
Sr. Systems Engineer, Spirit Airlines
Files.com With Databricks FAQ
What data engineering teams ask most when wiring file intake to Databricks.
Put Files.com Under Your Lakehouse
Receive, transform, and stage every file your clusters read, on one governed platform Databricks reads directly.
No credit card required • Free for 7 days • Live in minutes