Spark Reads Files.com Over s3a
Treat your Files.com site as an S3-compatible store, and Spark reads and writes your folders directly through the S3-compatible endpoint. No staging bucket, no copy step, no new library.
Ingest Files Into Databricks Straight From Files.com
Databricks can read from any S3-compatible endpoint — and Files.com is one. Spark’s s3a connector reads and writes your Files.com folders directly, with no staging bucket in between. Any source filesystem you mount behind those folders comes along for the ride.
No credit card required · 7-day free trial · Live in minutes
#1 in every G2 rating for MFT
Trusted by 4,000+ companies
Every protocol included
4,000+ organizations rely on Files.com every day
Real companies. Real file flows. Real results.
How Files.com Fits With Databricks
Databricks reads object storage. The data a lakehouse actually needs starts everywhere else: counterparties on SFTP, business systems exporting on schedules, teams with files on shares the cluster cannot reach. The standard answer is a landing bucket plus the glue you write around it: sync scripts, a decryption step, hand-carried loads for the awkward sources, and no monitoring on any of it.
Files.com is the landing zone Spark reads directly, with the file layer already built. The S3-compatible endpoint makes your site the store the cluster reads; mounts and the Agent bring in the filesystems Databricks can’t reach; and Automations, PGP, and the audit trail govern everything around the job.
Partner and vendor feeds land the same way whichever storage the warehouse reads: a drop point per sender, and every file in your own bucket the moment it arrives.
The Databricks File Flows Teams Actually Run
Direct s3a reads, on-prem sources reached, and client intake landed, all running on Files.com in production.
Any Source Filesystem Behind The Endpoint
Mount S3, Azure, SharePoint, or an on-premises share behind the same folders and Spark reads through them in real time. That pass-through gets a filesystem Databricks can’t reach into a job it can run.
The Share Databricks Cannot Read
Databricks can’t read an on-premises SMB share. The outbound-only Agent carries those files out through the firewall without an inbound rule, and they land where the cluster already reads. The analytics feed moves without rebuilding the estate behind it.
Client Data Landed For The Lakehouse
Clients deliver files however they can: SFTP, browser, whatever their side supports. Each lands in its own scoped folder, validates on arrival, and waits where the data science team’s jobs read. No engineer copies files by hand in the middle.
Output Delivered Back Down To Earth
Jobs write results back through the same endpoint, and Automations deliver them onward: to the analysts who work on local machines, to a counterparty’s server, or to the archive that keeps the run’s record.
The Feed That Went Quiet
A scheduled job over an empty folder succeeds at processing nothing. Delivery monitoring knows when each upstream feed is due and raises an alert when the window closes without a file, hours before anyone asks why the numbers look stale.
What The Databricks Connection Needs
Pointing a cluster at a new endpoint raises the same questions every time: credentials, permissions, governance, and formats.
The s3a Configuration
Point Spark’s S3A connector at the Files.com endpoint with an S3-compatible key pair. Set it at the cluster level to cover every notebook, or at the session level to test before committing a cluster change.
Keys That Inherit Real Permissions
The S3-compatible API key belongs to a Files.com user and inherits that user’s folder access exactly. Read permission loads data; write permission lets jobs put results back. Sign in as that user first and confirm it sees what the cluster should.
Where Unity Catalog Stands
This connection is cluster-scoped S3A access, not a Unity Catalog external location. Unity Catalog’s cloud-IAM role assumption isn’t part of the S3-compatible credential model. Governance over the exchanged files lives on the Files.com side: permissions, audit, retention.
The Formats Spark Expects
Whatever the job reads (CSV, JSON, Parquet, Avro, ORC), Automations and TransformScript can rename, convert, and route inbound files into that shape before the cluster ever lists the folder.
Customers Running Files.com With Databricks
A Global Asset Manager Eliminates Copy Jobs by Landing Vendor Files Where Databricks Reads Them Directly
Files.com preserves the SMB paths the batch estate watches, lets each vendor cut over on its own schedule, and lands analytics feeds in Azure with no second copy job.
Read The Story
A Claims Recovery Firm Routes Client Claims Data Through Files.com Into Databricks for Processing
Files.com gave the firm one governed way to meet client-mandated transfer requirements while keeping regulated claims data segregated and auditable.
Read The Story
An Economic Consulting Firm Automates Large Databricks-to-File-Server Transfers With Files.com
A one-way S3 sync through the Files.com Agent now carries new and updated cloud output onto on-premises storage without changing how analysts work.
Read The Story
A B2B Consultancy Runs Client Data Intake Through Files.com Over the Azure Storage Databricks Reads
Remote Server Mounts put a TLS 1.2-only, user-logged, directory-governed front door over the same Azure containers, with no migration and no second copy.
Read The Story
“Files.com's strengths are simplicity, ease of use, and the cloud connectors. We don't have to invent custom infrastructure for every partner.”
Tommy Chapley
Senior Software Engineer, Equifax
“Files.com is versatile — it can manage many different situations from a single platform. We've consolidated multiple tools onto it.”
Regis Litre
Chief Information Officer, Rag & Bone
“Files.com is robust and scales to a large enterprise. We get multiple files per minute, per second — and we're a 24/7 organization, so everything has to always be up.”
Nelson Miranda
Sr. Systems Engineer, Spirit Airlines
Files.com With Databricks FAQ
What data engineering teams ask most when wiring file intake to Databricks.
Databricks does not support SFTP natively. Spark reads from object storage, not from SFTP servers. Files.com closes the gap from both sides: counterparties deliver over SFTP, FTPS, AS2, or a browser, and Databricks reads those same files directly through the Files.com S3-compatible endpoint over Spark’s s3a connector.
Databricks connects to Files.com through Spark’s S3A connector, configured with the Files.com endpoint and an S3-compatible API key. Set it at the cluster level for every notebook, or at the notebook/session level for testing. From there, spark.read and spark.write work against your Files.com folders like any bucket.
The Files.com connection is cluster-scoped S3A access and is not governed by Unity Catalog. Unity Catalog external storage relies on cloud-provider IAM role assumption, which isn’t part of the S3-compatible credential model. Teams that need governed exchange run that governance on the Files.com side: per-user permissions, a full audit log, and retention rules on the folders the cluster reads.
Yes. Databricks can’t read an on-premises SMB share on its own, so the Files.com Agent carries those files to folders the cluster reads. The Agent runs inside your network and makes only outbound connections: no inbound firewall rule, no VPN, no rebuild of the systems that produce the data.
Yes. Remote Server Mounts present S3, Azure, Google Cloud, SharePoint, and more as Files.com folders, and s3a reads through them in real time: one endpoint for the job, with the bytes staying wherever they already live.
Yes. Files.com Automations rename, move, and route on arrival, and TransformScript converts CSV, XML, and JSON in the flow. The folder the cluster lists holds files already shaped for the read.
Files.com receives inbound data however the sender can deliver it: SFTP, FTPS, AS2, the S3 API, a browser upload, or an email inbox that turns attachments into files. Each counterparty gets its own scoped credential and folder, every arrival is logged, and PGP-encrypted files decrypt automatically before the job reads them.
Yes. With write permission on the key’s user, Databricks jobs write output through the same Files.com endpoint, and Automations deliver it onward: to counterparties over their protocols, to analysts’ working folders, or to an archive under retention policies.
Yes. Files.com runs SOC 2 Type II, encrypts in transit and at rest, and records every access in the audit log. A pre-approved HIPAA business associate agreement comes with Files.com Enterprise. The governance the S3A connection doesn’t carry, the platform underneath it does.
Files.com replaces the landing-bucket stack: a staging bucket, the sync scripts that feed it, the hand-carried loads from systems the cluster can’t reach, and the monitoring nobody built. One governed layer receives and stages the files, and the cluster reads it directly.
Put Files.com Under Your Lakehouse
Receive, transform, and stage every file your clusters read, on one governed platform Databricks reads directly.
No credit card required • Free for 7 days • Live in minutes