Nasdaq Data Link Brings Small Data Vendors Into Its Marketplace With Files.com—Without Running Its Own SFTP

Nasdaq operates the Nasdaq Stock Market and builds the technology behind markets around the world. More than 130 markets run on its systems. Alongside trading and listings, Nasdaq sells data: Nasdaq Data Link, the marketplace built on Quandl after Nasdaq acquired it in 2018, offers financial, economic, and alternative datasets to investment managers, banks, brokers, and corporates. Its catalogue indexes millions of time-series datasets drawn from more than 400 sources.

Much of that supplier data starts life as files delivered by a producer. The marketplace aggregates other companies' data, so its intake depends on file delivery from independent data producers. For smaller producers unable to host delivery infrastructure, Files.com became Quandl's branded SFTP front door. That endpoint remained in place after the acquisition and now handles roughly three million API transactions a day.

The Long Tail of Data Vendors Had Nothing to Deliver With

Data Link's suppliers split into two groups. Large vendors run their own delivery infrastructure, and Nasdaq connects out to their SFTP servers directly, point to point. The long tail does not. Small and alternative data producers often have no servers, no integration team, and no way to deliver files at all. Some only need to send samples.

Without a delivery channel for that segment, Quandl had two options, both bad. It could turn those suppliers away, losing exactly the alternative data the catalogue was built to carry. Or it could build the channel itself: stand up a partner-facing SFTP service, provision credentials for every vendor, and keep it all patched, monitored, and running around the clock. Quandl was a startup whose small engineering team existed to build the data product. Building the channel meant that team running file-transfer plumbing instead.

And whatever they stood up would have to hold still for years. Once a vendor's delivery script points at an endpoint, changing anything on the vendor's side is a lengthy process. The intake also sits at the head of the pipelines that feed the marketplace's clients, so an unstable endpoint would show up downstream, in what customers see.

What the Delivery Layer Had to Do

The requirements described the gap precisely. The endpoint had to carry Quandl's own domain, so vendors delivered to Quandl and not to a visible third party. Every vendor needed a separate credential and its own namespace, so no supplier could see another's data. Internal ETL jobs needed an API they could poll continuously to find what had landed. A vendor had to be able to deliver with whatever SFTP client it already had, installing nothing. And operating all of it had to stay off Quandl's engineering team's infrastructure workload.

Quandl selected Files.com to provide that vendor-facing delivery layer. The engineering team's reasoning was plain: it did not have the IT capacity to roll its own SFTP infrastructure, and buying the layer made more sense than building it.

A Credential Per Vendor, and ETL That Never Stops Asking

Each small vendor uploads over SFTP to an endpoint on Quandl's own branded domain, under its own namespaced credentials, into its own folder. Whatever client the vendor already runs is the client it uses. On the other side, Nasdaq's Ruby and Python ETL tooling polls the site continuously through an internal service account, over SFTP and the Files.com REST API, listing folders and checking file status to see what has arrived and pulling new files into downstream pipelines. Roughly three quarters of all traffic on the account is those listing and status operations. The channel is a live integration surface, not a folder someone checks by hand.

Using Files.com Automations, delivered files move into historical folders once they have been collected, so a vendor's folder holds only what is new and no supplier has to track what it already sent.

Three Million API Transactions a Day, and Nothing to Patch

With Files.com in production as the delivery layer, the pattern has held ever since.

• Onboarding a new data vendor is a credential and a folder, not an infrastructure project. The marketplace can take delivery from any supplier, including one that exists only to send a sample, regardless of what it runs on its own side.
• Daily API transactions grew from roughly two million to roughly three million in a year. That growth was absorbed without Nasdaq building, buying, or staffing any file-transfer infrastructure of its own.
• The pattern set up at Quandl carried unchanged through the acquisition into Nasdaq. Vendors never repointed a script. The same endpoint that served a startup now serves a global market operator.
• Nasdaq's data engineers run pipelines, not plumbing. There is no partner-facing server for them to patch, monitor, or staff.

The Front Door Carried From Quandl Into Nasdaq

Today, when Nasdaq Data Link signs a data supplier too small to run delivery infrastructure of its own, the vendor gets a credential on Files.com, the ETL jobs start finding its files, and the engineering team stays on the data product. A supplier's size stopped being a reason it couldn't be in the catalogue.