The Very Group Retires Its Custom SFTP-to-S3 Pipeline by Mounting S3 Directly in Files.com

The Very Group is the UK and Ireland's largest integrated digital retailer and flexible payments provider. Through Very and Littlewoods it sells fashion, electrical and home ranges from around 2,000 brands to 4.2 million active customers, and it finances much of that shopping through its own regulated consumer-credit business.

An operation like that runs on data as much as on parcels. The group's retail data platform lives in AWS: S3 buckets, access governed by the group's own IAM roles, a lakehouse fed by batch files. And much of the data feeding it comes from outside. External partners supply and collect files for the platform, and they connect the way external counterparties always do: over SFTP. So the group's partner data exchanges started life at a file-transfer perimeter and needed to end up in S3. The distance between those two points is where the problem lived.

A Pipeline Whose Only Job Was Moving Files Already Received

For years, the group's own code covered that distance. A partner file landed at the SFTP perimeter, a webhook fired, and an in-house process fetched the file and wrote it into AWS. The perimeter and the data platform were never actually connected. They were bridged by something the Data Platforms team had to build, run, and secure.

That bridge taxed the team three ways. Engineers maintained integration code that was pure plumbing, moving files between two systems the group already operated. The code ran on AWS resources that existed for no purpose other than shuttling files. And it authenticated into AWS with long-lived access keys, a class of credential the group's security team required to be rotated frequently, which turned every key into a recurring chore. Every partner exchange for the retail data platform depended on that middleware, so every new exchange added to the load.

The arrangement survived because neither endpoint could bend. Partners speak plain SFTP and nothing else. The data platform lives in S3, with access governed by IAM. Custom code was the only bridge available, until the transfer layer itself could speak natively to the group's storage.

In March 2024 it could, and the group's data engineering team scoped the change: retire the webhook-fetch-then-write process and have partner files placed directly into S3. The team framed the move as a modernisation, an improvement to the security of how they integrate with AWS, and a saving on AWS resources that would no longer be required. Files.com offered two routes, direct S3 integration or notification through AWS SNS, and the team chose S3 for a blunt reason: it would cost less engineer time to implement.

Mounting the Group's Own Buckets Into Files.com

What the team built made Files.com the SFTP face of the group's own AWS estate.

Using Files.com Remote Server Mounts, the team backed partner-facing folders directly with S3 buckets in the group's AWS accounts. A partner uploads over SFTP and the file is already in the group's storage, in a single step, with no intermediate copy and no code in between. The exchange runs both ways: partners collect outbound files from the same folders, reading straight out of the group's buckets.

The connections into S3 run under IAM policy, the group's own roles, rather than long-lived access keys. That removed from this path the very class of credential the security team had been forcing the team to rotate.

Two integration styles run in production, chosen per exchange. Direct mounts serve the integrations where a live view of the bucket is the right shape; Files.com Sync serves the ones where scheduled movement fits better. After running both, the team converted directories from Sync to mounts wherever the mount proved the better fit.

The whole arrangement is declared in Terraform. Remote servers, syncs and folder behaviours are written once in the Files.com Terraform provider and applied across development, test and production, so a small team configures the pattern once and repeats it everywhere.

Ingestion Runs on Files That Are Already Home

With files landing directly in S3, downstream processing keys off arrival rather than off a pipeline. For lakehouse ingestion, a control file signals when a complete batch is present, so processing starts only on whole batches and a half-loaded batch never runs.

The cutover was incremental rather than a single weekend. The first mounts went live in mid-2024, and exchanges moved across one at a time through 2026, with the legacy path retired as each integration landed on its mount.

The Fetch-and-Write Middleware Is Gone and the Pattern Repeats

With the mounts in production, The Very Group replaced a bespoke fetch-and-write pipeline with a standing property of its architecture: partner files land in the group's own storage because the transfer layer is mounted on it.

• The in-house fetch-then-write integration is retired, along with the AWS resources that existed only to run it. Nobody maintains code whose job was moving files between two systems the group already ran.
• Partner exchange for the retail data platform runs through one governed path. Four external partners drop and collect files that read and write directly against the group's S3 estate.
• The S3 connections run under the group's own IAM roles instead of long-lived access keys, so a set of credentials that demanded frequent rotation no longer exists on this path.
• The next exchange is a template, not a project. Adding one is a folder backed by a mount, declared in Terraform and applied identically across three environments. In 2026 the pattern crossed team boundaries: Data Platforms mounted a bucket owned by another squad and handed that squad working read, write and delete access to its own storage through Files.com.

The workload this carries is not a side channel. Today it is part of a live service.

One Layer Fewer Between Partners and the Platform

The Very Group did not adopt a new pipeline product. It deleted the pipeline. Files.com and the data platform now share the same storage, which means the glue code between an SFTP perimeter and cloud storage turned out to exist for one reason only: the transfer layer treated the storage as somewhere else. Mount the storage, and moving a file into AWS stops being a step anyone performs. It is simply where the file lands.