Skip to main content

Fred Hutch Automates Terabyte-Scale Genomic Delivery With Disposable Files.com Accounts

Files as large as 500 GB now move from on-premise storage to outside researchers without a hand-built delivery channel for every customer.
Fred Hutchinson Cancer CenterFiles.com

Fred Hutchinson Cancer Center is an independent, nonprofit cancer research and treatment institution in Seattle. It holds the only National Cancer Institute comprehensive cancer center designation in Washington state, and its researchers have won three Nobel prizes, the first for pioneering bone marrow transplantation. Alongside patient care and its own research, Fred Hutch runs a third, less visible line of business: Shared Resources, a set of core scientific facilities whose services are sold to investigators at outside academic and industry organizations around the world.

One of those cores is the Genomics & Bioinformatics shared resource. It operates as a service bureau. External researchers pay it to run whole-genome sequencing, exome sequencing, expression profiling, and other genomic services, and the bureau delivers the resulting data back. Every job it takes ends the same way: with terabytes of one customer's genomic data that have to leave the building, reach that customer, and reach no one else.

A 500 GB File, for One Customer Only

A modern sequencing run produces files that fit nowhere ordinary. Individual compressed files run to hundreds of gigabytes. One delivery the bureau described was roughly 2.7 TB across 44 files, with single files as large as 500 GB. Email cannot carry them, and consumer sharing tools cannot either. And every batch belongs to exactly one paying customer, in a research and healthcare context where a recipient must never be able to see another customer's samples.

Meeting those two requirements at once made every delivery an administration project. An administrator hand-built a credentialed account for each recipient, staged that customer's data to it from Fred Hutch's on-premise Isilon storage, watched for the download, and then cleaned up by hand: the account, the files, and the storage they occupied. Data nobody remembered to remove sat on paid storage indefinitely.

The entire external delivery operation is run by one person. So the real constraint on the bureau was never sequencing capacity. It was administration. The account work scales one-for-one with customers, and every hour of it came out of the same person's week. The bureau had outgrown hand-administered delivery: each new sequencing customer meant another account to build, another delivery to watch, and another teardown to remember.

Fred Hutch replaced that work with a script-driven Files.com workflow built around disposable, per-customer accounts.

Four Requirements Pulling in Different Directions

What a fix had to do was demanding on four fronts at once. Recipients are researchers at other institutions, so delivery had to run over a protocol their existing tools already speak, meaning SFTP rather than a bespoke client. The file sizes ruled out anything browser-shaped. Isolation had to be absolute, per recipient. And delivered data could not be allowed to accumulate: once a customer collected their samples, the storage had to come back.

A conventional SFTP server gets the protocol right and nothing else. Isolation and lifecycle stay manual. Someone still creates each account, scopes it, monitors it, and remembers to delete it, and with one operator that is precisely the work that does not scale.

Fred Hutch's Scientific Computing team wrote the required workflow down in five steps: create a user, create a matching folder, scope the user to that folder, expire the files on schedule, and delete the account after the recipient collects their samples. Every step had to be callable from a script, because no step could be allowed to consume an administrator.

Fred Hutch selected Files.com to run that lifecycle.

An Account That Exists for Exactly One Delivery

Files.com sits between the Isilon storage where sequencing data originates and the external customers who collect it, and its job is to make the recipient account disposable.

Datasets are pushed from the on-premise Isilon system up to Files.com. For each customer, the bureau creates a dedicated user account programmatically through the Files.com REST API. The account exists for one delivery and belongs to one recipient. The site runs in Files.com's HyperPrivacy mode with per-user home directories, so a recipient who logs in sees exactly one thing: their own samples. Isolation is not a permission somebody has to set correctly on every delivery; it is how the site is built.

Delivery runs over SFTP. An investigator at another institution collects a 500 GB file with whatever standard client they already use. Nothing to install, nothing to explain.

Cleanup is where the design pays for itself. Files.com Automations and folder-level file expiration govern the data lifecycle, so delivered files age out on schedule instead of waiting for someone to remember them. Once a recipient has collected their samples, the same REST API that created the account deletes it, and the storage it occupied is reclaimed. The loop finishes clean without a human in it.

One Person, the Entire External Customer Base

With that lifecycle in production, delivering a customer's genomic data stopped being an administration project and became a script.

  • One person runs the entire external delivery operation, from provisioning through delivery to teardown, for the bureau's whole external customer base, at batch sizes measured in terabytes.
  • The storage footprint no longer grows with the customer count. Every delivery reclaims its own storage on the way out, so serving the next customer never means paying to retain the last one's terabytes.
  • Private delivery no longer depends on anyone getting a permission right. A recipient can only ever reach their own home directory, on every delivery, by construction.
  • Taking on another sequencing customer costs an API call rather than an account-administration project.

Delivery Capacity That Scales With Customers, Not Administrators

That is the portable lesson in Fred Hutch's setup. A per-recipient account can be treated as disposable infrastructure, provisioned by API for a single delivery and deleted the moment its job is done, so that external data delivery scales with the number of customers served rather than the number of administrators or the terabytes retained.