Skip to main content

Calamos Investments Replaces SolarWinds Serv-U Without Rebuilding Hundreds of Batch Jobs

Files.com preserves the SMB paths behind Calamos’s batch estate, lets vendors cut over one at a time, and sends migrated analytics feeds to Azure without a second copy job.
Calamos InvestmentsFiles.com

Calamos Investments is a global asset manager with more than $50 billion under management. Founded in 1977 as a convertible-securities boutique, it is now the largest US manager of convertibles and one of the largest liquid-alternatives providers in the country, running mutual funds, ETFs, closed-end funds and UCITS vehicles for advisors, institutions and individual investors worldwide.

Every one of those funds runs on data that arrives from outside the firm. Between 60 and 100 custodians, fund administrators and market-data vendors deliver NAV, holdings, shadow accounting and trade files on daily, monthly and quarterly cycles. There is regulation behind that data, and there are live investment decisions on top of it: a start-of-day file that hasn't updated can show cash in a fund that is not actually there.

Replacing SolarWinds Serv-U could not mean rebuilding the hundreds of batch jobs that depended on it. Calamos needed to preserve the paths those jobs watched, let each vendor move separately, and stop requiring a second copy job for cloud analytics. Files.com gave the firm a way to do all three.

One Server, Two Jobs for Every Feed

For more than a decade, all of it landed in one place: an on-premise SolarWinds Serv-U SFTP server. ActiveBatch, the firm's global batch scheduler, watched that server's file share over SMB and fired hundreds of file-triggered jobs the moment data appeared, feeding the custom .NET code that loads Calamos's investment systems and reports.

Then the analytics stack moved to the cloud, and the arrangement started charging double. Databricks cannot read an on-prem SMB share, so every analytics feed needed a second leg: another job to copy each file onward into SFTP-fronted blob storage before Databricks could consume it.

I have to manage an ActiveBatch job, and I have to manage a Databricks job. That's double the work and double the risk.
Bill Heller, VP of Data Engineering, Calamos Investments

The server itself demanded patching, encryption key management, disaster-recovery replication across roughly 30 on-prem machines, and a set of failover IP addresses that every vendor had to whitelist. The economics had inverted, too: running an SFTP layer on top of blob storage was approaching the cost of a complete managed file transfer platform. Calamos had outgrown its landing point. The front door for the firm's most time-sensitive data belonged to an architecture the rest of the firm had already left.

We were sick of maintaining on-prem infrastructure related to this.
Jason Bertini, VP of Cloud & Infrastructure, Calamos Investments

Why Retirement Sat on the Sideline for Years

Nobody at Calamos defended the server. Retiring it had been discussed for years, and the project kept sliding, because the arithmetic was brutal.

Hundreds of jobs trigger on files appearing at exact SMB paths, and ActiveBatch cannot trigger on blob storage. Any replacement either landed files somewhere SMB-reachable or forced a rebuild of a batch estate assembled since 2007.

Every vendor would have to change hostname and credentials individually. Some connect by IP address and need firewall changes on their own side. Several are banks that accept configuration changes only once a quarter. A synchronized cutover of 60 to 100 vendors was never going to be scheduled.

And after 10 to 15 years, the estate had gone dark in places. Seventy-eight active SFTP accounts had accumulated. Each vendor process had two to four internal owners, and in many cases nobody had a current record of who the vendor contact even was. The team matched roughly 200 legacy jobs back to vendor names before outreach could start.

What forced the issue was the firm's broader cloud migration. On-prem file shares were being retired and analytics was moving off legacy SQL Server onto Databricks, so every new feed added another pair of jobs to manage and another workload anchored to the machine everyone wanted gone.

The replacement had four requirements: receive vendor files over SFTP under Calamos's own domain; write them into an SMB-reachable share preserving the existing folder and file path structure, so the batch estate kept running; write natively into Azure Data Lake Storage Gen2, so Databricks reads files directly and the copy jobs disappear; and run alongside the old server, so each vendor could move on its own schedule. Calamos selected Files.com to be that ingestion layer.

One Landing Point for the Batch Estate and the Data Lake

Files.com now fronts inbound vendor traffic on Calamos's own branded domain, running in parallel with the legacy hostname while vendors migrate one at a time. The design gives Files.com a single job: be the one landing point that both eras of the firm's data platform can consume.

Behind that front door, Files.com Remote Server Mounts connect folders directly to Azure storage, so a file's arrival on Files.com is its arrival in Azure. Feeds bound for the batch estate write into Azure Files, which is reachable over SMB, under the same folder and file path structure the old server used, so existing jobs need only a path change. Feeds bound for analytics land natively in Azure Blob and Data Lake Gen2, where Databricks reads them directly. One landing serves both destinations, and no feed needs a second job to exist in the cloud. The Files.com On-Premise Agent handles connectivity to systems still on-prem over private, outbound-only connections.

The migration itself runs on a build-once pattern. Each vendor gets a Files.com Partner folder in its final location while still sending to the old server, and a Files.com Automation pulls its files across. Everything downstream—including the Automations that route files and the Expectations that define required arrival paths and windows—is built against that final destination. When the vendor points its connection at Files.com, the interim pull is retired and nothing else changes. Files also flow back to the legacy server for downstream processes not yet moved, so a vendor's cutover never waits on downstream work, and downstream work never waits on a vendor.

Vendors Cut Over One at a Time, and Nothing Downstream Notices

With the Files.com front door in production, Calamos traded an all-or-nothing cutover it could never schedule for a vendor-by-vendor migration that nothing downstream notices.

  • Hundreds of ActiveBatch file-trigger jobs and the custom .NET codebase run unchanged against the new location. The only edit was the path they watch, which the team called the easiest shift of the entire program.
  • Analytics feeds are managed once, not twice. Files land natively in Azure Data Lake Gen2 and Databricks reads them there, so the parallel copy job, and its parallel failure mode, is gone from every migrated feed.
  • Each vendor cuts over on its own schedule, including the banks that accept changes once a quarter. A cutover is a hostname and credential change on the vendor's side and one retired copy job on Calamos's side.
  • The next vendor follows a pattern that already exists: a Partner folder, an Automation, an Expectation. The first migrations were built deliberately as the template for the rest of the population.

New feeds now skip the old server entirely; a recent market-data feed was stood up directly on Files.com and never touched the legacy machine. And the people closest to the files run their own piece of the platform: Data Engineering's eight-person on-call rotation troubleshoots file issues directly, with administrative rights over its own workspace and read-only visibility elsewhere, instead of routing every fix through a site administrator. A retirement that sat on the sideline for years is now a running program, emptying the old server one vendor at a time, with its decommissioning as the stated finish line.

Moving the Landing Point Without Moving the Paths

Today a vendor file arrives once, on Files.com, and reaches both of the worlds Calamos operates in: the batch estate the firm built up since 2007 and the Databricks platform its analytics are moving to. For the engineers who own the feeds, a feed is one job again instead of two, and standing up the next vendor means applying a pattern rather than designing one.

Calamos never had to choose between keeping an aging server and rebuilding hundreds of jobs. Files.com moved the landing point; the paths, and everything that triggers on them, stayed exactly where they were. That is what finally unstuck a retirement years in the discussion: the old server could be emptied one vendor at a time, and nothing downstream would ever notice.