Skip to main content

Agent High Availability

A single Agent installation is a single point of failure. When its host is rebooted, its process crashes, or its network path fails, every workflow that depends on that Agent waits until the host recovers. Agent High Availability removes that dependency with two levels of protection, and both are included with the Agent. Failover is automatic at both levels, moving traffic to a healthy connection or new work to a healthy node without manual intervention.

The first level of protection covers the connection between the Agent and Files.com, and it is automatic. Every Agent keeps a standby connection to Files.com alongside its active one and moves traffic to the standby immediately when the active connection fails. The same Agent process keeps running, transfers in progress keep going, and there is nothing to configure.

The second level of protection covers the Agent host itself, by running multiple Agent installations, called nodes, for one Remote Server. Each node is a separate installation on its own host, every node is live and eligible to serve, and Files.com moves new work to another node when the serving node becomes unavailable. Losing a host no longer stops the workflows that depend on the Agent.

Most Remote Servers run a single Agent installation, and that's good enough when a short outage only delays a scheduled job that catches up on the next run. Set up multiple nodes when workflows depend on the Agent continuously, such as partners delivering files into your network at all hours, or exchanges where a missed window means a missed business deadline. The connection-level protection comes with every Agent either way, with nothing to set up.

Agent High Availability shares its name with a different feature for mounted storage. If you're not sure which one you need, see Comparing Agent High Availability to Related Features below.

Setting Up Multiple Nodes

Before adding a node, have the pieces in place. Every node needs a host of its own, access to the same files as the other nodes, and a plan for roles if you want a preferred node rather than equal peers. When the Agent uses Direct Transfers, the new node's host also needs the same firewall rules and port forwarding as the existing nodes before it can serve transfers over the direct connection.

Install the Agent on each additional host using a copy of the existing installation's configuration file, then update the copy before starting the Agent service. The settings that connect an installation to its Remote Server, which are id, private_key, subdomain, and endpoint_override, must stay identical in every copy, because they are what make each installation serve the same Remote Server. The settings that describe an individual node must be changed in each copy. Give each node its own node_name, its own upload_temp_file_part_path directory, and its own availability_role when you are using the primary and standby roles. The root must resolve to the same shared storage from every host, and the path itself can differ when hosts mount that storage at different locations.

Update the copy even when the file you copied already carries high availability settings from an existing node. A copy started unchanged reports the first node's name and writes into the first node's upload cache directory, so change those values before the new node's first start.

Each installation identifies itself to Files.com as a separate node when it first connects, and Files.com discovers the nodes automatically. Two or more nodes running the default auto role activate multi-node routing on their own, with no enablement step. Install each additional node fresh instead of cloning a configured host.

Choosing Node Roles

The availability_role setting controls which node Files.com prefers. Every node is running, connected, and eligible at all times, so failover involves no promotion step. Files.com sends each new job to one selected node at a time rather than spreading jobs across nodes, and it keeps new work on the current healthy node rather than shuffling between equals.

With the default auto role, all nodes are equal peers and any of them can be selected. Set primary on one node and standby on the rest when a specific node needs to serve whenever it can. Files.com sends new work to a reachable primary ahead of auto and standby nodes, and when a recovered primary proves itself healthy again, new work returns to it.

How Node Failover Works

When the serving node becomes unavailable, Files.com confirms the failure and moves new work to another eligible node, typically within a few minutes. Workflows keep running without anyone reassigning anything.

Jobs already running on a failed node do not move mid-run. A running job lives inside its Agent process, so operations that can be recreated safely are recreated on a healthy node, and nothing is ever moved without being logged.

A client transfer that was in progress when a node failed may need to start again. The CLI App and Desktop App retry interrupted transfers automatically, and the retry runs through a healthy node. Other file transfer apps vary, with some including automatic retries and some not. Users of an app without them need to restart the interrupted transfer manually.

Safe Updates Across Nodes

Agent releases update one node at a time. The updated node must reconnect healthy on the new version before the release continues to the next node, and a failed update stops the release for the remaining nodes. A release that breaks one node leaves the others serving on the previous version instead of taking the whole group down.

Sharing Storage Between Nodes

Every node needs access to the same files, because any node can be the one that handles a transfer. Put the files on shared storage that every node's host can reach, and point each node at it the same way.

Attached shared storage, such as a SAN volume presented to every node's host through a cluster-aware file system, is the strongest choice, because each host works with the files at disk speed. Network attached storage, such as an SMB network share or an NFS mount, is the common choice when the files already live on a NAS or file server. Expect slower uploads with it, because the upload cache shares that volume, and network latency in the cache path adds overhead to every file part the Agent writes and reads, as covered in Agent Storage Requirements.

The root setting can point at the shared storage itself. For an attached volume, that is the volume's path on the host. For network attached storage on Windows, use the UNC path, such as '\\server\share\folder', because a mapped drive letter belongs to a user's sign-in session and the Agent service usually cannot see it. On Linux and macOS, mount the storage on the host and point root at the mount location.

The root can also be a local folder that reaches the shared storage indirectly. Mount the storage into a folder inside the root, or place symbolic links to it inside the root and turn on the follow_links setting. Symbolic links let one root fan out to several locations while keeping an identical layout on every node.

When the root lives on shared storage, the upload cache belongs on that same volume. The Agent stages upload parts in a cache directory and assembles them into the final file, and keeping the cache on the same filesystem volume as the root lets that assembly happen in place instead of copying the data a second time. It also keeps large upload buffers off the host's system drive. Set upload_temp_file_part_path on each node to an absolute path on the shared volume, giving each node its own directory. The Agent clears its cache directory at startup, so two nodes sharing one directory would wipe each other's staged uploads. Run files-agent service verify on each node to confirm the cache path. A node without a usable same-volume cache falls back to slower sequential part uploads rather than failing.

Give every node a view of the same live data. Two hosts that each hold a local copy kept aligned by replication can briefly disagree about the contents, so use a single shared location rather than replicas.

Monitoring Nodes

Each node reports its status to Files.com through the Agent's Remote Server. For every node, you can review its details.

Two other features provide similar functionality.

Agent High Availability vs. High Availability Mount

High Availability Mount provides redundancy for the storage side of a mounted folder by failing over between multiple storage backends when one becomes unreachable. Agent High Availability provides redundancy for the Agent side by running multiple Agent installations for one Remote Server, so losing an Agent host does not sever access. Choose based on which part needs the redundancy, the storage location or the Agent that connects to it.

Agent High Availability vs. Running Multiple Agents on a Single Host

Running Multiple Agents on a Single Host covers one machine hosting several Agents that each serve a different Remote Server. Agent High Availability is the reverse, with several machines each hosting a node of the same Remote Server's Agent. Nodes belong on separate hosts, because a second node on the same host goes down with the first when the host fails.