Sync Performance and Limits

A Sync's completion time depends on the capacity available to start jobs, the folder listings it compares, and the files it transfers. Plan job timing, folder structure, and remote connection capacity together so that the Sync can finish within your workflow's delivery window.

Sync Job and Connection Capacity

Files.com places no limit on the number of Syncs you can configure on the platform.

Remote Servers, however, almost always limit the number of inbound connections they accept. Contact the system administrator of the remote server to determine its limits.

Syncs that use different Remote Servers have separate remote connection limits, but still share the site-wide job limits below. Syncs that use the same Remote Server also share that server's connection capacity.

A Sync can transfer several files in parallel, so one running Sync can use several connections. A remote server that accepts five concurrent connections therefore does not necessarily support five Syncs running at once. Set the Remote Server's maximum connections to match its capacity, or choose a lower value to stay below its rate limits.

The exact number of Syncs that produces the best performance against a given remote server depends on automatic retries, sync intervals, folder depth and breadth, file count, file size, network performance, and remote system performance. Calibrate Sync settings, including the time interval between syncs, until you find the configuration that works best for your remote server.

Site-Wide Sync Job Limits

By default, no more than five Sync jobs start per minute across a site, and no more than five run at a time. These limits control the processing capacity used to compare folders and transfer files. Repeatedly scanning a large folder tree consumes resources even when few files have changed, so choose an interval that matches how quickly your workflow needs to discover changes.

For example, starting 1,800 Sync jobs at five per minute takes at least six hours. Long-running jobs can extend that time by occupying the five available running slots. Setting every Sync to a shorter interval does not bypass these shared limits.

If your requirement is to process a file in response to an event Files.com records, such as an upload, use an Automation with a file-action trigger. A triggered workflow responds to the relevant file event instead of repeatedly comparing two folder trees. Use a Sync when the job is to compare locations and bring their contents into agreement.

Higher limits are available when warranted. Contact our Support Team to review your workflow, required transfer frequency, and observed Sync duration. We can help determine whether an Automation better fits the process or additional Sync capacity is appropriate.

For customers who have upgraded this limit, the upgraded value applies both to the number of sync jobs that can start per minute and to the number of sync jobs that can run at a time.

Jobs vs. Transfers

Files.com runs one sync job at a time per configured sync. Sync jobs run until they are complete.

The larger the folder being synced, the longer sync jobs take to complete. In practice, the majority of sync jobs complete very quickly (on the order of seconds), but some customers have sync jobs that take hours.

A sync job runs one control process in a single thread (not parallelized) to determine which files need to be transferred. The actual file transfers are parallelized to achieve faster sync throughput.

In other words, we send multiple file upload and download requests at once, but we do not send multiple folder list requests at once.

Folder Size Limitations

Sync limits the number of items it can compare in a folder to prevent large listings from causing performance problems. The soft limit is 10,000 items per folder, and the hard limit is 100,000 items per folder.

When a folder exceeds the soft limit, the sync runs slowly and may be deprioritized behind other syncs. When a folder exceeds the hard limit, the sync fails entirely.

Managing Folder Size

The most direct way to stay under the folder limits is to divide a folder's contents into subfolders. When you organize manually, you can choose any division scheme that makes sense for your application, such as by date, by customer, or by filename prefix.

You can also configure the folder's settings to automatically organize files into subfolders by extension, created date, modified date, or regular expression.

Automations can move files into subfolders on an hourly or daily basis.

With the File Expiration folder setting enabled, Files.com automatically deletes files when they reach a specified age, which is another way to keep a folder's file count down.

Speed and Duration Factors

The overall time that a sync job takes depends on how many files and folders are being inspected, the quantity of files requiring transfer, the size of the files, and the transfer speeds of the source and destination systems, which are themselves shaped by network bandwidth, latency, throttling, and remote system performance.

Consumer-oriented remote servers, such as Dropbox and Google Drive, implement stricter rate limiting which results in slower syncs. Enterprise-oriented remote servers such as Azure, AWS, and GCP tend to have better throughput capabilities and therefore complete much more quickly.

There is no way to predict how long a sync will take to complete prior to running a sync. The time taken will be the sum of the duration needed to list the files and folders at the source plus the duration needed to transfer the data.

Protocol Factors

When the remote server offers both FTP over TLS (FTPS) and SFTP, Files.com recommends FTPS when transfer speed is the priority, especially for large files.

SFTP exchanges file read or write requests and responses over SSH throughout a transfer. That overhead, the amount of data the client can keep in flight, and network round-trip time can substantially reduce throughput even when bandwidth is available. SSH encryption and message processing also consume processing capacity; FTPS requires capacity for TLS encryption.

The size of the difference depends on the client, server, file sizes, connection limits, and network conditions. Compare representative transfers against your remote server before changing an established workflow, particularly when processing many small files.