Parallelism, Scheduling and Sync Speeds

Several factors affect the speed and transfer rates of a Sync.

Remote Server Concurrent Connections

When using a Sync with a Remote Server, you can configure the maximum number of connections to a Remote Server in the Remote Server settings. This allows you to configure a value that matches what the remote server can support, or to select a lower value to mitigate any rate limiting issues.

Jobs vs. Transfers

Files.com runs one sync job at a time per configured sync. Sync jobs run until they are complete.

The larger the folder being synced, the longer sync jobs take to complete. In practice, the majority of sync jobs complete very quickly (on the order of seconds), but some customers have sync jobs that take hours.

A sync job runs one control process in a single thread (not parallelized) to determine which files need to be transferred. The actual file transfers are parallelized to achieve faster sync throughput.

In other words, we send multiple file upload and download requests at once, but we do not send multiple folder list requests at once.

Speed and Duration Factors

The overall time that a sync job takes depends on how many files and folders are being inspected, the quantity of files requiring transfer, the size of the files, and the transfer speeds of the source and destination systems, which are themselves shaped by network bandwidth, latency, throttling, and remote system performance.

Consumer-oriented remote servers, such as Dropbox and Google Drive, implement stricter rate limiting which results in slower syncs. Enterprise-oriented remote servers such as Azure, AWS, and GCP tend to have better throughput capabilities and therefore complete much more quickly.

There is no way to predict how long a sync will take to complete prior to running a sync. The time taken will be the sum of the duration needed to list the files and folders at the source plus the duration needed to transfer the data.

Protocol Factors

When the remote server offers both FTP over TLS (FTPS) and SFTP, Files.com recommends FTPS when transfer speed is the priority, especially for large files. SFTP exchanges file read or write requests and responses over SSH throughout a transfer. That overhead, the amount of data the client can keep in flight, and network round-trip time can substantially reduce throughput even when bandwidth is available. SSH encryption and message processing also consume processing capacity; FTPS requires capacity for TLS encryption.

The size of the difference depends on the client, server, file sizes, connection limits, and network conditions. Compare representative transfers against your remote server before changing an established workflow, particularly when processing many small files.