Data Integrity

Data integrity issues may arise whenever the source and destination locations of a Sync are on different systems, particularly when a remote server is used as a source or destination.

Due to the wide variety of remote server types supported, Files.com does not check data integrity when files are transferred during sync.

For some types of Remote Server, the sync will skip files that have changed since the start of the sync.

We rely on the next run of the sync to again check whether the file size and file name match on the source and destination of the sync. If a file is found to be a different size at that time, Files.com will attempt to re-sync the file.

Sync requires the source and destination systems to accurately report file sizes. When a remote server cannot do that, syncs will not work correctly. Any remote server that doesn't accurately report file sizes is incompatible for use with the Sync feature. Use an alternative, such as an Automation or Remote Mount instead.

Concurrent Access

The largest threat to integrity is another process touching the same files at the same time. Race conditions occur when more than one Sync acts on the same source or destination location, or when an Automation or a Remote Mount targets a location a Sync is using.

Because these operations run at overlapping times, the order in which they start and finish is not predictable. The result is missing files, failed transfers, and partial operations. When an Automation and a Sync both act on the same files, one of them is likely to fail, and the risk is higher when either one deletes or moves files after copying. One job may try to copy a file another has already deleted, or two jobs may try to delete the same file.

Files on a remote server may also be in use by another user or process on that system. Syncing files while they are in use on the remote server produces incomplete copies, failed moves, and file corruption.

Do not point two Syncs at the same folder, and do not use the same folder in both an Automation and a Sync. Where the same location has to serve more than one purpose, give each Sync unique Include and Exclude patterns so the targets do not overlap, or combine the work into a single Sync or Automation so everything happens in a predictable order. Avoid using a location where files are still being written as the source of a Sync, because a source folder that a remote process treats as a working directory produces partial transfers even when no other Files.com job touches it.