File System Semantics

---

Files.com is a cloud-based managed file transfer platform that is optimized for business-to-business file transfer.

In general, we've tried to design the semantics of the system so that it "just works" in the majority of business-to-business file transfer needs. Since this domain has been around for a lot longer than Files.com (the first file transfer tools were first built in the early 1970s), we've paid close attention to how major existing software packages work and the general expectation around business-to-business exchanges in the real world.

Internally, Files.com implements a virtual filesystem and all filesystem logic is built into the Files.com application itself. That is, Files.com does not use Windows or UNIX file systems as a model for its path conventions. This page lists some of the key differences to be aware of.

Files.com uses its own Permissions scheme, which operates differently from POSIX or NTFS permissions. Please see our documentation on Permissions to understand how permissions work in Files.com.

With proper use of Files.com's permissions system, you should be able to achieve any access control scheme you require for your application.

When accessing Files.com through integrations such as FTP, SFTP, or WebDAV, those integrations will sometimes report POSIX permissions when you list a folder. These permissions are dummy values and should be disregarded.

Our research has found that the majority of business-to-business exchanges expect that filenames are treated on a case-insensitive basis, and we've found in practice that it is very common for the two sides of a business-to-business file transfer to disagree on the case conventions for a filename. For example, many EDI systems use all capital letters in filenames, and many others use all lowercase letters.

As a result of this, paths in Files.com are case-insensitive and accent-insensitive. Case differences and accent marks are discarded when comparing filenames for equality.

For example, Files.com will treat these file names as being identical to each other:

filename.txt

FILENAME.TXT

FiLeNaMe.TxT

FÎŁĘÑÂMÉ.TXT

Fïłèńämê.Txt

This means that uploading FILENAME.TXT to a folder that already contains filename.txt will cause filename.txt to be overwritten by FILENAME.TXT, or FILENAME.TXT to be renamed to FILENAME_2.TXT, depending on how your Overwrite behavior setting is configured. This also applies to any Remote Syncs and Automations that move, copy, or sync, files and folders with Files.com.

Despite being case-insensitive, Files.com is case-preserving, meaning that the path name will use the capitalization and accent marks associated with the original file or folder name.

Our behavior is similar to the default path behavior on Windows, which is still the dominant environment for most enterprises. Our behavior differs from popular "object" storage services such as Amazon S3, which typically use strict binary matching for object names.

When using a Remote Mount, the case sensitivity of that Remote Server is used instead of the Files.com case sensitivity.

For example, a Remote Mount to Microsoft Sharepoint, Box, or Dropbox, will be case insensitive while a Remote Mount to Microsoft Azure Blob, Amazon S3, or Google Cloud Storage, will be case sensitive.

When moving or copying files between Files.com and Remote Servers, whether using Remote Sync or an Automation, be aware that each system's case sensitivity will determine if files might be overwritten, or if folders might have their contents merged, based on their insensitive names.

This issue will only occur when moving or copying files from a sensitive system to an insensitive one.

When connecting to Remote Servers using SFTP, SFTP, or WebDAV, the case sensitivity and unicode sensitivity is determined by the remote server's operating system (OS).

Files.com path names use exactly one forward slash (/) between folders in the path, and never uses a slash at the beginning or end of the path. Path names may not contain a backslash (\).

File and folder names may not end with trailing whitespace characters, such as a space (" "), a tab (" " or "\t"), a new line (\n), or a carriage return (\r).

To maintain compatibility with remote servers, which typically use a slash character to delimit folder names, you may not use a slash character inside a specific file or folder name.

Files.com uses the "NFKC" (Normalization Form Compatibility Composition) algorithm for normalizing Unicode as part of path comparison. Although Files.com normalizes Unicode for path comparison, Files.com is Unicode preserving, meaning that the path name will be stored using the actual Unicode representation used when the file or folder is first created.

Files.com uses 2 algorithms for path normalization. Our Normalize algorithm is applied to all paths provided to the Files.com service to remove noncompliance with our path requirements. If you are building an SDK or manual API integration to Files.com, we recommend that you implement this algorithm prior to sending any paths to the Files.com API to ensure that they will be treated identically on the server side as to how you provided them.

Additionally, our Normalize For Comparison algorithm is used to compare two paths to determine whether they are the same. If you are building an SDK or manual API integration to Files.com which needs to determine whether two file paths are the same, we recommend that you also implement this algorithm.

The official Files.com SDKs implement both algorithms natively and we encourage the use of our SDKs rather than implementing either of these algorithms by hand. For completeness, we describe the algorithms here. Sample code for the following algorithms can be found in our SDKs.

Convert the path to UTF-8

Remove any characters with byte value of 0

Convert any backslash (\) characters to a forward slash (/)

Remove any trailing or leading slashes

Remove any path parts that are . or ..

Replace any duplicate forward slashes (such as /// with a single forward slash /)

Run the path through the Normalize Algorithm

Unicode Normalize the Path using Unicode NFKC algorithm

Transliterate and remove accent marks by using the official Files.com transliteration map specified below. Any instance of the first character in the map should be replaced with the remaining characters.

Convert the Path to lowercase using the case mapping found in Unicode 9.0. (Note: we are aware that this version of Unicode is fairly old and many modern programming languages now implement Unicode 15.0. The only differences affect two very rare languages and we have never seen these differences cause any actual issues in practice at Files.com. We suggest using whichever version of Unicode your environment supports, as that will most likely be fine.)

Remove any trailing whitespace (\r,\n,\t or the space character)

Any two paths with the same resulting string from this algorithm are considered the same file on Files.com.

TRANSLITERATION_MAP = "ÀA,ÁA,ÂA,ÃA,ÄA,ÅA,ÆAE,ÇC,ÈE,ÉE,ÊE,ËE,ÌI,ÍI,ÎI,ÏI,ÐD,ÑN,ÒO,ÓO,ÔO,ÕO,ÖO,ØO,ÙU,ÚU,ÛU,ÜU,ÝY,ßss,àa,áa,âa,ãa,äa,åa,æae,çc,èe,ée,êe,ëe,ìi,íi,îi,ïi,ðd,ñn,òo,óo,ôo,õo,öo,øo,ùu,úu,ûu,üu,ýy,ÿy,ĀA,āa,ĂA,ăa,ĄA,ąa,ĆC,ćc,ĈC,ĉc,ĊC,ċc,ČC,čc,ĎD,ďd,ĐD,đd,ĒE,ēe,ĔE,ĕe,ĖE,ėe,ĘE,ęe,ĚE,ěe,ĜG,ĝg,ĞG,ğg,ĠG,ġg,ĢG,ģg,ĤH,ĥh,ĦH,ħh,ĨI,ĩi,ĪI,īi,ĬI,ĭi,ĮI,įi,İI,IJIJ,ijij,ĴJ,ĵj,ĶK,ķk,ĹL,ĺl,ĻL,ļl,ĽL,ľl,ŁL,łl,ŃN,ńn,ŅN,ņn,ŇN,ňn,ʼn'n,ŌO,ōo,ŎO,ŏo,ŐO,őo,ŒOE,œoe,ŔR,ŕr,ŖR,ŗr,ŘR,řr,ŚS,śs,ŜS,ŝs,ŞS,şs,ŠS,šs,ŢT,ţt,ŤT,ťt,ŨU,ũu,ŪU,ūu,ŬU,ŭu,ŮU,ůu,ŰU,űu,ŲU,ųu,ŴW,ŵw,ŶY,ŷy,ŸY,ŹZ,źz,ŻZ,żz,ŽZ,žz"

The maximum length of a filename path (including all its parent folders) is 550 characters, including the file extension.

By default, file modification times are not editable by users in Files.com, as this can be a security concern.

However, some applications, such as file synchronization apps, require the ability to set the File Modification time.

Files.com offers a feature that enables this capability. To enable user-provided modified times in Files.com, type File "last modified date" semantics in the search box at the top of every page and click the matching result. Adjust the setting as needed.

Showing the user-provided date will affect FTP, SFTP, WebDAV and the web interface. For interactions via our API or SDKs, you will get access to both timestamps -- the user set timestamp as well as our timestamp of the actual most recent modification time of the file.

When previewing a file online, the following timestamps are shown:

Uploaded - The date and time that the file was uploaded to Files.com.

Modified - The date and time that the file was last modified on Files.com.

Modified on source system - The date and time that the file was last modified on the system that uploaded the file. This timestamp is only displayed if a date and time were provided by the source system during upload and if the time value differs from the Modified time value.

In POSIX-based filesystems and in NTFS, you can create pointers to files via hard links or symbolic links. Windows calls these Shortcuts. Files.com does not currently support either hard links or symbolic links.

With a POSIX-based filesystem, a move or copy operation can be destructive, overwriting an existing file if the destination already exists. By contrast, in Files.com such operations will fail if the destination already exists, preventing the existing file from being overwritten.

Files.com treats file uploads as atomic operations. In other words, a file must be uploaded to Files.com completely before the file will be shown in file listings and available to work with. This approach is necessary to ensure that a file is never accessed when in an incomplete state, and that file integrity is maintained with automated processes like webhooks, GPG encryption, Automations, and Remote Server Syncs.

This means that any attempts to access a file before the upload has completed will fail as if the file does not exist. If an existing file is being changed, downloading it during a change will result in the previous, unchanged version of the file being downloaded.

While file upload atomicity will be maintained when using the Files.com web interface and Files.com's native apps like our Command Line App (CLI), Desktop App for Windows/Mac, Mobile app for iOS/Android, and SDKs for developers, we cannot guarantee the same for file transfers performed using third party apps such as FTP apps. Therefore, we strongly recommend using Files.com native apps wherever possible.

The ability to treat uploads as atomic is limited by the storage solution being used to upload into when using the Remote Mount feature. For example, when using FTP or SFTP to connect to a Remote Mount, a network disruption between Files.com and the remote system can cause atomic uploads to fail, resulting in a partial or incomplete file being uploaded to the remote system.

If you are using our Remote Server Sync or Remote Server Mount capabilities, there may be additional restrictions related to the Remote Server. As a general rule, we will apply restrictions that are applied by Files.com as well as restrictions applied by the remote server.

For example, we will not allow spaces at the end of file/folder names on a Remote Server Mount, even if the remote supports them. We will also not support file/folder names that are not supported by the remote mount. Attempting to do this will result in an error at the time of file creation/upload.

Files may not be able to sync to a remote server if the filename is not accepted on the remote server. This scenario will result in failures appearing in Sync logs. Additionally, files may not be able to sync to Files.com from a remote server if the filename is valid on the remote but not on Files.com. For example, remote filenames ending in a space character or containing a slash character are not allowed on Files.com. This scenario will result in failures appearing in Sync logs.

Remote servers will likely prohibit certain characters from being used in file and folder names. If you see issues with invalid file or folder names, we recommend you modify those names.

Remote servers may also restrict the length of the file or folder name. Common length limits are 160 characters or 55 characters.

Bear in mind that some remote servers calculate the total length as being either just the file or folder name, including any extension. Or the sum of the whole folder path + file name + extension + temporary suffix.

To resolve this issue, shorten the name of files and folders so that they are below the length limits of the remote server.

On the Files.com side, the path limit is 550 characters for the sum of the full path, including folder names.