Storage locations

Choose where file contents are kept. Different spaces can use different locations.

Types

Where the files are

On the storage folder and on every Local folder location, each space is an ordinary folder, and its files are ordinary files with their own names. You can open /storage (the storage folder next to compose.yaml) and find everything there:

/storage/company/…              the company space "All files"
/storage/teams/<space name>/…   team spaces
/storage/users/<user name>/…    each user's "My files"

A space gets its folder when it is created, in the folder of the storage location it is created on, usually the default one (or /nas/teams/…, say, when it is created on a NAS added as a Local folder location). These are folder spaces: files copied into them by other programs, over SMB or by a scanner, show up in ThirtyFile, and changes made on the web are made in the folder.

• Names. A folder is named after the space or the user. Characters a file name can't have become _, and names ThirtyFile would hide (desktop.ini, say) get an _ in front. When the folder is already there, because another space has the same name, a folder with that name was made by hand, or a deleted space left it, the new one gets a number: teams/Sales (2).
• Renaming. Renaming a space doesn't rename its folder, so programs, SMB shares and backups that use the path keep working. Control panel › Spaces shows each space's folder.
• Deleting. Deleting a space, or a user with their files, removes them from ThirtyFile only: the folder and the files in it stay on the disk, for you to delete or keep. A new space or user with the same name gets a new folder, never the old files. Moving a deleted user's files to another space moves them on the disk.
• S3, SFTP and FTP keep files in a content store instead: names made of the file's content (ab/cd/…), with the names and folders only in ThirtyFile's database. Renaming a folder there would copy every file in it. Spaces created while one of these is the default location use the content store, and can be moved to another location later.

Adding a location

1. Open Control panel › Storage locations and click Add storage location.
2. Choose the type and fill in the address and the login details.
3. Click Test connection. It writes, reads back and deletes a small file. The location can be saved once the test succeeds. Keys or accounts that may write but not delete files fail it: ThirtyFile could never remove deleted files there. For a Local folder, saving creates the folder when it isn't there yet (see Disks and shares that aren't mounted). For S3, SFTP and FTP, saving puts a small .thirtyfile-location file there (see One place, one location).
4. Make it the default if new spaces should be created on it.

When you edit a location later, the saved password or key is kept as long as the server, account, bucket, SFTP host key and certificate check stay the same. Change one of them, or reset the host key, and enter the password or key again.

The default only decides where new spaces go: users' "My files", team spaces and, on a new server, "All files". When creating a team space you can choose another location, and new users' "My files" can have a location of their own (see Personal spaces). Each space stays on the location it was created on, so changing the default doesn't move any existing space, and their new files keep going where their older ones are. Only moving a space changes its location.

What the list shows

• Used space: the files of the spaces on the location. For folder spaces, the size of the files in their folders, as of the latest check; for the content store, the contents kept there. Contents that are the same in several files are counted once. Storage usage has the history.
• Free space: for the storage folder and Local folder locations, how much is free on the disk that holds the folder, and how big it is.
• Files and spaces: how many files the folder spaces on the location hold, as of their latest check, plus the contents its content store keeps (point at the number to see both), and how many spaces are on the location. Click the number of spaces to list them, with their kind, owner and size. What is in them isn't shown.

A location can't be deleted while a space is on it, while a space is being moved to or from it, or while its content store still keeps files. A folder space an administrator made from a folder of their choice (below) isn't on any location.

Provider notes

• AWS S3: leave the endpoint empty.
• Cloudflare R2: endpoint https://<account ID>.r2.cloudflarestorage.com, region auto.
• Self-hosted: RustFS works well. Choose "Self-hosted" and tick "Path-style URLs".

Moving a space to another location

Under Control panel › Spaces, select a space and choose Move to another location…. Any space can go to any other location: between content stores, from a folder into a content store, and into a folder. The dialog shows where the space is now and how big it is, and lists the other locations, with the free space of those on this server's disks. When the chosen location doesn't have enough room for the space, the dialog says so and the move can't start. A move that is resumed needs room only for what it hasn't copied yet, and a folder moved to its new place on the same disk needs none.

The move runs in the background, in three steps (here for a space in the content store; folder spaces and moves into a folder work alike, with the differences below):

1. Copy. Every file of the space, its earlier versions and the items in its trash are copied to the new location, and each copy is checked against the file's SHA-256 fingerprint. The space stays in use meanwhile: files uploaded or changed during the move are copied too.
2. Switch. Once everything is there, the space switches to the new location in one step. Its new files go there from then on.
3. Clean up. The old copies are deleted a minute later, so downloads that are reading them can finish.

Until the switch, the space is where it was: a move that stops or is cancelled changes nothing for the people using it. A file whose content is the same as another file's is kept only once in the whole system, so content a space shares with spaces elsewhere moves with it, and those spaces read it from the new location too.

Moving a folder space into S3, SFTP or FTP

A folder space (a space on the storage folder or a Local folder location, or a folder you chose) can be moved into the content store of an S3, SFTP or FTP location. Its files become content-store files there, with the same names, folders, share links, permissions, trash and earlier versions.

• Read-only meanwhile. From the moment the move starts until it is finished or cancelled, people can open, download and share the space's files, but not change them; the space shows a notice saying so. A paused or stopped move keeps it read-only, as what was copied must stay as it is: resume the move, or cancel it to open the space up again. A move waiting for its first turn doesn't make the space read-only yet.
• Changes in the folder. Files changed, added or removed in the folder from outside ThirtyFile during the move (over SMB, by a scanner) are found by a check of the folder right before the switch, and copied too. Better still, stop other programs from writing there until the move is finished.
• Names. The content store doesn't tell apart names that differ only in letter case, which a folder can hold side by side ("Report.docx" and "report.docx"). One of them gets a number, "report (1).docx", and the move's details say how many were renamed.
• The old folder. Once the space is on the new location, its folder is removed: each file only if it is still exactly what was copied, then the folders left empty. A file changed at the very last moment, and files ThirtyFile never shows (Thumbs.db, say), stay in the folder, and the move's details list them (for a personal space, only how many: its file names stay private).
• Room. Each file passes through a temporary file in ThirtyFile's data folder, which needs room for the largest file of the space. The move checks this, and that the folder is there, before it starts: a disk that isn't mounted is never taken for an empty space.

Moving a space into a folder

A space moved to the storage folder or a Local folder location (a NAS, say) becomes a folder space there: it gets a folder of its own, named as a new space's would be (company, teams/<name> or users/<user name>, with a number when that is taken), and is read-only while its files are copied.

• From S3, SFTP or FTP. Each file is written into the new folder with its date, checked against its SHA-256, and renamed into place; the trash goes to .thirtyfile-trash and earlier versions to .thirtyfile-versions, as in every folder space. Names a folder can't hold as they are (characters a file name can't have, names scans skip such as Thumbs.db, names too long for the disk) get another, and the move's details say how many. Once the space has switched, the content store lets go of its files, which are deleted a minute later unless another space uses the same content.
• From another folder. On the same disk, the folder is simply moved to its new place, at once. On another disk, everything in the folder is copied, with the trash, the versions and files ThirtyFile doesn't show, each file with its date and checked by its size. Changes made in the folder from outside ThirtyFile meanwhile are found by a check before the switch and copied too. Then the old folder is removed, as described above. When the new folder's disk doesn't tell letter case apart (a CIFS share, an exFAT disk) and the space holds names that differ only in letter case ("Report.docx" and "report.docx"), those items aren't copied and the move stops, listing them: rename one of each pair, then resume the move.
• A folder you chose. A folder space showing a folder of your choice, which is on no location, can be moved onto any location the same way. Its folder is then removed from where it was.

Cancelling a move into a folder removes the folder it made, with what was copied into it.

Several spaces, or everything on a location

• Several spaces. Under Control panel › Spaces, tick the spaces (the box in front of each, or the one in the header for all of them), then choose Move … spaces to another location…. Each space gets a move of its own, and they run one after the other. Spaces already on the chosen location, or already being moved, are left out, and the dialog says how many.
• Everything on a location. Under Control panel › Storage locations, choose Move everything to… on a location to move all of its spaces, personal ones included, to another. The location shows how far that is. Once nothing is left on it, it offers to make the location the spaces went to the default (when it was the default) and to delete it.

The free space of the location they go to is checked for all of them together, when it is a disk of this server.

Following moves

Control panel › Moves lists the moves that aren't finished, then the earlier ones. For each: the space (a user's "My files" is named after the user), where it moves from and to, how many files and bytes are copied, the speed and the time left, and its state. The page refreshes by itself while a move runs.

• One at a time. Moves run one after the other, in the order they were started; the others wait. At the same time lets up to eight run together, on a server with more processors and a fast network.
• Pause and resume. A paused move stops after the file it is copying and keeps what it copied. Resumed, it waits for its turn, then continues where it stopped. If ThirtyFile stops or restarts during a move, the move continues the same way when it starts again.
• Cancel. The space stays where it was, and what was copied to the new location is removed.
• Errors. A file that can't be copied is tried three times. When some still can't be copied, the rest are copied anyway, and the move stops as Stopped by an error without switching. Details lists those files (for a user's "My files", without their names, which administrators don't see) and the reason. Fix the cause, then resume the move to try them again, or cancel it. A new location that can't be reached stops the move the same way.
• History. Finished and cancelled moves stay on the list for 90 days (the newest 20 of each space whatever their age), with who started them and when. The activity log also records when each move starts, finishes, fails or is cancelled.

A space being moved can't be deleted, and neither can the user whose "My files" it is, until the move is finished or cancelled.

Copying everything to another location

Copy everything to…, in the … menu of a location under Control panel › Storage locations, copies all the spaces on that location to another location, and leaves the source as it is: no file is removed, the spaces stay where they are, and the default location doesn't change. Use it for an extra copy on a NAS, an off-site bucket or a disk you take away. For snapshots made again and again, and kept for a time, see Backups of files to another location; they are kept the same way, and restored the same way.

What is copied

• Every space on the location, personal ones included: its folders (empty ones too), files, the trash and earlier versions. The dialog lists the spaces and says how many files there are, how much of it is in the trash, and how much will be written to the other location.
• Each content once. Files with identical content are stored once, also across spaces. A space's content is copied wherever it is kept (identical content is stored once in the whole system, so a file of a space can be kept on another location).
• Who had access to each folder is recorded in the copy, for reference. Share links, favorites and uploads in progress aren't copied.
• Content on the location that only spaces elsewhere use isn't part of it: it belongs to those spaces.

Where the copy goes

Each copy gets a folder of its own on the other location, .thirtyfile-backups/<id>, which nothing else in ThirtyFile writes to. Nothing already there is replaced, and copies never mix. In it:

• objects/ab/cd/<sha256>: each content, named by its SHA-256;
• snapshots/<id>/manifest.jsonl: one line per space, folder, file and earlier version, with its path and the SHA-256 of its content;
• snapshots/<id>/complete.json: written last, with the manifest's SHA-256. A copy without it isn't complete, and is never offered for restoring;
• set.json: the copy's name, where it was copied from, and which ThirtyFile made it.

So the files can also be found without ThirtyFile: each line of the manifest names a file's path and the object holding its content.

The other location may be on the same disk or storage service as the source; the dialog then says that the copy doesn't survive a failure of it. Locations in the same place, or one inside the other, are refused. For FTP without encryption, the dialog says that the files travel unencrypted. On a disk of this server, the copy is refused when there isn't enough free space for it; for S3, SFTP and FTP, room left can't be told.

Files that change during the copy

The spaces stay usable while they are copied. The copy shows them as they were when their list of files was made (As of): files changed or deleted after that are copied as they were, since their content is kept from deletion until it is copied. Files of folder spaces are read one at a time and checked not to have changed while they were read; one that changed is read again. The folder is checked for changes made by other programs first. Each file of a folder space is whole and as the index had it, but files other programs changed at different times aren't one instant together.

Following, pausing and errors

Control panel › Backups lists the copies, what each is doing, and how far it got. Copies run in the background, one job at a time, and go on when the page is closed.

• Pause and resume. A paused copy stops after the item it is working on and keeps what it copied; resumed, or after ThirtyFile restarts, it continues where it stopped.
• Errors. A file that can't be read is tried three times. When some still can't be copied, the copy stops as Stopped by an error and isn't complete. Details lists them (for a personal space, without their names) and why. Fix the cause and Resume to try again. A location that can't be reached stops it the same way.
• Cancel. A copy that never completed is removed from the other location. The source isn't touched.
• Check the backup reads everything back from the other location and checks each content against its SHA-256, and each manifest against its complete.json. What is missing or damaged is listed.
• Delete backup removes its folder from the other location, after you confirm. It can be paused, not cancelled: a copy half deleted can't be restored from.

A location holding a copy can't be deleted, nor its folder changed, until the copy is deleted. Removing unused content and deleting files for good never look in .thirtyfile-backups, so they never remove a copy, and Browse doesn't open it (its manifests name the files of personal spaces). The activity log records when copies are made, restored from, checked and deleted.

Restoring from a copy

Select a copy and choose Restore…, then the space to restore. It goes into a new folder, Restored <space> <date>, at the top of the space itself, or of another company or team space you choose; nothing already there is replaced. Content the space's location still holds isn't copied again.

• A personal space only goes back into its owner's personal space, whole: administrators never see its files.
• What was in the trash is left out, unless you tick Also restore what was in the trash.
• The new folder takes the space's permissions. Earlier versions and the access recorded in the copy stay in the copy: restoring never gives anyone more access than the space gives.
• Names a folder can't hold are refused and listed; a name that is taken in its folder (A.txt and a.txt from a folder space, restored into a content store) gets a number, and the job says so.
• A restore that stops continues where it stopped, without making anything twice.

Replicas: the same files on several locations

A replica policy keeps the files of the spaces on one location on other locations too, checks them there, and reads them from there when the location fails. When the location is lost for good, one of the others can be promoted: the spaces then read and write there. Only administrators see and manage replicas, under Control panel › Replicas.

Making replicas

Choose New replicas… (or Replicate… in the … menu of a storage location), then:

• Replicate: the location whose spaces are copied, the primary.
• Keep copies on: one or more other locations, in order of priority. Each is brought up to date soon after changes (changes are gathered for a few seconds, then copied in the background), or on a schedule (every few minutes or hours, or daily at a time, in a time zone you choose). Locations in the same place, or one inside the other, are refused.
• Copies: how many copies to keep besides the original, 1 at first. They go on the first locations of the list that work; the others are spares. With fewer locations than copies, the policy says so and counts as needing attention.
• Spaces: every space on the location, also spaces added later, or the spaces you tick.
• Reading from copies, checking every few days (1 at first), when to tell administrators that changes have waited too long (24 hours at first), and a speed limit.

Each content is copied once per location, also when several files or spaces share it. Files, earlier versions and the trash are copied. On the other location the copies are kept where it keeps content (ab/cd/<sha256>), whatever kind of space they come from.

Folder spaces

Folder spaces (every space on the storage folder and on Local folder locations) are replicated too. Their files are read from the folder and checked while they are read: a file that changes meanwhile is read again. A file another program keeps writing (a log, a mailbox) doesn't make the sync fail: it is copied as it was last read whole, and the sync's details list it as changed while copied (a file never read whole has no copy yet and is listed the same way; files of personal spaces are listed without their names). The next sync reads it again. Changes made on the web are copied a few seconds later. Changes other programs make in the folder (over SMB, a scanner) are copied once ThirtyFile has seen them: about 12 seconds later on a disk of the server, and at the next regular check for changes on NFS or SMB mounts (every 15 minutes unless set otherwise). Each sync checks the folder for changes first.

How copies are kept

• Checked when written. Every copy is read back and compared with its SHA-256 before it counts. A copy already there that is damaged or has the wrong length is replaced.
• Checked again every few days, or with Check the copies: each copy is read back. A damaged or missing one is marked, and the next sync replaces it from the original or another good copy.
• Behind for a while. Copies are made a little after files change, not at the same moment. A file changed just before the location fails may not be on the others yet. Control panel › Replicas shows, for each location, whether it is current, how long changes have waited, and how many contents it holds of those it should, as its last sync counted them (after the policy changes, a few seconds later). Its list of syncs and checks keeps finished ones for 90 days, at most the newest 1,000 of each location, and the newest 20 whatever their age.
• A location that can't be reached waits, is tried again every 5 minutes, and catches up once it is back. A sync that fails is tried again after an hour, or at once with Sync now. Syncs can be paused, resumed and cancelled; they continue where they stopped, also after ThirtyFile restarts.
• Never deleted by mistake. Emptying the trash, deleting files for good, removing unused content and deletions waiting to be retried never delete a replica. A copy is only let go once nothing uses its content any more, and only by the policy that made it.
• Administrators are told (in the app and by email, notification kind Replicas) once when a policy needs attention (the spaces' own location or a location with copies can't be reached, a sync fails, copies are damaged, there are too few locations, or changes waited longer than allowed), and once when it is fine again.

When the primary fails

When the content of a file can't be read from the primary (the location is offline, the content is missing there, or a folder space's folder isn't there), it is read from a checked copy on another location instead, if the policy allows reading from copies. The copy is the same content, named by its SHA-256, so it is never an older version of the file than ThirtyFile knows. For a folder space, that is the file as the index last had it: a change another program made in the last seconds before the disk failed, which ThirtyFile hadn't seen, isn't in the copy. Each time is noted in the server log, and in the activity log at most every 10 minutes per location (Read from a replica).

Only reading falls back. While the primary is offline, uploads and changes to its spaces wait, as described under When a location is offline: a change is never written only to a replica.

Promoting a location

When the primary is lost for good, or is being retired, choose Promote… and the location to promote. Promotion is never done by itself. Before anything happens, the dialog says whether both locations can be reached, how far the chosen one is behind, how many contents move and which spaces, and how many contents aren't there.

• A location that is current is promoted at once.
• A location that is behind is refused while the primary can still be reached: sync it first. While the primary can't be reached, it can be promoted when you tick that files whose content isn't there can't be read until the old primary is back. If the missing-content count changes while promotion is being checked, nothing is changed: review the current state and try again.
• Promoting changes no file. The spaces then read and write on the promoted location, which becomes the primary. Running syncs stop, and a change that was being saved to the old primary at that moment is refused and can be tried again.
• A folder space whose files and earlier versions all have a checked copy on the promoted location becomes a space of its content store there, as a move into S3, SFTP or FTP would make it: its files are no longer in a folder that other programs can change. Its folder on the old primary stays as it was, untouched; delete it by hand once you no longer need it. To have it in a folder again, move the space into a folder later. A folder space not wholly on the promoted location stays where it is, as it is, and the dialog names it.
• The old primary keeps everything it had and is never written to as a primary again. It becomes a location with copies, marked Old primary, being checked: when it is back, its copies are read and checked, damaged ones are replaced, and it counts as a copy once it is current. Nothing moves back on its own. To go back, promote it again once it is current.

After a promotion, check the policy's list of locations: the promoted one is no longer in it, so add another location if you want as many copies as before.

Removing a location or a policy

Removing a location from the list, keeping fewer copies, or deleting a policy stops new copies, but the copies already made stay, and are kept from deletion. Control panel › Replicas then shows how many copies on each location no policy wants, with Remove them…. Each is checked again right before it is deleted: a copy a policy still wants, content that is the original there, and copies of the files of a folder space whose location can't be reached now stay. A location that holds replicas, or that a policy copies from or to, can't be deleted until then.

Folder spaces: a folder on the server as a space

Every space on the storage folder or a Local folder location is a folder space (see Where the files are). You can also show an existing folder on the server, such as a NAS share, as a space. Its files stay where they are, as ordinary files, and can also be changed there, for example over SMB or by a scanner.

1. Mount the folder into the container, for example -v /srv/shared:/mnt/shared.
2. Under Control panel › Spaces, click New folder space and enter the path inside the container, /mnt/shared.
3. Add members as for any team space.

ThirtyFile finds changes made in the folder when someone opens it (not again within a few seconds, not while a change to the space is being made, such as copying a large folder into it, and not in watched spaces, see below: the list then shows what ThirtyFile knows), and checks every folder space every 15 minutes (change this under Control panel › General), or right away with Check for changes (a check of a large folder shows its progress in the list and in a message at the bottom, and goes on if you leave the page). Files and folders moved or renamed in the folder keep their share links and permissions. Files and folders a check finds have no uploader (the Uploaded by column and Created by in the details pane show —) and aren't in anyone's Recent, unless they open or edit them; items found by earlier versions of ThirtyFile keep the space's owner as their uploader. ThirtyFile tells a moved item from a new one by when the disk says it was created. Where the disk doesn't record that (some network shares), a moved file is still recognised by its size and date, but a moved folder counts as removed and added again, and loses its share links and permissions.

On a supported local Linux file system, changes show up by themselves about 12 seconds after they are made: ThirtyFile watches the folders for changes. Those spaces get the regular check only a quarter as often (every hour with the usual 15 minutes), for what watching can miss. NFS, SMB, 9P and FUSE/virtiofs mounts, including host folders shared by Docker Desktop, may not report external changes. Unrecognized file systems also use the safer fallback: opening a folder checks it again (at most once every five seconds), and scheduled checks keep their normal interval. No probe file is written to infer that host events work. With very many folders, Linux's limit on watched folders (fs.inotify.max_user_watches, often 8,192 or more) can be reached; the log then says so, and refresh-on-open and regular checks find the rest. Raise it on the host with sysctl fs.inotify.max_user_watches=524288. Every folder counts, in all spaces together (every user has a space of their own).

Docker Desktop on Windows and macOS. Changes made in the host's shared folder are found when its folder is opened or refreshed in ThirtyFile, and by the regular check; they need not appear instantly. A file still being written appears once it settles. Renames or deletions can start a background check to distinguish a move elsewhere in the space from a deletion. Control panel › Spaces shows whether changes appear within seconds or are checked when opened, with the actual scheduled interval. Setting the interval to zero turns off scheduled checks, but opening a folder still checks it when watching is unavailable. Check for changes remains available for a full check at any time.

While a large folder space is being checked, Control panel › Spaces shows how far it got, and the log says how long reading the folder and updating the index took when it took more than 10 seconds. As a guide, on an SSD the first check of a folder with 200,000 files takes about half a second to read and 25 seconds to index; checking it again when nothing changed takes a second or two. Network mounts are slower to read, since every file's size and date are asked for over the network: keep the regular check at 15 minutes or more for large NFS or SMB folders.

Everything done on the web changes the folder itself: uploading, creating folders, renaming, moving, copying, editing and deleting. Moving to a space on another disk, or to a space that isn't a folder space, copies the files, checks them, then removes the originals; a file changed in the folder meanwhile stays there, and the next check for changes shows it. A message at the bottom of the page shows how far a long move or copy got; it goes on on the server even if the page is closed.

• Trash. Deleted items go to a hidden .thirtyfile-trash folder inside the space's folder, so they can be restored. Emptying the trash, or deleting an item from it, removes the files.
• Earlier versions. When a file is saved over on the web, replaced by an upload or restored to an earlier version, the file it was is kept in a hidden .thirtyfile-versions folder inside the space's folder (as a hard link where the disk allows one; elsewhere the file itself is moved there, so even a large file is kept at once), which ThirtyFile never shows as files. Changes made directly in the folder, over SMB say, don't keep versions. When a file is deleted for good, its versions go with it. A file in that folder that no version refers to is removed by a check for changes once it is a day old.
• Editing at the same time. When a file was changed in the folder while someone edited it on the web, it isn't overwritten: the web version is saved next to it as name (conflict copy), in the system's default language (English when that follows the browser), whoever saved it.
• Letter case. On a disk that doesn't tell letter case apart (exFAT, NTFS, most SMB shares), a folder named in other letter case is the folder already there. Uploading a folder Photos where photos is puts its files into photos, and New folder Photos says it already exists.
• Downloads. A download that is resumed notices a file that was replaced in the meantime and starts over. A file replaced while it is being downloaded can arrive mixed; download it again.
• Read-only. Tick Read-only when creating the space, or use Make read-only on it later, to allow browsing, downloading and sharing only.
• File owners. ThirtyFile writes files as the user in THIRTYFILE_RUN_AS (1000:1000 unless set). That user needs permission to write in the folder. When other programs (a Samba share, say) write files as another user, give both the same group and make the folder group-writable, or set THIRTYFILE_RUN_AS to the user those programs use.
• Deleting a folder space removes it from ThirtyFile only; the folder and its files stay on the server.
• A folder can belong to one space or location only: one that contains, or is inside, another space's folder, a Local folder location's folder or ThirtyFile's data folder is refused, so the same files never show up twice. The system's own folders (such as /etc, /run or /usr, and the disk's root) are refused too.
• Temporary files of office programs and systems (~$…, Thumbs.db, .DS_Store) aren't shown. Neither are ThirtyFile's own files there, whose names start with .thirtyfile-. These names can't be given to items on the web or over WebDAV. Extracting a ZIP file leaves out what has such a name, and moving or copying from another space into a folder space is refused while it holds any. Symbolic links, names that aren't valid text and other disks mounted inside the folder are skipped; Check for changes lists them (in other people's personal spaces it only counts them, and doesn't show the space's folder).
• A file that is still being copied into the folder appears once it has stopped changing for a few seconds.
• Disks and shares that aren't mounted. A space's folder holds a small hidden file, .thirtyfile-space, with the space's id, from when the space is created. A folder without it that is suddenly empty is taken to be a disk or share that isn't mounted: the space keeps its items, shares and history, and Check for changes says so. Mount it again and check. Changes made on the web are only made in a folder that holds the space's own file: when the folder is missing, or a different disk with a folder of the same name is mounted there, uploads and changes are refused with The folder isn't there, or a different disk is mounted there, and the folder is never made again. A folder holding another space's file isn't checked either. Files are read only from a folder that holds the space's own file. A read-only space's folder may also have none, since it can't always be written there. When the file is missing, Check for changes puts it back only if the folder still holds the space's items. Otherwise it reports the folder and changes nothing. A folder inside the space that can't be read keeps its items the same way. To empty a space, delete its items in ThirtyFile.
• When something goes wrong while several items are deleted, moved or restored at once, the ones already changed in the folder are put back, so nothing is half done. If ThirtyFile stops in the middle of moving an item into a space on the same disk, the item is put back in that folder, under its own name, at the next check a day later. If it stops while copying an item to another disk, the original is still where it was, and the unfinished copy is removed at that check. Renaming an item, moving it to another folder of its space, moving it to the trash or out of it, and keeping a file as an earlier version are written down first in a hidden .thirtyfile-journal folder: if ThirtyFile stops halfway, the next check for changes puts the item back where it was, with its shares, access and history, also on disks that don't tell items apart by an identity (Windows, exFAT). When something else took its old name meanwhile, the item stays where it was moved to, and keeps them there. When a save had already put the new content in place, that content stays and what the file had is kept as an earlier version.
• Items in the trash stay there when the folder they were in is deleted or renamed on the server. Restored, they go to the space's top folder.

Checking what a location holds

The … menu of each location under Control panel › Storage locations has three more actions. Only administrators have them.

Test step by step

Runs every step files go through and lists each one with how long it took:

1. Connect to the location.
2. Write a small file, then read it back and compare.
3. Upload a larger file (24 MB) the way uploads are stored, which on S3 is an upload in parts, then download it and compare. Both show their speed in MB/s.
4. Delete the test files and check that they are gone.

The test files go in a .thirtyfile-check folder and are deleted afterwards, also when a step fails. The test takes at most two minutes: on a slow connection the larger file is skipped rather than left half done. A failed step says why, for example that the keys can't delete files.

Browse

Shows what the location holds, one folder at a time and 200 items per page, with sizes and dates. Nothing can be changed here. Single files can be downloaded as they are stored, except content waiting to be deleted and copies still being written.

• Content store (ab/cd/…, under blobs on S3, SFTP and FTP): each item says whether a file uses it, an earlier version, a file in the trash, whether it is a replica of content kept on another location, whether it is waiting to be deleted, or whether nothing uses it. Where a file uses it, the space and the file's path are shown.
• Folder spaces on the storage folder and Local folder locations are marked with their space.
• Personal spaces stay private. Other people's personal spaces are named ("Personal space of anna"), but their files aren't listed, their folder can't be opened here, and their content can't be downloaded. This holds however the folder's name is written (USERS/Anna for users/anna), while the space is being moved (the folder it is being copied into, the content already copied, and its old folder until everything there is removed), and for replicas of its files on other locations, which are marked with the space.
• Symbolic links are listed but never followed, also on SFTP and FTP, and also in the middle of a path.
• The .thirtyfile-backups folder, which holds copies, can't be opened here: its lists of files name the files of personal spaces. Copies are managed under Control panel › Backups.

Find unused content

Looks through the content store of the location for content that no file, earlier version or item in the trash uses, for example content left behind when a server stopped halfway through an upload. It runs in the background and lists what it found, with the total size. Content written in the last 24 hours is left alone, since uploads store their content before recording it, and so is content already waiting to be deleted.

Remove N items deletes exactly the content on the list, after you confirm. Each item is checked again right before it is deleted: one that was used or written again in the meantime is kept. The removal is recorded in the activity log.

On S3, SFTP and FTP, searching and removing are refused when the place's .thirtyfile-location file names another location or another ThirtyFile installation: that content isn't unused, it belongs to them.

One place, one location

Two locations in the same place would each take the other's files for unused content. So a location can't be added, or changed, to a place another location already uses: the same bucket and prefix on the same service, or the same folder on the same SFTP or FTP server (a relative folder counts per account).

The place of an S3, SFTP or FTP location also holds a small .thirtyfile-location file: the location's id on the first line, the ThirtyFile installation's id on the second. It is written when the location is added. A location added before these files existed gets its file at its next successful connection check. A place whose file names another location, or another installation of ThirtyFile, can't be added, and its content isn't searched or removed. Deleting a location removes its file.

Storage usage and performance

Control panel › Storage usage shows how much is stored, how that changes over time, and how storage operations go. Only administrators see it. At the top are the totals by kind of space, the trash and earlier versions. Below them, each location has a card, and so does the data folder.

What the figures mean

Files changed in a folder space by other programs are counted after the next check for changes.

Operations

Every read, write, delete, listing and connection check on a location is timed and counted, and so are the files of folder spaces that are read or uploaded. Renaming, deleting and scanning inside folder spaces aren't measured. The counting happens in memory and is written once every five minutes, so it doesn't slow down file operations. Up to five minutes of counts are lost when the server stops.

• Kinds of work. People using files are downloads, previews, uploads and saves. Background work covers moves of spaces, compressing and extracting, long copies and moves, and deleting unused content. Checks and tests are the connection checks made every 20 to 30 seconds, and the step-by-step test. They are kept apart, so a move doesn't make upload times hard to read, and a test isn't taken for real traffic.
• Time per operation. Shown as the median (p50) and the 95th percentile (p95): half the operations were faster than p50, and 95 in 100 faster than p95. Percentiles are accurate to about 10%. A read is timed until it can start, and the transfer after that isn't included. A write is timed until the content is stored. On the same disk that is a rename, so it is quick.
• Throughput. Bytes read and written per second, averaged over each period.
• Failed or timed out. Operations that failed, plus those that took too long or were given up, for example when a person left during an upload. A download that breaks off partway counts as failed too. Errors people ran into are under Control panel › Activity log, on the Errors tab, and moves under Control panel › Moves. The page links to both.
• Idle. A period without operations has no point, and the chart shows a gap there. It never shows 0 ms.

History

Choose a location (or all of them), a range and a kind of work. Every chart states its unit and the period each point stands for. Pointing at a chart, or moving along it with the arrow keys, shows the values of one period. Show as a table lists them all. Download CSV saves what is shown.

Capacity is measured every 15 minutes and shortly after the start. The last measurement of an hour or a day stands for it. Operations are added up per hour and day, so the history stays small. Times are shown in your browser's time zone. A card whose latest measurement is more than 30 minutes old is marked Out of date.

Time to full draws a straight line through the disk's use over the last 30 days. It is only shown for a disk whose size is known, with at least 3 days of measurements, while the disk is filling up at a steady pace. Otherwise the page says why it isn't shown.

Alerts

Under Alert thresholds, choose when the page raises an alert: when a disk is fuller than a percentage (90% at first), or when more than a percentage of a location's operations failed or timed out in the last hour (5% at first). The failure alert needs at least 20 operations in that hour. Enter 0 to turn one off. Offline locations and measurements that are out of date raise an alert too. Alerts are shown at the top of the page, and they aren't sent by email.

When a location is offline

Browsing, searching and organising folders keep working. Opening, downloading and uploading are paused, and resume by themselves when the location is back. Files with a checked replica can still be opened and downloaded. The space is marked as offline in the meantime. In a folder space, whose folders are on the disk, organising folders is paused too.

Disks and shares that aren't mounted

A Local folder location is often a disk or a network share mounted on the server, such as /mnt/nas/thirtyfile. When it isn't mounted, its folder is missing or is an empty folder on the server's own disk. ThirtyFile never writes there: files would land on the wrong disk and be hidden once the share is mounted again.

• The marker file. Adding a Local folder location, or changing its folder, creates the folder when it isn't there yet and puts a small file in it, .thirtyfile-location, holding the location's id. Test connection before saving creates nothing. Nothing else ever creates the folder.
• Unavailable. From then on, the location is used only while its folder holds that file. Otherwise the list shows it as offline with The folder isn't there, or a different disk is mounted there, and its spaces refuse uploads and changes with the same message. Nothing is created or written, not even a space's folder.
• Back again. Mount the disk or share again: the location is checked every 20 seconds while it is offline, and is used again as soon as the file is back, without a restart.
• A disk that stops answering. A network share whose server went away can leave the disk mounted but silent. ThirtyFile then waits a minute at most for it (long enough for a disk to spin up), and 20 seconds once a change is under way, before refusing the change with The disk of the folder on the server doesn't answer. Until it answers again, changes to its spaces are refused at once, while everything else keeps working. Checks of the location and the health check don't wait for it either. The content store of a Local folder location is treated the same way: a step on its disk that takes more than a minute is given up with The storage service is temporarily unavailable, and until the disk answers again, its files are refused at once while the rest of ThirtyFile keeps working.
• The file went missing. A folder that has items but no .thirtyfile-location (a copy that left out hidden files, say) is offline with a message that names the line the file must hold: the location's id. If it is the right folder, create the file with that line in it; the location is used again at the next check.
• Another location's folder. A folder whose .thirtyfile-location names another location is refused. Deleting a location removes its file, so its folder can be used again. A folder that still holds the location's own file, when you change its folder back, is taken as it is.
• Changing the folder. While spaces or files are on the location, its folder can only change to a copy of it: copy everything in the old folder to the new one first, .thirtyfile-location included, then change the folder. The spaces' folders follow. A new, empty folder is refused, since the spaces' files would stay behind in the old one. A location with nothing on it can change to any folder.

The storage folder (/storage in Docker, set with THIRTYFILE_STORAGE) gets the same file, holding local, at the first start of a new server: when the folder is new or empty. Items a disk or NAS makes by itself (lost+found, #recycle, @eaDir, .DS_Store, System Volume Information, $RECYCLE.BIN) don't count. Once files or spaces are on it, a missing or empty storage folder is never taken again: ThirtyFile doesn't start and says the volume may not be mounted. If the storage folder has files in it but no .thirtyfile-location, or one of another location, ThirtyFile doesn't start either: it may be a different disk than the one it used. Check the volume or mount. When it really is the right folder, create the file with the word local in it.