DMF and archiving

Projects can apply for archive storage space. Archive storage spaces for projects are separate folders on Lustre under /project/archive. DMF handles the policies that are in place for archive storage spaces and is backed by a Spectra Tfinity tape library.

Policies for archive storage spaces:

  • migrate files that are at least 100MB, older than 1 hour, and have not been migrated yet (check every two hours)

  • release blocks in the filesystem for files that are already on tape and older than 4 hours (check every two hours)

Note

Released blocks do not count towards your block quota.

Warning

Released blocks do not have an effect on your available Inodes.

Checking file archive status

You can inspect whether a file has been migrated (archived), released, or still pending by using the lfs hsm_state command. This tool queries the HSM (Hierarchical Storage Management) status managed by DMF.

Example 1 — File already archived and released:

[testuser@vn01]$ lfs hsm_state testfile5
testfile5: (0x0000000d) released exists archived, archive_id:1

  • archived – the file’s contents are safely stored on tape

  • released – the Lustre file blocks were freed (only metadata remains on disk)

  • exists – the file’s metadata is still present in the filesystem

  • archive_id:1 – identifies the DMF archive target

Example 2 — File waiting for migration:

[testuser@vn01]$ lfs hsm_state ubuntu-25.04-desktop-amd64.iso
ubuntu-25.04-desktop-amd64.iso: (0x00000000)

  • (0x00000000) indicates that the file is not yet archived and no HSM actions have been performed.

Useful commands

Check the state of all files in the current directory:

lfs hsm_state *

List only those that are already archived:

lfs hsm_state * | grep archived