🪴 Zero's Garden

ZFS

3 min read

zpool create \
      -o ashift=12 \
      -O compression=zstd \
      -O atime=off \
      -O xattr=off \
      -O dnodesize=auto \
      <...>
  1. ashift=12 sets pool sector size. The typical case for setting this property is when performance is important and the underlying disks use 4KiB sectors but report 512B sectors to the OS (for compatibility reasons); in that case, set ashift=12 (which is 1<<12 = 4096).
  2. compression=zstd enables zstd compression. lz4 is recommended for generic workload, while zstd has better compression.
  3. atime=off turns access time update off.
  4. xattr=off turns extended attributes off.
  5. dnodesize=auto to use the large_dnode feature.

Datasets

zfs craete <dataset>

Snapshots

# Create snapshot
zfs snap <volume>@<snapshot_name>
# Delete snapshot (dry-run)
zfs destroy -vn <volume>@<snapshot_name>
# List snapshots
zfs list -t snapshot
# Restore snapshot
zfs rollback <volume>@<snapshot_name>

Dataset properties

Record size

Note

The default 128KiB is good enough for most cases. See also ZFS Compression Test.

General rules of thumb:

  • 1MiB for general-purpose file sharing/storage
  • 1MiB for BitTorrent download folders—this minimizes the impact of fragmentation!
  • 64KiB for KVM virtual machines using Qcow2 file-based storage
  • 16KiB for MySQL InnoDB 
  • 8KiB for PostgreSQL

See https://klarasystems.com/articles/tuning-recordsize-in-openzfs/.

Compression

# List logical and compressed size
zfs list -o name,logicalused,used,compressratio
# List current compression config
zfs get compression
# Set zstd compression
zfs set compression=zstd pool[/component]
# Inherit from parent
zfs inherit compression pool[/component]

Applying to existing data

compression and deduplication can be applied to existing data with filerewrite, but recordsize change can not be applied in-place.

Adding disks

zpool attach adds a new device to an existing vdev in the pool, mirroring its content during the resilver process that is started immediately. This does not expand the pool’s capacity.

zpool add adds a new vdev to the pool, expanding its capacity. You may also use this command to add a separate intent log or cache device on SSD to improve performance.

Rescue mount

Linux

mount -o zfsutil -t zfs <dataset> <mountpoint>

ZFS on Linux features and gotchas

  • 2.3.0: RAIDZ expansion, fast dedup, direct IO, long names.
    • Available in Ubuntu since 25.04 (Plucky Puffin)
    • Direct IO is O_DIRECT support.
    • Fast dedup is still resource-intensive.

WARNING

All services depending on a ZFS filesystem should add a dependency on zfs.target to prevent early termination of critical ZFS services like zfs-zed.service. Otherwise your servicews may lose access to ZFS filesystems in their shutdown process and become stuck on I/O.

[Unit]
Wants=zfs.target
After=zfs.target

On Ubuntu, /etc/cron.d/zfsutils-linux schedules scrubbing on the second Sunday of every month.

References

Graph View