2. tools
  3. btrfs

    btrfs

    Alternative written in Rust with huge potential: stratisd

    FAQ

    Subtools

    • btrbk: automatic backup through automatic snapshots (daily/weekly/etc..)
    • btrfsmaintenance: automate a few maintenance tasks. This means the scrub, balance, trim or defragmentation.
    • btrfs-backup: pure-bash automatic snapshots

    Create partition with raid

    Note: requires package btrfs-progs for debian

    List subvolume

    btrfs subvolume list -qu --sort ogen /home

    Used space

    Data, System, Metadata

    btrfs filesystem df /home

    Note: don't worry if metadata almost reach the available space, btrfs automatically expand it.

    Volume/Subvolume

    Show the exclusive and shared space used

    btrfs filesystem du -s /home

    Note: can take some time to compute

    Stats about errors

    btrfs dev stats /home

    Scrubg (checksum)

    btrfs scrub is used to scrub a btrfs filesystem, which will read all data and metadata blocks from all devices and verify checksums. Automatically repair corrupted blocks if there’s a correct copy available.

    btrfs scrub start /home
btrfs scrub status /home

    Copy btrfs filesystem from a device to another

    Raid or not, this solution should work.

    Whole volume (or one subvolume without taking deduplicating in consideration)

    # Get a list of subvolumes ordered
btrfs subvolume list -qu --sort ogen /media/old

# Make all subvolumes read-only
btrfs property set -ts /media/old ro true

btrfs send /media/old/sync/ | btrfs receive /home/

    Note: On rock64 board (popcorn) i was not able to properly run a copy (returns ioctl error at a certain point). I suspect trouble and unstability with debian:10 (homemade build for arm) and the usb driver. In dmesg I had such error BTRFS warning (device sda1): csum failed root 605 ino 826615 off 84135936 csum 0x4b288805 expected csum 0xb59b103c mirror 1 which could be related either by damage on the btrfs fs, to verify with scrub, or due to the memory (e.g., memtester should be ran to check). see discussion

    source

    Multiple specific subvolumes, with deduplicating

    # (if its the first time) Take the oldest subvolume (sharing common data with others)
btrfs send /media/<subvol_oldEST> | btrfs receive /home/

# send based on a parent 
btrfs send /media/<subvol> -p /home/<subvol_oldER> | btrfs receive /home/

    Questions

    How does btrfs receive find parent subvolume

    Use btrfs sub show on the subvolumes you're using and note the UUID values.

Use btrfs receive -v and note at the top of the spew you'll see receiving snapshot uuid= and parent_uuid, these will match the subvolumes on the sending machine, and match the receiving subvol's 'received uuid'. (as an aside, this is why I don't recommend changing ro properties of subvols, as this doesn't clear the received uuid and can mess up your incremental backups).

If you're cloning (-c) multiple subvols including parents, you don't even need to tell it which is the parent, it can work that out itself.

    source

    How to handle raid

    To my point of view, use lower level such as mdadm, however note that it is possible to make a raid with btrfs tools. e.g. create btrfs partition on two different devices mkfs.btrfs /dev/sda /dev/sdb, mount one mount /dev/sda /mnt, say explictely its raid1 for those two partitions btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt. source. However I'm not sure how reliable it is ? kernel based seems more reliable I guess.

    Experience with a drive replacement

    I tried a classical btrfs send /media/old/sync/ | btrfs receive /home/ but it did not work as expected (csum error from old device).

    I also tried btrfs restore --verbose --snapshots --xattr --metadata /dev/sda1 /home (where sda1 represent the btrfs fs from old device), but due to csum error it did not work either, even with --ignore-errors.

    However, I was able to get back some old snapshot with somethink like this btrfs send /media/old/btrbk_snapshots/sync.20201101/ | btrfs receive /home/.

    Problem with this snapshot is that it was not related with the recent sync as a parent i.e., 54087ccd-ae1f-3b48-923e-653aec83adbf:

    root@rock64-backup:~# btrfs subvolume list -qu --sort ogen /home
ID 257 gen 775 top level 5 parent_uuid -                                    uuid 54087ccd-ae1f-3b48-923e-653aec83adbf path sync
ID 381 gen 195 top level 5 parent_uuid 54087ccd-ae1f-3b48-923e-653aec83adbf uuid 6f4ab5ba-3a26-5146-9cf7-8cf9c2e9ecf3 path btrbk_snapshots/sync.20210325
ID 382 gen 197 top level 5 parent_uuid 54087ccd-ae1f-3b48-923e-653aec83adbf uuid f3af6ca1-5895-bd4e-b234-7e5de4c38e27 path btrbk_snapshots/sync.20210326
ID 429 gen 449 top level 5 parent_uuid -                                    uuid 0387b101-2613-3c4d-bd01-4d2c3cabdedb path btrbk_snapshots/sync.20201101

    In consequence, I had a waste of space, it didn't shared any data (see below). I should have spend more time with the option -p and -s concerning btrfs send, as well as investigating deduplication, but it felt a waste of time. I just end up with keeping the old drive off-line for while (the time that the new one gather enough data).

    root@rock64-backup:~# btrfs filesystem du -s /home/btrbk_snapshots/sync.20201101/
     Total   Exclusive  Set shared  Filename
 120.20GiB   120.20GiB       0.00B  /home/btrbk_snapshots/sync.20201101/

    Note: shared should have been different than 0.