Snapper btrfs snapshots on Gentoo, need pointers

ihasama · October 11, 2024, 8:00am

I’m learning to use Snapper with my btrfs to make snapshots of my root. Snapper is used in openSUSE and available to other linux oses too. My system is Gentoo and I’ve followed the wiki
and got it working. At the moment it makes a new snapshot everytime I update my system.
here’s the status of my system before I did first update after followed the wiki:

$ snapper list
# │ Type   │ Pre # │ Date                             │ User │ Cleanup │ Description │ Userdata
──┼────────┼───────┼──────────────────────────────────┼──────┼─────────┼─────────────┼─────────
0 │ single │       │                                  │ root │         │ current     │
1 │ single │       │ Thu 10 Oct 2024 08:59:47 PM EEST │ pete │         │             │

then after sync&update (there was 7 packages to update)

$ snapper list
 # │ Type   │ Pre # │ Date                             │ User │ Cleanup │ Description                                                                   │ Userdata
───┼────────┼───────┼──────────────────────────────────┼──────┼─────────┼───────────────────────────────────────────────────────────────────────────────┼─────────
 0 │ single │       │                                  │ root │         │ current                                                                       │
 1 │ single │       │ Thu 10 Oct 2024 08:59:47 PM EEST │ pete │         │                                                                               │
 2 │ pre    │       │ Fri 11 Oct 2024 10:34:15 AM EEST │ root │ number  │ Upgrading to dev-lang/spidermonkey-115.16.0 replacing version(s) 115.15.0     │
 3 │ post   │     2 │ Fri 11 Oct 2024 10:34:18 AM EEST │ root │ number  │ Upgrading to dev-lang/spidermonkey-115.16.0 replacing version(s) 115.15.0     │
 4 │ pre    │       │ Fri 11 Oct 2024 10:34:29 AM EEST │ root │ number  │ Upgrading to dev-libs/ell-0.69 replacing version(s) 0.66                      │
 5 │ post   │     4 │ Fri 11 Oct 2024 10:34:32 AM EEST │ root │ number  │ Upgrading to dev-libs/ell-0.69 replacing version(s) 0.66                      │
 6 │ pre    │       │ Fri 11 Oct 2024 10:34:34 AM EEST │ root │ number  │ Upgrading to net-libs/libpsl-0.21.5 replacing version(s) 0.21.5               │
 7 │ post   │     6 │ Fri 11 Oct 2024 10:34:37 AM EEST │ root │ number  │ Upgrading to net-libs/libpsl-0.21.5 replacing version(s) 0.21.5               │
 8 │ pre    │       │ Fri 11 Oct 2024 10:35:03 AM EEST │ root │ number  │ Upgrading to net-firewall/ufw-0.36.1-r1 replacing version(s) 0.36.1           │
 9 │ post   │     8 │ Fri 11 Oct 2024 10:35:10 AM EEST │ root │ number  │ Upgrading to net-firewall/ufw-0.36.1-r1 replacing version(s) 0.36.1           │
10 │ pre    │       │ Fri 11 Oct 2024 10:35:48 AM EEST │ root │ number  │ Upgrading to www-client/firefox-bin-131.0.2 replacing version(s) 131.0        │
11 │ post   │    10 │ Fri 11 Oct 2024 10:35:52 AM EEST │ root │ number  │ Upgrading to www-client/firefox-bin-131.0.2 replacing version(s) 131.0        │
12 │ pre    │       │ Fri 11 Oct 2024 10:36:13 AM EEST │ root │ number  │ Upgrading to mail-client/thunderbird-bin-128.3.1 replacing version(s) 128.3.0 │
13 │ post   │    12 │ Fri 11 Oct 2024 10:36:17 AM EEST │ root │ number  │ Upgrading to mail-client/thunderbird-bin-128.3.1 replacing version(s) 128.3.0 │
14 │ pre    │       │ Fri 11 Oct 2024 10:36:29 AM EEST │ root │ number  │ Upgrading to app-portage/emlop-0.7.0 replacing version(s) 0.6.1               │
15 │ post   │    14 │ Fri 11 Oct 2024 10:36:32 AM EEST │ root │ number  │ Upgrading to app-portage/emlop-0.7.0 replacing version(s) 0.6.1

I think this is a bit too much to have a pre and post snapshot of every updated package. Optimal would be one snapshot before update and one snapshot after update of all packages.

Maybe I remove the /etc/portage/bashrc script (seen in the wiki link) for now and just make a scheduled snapshot for now.

This snapper thing is very cool but I need to read more about it… Do you have any tips / tricks?

nevj · October 11, 2024, 11:33am

Have you tried recovering fronm a snapshot? You need to know recovery will work.

ihasama · October 11, 2024, 4:56pm

You’re right. Although I don’t think snapper is enough for backup. I think it as a quick recovery option and still take real backups to cloud and USB.

I’m still just starting to understand it. Now reading the openSUSE manual:

nevj · October 11, 2024, 11:27pm

Right… like Timeshift, only hopefully without making assumptions and doing things behind your back.

ihasama · October 12, 2024, 7:27am

a quick update: I installed app-backup/grub-btrfs and now I can boot from grub to a snapshot. GitHub - Antynea/grub-btrfs: Include btrfs snapshots at boot options. (Grub menu)

I can boot from grub menu to my snapshots.

nevj · October 12, 2024, 11:16am

That is neat.
I suppose a snapshot is a .img file? If that is the case , what grub would do is loop mount the snapshot file, so it looks like a device.

ihasama · October 12, 2024, 12:25pm

it’s just a full snapshot of my root and there is also the /boot/ in the snapshot. It’s not read-only, I’ve chown -R /.snapshots

here’s the latest snapshot of my /boot:

ls -al /.snapshots/27/snapshot/boot/
total 388544
drwxr-xr-x 1 root root      990 Oct  4 07:43 .
drwxr-xr-x 1 root root      302 Oct 11 10:15 ..
-rw-r--r-- 1 root root        0 Jun 16 14:28 .keep
-rw-r--r-- 1 root root  9475938 Jun 17 13:00 System.map-6.6.32-gentoo-dist
-rwxr-xr-x 1 root root  9411543 Aug 22 18:01 System.map-6.6.47-gentoo-dist
-rw-r--r-- 1 root root  9413378 Oct  4 07:39 System.map-6.6.51-gentoo-dist
-rw-r--r-- 1 root root  9413378 Sep 27 10:54 System.map-6.6.51-gentoo-dist.old
-rw-r--r-- 1 root root    77312 Sep 22 16:45 amd-uc.img
-rw-r--r-- 1 root root   266734 Jun 17 13:00 config-6.6.32-gentoo-dist
-rwxr-xr-x 1 root root   266812 Aug 22 18:01 config-6.6.47-gentoo-dist
-rw-r--r-- 1 root root   266814 Oct  4 07:39 config-6.6.51-gentoo-dist
-rw-r--r-- 1 root root   266814 Sep 27 10:54 config-6.6.51-gentoo-dist.old
drwxr-xr-x 1 root root      102 Oct  7 17:09 grub
-rw-r--r-- 1 root root 77038809 Jun 17 14:07 initramfs-6.6.32-gentoo-dist.img
-rwxr-xr-x 1 root root 66414539 Aug 22 18:01 initramfs-6.6.47-gentoo-dist.img
-rw-r--r-- 1 root root 68048193 Oct  4 07:39 initramfs-6.6.51-gentoo-dist.img
-rw-r--r-- 1 root root 67924532 Sep 27 10:54 initramfs-6.6.51-gentoo-dist.img.old
-rw-r--r-- 1 root root 16665600 Oct  4 07:43 intel-uc.img
-rw-r--r-- 1 root root 15723504 Jun 17 13:00 vmlinuz-6.6.32-gentoo-dist
-rwxr-xr-x 1 root root 15707120 Aug 22 18:02 vmlinuz-6.6.47-gentoo-dist
-rw-r--r-- 1 root root 15715312 Oct  4 07:39 vmlinuz-6.6.51-gentoo-dist
-rw-r--r-- 1 root root 15715312 Sep 27 10:54 vmlinuz-6.6.51-gentoo-dist.old

The difference is in /boot/grub as in the snapshot there is no grub-btrfs.cfg file.

pete@gentoo ~ $ ls /boot/grub/
fonts  grub-btrfs.cfg  grub.cfg  grub.cfg~  grubenv  locale  themes  x86_64-efi

pete@gentoo ~ $ ls /.snapshots/27/snapshot/boot/grub/
fonts  grub.cfg  grub.cfg~  grubenv  locale  themes  x86_64-efi

So the boot process uses /boot/grub and there is grub-brtfs.cfg which adds a boot menu item “Gentoo” where it lists all snapshots. I can choose at boot to continue normally, use Debian(dual boot) or go to the snapshot menu. The GitHub link I posted earlier has more info.

nevj · October 12, 2024, 12:40pm

Oh, so it is like an rsync copy.
I guess grub can boot that quite easily… it is the same as an installed OS as far as grub is concerned.

Sorry, but I like to understand how grub does things.

ihasama · October 12, 2024, 12:46pm

me too
here’s the start of the 41_snapshots-btrfs:

cat /etc/grub.d/41_snapshots-btrfs
#! /usr/bin/env bash
#
# Written by: Antynea
# BTC donation address: 1Lbvz244WA8xbpHek9W2Y12cakM6rDe5Rt
# Github: https://github.com/Antynea/grub-btrfs
#
# Purpose:
#   Improves Grub by adding "btrfs snapshots" to the Grub menu.
#   You can boot your system on a "snapshot" from the Grub menu.
#   Supports manual snapshots, snapper, timeshift ...
#   Warning : booting on read-only snapshots can be tricky.
#   (Read about it, https://github.com/Antynea/grub-btrfs#warning-booting-on-read-only-snapshots-can-be-tricky)
#
# What this script does:
# - Automatically List snapshots existing on root partition (btrfs).
# - Automatically Detect if "/boot" is in separate partition.
# - Automatically Detect kernel, initramfs and intel/amd microcode in "/boot" directory on snapshots.
# - Automatically Create corresponding "menuentry" in grub.cfg.
# - Automatically detect the type/tags and descriptions/comments of snapper/timeshift snapshots.
# - Automatically generate grub.cfg if you use the provided systemd service.
#
# Installation:
# - Refer to https://github.com/Antynea/grub-btrfs#%EF%B8%8F-installation
#
# Customization:
#  You have the possibility to modify many parameters in /etc/default/grub-btrfs/config.
#  Read more here https://github.com/Antynea/grub-btrfs#installation- an in the manpage
#  'man grub-btrfs'
#
# Automatically update Grub
#  If you would like grub-btrfs menu to automatically update when a snapshot is created or deleted:
#  - Refer to https://github.com/Antynea/grub-btrfs#-automatically-update-grub-upon-snapshot.
#
# Special thanks for assistance and contributions:
# - My friends
# - All contributors on Github
#

nevj · October 12, 2024, 12:53pm

OK, I get it, it makes a custom entry in grub.d

ihasama · December 11, 2024, 8:20pm

Here’s my first real use case after crash:

I updated(upgraded) my system today and after reboot Hyprland crashed at launch.
Snapper to resque!
snapper list to see the pre Hyprland update snap number (182)
snapper modify 182 --read-write to boot to snap with read-write permissions

Then reboot to snap 182 from grub and:

sudo emerge -a =gui-wm/hyprland-0.42.0-r1 to re-install the previous working Hyprland version.
sudo revdep-rebuild to make sure all is ok for the new old version of Hyprland
Gentoo’s package manager Portage has a nice feature to force it to keep an old version even if there’s newer version of that package: sudo nano /etc/portage/package.mask/hyprland and add a line >=gui-wm/hyprland-0.42.0-r2 to mask the newer version(s). Masking means that when I update the Hyprland wont update to that version. Need to remember to remove that masking some day.

Last to update everything again just in case: sudo emerge -uDN @world there wasn’t a new hyprland package to install! Then sudo emerge --depclean and reboot.

Fingers crossed!

ihasama · December 11, 2024, 8:29pm

Well, it didn’t work
Luckily I have Cinnamon as a backup DE, so TTY login and startx to get going. One thing with Hyprland is that it is very fast moving WM and it just crashes sometimes. Maybe it’s been fixed tomorrow… Anyways, snapper is great tool to get an earlier (working) snapshot to boot to if there’s any issues.

nevj · December 11, 2024, 10:20pm

Why ?
There must be some other package involved in the hyperland fault, probably a dependency. You need to mask that too, if you can identify it.

ihasama · December 12, 2024, 6:27am

There’s maybe something else what affects Hyprland. Clang, llvm, rust and openRC were upgraded.

Hyprland error log says getCurrentCRTC: No CRTC 0

If you haven’t updated Gentoo this week maybe it’s better to wait for a while.

ihasama · December 12, 2024, 11:49am

I got the Hyprland working. It’s the r1, not newest r2 version:

genlop -i hyprland
 * gui-wm/hyprland


   Total builds: 1
   Global build time: 5 minutes and 50 seconds.

   Info about currently installed ebuild:

   * gui-wm/hyprland-0.42.0-r1
   Install date: Fri Oct 25 10:23:29 2024
   USE="X -legacy-renderer -systemd"
   CFLAGS="-O2 -pipe -march=x86-64-v3 -flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing"   CXXFLAGS="-O2 -pipe -march=x86-64-v3 -flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing"   LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"

I’ll wait for a couple of days now before trying to update it.

ihasama · December 12, 2024, 5:36pm

My desktop with Gentoo and systemd works fine after Hyprland upgrade to the 0.42.0-r2 so it probably is an openRC (which was also upgraded) issue with my laptop.

EDIT: I’ll add the process how to use the working snapshot as a new default grub entry where the system will boot if you don’t press any button during boot. First I followed this link’s instructions:

When I ran the command I got this error message:
sudo snapper rollback: Cannot detect ambit since default subvolume is unknown. This can happen if the system was not set up for rollback. The ambit can be specified manually using the --ambit option.

Then a bit of searching and found the solution:

$ sudo snapper --ambit classic rollback <id>

Now my system boots normally and all works. Next:

sudo emerge -uDN --with-bdeps=y @world

Local copy of remote index is up-to-date and will be used.

Local copy of remote index is up-to-date and will be used.

These are the packages that would be merged, in order:

Calculating dependencies … done!
Dependency resolution took 38.10 s (backtrack: 0/20).

[ebuild U ] sys-apps/openrc-0.55.1::gentoo [0.54.2::gentoo] USE=“netifrc pam sysv-utils sysvinit unicode -audit -bash -caps -debug -newnet -s6 (-selinux)” 0 KiB
[ebuild U ] gui-wm/hyprland-0.42.0-r2::gentoo [0.42.0-r1::gentoo] USE=“X -legacy-renderer -systemd” 0 KiB

Total: 2 packages (2 upgrades), Size of downloads: 0 KiB

!!! The following binary packages have been ignored due to non matching USE:
=sys-apps/openrc-0.55.1 -sysv-utils
NOTE: The --binpkg-respect-use=n option will prevent emerge
from ignoring these binary packages if possible.
Using --binpkg-respect-use=y will silence this warning.

Would you like to merge these packages? [Yes/No]

Fingers crossed

ihasama · December 12, 2024, 8:04pm

well after updates and reboot hyprland didn’t work. So I’ll stay on the working versions of openrc and hyprland for now. I’m glad that the btrfs and snapper work! This was a real test of the snapshots and it worked

nevj · December 12, 2024, 10:34pm

Agree, that is an achievement.

I am puzzled about your OpenRC… it seems to be using SysVinit components?
My Gentoo is the same.
I read that it is possible to have OpenRC on its own, without sysVinit… ie OpenRC supplies the init process, the startup, and the service management.
There may be some options there that you could fiddle to get around it

ihasama · December 13, 2024, 5:57am

My OpenRC is not modified any ways. It’s just like how you would install a fresh Gentoo system by the handbook. I don’t know more, sorry.

Next thing I’ll try is to upgrade Hyprland but leave OpenRC as it is now. Then I’ll see if it’s an Hyprland issue or OpenRC issue.

nevj · December 13, 2024, 6:17am

"Users of OpenRC can have sys-apps/openrc compiled with USE=sysv-utils and system (most importantly bootloader) configured to use openrc-init. This way OpenRC doesn’t depend on sysvinit. "

from the Gentoo sysvinit Wiki
I would want to be sure it is an OpenRC issue before trying anything like this.