Planning For a Fresh Install, Any Suggestions?

Discussion
1

I have enjoyed following the discussions on “how to choose your next Linux distro”. Something mentiond by various contributors was how a fresh install can be faster and better clean all leftovers from previous installs.

Personally, I have mostly stuck to openSuse for years, and upgraded to latest revision, most recently to Slowroll. Some times I went for a fresh install for a cleaner system. In everycase the upgrades were fast, went without a hitch and the distro remained solid and stable, thank you OpenSuse.

I normally choose to install /home on a separate partion for quicker dup/fresh install, but getting back to before actually takes me a lot of time. I must be missing some tricks for a quicker fresh install:

  • a list of installed (main) apps: I try to keep a list but often miss secondary apps installed subsequently or needed for other uses. Is there a way to generate a list of main installed apps (not all +1,700 apps!) to check after dup?… or even to add to a zypper script to update?
  • preferences: settings before dup/fresh install may not be compatible or miss new features. Is there a quicker alternative to manually updating all preferences (and trying to remember all those colour settings that worked best!, etc )?
  • critical files: where best to keep files, (passwords!), etc that you need to get the fresh install fully up and running - ANother partition, USB, and what should be kept there for next fresh install ?

Any pointers for a quicker fresh install next ?

5 Likes
2

RG, your questions ring some bells with me. My solution was to plug in an external HDD. I created a folder called Perpetual Files. It contains my contact lists (as .vcf), my calendar (as .ics), and a dozen or so similar files that I’ll need for the next new install. I use a password manager from Protonmail that stores all my passwords far away. The external drive has a 1T capacity, so any file that might have use past the next hour or so gets filed there. No data in the new installation–all on the external drive.

4 Likes
3

I think Slowroll is a rolling release… that means you should never have to do a fresh install … but you may choose to as a means of getting a cleanup.

In fixed release distros like Debian, what I do is put the fresh install on a spare partition and multiboot both of them… then I have the old distro for reference while I am setting up the new release. When I am confident, I delete the old one.

I dont usually keep /home on a separate partition, but I put very little in it. All my data goes on a large data partition that I use from several distros. That separates data from all the dot files that clutter up home directories. That is similar to what @berninghausen does.

6 Likes
4

Thank you for the feedback.

I used to set up a separate partition /D on the internal HDD. I would put there all data and mount manually for additional separation. Following your comments I think I will go back to that arrangement, a bit similar to @berninghausen and @nevj solution but I prefer to use external HDD for backup.

For my passwords and similar I use KeePassXC. I keep it in an encrypted hidden directory, with YubiKey 2FA and an external backup. A bit cumbersome, specially after a fresh install, but has been solid and reliable for me. I do have an account with BitWarden but I am yet to fill confident to file there the some passwords!

I can see the dual boot solution provides a solid reference to fix the fresh install but it would still requires a lot of going back and forward between the two.

4 Likes
5

I fiddle a lot with my system before I put it to “production”.
At the moment I’m on Debian 12, my next will be Trixie.
It’s not yet ready.

So in thr beginning I start with a clean vanilla default installation.
I remove this and that, I install apps I need. Modify configs, etc.
Whatever I do, I put in a script.
That involves adding repositories, tweak some settings, actually install something.
At the end I’ll have a script, which transforms a default vanilla install into something that’s customized for me.
So fresh reinstalling is just do a normal install, and then run that script.
For you to get the idea, I show few of the steps in my script, but not the whole.
You need to make your own.

This is to set up Debian repos:

cat <<_EOF >/etc/apt/sources.list
# See https://wiki.debian.org/SourcesList for more information.
deb http://deb.debian.org/debian trixie main contrib non-free  non-free-firmware
deb-src http://deb.debian.org/debian trixie main non-free-firmware

deb http://deb.debian.org/debian trixie-updates main contrib non-free non-free-firmware
deb-src http://deb.debian.org/debian trixie-updates main non-free-firmware

deb http://security.debian.org/debian-security/ trixie-security main contrib non-free non-free-firmware
deb-src http://security.debian.org/debian-security/ trixie-security main non-free-firmware

_EOF

There are other repos, but setup in a different list file, such as for spotify:

##Spotify repo
curl -sS https://download.spotify.com/debian/pubkey_C85668DF69375001.gpg | gpg --dearmor |  tee /usr/share/keyrings/spotify.gpg
echo "deb [signed-by=/usr/share/keyrings/spotify.gpg] http://repository.spotify.com stable non-free" > /etc/apt/sources.list.d/spotify.list

Actually installing Spotify happens later in the script, basically just an apt install spotify-client -y

Tweaking settings is like that, for example enable mpv to use quicksync codecs:

#mpv quicksynchez:
mkdir -p /etc/mpv
cat << _EOFMPV > /etc/mpv/mpv.conf 
hwdec
hwdec-codecs=all
vd=h264_qsv,hevc_qsv,mpeg2_qsv,mjpeg_qsv,vp8_qsv,vp9_qsv,vc1_qsv,av1_qsv
_EOFMPV

Some other tweaks, involvind to load i2cdev:

#instruct dconf to store data in text:
echo "service-db:keyfile/user" > /etc/dconf/profile/user

##system configuration changes
echo fs.inotify.max_user_watches=524288 > /etc/sysctl.d/fsnotify.conf
echo vm.min_free_kbytes=8192 > /etc/sysctl.d/vmminfree.conf

## USB attached disks will be writeable without root privileges
echo 'KERNEL=="sd*", SUBSYSTEMS=="usb", MODE="0666"' | tee /etc/udev/rules.d/99-usb-storage.rules

#load on boot i2c for ddcutil
echo i2c-dev >> /etc/modules

All this tweaks and changes to the system is according to my liking and needs, your liking is prabably different. You can create your own script, which when you run, it transforms a default install to something you really love.

4 Likes
6

I always format the disk and install clean from a download.

Having copied my hard disk documents, images, music etc. NOT using backup but real copies.

7

@kovacslt Thank you, I had no idea it was possible to use a script to update/modify a base install!. Sad really: thinking about it, it must be obvious, and I guess it must be the standard method used to upgrade/update systems in an organization!. But I can see how it would improve the speed and consistency even on the adaptions of my systems at home after upgrade/installs. I will set it up, starting very slowly with my preferred OpenSuse tweaks.

@callpaul.eu Sorry not clear the distinction - I thought a backup is a copy of the file(s) perhaps more extensive and maybe compressed?

2 Likes
8

This is a seperate talking point between us, as to what you should or should not do, both have merit and pitfalls. So I will try to give a clear un biased reply.

Backup, this uses a software tool that copies from your system the files, directories, images you select to include. It then compression copies to another drive. In principal it uses less space on the destination drive and could be considered faster. It can also be set to backup entire drives so everything is copied.

My comments against backup is that you have to use the same software to extract your backup, so not always cross platform. Its also quite difficult to just restore 1 image from the backup. In some cases you are forced to have the same folder layout on the new drive to restore into.

I do more cross platform work, moving clients from windows to linux so dont have the same tools on both systems.

I prefer copy and paste or drag and drop copy from old system to another disk, then when new system is in place copy the stuff back.

My method uses the same space on a disk as the original no compression.

But i can copy to where I want not always the same folder structure, I choose. Plus I can get one file or image back easy.

But its all personnel choice, I am lucky that I have loads of spare hard disks, have time to copy what I want and use as I like to work, not all have this possible.

I copy from a windows system that may have really long complex paths to get original image and then easy drop into the linux image folder.

5 Likes
9

Thank you @callpaul.eu, perfectly clear. I use a basic rsync-based script for backup but it is simpler and quicker to have file copies ready. I will for my next fresh install!

3 Likes
10

I think what you do @callpaul.eu is not really backup and restore, but transfer data from one system to another. Isn’t it?

1 Like
11

Yes and NO

For clients yes so they have a copy just in case

But for myself I copy using this method

1 Like
12

Hi all, :waving_hand:

What I do (amongst many other things) as preparation for a fresh install is create a list of post-installed programmes / packages.
One often tends to forget about keeping a list of them.

There are the obvious ways of looking at post-installed packages, i.e. packages you added yourself after the setting up of your system.

But generally those include not only the packages themselves but also their dependencies, like libraries etc.
What if those changed with the new installation of your Linux system?
In this case the list won´t do much good.

Therefore the following command may come in handy:

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n ‘s/^Package: //p’ | sort -u) > post-installed_packages.txt

The command has to be issued from within your old system, of course.
It produces a list (text-file) with all the programmes / apps you installed yourself after setting up your (old) system.
And it lists the programmes only, without their dependencies. :wink:

My list looks like this (excerpt):

abiword
alpine
apt-rdepends
at
baobab
barrier
bat
bison
bucklespring
byobu
calcurse
calendar
calibre
cheese
cherrytree
[...]

Immensiely useful for post-installation with apt.

Only caveat:

The file
/var/log/installer/initial-status.gz
has to be present on your old system.
Not quite sure about it, but I think not every distro provides this file by default.
Mine (Linux Lite) does.

Many greetings from Rosika :slightly_smiling_face:

3 Likes
13

Hi @Rosika that’s so right - after a while it is easy to forget the bits and pieces installed to get things working the way you want/need. I have been looking to a way to generate a list of the main packages installed on my system. Unfortunately I don’t know enough to convert your command into “zypper speak”, but from searching it seems that on zypper/rpm you can only list everything installed inc libraries etc. The list then needs to be searched/sorted/trimmed - time consuming and not very helpful. So I have compiled manually a list from memory, checked against the package manager. Hopefully not much forgotten!.

I am hoping to write a script to load all packages on that list that were not installed in the initial base fresh install.

I am also compiling a list of info readily available after the fresh reinstall to get going quickly: fstab, a few passwords, email client settings, bookmarks, …

Regards and thanks for all feedback,

Ramiro

2 Likes
14

Hi Rosika,

Thanks! I didn’t know apt-mark showmanual maybe there’s same kind of list on other package managers. On Gentoo you can check the world file: cat /var/lib/portage/world

2 Likes
15

That is the valuable feature of apt-mark showmanual

You only want to know the packages you installed, not the dependencies. The dependencies will look after themselves

In Void Linux use
xbps-query -m to list all manually installed packages.

In Arch based distros use

pacman -Qe to list manually installed packages

I have not used OpenSUSE, but it seems the best you can do is

grep install /var/log/zypp/history

then review it either by hand or with a script.

3 Likes
16

Thank you @nevj I did find the zypp history, but it contains a very long chronological list of every packages and revision installed. I could not see any flags to identify manual installs, so I couldn’t use it. The package manager GUI is actually quite good, and I found it was quicker to use it to scroll through for a quick visual check.

I have now done a fresh install. I picked up the various comments received here (thanks again!) and it went surprisingly quickly, almost without a hitch and only a couple of missing packages managers that I had forgotten on my list.

Regards

3 Likes
17

Hi again, :waving_hand:

@RG1 :

You´re raising an intersting point. Although I don´t use openSUSE myself I found it so intriguing that I did some research on it.

You’re right — in zypper/rpm it’s not as straightforward as on apt, because the package manager doesn’t always track “manual vs dependency” installs in a user-visible way.

If your version of zypper supports it, you may try:

zypper search --installed-only --type package --user-installed

That gives you a list closer to what apt-mark showmanual does on Debian/Ubuntu systems.
It seems to be roughly equivalent to apt-mark showmanual.
(--user-installed was introduced in newer zypper versions, I guess, older ones might not support it.)

If that option isn’t available, the best fallback is to create a baseline list right after a fresh install and later compare against it. Example:

After a fresh install:

rpm -qa --qf '%{NAME}\n' | sort -u > baseline.txt

Later on (after customizing the system):

rpm -qa --qf '%{NAME}\n' | sort -u > current.txt
comm -23 current.txt baseline.txt > post-installed_packages.txt

That way you get a list of the packages you added after setting up your system - your real “post-installed” apps.

No idea whether that might be good enough for you. It´s the best I could come up with. :wink:

@ihasama :

Thanks for the additional info.

@nevj :

Yes, Neville. I like it a lot, too.
Plus:

Not every distro provides /var/log/installer/initial-status.gz.

That file is Debian/Ubuntu specific, and since Linux Lite is Ubuntu-based, I fortunately have it on my system.
It contains the package status snapshot right after installation, which makes it perfect for comparisons like the command I posted.

Non-Debian based distros (Fedora, Arch, openSUSE, etc.): They use different package managers and installers, so no such log exists.

Perhaps the workaround (see above) might do some good.

Many greetings to all from Rosika :slightly_smiling_face:

4 Likes
18

Thanks @Rosika . The CLI package manager zypper command does not recognise the –user-installed flag nor any that would suggest a similar purpose. The GUI package manager (Yast, soon to be replaced by Myrlyn) has quite good sorting options - but nothing that gives the manual install or excludes the dependencies. And looking at the history file I couldnt identify anything that could be used for that purpose including “installed” that seemed to apply to all packages.

The base install list is a good idea, but I moved recently to Slowroll, effectively a distro upgrade everytime and such base list would be out of date in a few weeks if not days!.

How is it done in an office with dozens or more PCs to upgrade/update?. Script to all run over the weekend plus a Monday morning email with a few “helpful” notes, and tough luck with any individual tweaks ?. My experience was more don’t update anything unless absolutely critical, but that left users doing their own “updates”!

2 Likes
19

They do one PC and clone it.

With a rolling release you dont need the list. Everything you install will stay there.
In the event of a disaster, roll it back to your last backup… dont do a fresh install.

2 Likes
20

Do you suppose Zypper doesn’t include the history type feature Rosika mentioned because it wouldn’t make sense on a rolling release?

3 Likes