What is virtiofsd?

nevj · November 23, 2023, 9:56am

I went to start my Debian12 VM in virt-manager.
I get the following in a popup

Error starting domain: operation failed: Unable to find a satisfying virtiofsd

So I searched in my Void host for virtiofsd

# xbps-query -Rs virtiofsd
[*] virtiofsd-1.8.0_1 Virtio-fs vhost-user device daemon

and installed it

xbps-install virtiofsd

Then I expected to see an entry in /etc/sv because it is a daemon… not there?
Nothing in which virtiofsd
Nothing in man virtiofsd
So I retry starting the Debian12 VM… and it works.

SUCCESS, but
Why do I need virtiofsd right now… I have had that Debian VM for months.
Why do all my other VM’s start without virtiofsd? Why only Debian affected ?
and
where is the installed virtiofsd in my system? It is neither a daemon, nor a command so what is it?
I tried this

$ ps ax | grep virtiofsd
 9580 ?        S      0:00 /usr/libexec/virtiofsd --fd=28 -o source=/common
 9590 ?        Sl     0:00 /usr/libexec/virtiofsd --fd=28 -o source=/common
21234 pts/15   S+     0:00 grep virtiofsd

so it is one of those weird things in /usr/libexec
Does anyone know how they are used?

If I shutdown Debian it disappears from the ps

# ps ax | grep virtiofsd
 3935 pts/16   S+     0:00 grep virtiofsd
#

and I actually have an hyperbola VM running , so Hyperbolavm does not need virtiofsd, only Debianvm???

I start FreeBSDVM… and now virtiofsd appears in a ps just like with Debian… but FreeBSDvm did not need it… it started without it.

It has something to do with my filesystem called /common which I export with nfs from the Void host to the guests.
I export it to FreeBSD and Hyperbola. I use it (ie mount it) in Hyperbola… so why does hyperbola not need virtiofsd ?.

It would seem that the OS that does do a virtiofs mount ( ie Debian) needs the virtiofs daemon, while the OS that does an nfs mount ( ie Hyperbola) does not need virtiofsd.

Sudden changes like this are a mystery.

I think I have it
In /etc/exports in the host I have

# cat /etc/exports
# /etc/exports
#
# See exports(5) for more information.
#
# Use exportfs -r to reread
# /export	192.168.1.10(rw,no_root_squash)

/common   192.168.122.45(rw,no_subtree_check,no_root_squash)
/common   192.168.122.57(rw,no_subtree_check,no_root_squash)
/common   192.168.122.41(rw,no_subtree_check,no_root_squash)

Those 3 IP numbers are MX ( which no longer exists as a VM) , hyperbola, and FreeBSD.
It is out of date… no Debian entry at all

So the new ‘fixed’ /etc/exports has

# hyperbola
/common   192.168.122.57(rw,no_subtree_check,no_root_squash)
# FreeBSD-2
/common   192.168.122.41(rw,no_subtree_check,no_root_squash)
# Debian12
/common   192.168.122.145(rw,no_subtree_check,no_root_squash)
# Antix
/common   192.168.122.59(rw,no_subtree_check,no_root_squash)

That should be right, but the question now is… do I need it.
Does virtiofsd take its place? I doubt it.
My Debian12 VM does not try to do a virtiofs mount in its /etc/fstab/… I have it commented out. So Debian was not trying to mount a non-exported filesystem. So what was it trying to do that required
virtiofsd? I dont know.

The only thing I know that changed recently was deleting the MX VM and forgetting to take its line out of /etc/exports.
Why would that affect Debian?

Rosika · November 23, 2023, 4:51pm

Hi Neville;

I don´t know anything about virtiofsd but I took a look at my host (Linux Lite) in order to be able to compare.

locate virtiofsd
/usr/lib/qemu/virtiofsd
/usr/share/man/man1/virtiofsd.1.gz
/usr/share/qemu/vhost-user/50-qemu-virtiofsd.json
/var/log/libvirt/qemu/debian10-fs0-virtiofsd.log

…

Almost looks like it.
If you take a look at my locate results there´s just one log-file present:

debian10-fs0-virtiofsd.log

pointing to the Debian VM…
…in spite of the fact that I´ve got Bodhi Linux and FreeBSD as VMs as well…

Sorry that I can´t be of more help, Neville.

Cheers from Rosika

nevj · November 23, 2023, 10:57pm

Hi Rosika,
That actually is a lot of help, thank you.
So Linux Lite has virtiofsd and it is in /usr/lib/qemu

Given that , I suspect now that Void has lost or misplaced virtiofsd… or that it never had it and my Debivm somehow worked without it

When I installed virtiofsd in Void, it went into /usr/libexec. I dont understand libexec.

If you take a look at my locate results there´s just one log-file present:

debian10-fs0-virtiofsd.log

pointing the the Debian VM…
…in spite of the fact that I´ve got Bodhi Linux and FreeBSD as VMs as well

That is a good clue. Debian is somehow different. It might be systemd causing it to differ.

I think my looking into NFS was a red herring…this has something to do with virtiofs, not nfs.

Regards
Neville

Double checked. I definitely have the virtiofs mount commented out in my Debian

$ cat /etc/fstab
# /etc/fstab: static file system information.
....
#mnt-tag-common /mnt/common virtiofs rw,relatime 0 0
#mnt-tag-common /mnt/common 9p rw,relatime 0 0

It is not using virtiofs.
But the daemon is running in Void while DebianVM is active

$ ps ax | grep virtiofs
13836 ?        S      0:00 /usr/libexec/virtiofsd --fd=28 -o source=/common
13841 ?        Sl     0:00 /usr/libexec/virtiofsd --fd=28 -o source=/common

And Void does not have /lib/qemu. All the virt... executables are in /usr/libexec.

And my locate finds more logs than yours

# locate virtiofsd
/usr/libexec/virtiofsd
/usr/share/doc/virtiofsd
/usr/share/doc/virtiofsd/README.md
/usr/share/doc/virtiofsd/xattr-mapping.md
/usr/share/licenses/virtiofsd
/usr/share/licenses/virtiofsd/LICENSE-APACHE
/usr/share/licenses/virtiofsd/LICENSE-BSD-3-Clause
/usr/share/qemu/vhost-user/50-qemu-virtiofsd.json
/var/cache/xbps/virtiofsd-1.8.0_1.x86_64.xbps
/var/cache/xbps/virtiofsd-1.8.0_1.x86_64.xbps.sig2
/var/db/xbps/.virtiofsd-files.plist
/var/log/libvirt/qemu/MXahs-fs0-virtiofsd.log
/var/log/libvirt/qemu/debian12vm-fs0-virtiofsd.log
/var/log/libvirt/qemu/freebsd13.1-2-fs0-virtiofsd.log

The MX log is defunct… I have removed the MV VM. But my FreeBSD makes a qemu log ? I have no logs from Hyperbola or Antix or GhostBSD. I wonder does it depend on whether qemu-ga is running in the guest?

locate is useful. I had to install it in Void.

Second update:
Wrong again. My VM’s all have logs

[nevj@trinity qemuvm]$ cd /var/log/libvirt/qemu
[nevj@trinity qemu]$ ls
LMDE-5.log                    dragoravm.log                    linux2022.log
MXahs-fs0-virtiofsd.log       freebsd13.1-2-fs0-virtiofsd.log  openbsd7.3.log
MXahs.log                     freebsd13.1-2.log                win10.log
antix22runit.log              freebsd13.1.log                  win11.log
debian12vm-fs0-virtiofsd.log  freebsd13.2.log                  win11vm.log
debian12vm.log                ghostbsdvm.log                   win7.log
dragonflybsd5.6.log           hyperbola03.log

even the ones I have deleted.
But only 3 have virtiofsd logs

Rosika · November 24, 2023, 1:52pm

Hi Neville,

thanks for the update.

Neville Jackson:

If you take a look at my locate results there´s just one log-file present:

debian10-fs0-virtiofsd.log

pointing the the Debian VM…
…in spite of the fact that I´ve got Bodhi Linux and FreeBSD as VMs as well

That is a good clue. Debian is somehow different. It might be systemd causing it to differ.

O.K., but if systemd was the cause then Bodhi Linux should still have a log entry, as it makes use of systemd as init software as well. I just checked here.
Remains a strange thing, I guess… .

Taking up your example I looked on my host as well:

ll /var/log/libvirt/qemu/
total 1,6M
-rw------- 1 root root  609 Jul 10 16:10 debian10-fs0-virtiofsd.log
-rw------- 1 root root 675K Nov 12 16:25 debian10.log
-rw------- 1 root root  26K Sep  9 16:43 freebsd13.0.log
-rw------- 1 root root  42K Okt 19 18:29 freebsd13.1-2.log
-rw------- 1 root root 269K Okt 29 15:22 freebsd13.1-3.log
-rw------- 1 root root 112K Sep 15 17:11 freebsd13.1.log
-rw------- 1 root root  19K Mär 14  2023 ubuntu18.04-2.log
-rw------- 1 root root 431K Nov 16 16:01 ubuntu18.04.log

And again: It´s only Debian which has a virtiofsd.log, unlike the other entries…

I´m not much help. Sorry, Neville.

Cheers from Rosika

nevj · November 24, 2023, 11:05pm

Hi Rosika,
I bet you have gemu-ga running in the Debian VM, but not in Bodhi or Freebsd?
Regards
Neville

nevj · November 25, 2023, 9:40am

The official page in virtiofsd is here
https://qemu.readthedocs.io/en/v7.2.2/tools/virtiofsd.html
it is not helpful.

There is a qemu page on removed features
https://www.qemu.org/docs/master/about/removed-features.html

It says


virtiofsd (removed in 8.0)

There is a newer Rust implementation of virtiofsd at https://gitlab.com/virtio-fs/virtiofsd; this has been stable for some time and is now widely used. The command line and feature set is very close to the removed C implementation.

My Void has qemu-8.1.0_1
That explains why virtiofsd has been removed from qemu.
But why would they remove it from qemu, then supply it as a separate package?
I have been unable to determine if the new virtiofsd separate package is the rust version … it depends in glibc and libgcc so maybe not?

Rosika · November 25, 2023, 1:48pm

Hi Neville,

I just started my Debian VM to have a look:

rosika2@debian ~> ps aux | grep qemu-ga
rosika2   1655  0.0  0.0   6084   888 pts/0    S+   14:35   0:00 grep --color=auto qemu-ga
rosika2@debian ~> locate qemu-ga
rosika2@debian ~>

So qemu-ga doesn´t seem to be running… .

Now I took a look at the FreeBSD VM.

rosika@freebsdTest ~> ps aux | grep qemu-ga
root       912   0.0  0.8 18752  7800  0  S    14:43   0:00.02 sudo qemu-ga
rosika     916   0.0  0.0   436   256  0  R+   14:43   0:00.00 grep --color=auto qemu-ga
root       913   0.0  0.8 18752  7784  1  Ss+  14:43   0:00.00 sudo qemu-ga
root       914   0.0  0.4 17660  4328  1  S    14:43   0:00.01 qemu-ga

but on the host, it´s still the same:

rosika@rosika-Lenovo-H520e ~> ll /var/log/libvirt/qemu/
total 1,6M
-rw------- 1 root root  609 Jul 10 16:10 debian10-fs0-virtiofsd.log
-rw------- 1 root root 680K Nov 25 14:38 debian10.log
-rw------- 1 root root  26K Sep  9 16:43 freebsd13.0.log
-rw------- 1 root root  42K Okt 19 18:29 freebsd13.1-2.log
-rw------- 1 root root 273K Nov 25 14:41 freebsd13.1-3.log
-rw------- 1 root root 112K Sep 15 17:11 freebsd13.1.log
-rw------- 1 root root  19K Mär 14  2023 ubuntu18.04-2.log
-rw------- 1 root root 431K Nov 16 16:01 ubuntu18.04.log

and:

rosika@rosika-Lenovo-H520e ~> locate virtiofsd
/usr/lib/qemu/virtiofsd
/usr/share/man/man1/virtiofsd.1.gz
/usr/share/qemu/vhost-user/50-qemu-virtiofsd.json
/var/log/libvirt/qemu/debian10-fs0-virtiofsd.log

O.K., that´s just for information.
Not much help from me - again. Sorry, Neville.

Cheers from Rosika

nevj · November 25, 2023, 10:27pm

Hi Rosika,
So you started qemu-ga in Freebsd, and it did not make a log happen in the host.
Maybe there was nothing to log?
Please look again when you next start Freebsd. You may get a log message on startup.
also
you started qemu-ga without flags
try putting this in /etc/rc.conf

qemu_guest_agent_enable="YES"
qemu_guest_agent_flags= -d -v -l /var/log/qemu-ga.log -p /dev/vtcon/org.qemu.guest_agent.0 -m virtio-serial

I think I am wasting your time. I have Antix and Ghostbsd with qemu-ga, and neither of those make a virtiofsd log.
Forget it.

There is also virtqemud it runs all the time in the host, but virtiofsd seems to only run when required… what triggers it is a mystery.
Regards
Neville

Rosika · November 26, 2023, 1:59pm

Hi Neville,

That could be one explanation, of course.

O.K., next time I start FreeBSD I´ll try to remember to take a closer look.

Interesting. I didn´t know about that.
I found something similar here (in German though). I´ll give it a try.

Thanks a lot and cheers from Rosika

nevj · November 26, 2023, 2:20pm

That looks OK.
service command only starts it for the current session
You need those lines in rc.conf to make it permanent.

The Freebsd way of starting services is even simpler than runit. I like it.

Rosika · November 26, 2023, 2:24pm

Thanks, Neville.
I´ll see to it that changes are made in rc.conf.

I see you´re still up.
It´s so late, or rather: early in the morning. I hope you´ll be able to find some rest after all.

Cheers from Rosika