Incron: monitoring folders recursively

Hi all, :wave:

does anyone of you use incron?
In fact it´s a fantastic utility for monitoring changes of dedicated folders or files.

It works well; yet I have a general question of how it works when monitoring folders recursively.

I´ve been looking around the internet for quite a while now and found various sources stating that incron doesn´t support recursive monitoring of folders.

Hmm, I think that might not be entirely true anymore. :thinking:

On incrontab(5) — Arch manual pages I found the hint:

Also, there is the symbol recursive=false. This symbol limits the observation on the specified directory and does not include subdirectories.

This should imply that otherwise (per default) recursive observation is active… :wink:

practical example:

I want to monitor a certain folder “finance”. In it there are various sub-folders for “taxes”, “insurance” etc. Within them there are more sub-folders.

Whenever I add or modify a file within any of these sub-folders a short signature tune should be played (behalten_Pausenzeichen_SWF1_1974.wav).

BTW:
the folder finance is soft-linked, but I don´t think that should be of greater importance.

With crontab -e I set up the rule:

/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/finance/ IN_CLOSE_WRITE firejail mplayer /home/rosika/Downloads/behalten_Pausenzeichen_SWF1_1974.wav

And indeed it works as desired … even with sub-folders. :+1:

However: what I noticed was the following:

In the rule I set up it seems to be important to include the final slash in the path:
/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/finance/
Otherwise the command is not executed.

My question are:

  • am I right in assuming this (final slash needed)? At least my practical tests would suggest it.
  • can anyone confirm that recursive monitoring is functional now? Again my test would suggest that.

Just curious to listen to what you kind folks think about it. :smiley:

Many thanks in advance and many greetings.
Rosika :slightly_smiling_face:

P.S.:

other sources:

Incron › Wiki › ubuntuusers.de (in German)

2 Likes

It depends on the program operating on the folders. With many classical programs, like cp, rsync and the like it’s important to specify, if you want to “handle” the links or just treat them as simple files.
I would expect newer programs to “handle” them by default, but back then it was normal to treat them specially.
How it is with incron, I don’t know.

In really old programs, it was pretty much universal, that my-name is a file and my-name/ is a folder. It’s a distinction, which then was applied in the programs’ functionalities. So, in general, if you wanted to operate on a folder, you attached that trailing slash, to be absolutely clear, you want to operate on a folder. If the slash was missing, you implied operation on a file.
Later, however, this became less and less common. Today, it’s not common to distinct the two like this, anymore.
Nowadays, the type is always auto-detected or explicitly specifically specified and that’s what counts. It’s usually not necessary anymore, to explicitly attach that trailing slash, to make clear that operation is supposed to be executed upon a folder, not a file.

It especially seems strange in this case, because the first argument is supposed to be an absolute path anyway, according to the documentation. So, it wouldn’t even make sense to make that distinction, as it must be a path, either way.

I would test that, by adding files to several different sub-folders at several levels.

Example:

#!/bin/bash

my_long_path_1='/why/is/this/path/so/long'
my_long_path_2='/why/is/this/path/so/so/long'
my_long_path_3='/why/is/this/path/so/so/so/long'

mkdir -p "$my_long_path_1"
mkdir -p "$my_long_path_2"
mkdir -p "$my_long_path_3"

touch "${my_long_path_1}/test_1"
touch "${my_long_path_2}/test_2"
touch "${my_long_path_3}/test_3"

If a notification appears for all these files, then it’s pretty much clear, that things work, as expected.

1 Like

Hi @Akito, :wave:

thank you so much for your detailed response. It´s highly appreciated, as always. :heart:

Yet it seems to me things get weirder the more I deal with this subject. :thinking:

1.)

Curious thing:
when booting into my system (after a reboot as well) incron won´t work.
If I create or alter a file (e.g) a text-file incron doesn´t trigger the action defined in incrontab.
So I looked up systemctl status incron and got the following result:


● incron.service - file system events scheduler
     Loaded: loaded (/lib/systemd/system/incron.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-03-24 14:22:03 CET; 4min 55s ago
       Docs: man:incrond(8)
    Process: 892 ExecStart=/usr/sbin/incrond (code=exited, status=0/SUCCESS)
   Main PID: 950 (incrond)
      Tasks: 1 (limit: 4509)
     Memory: 2.9M
     CGroup: /system.slice/incron.service
             └─950 /usr/sbin/incrond

Mär 24 14:21:57 rosika-10159 systemd[1]: Starting file system events scheduler...
Mär 24 14:22:03 rosika-10159 incrond[892]: starting service (version 0.5.12, built on Mar 22 2020 15:44:16)
Mär 24 14:22:03 rosika-10159 systemd[1]: Started file system events scheduler.
Mär 24 14:22:03 rosika-10159 incrond[950]: loading system tables
Mär 24 14:22:03 rosika-10159 incrond[950]: loading user tables
Mär 24 14:22:04 rosika-10159 incrond[950]: loading table for user rosika
Mär 24 14:22:05 rosika-10159 incrond[950]: access denied on /media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/F_M_neu/ - events will be discarded silently
Mär 24 14:22:06 rosika-10159 incrond[950]: cannot create watch for user rosika: (2) No such file or directory
Mär 24 14:22:06 rosika-10159 incrond[950]: ready to process filesystem events

That doesn´t look good. :slightly_frowning_face:

Why should access be denied regarding the defined path… :question:

Plus:

cannot create watch for user rosika: (2) No such file or directory

Now I restarted incron: sudo service incron restart

I got the following messages:

systemctl status incron
● incron.service - file system events scheduler
     Loaded: loaded (/lib/systemd/system/incron.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-03-24 14:29:58 CET; 1s ago
       Docs: man:incrond(8)
    Process: 3117 ExecStart=/usr/sbin/incrond (code=exited, status=0/SUCCESS)
   Main PID: 3118 (incrond)
      Tasks: 1 (limit: 4509)
     Memory: 2.0M
     CGroup: /system.slice/incron.service
             └─3118 /usr/sbin/incrond

Mär 24 14:29:58 rosika-10159 systemd[1]: Starting file system events scheduler...
Mär 24 14:29:58 rosika-10159 incrond[3117]: starting service (version 0.5.12, built on Mar 22 >
Mär 24 14:29:58 rosika-10159 incrond[3118]: loading system tables
Mär 24 14:29:58 rosika-10159 systemd[1]: Started file system events scheduler.
Mär 24 14:29:58 rosika-10159 incrond[3118]: loading user tables
Mär 24 14:29:58 rosika-10159 incrond[3118]: loading table for user rosika
Mär 24 14:29:58 rosika-10159 incrond[3118]: ready to process filesystem events

This looks much better. :smiley:

I tried it out and now incron works as desired. Creating a file, modifying and saving it, and copying a file would trigger the signature tune defined in incrontab.

BUT:
Why do I have to restart the incron service in the first place :question:
I mean: before the restart it was working as well (loaded, active and running) but it showed the access denied message.
After a restart: no such message… :thinking:

2.)

I followed the examples you kindly provided in you bash-script. Thanks a lot for that. :heart:

All the (sub-)directories were created. BUT:

none of this happened. Not one time … :astonished:

I did it this way:

my_long_path_1='/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/F_M_neu/prov22/why/is/this/path/so/long'
my_long_path_2='/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/F_M_neu/prov22/why/is/this/path/so/so/long'
my_long_path_3='/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/F_M_neu/prov22/why/is/this/path/so/so/so/long'
mkdir -p "$my_long_path_1"
mkdir -p "$my_long_path_2"
mkdir -p "$my_long_path_3"
touch "${my_long_path_1}/test_1"touch "${my_long_path_1}/test_1"
touch "${my_long_path_2}/test_2"
touch "${my_long_path_3}/test_3"

So all new (sub-)directories are within the newly created foleder prov22 which itself resides within the directory watched by incron.

But wait: it still gets weirder … :blush:

The path
/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/F_M_neu/

already holds a plethora of subfolders and sub-sub-folders etc. It´s here where my taxes and bills folders etc. reside.
So these (sub-)folders already exist. :exclamation: :exclamation: :exclamation:

Whenever I put a new file in them (or modify and save e.g. a text file) incron works and the signature tune is played. :astonished: :smiley:

Just not with the recently created (sub-)folders from your example.

I´m pretty baffled considering those two points and it seems I have no explanation for them at all.

Many thanks again and many greetings.
Rosika :slightly_smiling_face:

If that happens after a reboot, then it probably means, that the service is started, before something else is started, which removes the access denied problem.

There are ways to auto-start systemd services after other ones, as dependencies.

Not sure, what the reason is, but the first thing, that comes to my mind is permission issues. Maybe check out possible differences with ls -alh and compare files that work and the files which are not detected.

There will be a solution, if we go into the right direction, step by step.

1 Like

Hi @Akito, :wave:

thanks so much for your detailed response and for the plethora of links. :heart:

I´ll look into them and thereby try to resolve the problem.

One thing beforehand:

I´ve issued the systemd-analyze blame command and got the following result:

35.911s snapd.service                                                                            
34.701s tor@default.service 
# [... and pretty much at the end of the file:]
9ms sys-kernel-config.mount                                                                  
    9ms snapd.socket                                                                             
    5ms incron.service                                                                           
  463us clean-mount-point@media-rosika-28BC\x2dDAFC.service                                      
  356us clean-mount-point@media-rosika-A492\x2dCD29.service                                      
  340us clean-mount-point@media-rosika-f14a27c2\x2d0b49\x2d4607\x2d94ea\x2d2e56bbf76fe1.service  
  251us clean-mount-point@media-rosika-74C1\x2d30C7.service

Perhaps I´m all wrong here but wouldn´t that suggest that incron.service (5ms) is loaded after
/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/ (340 us)
is mounted?
So the succession should be right at least in this respect… :question:

Well, I´ll look into your links now.

In the meanwhile thanks a lot.

Many greetings from Rosika :slightly_smiling_face:

Seems like it. Maybe it’s not the only thing that needs to be loaded first? Not sure.

Maybe a little trial and error can already deliver some results, leading into the right direction.

1 Like

Thanks for the confirmation, @Akito . :+1:

Yes, that´s what I´m thinking as well.
I´ll try to investigate further.

Curious thing though:

I now did the following:

systemd-analyze plot > bootchart.svg

and loaded the .svg-file into a firefox tab. That way it´s easy to scan for search words.
Now I can safely say that incron doesn´t appear here at all.
Perhaps the chart is too crude to display anything between 0.0s and 1.0s…

Thanks so much.

Many greetings.
Rosika :slightly_smiling_face:

Hi, :wave:

not a solution yet but at least a hint, I guess: :blush:

Looking through some of the links lead me to investigate on my system:

locate incron.service | grep lib
[...]
/usr/lib/systemd/system/incron.service

Accordingly:

cat /usr/lib/systemd/system/incron.service
[Unit]
Description=file system events scheduler
Documentation=man:incrond(8)

[Service]
Type=forking
PIDFile=/run/incrond.pid
ExecStart=/usr/sbin/incrond

PrivateTmp=true

[Install]
WantedBy=multi-user.target

I guess the last part WantedBy=multi-user.target seems important.

This one (as I found out by looking at bootchart.svg) is started after 83s whereas incron.service ist started already after 5ms.

If my assumption is correct, it seems all I´d have to do is looking for a way of getting
incron.service started after multi-user.target:thinking:

I´ll keep looking.

Many greetings
Rosika :slightly_smiling_face:

EDIT:

Hmm, on second thoughts I´m not quite sure:

WantedBy=multi-user.target

Isn´t it rather so that multi-user.target needs incron.service to be running?

Indeed. It’s a bit confusing.

At first, I would try to make it start as late as possible. At least one of the links provided above talks about the topic of running a service last of all.

1 Like

Hi @Akito, :wave:

thanks again.

O.K., I was looking around and the descriptions provided on How do you make a systemd service as the last service on boot? - Super User seem a bit scary to me. I´m indeed afraid of messing things up on my system. Sorry. :slightly_frowning_face:

Taking a look at Start systemd service after specific service? - Stack Overflow it seems:

In systemd it is advised to use Before= and After= to order your services nicely around the other ones.

that should be the recommended way to go.

Still I´m not quite sure…

The current entry for incron.service is:

[Unit]
Description=file system events scheduler
Documentation=man:incrond(8)

[Service]
Type=forking
PIDFile=/run/incrond.pid
ExecStart=/usr/sbin/incrond

PrivateTmp=true

[Install]
WantedBy=multi-user.target

If I change it to

[Unit]
Description=file system events scheduler
Documentation=man:incrond(8)
After=multi-user.target  # no idea if this is right ????

[Service]
Type=forking
PIDFile=/run/incrond.pid
ExecStart=/usr/sbin/incrond

PrivateTmp=true

[Install]
WantedBy=multi-user.target  # should this remain here as well ????

I´m still not sure whether this would work and if it´s the correct way of changing incron.service :thinking:

Hmm,

but what about the following workaround, which wouldn´t require to change anything in the (crucial) definitions :question:

Whenever I start my system (after boot is finished) I punch in a certain command as per default (fish syntax):

ping ubuntuusers.de -c 3; and sleep 2; and conky -c /home/rosika/Desktop/Kopie_Skripte/new/mod_4bconky_enx001e101f0000.txt

This command gives me a feedback that my internet connection is established and starts the system monitor conky.

If I append sudo service incron restart to that command, i.e.:

ping ubuntuusers.de -c 3; and sleep 2; and conky -c /home/rosika/Desktop/Kopie_Skripte/new/mod_4bconky_enx001e101f0000.txt; and sudo service incron restart

that should take care of settings things right for incron, if I´m not completely mistaken. :thinking:

The beauty of fish is its autocompletion and colour highlighting.
I wouldn´t have to remember the whole of the command. I just type e.g. “ping”, then arrow up and “enter” and the command is executed.
There´s not even the need to create an alias for that. :blush:

Well, as I said, it would be some kind of workaround.
What do you think about it? Would there be any argument against it :question:

Many thanks again and many greetings
Rosika :slightly_smiling_face:

I think, it’s good to poke around a bit and try things out. I’m certain, we are close to the solution, so it should help already to have a trial & error session, showing how things go with different settings.

Hi @Akito, :wave:

thanks for the heads-up. :heart:

First of all I´d like to set my workaround-command right.
I couldn´t sleep well last night and one thing keeping me awake was realizing the command I provided wasn´t correct with respect to the right sequence of “sub-commands”.

If, like in the example I´ve given, sudo service incron restart was the last command to be executed it would come into action only after the preceding command has come to end.
That would be the conky process.
That´s not wanted after all and thus the correct command should be:

sudo service incron restart; and ping ubuntuusers.de -c 3; and sleep 2; and conky -c /home/rosika/Desktop/Kopie_Skripte/new/mod_4bconky_enx001e101f0000.txt

Sorry for the confusion. :slightly_frowning_face:

I tried it out and in fact it works as desired. The finance folder is monitored by incron and changes therein (and the sub-sufolders) are met with playing the signature tune. :smiley:

So I can confirm the workaround would be a “solution”. :blush:

But of course in order to look for a more elegant solution:

I´ll keep digging. :wink:

In the meantime I stumbled over an interesting source: configuration - How do I override or configure systemd services? - Ask Ubuntu .

I guess there are some important points to think about:

1.)

Packages ship unit files typically in /lib/systemd/system/

I found incron.service twice, actually:

rosika@rosika-10159 ~> ll /lib/systemd/system | grep incron
-rw-r--r-- 1 root root  209 Feb 19  2019 incron.service

rosika@rosika-10159 ~ [2|1]> ll /usr/lib/systemd/system | grep incron
-rw-r--r-- 1 root root  209 Feb 19  2019 incron.service

It turns out they´re identical:

env LANG=en_US.UTF-8 diff -s /lib/systemd/system/incron.service /usr/lib/systemd/system/incron.service
Files /lib/systemd/system/incron.service and /usr/lib/systemd/system/incron.service are identical

2.)

These are not to be edited. Instead, systemd allows you to override these files by creating appropriate files in /etc/systemd/system/.

3.)

For a given service foo, the package would provide /lib/systemd/system/foo.service.
You can check its status using systemctl status foo, or view its logs using journalctl -u foo. To override something in the definition of foo, do:

sudo systemctl edit foo

4.)

This creates a directory in /etc/systemd/system named after the unit, and an override.conf file in that directory (/etc/systemd/system/foo.service.d/override.conf).

5.)

You can add or override settings using this file (or other .conf files in /etc/systemd/system/foo.service.d/). This is also applicable to non-service units - you could do systemctl edit foo.mount, systemctl edit foo.timer, etc.

6.)

Usually, if you edit a systemd unit file, for it to take effect, you need to run:

sudo systemctl daemon-reload

However, systemctl edit automatically does this for you.

Actually I didn´t know the exact proceedings and hope that this is the right way to go.
If so, I might venture an “experiment”… :blush:

But I´m still not sure whether the way I want to change things as I pointed out in post #10
would be correct.
Basically:

under [Unit]: “After=multi-user.target”

and

under [Install]: “WantedBy=multi-user.target” # should this be left as it is :question:

Well, I guess there´s nothing left to do than to try…

Many thanks, @Akito and many greetings.
Rosika :slightly_smiling_face:

1 Like

incron looks like it could be a potential answer to an ongoing issue I have…

I have some hidden “secrets” (in files) that get sync’d by Resilio Sync (aka RSL), but resilio is SHIT at permisssions, e.g.

The file “SECRET” has 0600, I update something it, and RSL happily goes and updates it on ALL the other machines using that sync folder - BUT - it cuts the files BACK to 775 on ALL the sync targets - and I DO NOT WANT that - it’s a SECRET! So - I then have to run a script that finds ALL the sync targets and updates ALL the files to 0600, except for one file, an expect script, that I need to have 0700!

I can’t have have a universal umask for that folder ('cause one file need execute) :

I also need to have the parent folder set to 0700, if it’s 0600 then I can’t traverse to it…

I was going to try and solve this using ansible, but it got too hard and a shell script that SSH’s to each targer sorta/kinda works, but it’s a bit clunky, and inelegant…

Excellent analysis & great research. :clap: :grinning:

Keep going, you are on the right track.

Not sure about that, either. Never had to use this explicitly. Would need to test, just as you already do, anyway.

Indeed. It’s easiest and quickest to just try now.

Hi all, :wave:

@Akito:

Thanks so much for the compliment. :heart:
I´ll keep going but the phenomenon we´re dealing with here is really something else.

So I thought: why not getting even further help by posting a thread in the (renowned) German-spreaking forum at ubuntuusers.

The topic is located here: einen bestimmten Dienst als letzten starten (hier: incron.service) › System einrichten und verwalten › Systemverwaltung, Installation, Aktualisierung › Forum › ubuntuusers.de .

There´s an ongoing discussion here and up to now I´ve learnt quite a lot but the final breakthrough hasn´t been achieved yet :wink:.

As soon as I can come up with something substantial I´ll post it here. :slightly_smiling_face:

In the meantime: be thanked a lot. :heart:

@daniel.m.tripp:

That sounds great, Dan.

It seems quite a complicated scenario you´re dealing with.
Perhaps incron can be helpful in your case as well.

Many greetings to all of you.
Rosika :slightly_smiling_face:

Did you try this?
https://forum.ubuntuusers.de/topic/einen-bestimmten-dienst-als-letzten-starten-hi/#post-9306845

As a workaround/test, you could also add a sleep 500 or something like that, to the service or just inside the script, which the systemd service is running. This way, you can artificially delay the startup. If it works, then you can be sure, that it is related to how early it is started.
However, if it still won’t work, then you know, that it does not matter how early it starts, but there is a different problem with getting the mount ready, or something related to that.

1 Like

Hi @Akito, :wave:

thanks so much for your suggestion.

Well, in the meantime we tried the following:

create /lib/systemd/system/incron.service.d/reqmount.conf with:

[Unit]
RequiresMountsFor=/media/rosika/f14a27c2-0b49-4607-94ea-2e56bbf76fe1/LINK-FARM/F_M_neu

This wasn´t met with any success. So as a next step:

  • deactivate the incron.service-unit
  • create an incron.timer unit with the following contents:
[Unit]
Description=Starts incron 60 seconds after (re)boot

[Timer]
OnStartupSec=60sec
Unit=incron.service

[Install]
WantedBy=multi-user.target
  • activate incron.timer

Yet the problem still remains. :slightly_frowning_face:

What would work however is the following:

  • sudo systemctl disable incron
  • after reboot: sudo systemctl start incron

So manually starting the incron service would work as desired.

I was thinking of packing it into an already existing command of mine which would then be (fish syntax):

sudo systemctl start incron; and ping ubuntuusers.de -c 3; and sleep 2; and conky -c /home/rosika/Desktop/Kopie_Skripte/new/mod_4bconky_enx001e101f0000.txt

… i.e.: starting incron - verifying my internet connection - running the conky system monitor.

It´d take no longer than 1 - 2 secs. to issue the command (due to fish). :wink:

Well, the original question (why ist it that …) wouldn´t be solved but at least: some kind of workaround…

Many greetings.
Rosika :slightly_smiling_face:

To make absolutely sure, the test delivers correct results, I would set the delay to at least 500 seconds, just to be sure, that nothing else is still starting, which incron depends on.
If it still won’t work after 500 seconds, then it’s not related to another service starting too early, as I explained in the earlier post.

This, again, seems like it’s fine if all other services are started. I’m sure, that by the time this service is started manually, all others are already started for a long while.

I still think, we are close to the actual solution. There needs to be some dependency or whatever. Maybe incron has some FAQ or documentation on this issue or at least hints on what services need to be running first.
Maybe the maintainers might also be of help.

1 Like

Hi @Akito, :wave:

thanks again for your suggestions.

O.K. That sounds reasonable. That would be 8.33 mins then.

Yes, quite right. My respective test was successful; so I can safely say it works this way. :wink:

I was looking around and I guess this would be the path to follow:

Thanks again so much. I´ll keep you posted.

Many greetings from Rosika :slightly_smiling_face:

Hi @Akito, :wave:

SUCCESS: :smiley:

It seems you were very, very right in assuming:

I changed the incron.timer unit to 500 secs and waited for 9 mins (actually 14 mins as I had some other stuff going on)
The result looks promising:

systemctl status incron.service
● incron.service - file system events scheduler
     Loaded: loaded (/lib/systemd/system/incron.service; disabled; vendor preset: enabled)
    Drop-In: /usr/lib/systemd/system/incron.service.d
             └─reqmount.conf
     Active: active (running) since Tue 2022-03-29 17:00:01 CEST; 7min ago
TriggeredBy: ● incron.timer
       Docs: man:incrond(8)
    Process: 13771 ExecStart=/usr/sbin/incrond (code=exited, status=0/SUCCESS)
   Main PID: 13789 (incrond)
      Tasks: 1 (limit: 4509)
     Memory: 2.7M
     CGroup: /system.slice/incron.service
             └─13789 /usr/sbin/incrond

Mär 29 17:00:01 rosika-10159 systemd[1]: Starting file system events scheduler...
Mär 29 17:00:01 rosika-10159 incrond[13771]: starting service (version 0.5.12, built on Mar 22>
Mär 29 17:00:01 rosika-10159 incrond[13789]: loading system tables
Mär 29 17:00:01 rosika-10159 systemd[1]: Started file system events scheduler.
Mär 29 17:00:01 rosika-10159 incrond[13789]: loading user tables
Mär 29 17:00:01 rosika-10159 incrond[13789]: loading table for user rosika
Mär 29 17:00:03 rosika-10159 incrond[13789]: loading table for user root
Mär 29 17:00:03 rosika-10159 incrond[13789]: ready to process filesystem events

… and it works. :+1: :+1: :+1:

So it really seems other important dependencies take more than 60 secs to load …

Thanks so much :heart:

Many greetings.
Rosika :slightly_smiling_face: