Recursive Inotify Daemon

There aren’t many Inotify daemons available that can work recursive and offer a descent flexibility regarding Inotify signals. For basic rsync operations, lsyncd isn’t that bad but it isn’t that flexible for system administration. The fact that it is written and configured in Lua, makes it actually really complex (considering that you don’t know to develop in Lua). Most system administrators do know or at least understand Perl.

Linux::Inotify2 is a CPAN module which allows to catch Linux Inotify signals (duh) and react upon them, in realtime. Combined with the AnyEvent CPAN module, you can use a plethora of event loop libraries available in your system, to have an amazingly fast inotify daemon.

The first thing we’ll discuss, is how to turn a regular Perl script into a standalone daemon process.

The process will get forked and the default IO handlers will be redirected. A PID file will also be created, so that the process can killed easily afterwards.
For example:

Next we will create a inotify signal handler and search for directories to watch for changes. In our example, we will watch every subdirectory of the ‘/data/vault’ directory (the parent directory itself is not included in this example):

The ‘find’ subroutine comes from the File::Find module, which is a standard Perl module.
For every directory found, the ‘create_watcher’ subroutine will be called, with the full path of the found directory.

The watcher code will watch for the following inotify signals:

  • IN_CREATE
  • IN_CLOSE_WRITE
  • IN_MOVE
  • IN_DELETE

If a new directory is created within ‘/data/vault’, the ‘create_watcher’ subroutine will be called again for this new directory. For files, the code will check what inotify signal was spawned and act accordingly.

The last to do is to initialize the event loop, using AnyEvent:

The above example shows how the inotify object ($inotify) is used in the event listener.

These examples explain how to create a simple inotify daemon, which can recursively go through directories and then watch these directories for certain inotify signals.

The full script:


Comments

Recursive Inotify Daemon — 14 Comments

  1. Pingback: Linux : SL Tech Umbrella

  2. Thanks for this nice idea!

    However, I’m having some problems. For some reason, while the script works as expected when monitoring the initial directory_tree and seems to be adding newly created subdirectories to its watch list, when new files are created inside those aforementioned directories, it does not notify correctly.

    e.g.
    /data/vault
    touch /data/vault/testfile (script notifies correctly)
    mkdir -p /data/vault/testdir (script notifies and adds testdir to its watch list, or at least that’s what it says)
    touch /data/vault/testdir/testfile (script FAILS to notify correctly)

    Any ideas?

    Cheers, Nick

    • Hi Nick.

      That is strange, and I’m unable to reproduce your problem. On my PC, the script gives the following output:

      The first two IN_CLOSE_WRITE statements were created using ‘echo “bla” > file’ and the last one was a ‘touch’.

      What was the output of the script (in /tmp/log.txt) on your side?

      Cheers,
      Johnny

      • 1. touch /data/vault/testfile
        2. mkdir -p /data/vault/testdir
        3. touch /data/vault/testdir/testfile

        IN_CLOSE_WRITE /data/vault/testfile
        Watching /data/vault/testdir

        Third command does not produce any output

        Is there a limit to the number of subdirectories inotify can watch? Cause mine is huge. Maybe that’s the problem.

          • Cool! It must be really huge then, cause overhere it is set by default to 65536:

            # cat /proc/sys/fs/inotify/max_user_watches
            65536

            Cheers!

  3. Hi there, First of all please accept my huge gratitude for providing this script, it is a great help and i was looking for something like this very desperately.
    But as I haven’t used perl before so i need your help regarding the error message i receive, when I try to invoke the script.
    It says:
    ” Can’t locate AnyEvent.pm in @INC (@INC contains: /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/5.8.8 .) at notifier.perl line 6.
    BEGIN failed–compilation aborted at notifier.perl line 6.”
    What do i need to do to fix this problem.
    Secondly i wanted further help regarding
    1. How to add more events. like “File modification event”
    2. Connecting this script to a postgresql data for logging the events with corresponding timestamp to the database.

    Many thanks once again for all the help.

    Cheers,

    Malik Junaid

  4. Pingback: Monitor creation of hidden system files

  5. Hi Johnny,

    this is great, it work like a charm 🙂

    A small change you may consider:

    > use File::Basename;
    > my $basename = basename($0);
    > my $PID_FILE = “/var/run/$basename.pid”;

    otherwise it won’t start unless you are in the same directory as the script itself

    (I have added a init script an run it as a daemon at system startup)

    Best Regards

    Yiorgos

  6. The -f check you do for the IN_DELETE and IN_MOVED_FROM will not work because at that point (firing of the inotify2 event) that pathname no longer exists. Your example daemon will never print out those two notices.

    Also, you don’t remove any watchers when a directory is deleted. 🙂

    • for IN_DELETE and IN_MOVED_FROM you can use following change.

      if(-d $filename && $e->IN_CREATE) {
      create_watcher($inotify, $filename);
      return
      }
      elsif($e->IN_DELETE) {
      print __LINE__ . “: IN_DELETE $filename\n”;
      }
      elsif($e->IN_MOVED_FROM){
      print “IN_MOVE_FROM $filename\n”
      }
      elsif(-f $filename){
      if($e->IN_CLOSE_WRITE){
      print “IN_CLOSE_WRITE $filename\n”
      }
      #elsif($e->IN_MOVED_FROM){
      # print “IN_MOVE_FROM $filename\n”
      #}
      elsif($e->IN_MOVED_TO){
      print “IN_MOVED_TO $filename\n”
      }
      #elsif($e->IN_DELETE){
      # print “IN_DELETE $filename\n”
      #}
      }

  7. This does not seem to be detecting deletion of files or directories (as noted in other comments), seems like the code should be working for that. Has anyone gotten this to work for deletion? It is working great for creates…

Leave a Reply

Your email address will not be published. Required fields are marked *