Watch a directory for uploaded files

In some situations, you need to watch a directory for uploaded files and move/process them immediately. Many scripts work in a polling kind of way, which will check every x amounts of seconds for new files. This is nowadays completely out of fashion and generaly not cool.

A real watcher script uses the OS’s built-in features and the Linux OS is therefor the number one choice for these kind of operations.
The Linux kernel has a feature called inotify. inotify is an inode-base file system notification mechanism, which can be used in almost every real programming language.

The following example uses the Perl programming language. The Perl module that will be used is called Linux::Inotify2, which is able to catch the inotify notifications.

The example script will watch a certain directory for files (which will be uploaded using HTTP, FTP, SFTP or which will be just moved into that specific directory).
It will also use other Perl modules, such as:
– AnyEvent: for event looping
– File::Copy: to move the uploaded to another directory
– Digest::MD5: to check if a file has been already uploaded, we don’t want duplicates
– Storable: to keep a state file of uploaded files
– use features: to use to cool new features of Perl

The first part of the script calls all the required modules and configures some variables that will used in the script.

Next the script will check if there is already a statefile on disk and if so, the script will load it.
It will also catch the SIGTERM and SIGINT signals (when the script is killed or stopped) and store the statefile when the signals are caught.

Now the fun part starts. The script will
– initialize the event loop
– create a watcher
– wait for events.

Only three notification signals will watched:
– IN_MOVED_TO
– IN_CLOSE_WRITE
– IN_DELETE
Other signals are not important. When the IN_CLOSE_WRITE is received, so when a file is uploaded or moved into the directory, our action will be called. For the sake of the example, the script will just move the new file into a new directory if it passes some tests.

The action used in the IN_CLOSE_WRITE notification, is defined below. It is pretty straightforward is left for the reader to analyze.


Comments

Watch a directory for uploaded files — 1 Comment

  1. Pingback: Information about inotify in Linux based Operating Systems | Admin Junkie

Leave a Reply

Your email address will not be published. Required fields are marked *