Retrieve a file from an authenticated website (in Perl)

At my daily job as a Perl developer, I was asked to write a Perl script which would download a backup file from a certain web interface. The file however, was in a secured part of the web site. This means, that the script would first have to authenticate against the web site and then afterwards retrieve the requested file.

Thanks to the great LWP Perl module, this is again a rather easy and simple job.

Let’s start by explaining the script step-by-step:

1. Call your modules

   #!/usr/bin/perl
   use strict; use warnings;
   
   use LWP;
   use HTTP::Cookies;
   use HTTP::Request;
   use File::Path qw{mkpath};
   

   The modules HTTP::Cookies and HTTP::Request are required to create the cookie jar file and to make a HTTP request (duh!). These two modules will create the objects that are required for the LWP user-agent object we are about to create.

2. Check your temporary path:

   my $HOME = $ENV{HOME} . '/tmp';
   mkpath $HOME   unless -d $HOME;
   

   We will need this location to save our cookie jar file and the file we want to download

3. Create a cookie jar and a LWP user-agent:

   # create a cookie jar on disk
   my $cookies = HTTP::Cookies->new(
           file     => $HOME.'/cookies.txt',
           autosave => 1,
           );
   
   # create an user-agent and assign the cookie jar to it
   my $http = LWP::UserAgent->new();
   $http->cookie_jar($cookies);
   

   No rocket science here, just create the objects!

4. Send the authentication request:

   # try to log in
   my $login = $http->post(
           'https://www.example.com/auth/login.pl', [
               username => 'user',
               password => 'secret', ]
           );
   

   Once authenticated, the web site will create a cookie containing a session ID. This cookie will allow us to download the file we need from the secured part of the web site.

From this point on, it is required to check whether the log in succeeded and then continue with downloading the file. If the log in failed, an error message should be displayed with the reason on the failure.

# check if log in succeeded
if($login->is_success){
    print "- logged in successfullyn";
    print "- requesting file, might take a whilen";

    # make request to download the file
    my $url       = 'https://www.example.com/auth/files/backup.zip';
    my $file_req  = HTTP::Request->new('GET', $url);
    my $get_file  = $http->request( $file_req );

    # check request status
    if($get_file->is_success){
        print "--> downloaded $url, saving it to filen";

        # save the file content to disk
        open my $fh, '>', $HOME.'/backup.zip' 
                                         or die "ERROR: $!n";
        print $fh $get_file->decoded_content;
        close $fh;

        print "nsaved file:n";
        print "-------------n";
        print "filename:  ".$get_file->filename."n";
        print "size:      ".(-s $HOME.'/backup.zip')."n";
    }
    else {
        die "ERROR: download of $url failed: " . $get_file->status_line . "n";
    }
}
else {
    die "ERROR: login failed: " . $login->status_line . "n";
}