dark

Retrieve a file from an authenticated website (in Perl)

blank

At my daily job as a Perl developer, I was asked to write a Perl script which would download a backup file from a certain web interface. The file however, was in a secured part of the web site. This means, that the script would first have to authenticate against the web site and then afterwards retrieve the requested file.

Thanks to the great LWP Perl module, this is again a rather easy and simple job.

Let’s start by explaining the script step-by-step:

  1. Call your modules
    #!/usr/bin/perl
    use strict; use warnings;
    
    use LWP;
    use HTTP::Cookies;
    use HTTP::Request;
    use File::Path qw{mkpath};
    

    The modules HTTP::Cookies and HTTP::Request are required to create the cookie jar file and to make a HTTP request (duh!). These two modules will create the objects that are required for the LWP user-agent object we are about to create.

  2. Check your temporary path:
    my $HOME = $ENV{HOME} . '/tmp';
    mkpath $HOME   unless -d $HOME;
    

    We will need this location to save our cookie jar file and the file we want to download

  3. Create a cookie jar and a LWP user-agent:
    # create a cookie jar on disk
    my $cookies = HTTP::Cookies->new(
            file     => $HOME.'/cookies.txt',
            autosave => 1,
            );
    
    # create an user-agent and assign the cookie jar to it
    my $http = LWP::UserAgent->new();
    $http->cookie_jar($cookies);
    

    No rocket science here, just create the objects!

  4. Send the authentication request:
    # try to log in
    my $login = $http->post(
            'https://www.example.com/auth/login.pl', [
                username => 'user',
                password => 'secret', ]
            );
    

    Once authenticated, the web site will create a cookie containing a session ID. This cookie will allow us to download the file we need from the secured part of the web site.

From this point on, it is required to check whether the log in succeeded and then continue with downloading the file. If the log in failed, an error message should be displayed with the reason on the failure.

# check if log in succeeded
if($login->is_success){
    print "- logged in successfullyn";
    print "- requesting file, might take a whilen";

    # make request to download the file
    my $url       = 'https://www.example.com/auth/files/backup.zip';
    my $file_req  = HTTP::Request->new('GET', $url);
    my $get_file  = $http->request( $file_req );

    # check request status
    if($get_file->is_success){
        print "--> downloaded $url, saving it to filen";

        # save the file content to disk
        open my $fh, '>', $HOME.'/backup.zip' 
                                         or die "ERROR: $!n";
        print $fh $get_file->decoded_content;
        close $fh;

        print "nsaved file:n";
        print "-------------n";
        print "filename:  ".$get_file->filename."n";
        print "size:      ".(-s $HOME.'/backup.zip')."n";
    }
    else {
        die "ERROR: download of $url failed: " . $get_file->status_line . "n";
    }
}
else {
    die "ERROR: login failed: " . $login->status_line . "n";
}
2 comments
  1. hi i have a question , i have to go in a website daily go to manually to inside the website and download a file can yuo get me ideas

  2. hi i have script download file in a website follow me ..

    #!/usr/bin/perl

    #####
    # Code Perl Download file
    # BY: N-Cen-Dainamix
    #####

    use LWP;
    use LWP::Simple; # socket for download

    my $url = “http://filedownload.rar”; # url
    my $name = “file.rar”; #name

    print ” Download .. \n”;
    getstore($url,$name);
    print ” Download finish .. \n”;

    # end code

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Previous Post

A Timesheet to Google Calendar Importer (in Perl)

Next Post

Perl, Facebook and GMail Contacts

Related Posts