Configuring HttpArchive + Webpagetest (Part 3: HttpArchive)

It’s time to install HttpArchive! So just as with Webpagetest, there’s some requirements for HttpArchive as well.

HttpArchive must run on a unix-based machine, as HttpArchive uses pcntl, a threading function in PHP currently available only on unix-based machines. For this guide I will be using Ubuntu.

The following will be needed on your machine:

  • Apache2+
  • PHP5 or above
  • MySQL
  • Subversion
  • pcntl (PHP)

Most of these can be installed with a package manager. However with pnctl, one must manually download the source, and either configure PHP with the pcntl argument, or compile and install the pcntl extension manually. I found an Ubuntu forum post from skout23 that explains a very easy way to install pcntl for Ubuntu users. However I’m sure aside from the package manager, BSD based Linux users can do the exact same thing: http://ubuntuforums.org/showthread.php?t=549953

Here’s the relevant code for Ubuntu users:

mkdir php
cd php
apt-get source php5
cd php5-(WHATEVER_RELEASE)/ext/pcntl
phpize
./configure
make

And don’t forget to restart Apache afterward!

Once everything is configured properly, you can checkout the HttpArchive source from the googlecode repository:

http://httparchive.googlecode.com/svn/trunk/

In addition, unless you want to download the downloads folder (which contains over 1GB of data from the sites that HttpArchive tracks), it would be best to checkout non-recursively, then check out all other folders:

$svn co -N http://httparchive.googlecode.com/svn/trunk/ .
$cd trunk
$svn up images
$svn up bulktest

Next, we will modify the settings.inc folder with the following information:

  • $gMysqlServer = "YOUR_SERVER"
  • $gMysqlDb = "YOUR_DATABASE"
  • $gMysqlUsername = "ACCOUNT_USERNAME"
  • $gMysqlPassword = "ACCOUNT_PASSWORD"

Finally, Apache needs to interpret the .js files with PHP before being served to the user. There exists a directive inside the .htaccess file in the root of the repository that already accounts for this. However, I had issues with this particular part, so I had to add the directive into the php.conf file under /etc/apache2/mods-enabled (you will need root permissions to modify this file):

<Filesmatch "(filmstrip|harviewer|interesting|interesting-images).js">
SetHandler application/x-httpd-php
</FilesMatch>

And you've configured HttpArchive! Unfortunately it won't really work without any data, but we'll talk about the final steps in part 4: Configuring the two to work with each other!

Advertisements

6 comments

  1. Tom · December 7, 2011

    Great so far!

    I’m about to start tackling step #3 – but have noticed that you haven’t managed to do a write up on step #4 yet.. Is this forthcoming, or would you be able to point me in a helpful direction?

    Cheers!

  2. Tom Taylor (@tom_taylor) · April 17, 2012

    Hey toumorokoshi! You’ve not posted part 4 yet I see! Can you get in touch? would be good to see how httparchive integrates with webpagetest! Thanks!

    • toumorokoshi · May 8, 2012

      Hi Tom,

      sorry about that, It’s been a while since I’ve worked with HttpArchive, and the configuration is a little bit scattered as I recall. I was hoping to wait until it was more centralized, but I’ll try to write a tutorial up soon.

      • alfie · August 31, 2012

        any possible to share your private instance’s source code?I found that it is hard to get it run smoothly according to your post.

      • toumorokoshi · September 8, 2012

        Sorry, It’s a private instance in the strictest sense, so I’m not allowed to share my source. That said, it might not be best to follow this guide for the HttpArchive part: It’s been a long time since I’ve worked on I believe there’s enough small changes to make my instructions no longer apply. Sorry 😦

  3. asdasd · November 7, 2012

    damn it, where is part 4

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s