Lessons learned from setting up Sketchy

Have you ever wanted to pass a URL off to a program and have it return a screenshot of that site?  This is incredibly useful for things like DFIR, allowing you to get an initial look at a page without having to poke at it with a potentially vulnerable browser.  I've used various tools to try to take screenshots of sites that either have a Javascript-based redirect at initial load or are AJAX-based and these tools always failed me.

Earlier this summer, I heard about a suite of tools released by Netflix.  This suite included Sketchy, a conglomeration of python, Flask, phantomjs, gunicorn, celery and redis.  Sketchy uses lazy-rendering within phantomjs to allow it to take screenshots of AJAX-heavy sites.

Based on the writeup by the Netflix crew, I was hopeful this would solve the problem once and for all.  I finally had time this week to sit down and play with Sketchy.  There were a few bumps along the road, so I decided to put down what I did here in case anybody else is interested in getting Sketchy working.

Installation

I installed Sketchy in my Kali linux VM.  The installation was straight forward.  Use git to clone the Sketchy repository to your machine.  I chose to put mine in /opt/sketchy.  With an up-to-date Kali installation, simply running ubuntu_install.sh will pull down all the necessary dependencies and build your environment for you.  If you don't want to trust a script to do this for you, the dependencies are clearly noted in the manual install section of the wiki.

User Setup

The Sketchy wiki doesn't discuss this, but if you're going to run Sketchy as root, celery will complain about being started as UID 0.  To get around this, I created a standard privilege user named sketchy, a group named sketchy, and made the sketchy user a member of the sketchy group.  I then changed ownership of the Sketchy install directory and all files and subdirectories to the sketchy user and sketchy group.

Database Setup (and the first hiccup)

By default, Sketchy creates a SQLite database to store information.  While the wiki recommends a different RDMBS such as MySQL, for low volume purposes you should be fine using the default database.  This was where I ran into a problem which would confound me for some time.

If you use Kali, you're probably running most of your commands as root.  If you are, when you set up the database using `python manage.py create_db'.  If you proceed down this path and follow the Test Startup instructions, everything will work fine, however you will get an Internal Server Error message if you try to follow the Production startup instructions.

In my case, production startup failed to render images because the database was set up by root but gunicorn was running under a reduced-privilege user (to be discussed later).

To get around this, I created a tmp directory within my Sketchy install as /opt/sketchy/tmp.  In order for manage.py to create the DB in this directory, I modified config-default.py to point to the new location:

# Database setup
SQLALCHEMY_DATABASE_URI = 'sqlite:////opt/sketchy/tmp/sketchy.db'

If set up the database as root, you'll want to change ownership of the new database to sketchy.sketchy to allow gunicorn to update it.

Configuration

The Sketchy wiki indicates you should remove ":8000" from the HOST variable in config-default.py.  I did not find it necessary to remove this to allow Sketchy to work properly.

Make sure to update your PHANTOMJS location according to your local system.  The setup script detected I had phantomjs installed, however config-default.py was looking for it in /usr/local/bin instead of /usr/bin.

supervisord.ini

There isn't too much to change in this file.  For [supervisord], you may want to store your log files in /var/log.  Changing the loglevel to debug will help you identify issues.  In both the [program:celeryd] and [program:gunicorn] sections, set the directory to your Sketchy installation directory and change the user to the account you created to run the daemons (I used sketchy).  I also changed the address gunicorn was using to 127.0.0.1 to prevent it from listening on any network interface.

Conclusion

Sketchy definitely has a place in my toolkit.  I haven't found anything that will reliably screenshot pages that use Javascript to redirect to another page or things that are AJAX-based.  Sketchy fits the bill perfectly for that use case.  The performance isn't bad, but it's not a speed demon either.  I haven't spent any time looking into optimizing the various components used to see if I can get better performance.

I'll be posting another blog soon about how I use Sketchy as an internal penetration tester to reduce the amount of time I spend performing website reconnaissance and looking for information disclosures.