Bash as a Static Site Generator

By Dylan Thinnes, Last Edited Sat 21 Sep 10:45:08 BST 2019

Originally, this site was served using Ruby on Rails, deployed on Heroku. I have since migrated to a few custom Bash scripts that serve as a quick and dirty templating engine, minifier, and build system, which then upload to Github Pages.

In this post, I’m going to go into a little bit of the why behind this life-altering site-altering decision.

The Issues

The original Rails implementation was a problem for me for a few reasons:

0. High Latency

Heroku’s free tier regularly puts your site to sleep if it doesn’t have frequent visitors.

No matter how small I made the site, it would take 2 - 10 seconds to respond to a new request if that was the first request in a while. This is this terrible for user retention; if I connected to a site this slow to respond, I’d assume it’s down.

With a static site hosted on S3 or Github Pages, this wouldn’t be an issue.

1. Slow Build Time

When Rails needed to generate a page, it would take as long as 500ms to complete the build.

This could be solved by “pre-generating” a page, but I found caching fragments and generating static pages ahead of time to be somewhat annoying to configure with Rails.

With a static site, all pages would be pre-generated by default and load without that 500ms delay.

2. More than I needed

I was ultimately using a massive web framework with many bells and whistles to do some very rudimentary page templating and serve some simple static assets that would be better off in an S3 bucket.

I didn’t need ActiveRecord, since my schemas barely changed and had little code dependent on them.

I didn’t need ActiveCable, since I don’t have websockets.

I didn’t even need any ability to respond to POST, since this website is firmly locked on Web 1.0.

3. Arduous Updates

All dynamic content was built from ActiveRecord entries, which were in turn updated from txt files, one per project/article. Any other content was created in Rails’ views. To update any of this after the fact, I had to ssh into the Heroku box, pull the committed changes, and merge them in to the database by running some helper scripts.

I would be happy to do this if I were paid to, or if I had something significant to gain from it, but it is entirely too much effort for something that is supposed to be personal and care-free.

In a static site, all dynamic content could be hosted locally or in the repo in an easily read and hand-modified format.

4. Little to Learn

The biggest reason I start projects is to learn something new that I can apply. While Rails is an interesting framework to learn, using its systems and learning its ideosynracies are not very valuable beyond having a job in Rails itself.

If the build system were written in something custom, I would have the pleasure of learning about simple build systems, namespacing build scripts, deciding on a text-based format for my blog posts & other dynamic content, implementing a templating language, and versioning my static files.

5. There is one way to do it

I quite like going off the beaten path with my website features, by generating pages in hacky but ultimately working ways. Rails does not make that easy, since one of its points as a framework is to provide a safe environment for writing, generating, and serving pages.

Rails’ “omakase” philosophy is a good one for production environments with teams and goals, but it does not make for particularly fun experimentation. That isn’t a negative in Rails - it is simply down to what I’m trying to achieve through my site.

The Fundamentals

For the most part, the migration was only meant to be a change in host and build process. The styles, the javascripts, and the layout were to remain identical, or as identical as possible.

Most importantly, my site is strongly dedicated to being as tiny as possible while still being sleek and fairly pretty.

‘No Layout’ Mode

To achieve this, the site has a few tricks. The first trick is to have two endpoints for each page – one with just the content, and one with the rest of the page (such as the navbar and name) built around the content.

As a (simplified) example, for this blog post the server would have two endpoints:

<!-- /blog/bash-static-site.html?nolayout=true -->
<h1>Bash as a Static Site Generator</h1>
<p>By Dylan Thinnes</p>
<p>
    Originally, this site was served using...

<!-- /blog/bash-static-site.html -->
<body>
    <head>
        <title>Bash as a Static Site Generator</title>
        <style>
            ...
        </style>
    </head>
    <div id="content">
        <!-- Here is the original nolayout blog post -->
        <h1>Bash as a Static Site Generator</h1>
        <p>By Dylan Thinnes</p>
        <p>
            Originally, this site was served using
        ...
    </div>
</body>

Then, when JavaScript is enabled, each local link is overridden. When the link is clicked, an XHR request is made instead for the ‘nolayout’ version, which is then inserted into the page.

This memoizes content (so we don’t request it again), strips away redundant formatting data on subsequent pages (slimmming request size), and allows me to have a pretty fade-out-fade-in effect (arguably the most important benefit).

Templating and Static URL Hashing

To keep sites easier to maintain, pages need to be able to call functions and insert their results into the page.

In Rails, this is done with preprocessor tags, <%= and =%>, which execute what’s between them before outputting the files. In the bash version, this pretty easily emulated with a grep and replace.

<h2>This is a table of results</h2>
<!-- Now, we insert the table by running some custom "gentable.sh" script -->
<%= ./myscripts/gentable.sh =%>

A specialized use for the preprocessor is for static external assets. When loading static external assets, such as scripts, styles, and images, any requested url should be replaced with a hashed version that identifies its current version. For example, html that would normally be like this:

<img url='/my-profile.png' />
<script src='/links.js' />

should automatically be translated to:

<img url='/my-profile-509adde4.png' />
<script src='/links-aefc1278.js' />

We can do this by writing a hashing function which, given a filename, outputs the hashed version and moves the static asset to the hashed name.

Then, hashing links is as simple as running the command using the preprocessor in any html that would otherwise have the link. To build on the previous example, we’d replace the old html with:

<img url="<%= hashing_function '/my-profile.png' =%>" />
<script src="<%= hashing_function '/links.js' =%>" />

Going Beyond the Fundamentals

Now that my scripts emulate most of core features that are needed to implement the site, I proceed to more interesting scripts and page generation, most of which can be seen in the repo for this website.