Moving From Hugo to Write.as

With the migration pretty much complete, I figured it'd be fun to write about how I did it.

I like static site generators — so much so, that I built a custom theme for one called Hugo. But lately, I've gotten a bit tired of how fiddly it all is — the command-line incantations, being tied to a desktop to publish a quick thought, breaking changes in the publishing tool.

So, as I've written before, I started using a hosted solution called Write.as1 for my writing, because, as I wrote in that post:

I've come to realize that the problem with fiddling is that it's a distraction from doing.

I even got myself a .blog TLD for the the new blog. Because I'm fancy like that. And because I have other plans for my original .com site.

But mostly because I'm fancy.

Of course, now I had two problems blogs: one active, with a combination of short updates and long-form posts like this one, and an abandoned one2 — not a good look, in my opinion. Time to merge the two.

Enter The Importer

Hugo sites generally keep your writing in directories under a content directory — e.g., content/posts. Most Hugo sites will only ever have just one source-content directory, but you could have several:

…and so on.

Under these directories, you essentially have a set of Markdown files —one for each post— with some front matter that sets some metadata for the post, like the title, the publish date, whether or not it's a draft, and so on.

So, really, what I needed to do was parse each post on angelostavrow.com —all 146 of them— and upload the content to angelostavrow.blog via the Write.as API. Piece of cake. Easy as pie.

I discussed the idea with Matt and, lo and behold, the idea of a simple command-line tool that helps people import their Hugo-powered blog to Write.as/WriteFreely was born.

If you came here looking for a link to the tool, great — you can find it here. Check out the README, and be sure to note the v1.0 limitations.

If you want to learn more about how it was built, read on.

The Details

The tool is a command-line app built in Go — a programming language that I'm not especially familiar with, but is what both Hugo and WriteFreely/Write.as is built with. You invoke it from the root directory of your Hugo site with a few parameters, and it'll walk the source content directory, parse each Markdown file it finds, and upload each to your Write.as (or WriteFreely) blog.

Optionally, for Write.as Pro subscribers using Snap.as for image hosting, you can have the tool try to upload locally-hosted images to Snap.as and replace the links in your Hugo post.

The bulk of the heavy lifting is done by three dependencies:

  1. The go-writeas module, for uploading posts to Write.as/WriteFreely;
  2. The cli module, for command-line goodness; and
  3. Hugo's own parser module, for parsing site configuration files and post front matter.

There are also dependencies for uploading to Snap.as and for testing assertions, but those aren't essential for the tool to work.

Authentication

You pass in your Write.as/WriteFreely username when you invoke the tool, and it then prompts you for your password. Enter it, and the tool authenticates you with the API, retrieves an access token for subsequent requests, and forgets your password once the SignIn() method terminates. If the tool encounters an unrecoverable error, it invalidates that token, and then exits.

Parsing Posts

Parsing starts by checking the Hugo site's config file for two things: the site's language, and its base URL. Once it's got that information, it walks the source content directory and parses each Markdown post it finds:

  1. First, it parses the front matter to determine, primarily, the title and publish date. It also takes all tags and categories it finds, translates them to camel case, and stores them as an array of hashtag strings.
  2. With that done, it proceeds to parse the post content. For most of Hugo's built-in shortcodes, it translates the link to a basic URL.
  3. If image upload to Snap.as is enabled, each image link it finds (both Markdown and HTML syntax is supported) is checked to determine if it's a remote or local asset. If it's local, it looks for the image in the static folder of the Hugo site, uploads it to Snap.as, and replaces the image source URL with the new Snap.as URL.

A nice detail with the shortcode translation is that, if you're a Write.as Pro subscriber, a Hugo shortcode like this:

{{< tweet 1369386140931928075 >}}

turns into an embed like this:

And if you're not a Write.as Pro subscriber, or you're using WriteFreely, it just becomes a simple link to the tweet:

https://twitter.com/twitter/status/1369386140931928075

The Upload

Each post, once parsed, is added to an array of PostToMigrate values, which the parser returns once its work is done. That's then handed off to the publisher, which uses it as a queue: each value is converted to a writeas.Post value, and then handed off for upload.

All of this work is done serially; since this is a one-time process, there's not a huge gain to be had in spinning up multiple threads to parse and upload multiple posts at once. I was able to migrate 146 posts —including the upload of 26 local images to Snap.as— in under two minutes on my five-year-old MacBook Pro.

Of course, it you want to parse and upload in parallel, you can contribute the changes to the project!

Cleaning Up

In my specific case, because I had a lot of absolute URLs in my older posts pointing to angelostavrow.com/posts/title-of-thing, and now they live at angelostavrow.blog, I need to go back and manually update those links.

I'll also need to add some kind of redirect to keep old links pointed to the right content on the old site, which will be part of updating it to a personal landing page.

But otherwise, I'm pretty happy with how it all worked, and it's really, really nice to have my writing together again.

#writeas #hugo #migration

Footnotes

[1] — A disclaimer: I'm working with the Write.as team on an iOS/macOS client, but I want to be clear that there was never a requirement to move my writing to the platform. I did it because it's a really nice platform for writing.

[2] — Not just abandoned, but my .com URL is the main entry point for my online presence. So, yuck, I better update that landing page, quick.

Discuss...