Joplin to Hugo CI/CD

Managing a static website from any device running Joplin notes.

The software stack:

Joplin is an open source note taking application that I have been using for a few years now.

• Configure sync target on a host with joplin cli

Hugo is a markdown based static website generator.

Home Assistant for control plane.

Mosquitto for mqtt messaging.

Goals:

The goal is to manage and publish web content from mobile devices with a Local-First approach. Given the markdown format support by both joplin and hugo, integrating the two made sense.

Why? I thought it’d be a fun way to encourage myself to post more. No excuses if I can write and post with offline first storage and seamless device sync.

Pipeline:

Joplin configured with sync

• WebDAV works well for me

Notebook to Website Pipeline

• Home Assistant Script publishes MQTT message to build and sync
• Python script mqtt-publish.py receives message and executes publish.sh bash script
• publish.sh
  • cleans directories
  • uses joplin cli to sync
  • export a notebook to markdown
  • copy posts and resources to hugo site
  • generates static site with hugo
  • rsync site to web server

More info, scripts, and snippets to follow. Stay tuned.

scripts

mqtt-publish.py:

import paho.mqtt.client as mqtt
import json
import subprocess

def on_connect(client, userdata, flags, rc):
    print(f"Connected with result code: {rc}")
    # Subscribe to the topic inside the callback to ensure it's renewed on reconnect
    client.subscribe("websites")

def on_message(client, userdata, msg):
    print(f"Received message on {msg.topic}: {msg.payload.decode('utf-8')}")
    jsmsg = json.loads(msg.payload.decode('utf-8'))
    print(f"decoded {jsmsg}")
    if jsmsg['publish'] == 'www.jalder.com':
      result = subprocess.run(["./publish.sh"], capture_output=True, text=True)
      print(result.stdout)
      print(result.stderr)

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

# Connect to broker (host, port, keepalive)
client.connect("localhost", 1883, 60)

# Start the loop to process network traffic
client.loop_start()

try:
    while True:
        # Keep the main thread alive
        pass
except KeyboardInterrupt:
    client.disconnect()
    client.loop_stop()

publish.sh:

#!/bin/bash

# Sync Joplin
node ~/joplin/node_modules/joplin/main.js sync

# Clean up local sync dir

rm -rf sync/www.jalder.com
rm -rf sync/_resources

# Export www.jalder.com notebook to local sync dir
node ~/joplin/node_modules/joplin/main.js export --notebook www.jalder.com --format md sync

# Copy posts to content/posts
rsync -avz sync/www.jalder.com/ content/posts/

# Publish with Hugo
hugo

# Sync _resources for embedded images and attachments
rsync -avz sync/_resources public/

# Strip exif
exiftool -all= public/_resources/*png

# Sync to web server
rsync -avz public/ user@bastion:/usr/share/nginx/www/public/

screenshots

Joplin notes on mobile, editing a post.

Script configured in Home Assistant

Button to publish website in Joplin to Webserver