Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

SparkleMuffin

SparkleMuffin is a self-hosted bookmark manager and feed aggregator.

It provides a minimalist interface thait aims at being simple to use, clutter-free and accessible.

Documentation

The documentation is split in two main sections:

  • a User Guide that showcases SparkleMuffin’s features and how to use them;
  • a Developer Guide that provides information on how SparkleMuffin works, and how to contribute to the project.

Contributing

SparkleMuffin is free and open-source, licensed under the MIT License. You can find the source code on GitHub, and issues and feature requests can be posted on the issue tracker.

If you would like to contribute, please read the Contributing guide!

What status is the project currently in?

SparkleMuffin is still alpha software, and should not be considered ready for production use.

Change Log

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

UNRELEASED

Security

  • Bump GitHub Actions to their latest release, and pin them by SHA + version comment
  • Enable Dependabot for GitHub Actions, Go modules, Node.js packages and the SQLFluff uv project

Added

Packaging and automation

  • Add root certificates for Let’s Encrypt Generation Y certificate authorities YE and YR
  • Setup GitHub Actions to store and reuse Docker cache mounts (NPM, Go)
  • Setup GitHub Actions to build multi-platform Docker images (amd64, arm64)

Changed

Packaging and automation

  • Bump golangci-lint
  • Rework SQLFluff setup:
    • Pin with pyproject.toml
    • Install with uv, run with uvx and Python 3.14
    • Bump to v4.2.2
    • Silence new rule warnings for existing migrations with noqa comments
  • Rework mdBook setup:
    • Bump to v0.5.4
    • Update configuration, remove deprecated and commented options
    • Switch to lychee to check documentation links

Documentation

  • Document the Copywrite and Vulnerabilities GitHub Actions workflows
  • Update the CI Workflow description, link how-to guides from each workflow section
  • Update and reword conceptual, how-to and reference guides to match current tooling configuration
  • Fix broken documentation links and reword the README
  • Link the CI badge to its workflow run, add a badge for the GHCR Docker image

WWW

  • Migrate the application from Bulma to Bootstrap 5

v0.7.0 - 2026-07-26

Security

  • Update Chi HTTP middleware to read the remote client IP address from a header set by a reverse proxy
  • Prevent the HTTP client used to fetch syndication feeds from performing requests to unroutable or blocked IP address ranges.
  • Fully invalidate user sessions on logout and password change
  • Periodically invalidate expired user sessions
  • Rate-limit login attempts by resolved client IP and by user email
  • Set the Secure and SameSite attributes for session cookies

Fixed

Packaging and automation

  • Add missing command-line flags to Make targets used for local development

Feeds

  • Ensure each feed has a unique slug

Added

Monitoring

  • Expose Prometheus metrics for periodic feed synchronization tasks

Changed

Packaging and automation

  • Update direct and transitive dependencies

Bookmarks

  • Improve test coverage for bookmark operations
  • When attempting to add a bookmark for a URL that belongs to an existing bookmark, display a conflict resolution form

Feeds

  • Improve test coverage for feed operations
  • Bump gofeed to v1.4.0
  • If a feed entry URL only consists in a relative path, attempt to build its full URL from the feed’s base URL

User

  • Refactor model validation for user creation and update
  • Require a minimum password length (8 characters)

WWW

  • Add helper to render a single named HTML template
  • Add method to retrieve a given feed entry
  • Use Alpine.js to show/hide navbar menus on hover
  • Use HTMX to partially refresh the bookmarks list on:
    • Navigating result pages
    • Searching bookmarks
  • Use HTMX to partially refresh the tag list page on:
    • Navigating result pages
    • Searching tags
  • Use HTMX to partially refresh the feed list on:
    • Changing the All / Read / Unread filter
    • Toggling entry summaries
    • Marking all entries as read
    • Toggling an entry’s read status
    • Navigating result pages
    • Searching entries
    • Selecting a given category
    • Selecting a given subscription
  • Rework edition forms as modals:
    • Bookmark edition form
    • Bookmark tag edition form
    • Feed category edition form
    • Feed subscription edition form
  • Homogenize edition form layout, use horizontal fields
  • Rework deletion forms as modals
  • Map domain errors to user-facing error messages

v0.6.1 - 2026-04-23

Security

  • Replace custom Cross-Site Request Forgery (CSRF) helpers with Go 1.25’s CSRF protection middleware
  • Define a Content Security Policy for Web browsers to enforce
  • Do not expose sensitive configuration keys as Docker ENV variables
  • Require a non-empty HMAC key to be configured to generate session tokens
  • Require a non-empty database password to be configured
  • Ensure golang-migrate is using the pgx/v5 database driver

Added

Monitoring

  • Add a /health endpoint and Docker HEALTHCHECK command

WWW

  • Add a JavaScript asset pipeline to the esbuild configuration

Changed

Packaging and automation

  • Build with Go 1.26
  • Update direct and transitive dependencies
  • Update all GitHub Actions
  • Update golangci-lint

CLI

  • Change monitoring server port to 8090
  • Let the monitoring server listen to all interfaces by default

WWW

  • Extract Awesomplete tag completion as a JavaScript module
  • Extract EasyMDE editor initialization as a JavaScript module
  • Do not define Go templates that only contain a JavaScript script block
  • Rework the base template to allow content pages to define an optional scripts template
  • Rework Goldmark / Chroma usage to use CSS classes instead of outputting inline style information

v0.6.0 - 2025-12-04

Added

Feed

  • Add display preferences
    • Filter displayed entries by reading status (all / read / unread)
    • Toggle entry summary visibility
  • For each displayed entry, add a link to the corresponding subscription
  • Collapse and expand subscription categories
  • Set a timeout for scheduled synchronization tasks

WWW

  • Add user preferences page
  • Propagate request and task context to the Services and Repositories

Packaging and automation

  • Add Make target to reindex the development database and refresh collation version
  • Update .gitignore and .dockerignore to exclude JetBrains IDE configuration

Changed

CLI

  • Allow to migrate to a specific database schema version
  • Simplify database migration logging

Feed

  • Handle cases where an entry’s publication and/or update date(s) are zero, and fall back to the current date/time

Packaging and automation

  • Update direct and transitive dependencies
  • Bump github.com/jaswdr/faker to v2.9.0
  • Enable esbuild watch mode for live asset rebuilding
  • Run assets pipeline and Go server concurrently to avoid live server interruptions

Repository

  • Document query and transaction helpers
  • Cleanup SQL queries

WWW

  • Indicate browsers that only a light theme is available
  • Homogenize all forms to use horizontal fields
  • Rework the feed subscription management page
  • Rework the bookmarks tag management page

Fixed

Feed

  • If we fail to parse an XML feed due to invalid Unicode characters, filter them out and attempt to parse the feed again

WWW

  • Fix <time> blocks in bookmark and feed list pages
  • Use the same plus sign icon for bookmark and feed navbar menu items

v0.5.1 - 2025-08-16

Added

Documentation

  • Add high-level roadmap using the Now / Next / Later format

WWW

  • Add robots.txt route

Changed

Packaging and automation

  • Add Make targets to dump and restore the database
  • Upgrade PostgreSQL to version 17
  • Build with Go 1.25
  • Update direct and transitive dependencies
  • Switch to Debian 13 Trixie for the final image

WWW

  • Resolve assets with Node 24 LTS
  • Update Web manifest
  • Update FontAwesome to 7.0.0
  • Log view rendering errors
  • Rework home view rendering
  • Ensure services are initialized before starting the server
  • Skip the user session middleware for static pages and assets

Fixed

WWW

  • Prevent feed entries containing long sequences of non-whitespace characters from overflowing
  • Do not render feed entry text as Markdown
  • Fix pagination for the one but last page

v0.5.0 - 2025-06-01

Security

Added

Bookmarks

  • Export as a JSON document

WWW

  • Resolve and deploy frontend assets with NPM and esbuild

Changed

WWW

  • Log pagination errors as warnings
  • Rewrite HTML templates and CSS with the Bulma framework
  • Rework site navigation using navbar dropdowns
  • Rework search forms
  • Rework result pagination templates and helpers

Packaging and automation

  • Build with Go 1.24
  • Update CI workflow
  • Update direct and transitive dependencies
  • Run the modernize analyzer and apply fixes
  • Bump golangci-lint to v2
  • Setup golangci-lint to run additional linters

Removed

Feed

  • Remove unused importing.Repository type

v0.4.3 - 2025-01-05

Fixed

Feeds

  • Ensure truncating entry descriptions does not result in invalid UTF-8 code points

v0.4.2 - 2024-12-21

Changed

Feeds

  • When deleting a category or subscription, propagate the deletion to feeds with no remaining subscriptions

Security

v0.4.1 - 2024-12-14

Security

v0.4.0 - 2024-12-10

Added

Database

  • Add PostgreSQL integration tests for feed operations

Changed

Database

  • Split PostgreSQL repository into dedicated domain repositories
  • Update testcontainers configuration to use a tmpfs volume and disable WAL features to speed up integration tests

Feeds

  • Update page title to display the subscription alias (if set) or the feed title
  • Update listed entries to display the subscription alias (if set) or the feed title

WWW

  • Relocate HTTP packages to internal/http
  • Relocate version detection helpers to internal/version

v0.3.1 - 2024-12-07

Fixed

Feeds

  • Fix HTML templates after renaming querying models

v0.3.0 - 2024-12-07

Added

Database

  • Add dedicated tests for PostrgeSQL database migrations (up/down)

Documentation

  • Add custom CSS to display wider content on large screens

Feeds

  • For each entry in the list, display the title of the corresponding feed
  • Save and display feed descriptions
  • Extract keywords (significant terms) from entry content/description with TextRank
  • Add full-text search based on feed and entry metadata
  • Store and compare the hash (xxHash64) of the feed data to avoid unnecessary database upserts
  • Document the feed polling and caching strategy
  • Allow users to set an alias title for feed subscriptions

Changed

CI

  • Lint and format SQL files with SQLFluff
  • Publish HTML documentation to GitHub Pages

Documentation

  • Update documentation structure to follow the Diátaxis approach
  • Disable mdBook file auto-creation
  • Check for broken links with mdbook-linkcheck

www

  • Update the home page
  • Render HTTP 4xx errors as HTML views

Fixed

Docker

  • Install the ca-certificates package in the Docker image for TLS connections

Feeds

  • Ensure entry publication and update dates are non-zero
  • Ensure entry publication and update dates are not in the (far) future (limit: 2 days)
  • In the subscription edit form, ensure the correct category is selected

v0.2.0 - 2024-11-14

Added

Feeds

  • Subscribe to Atom and RSS feeds
  • Categorize subscriptions
  • Display subscriptions
  • Periodically synchronize subscriptions
  • Import existing subscriptions from OPML
  • Export subscriptions

Changed

Bookmarks

  • Enforce CSRF validation for import and export forms

PostgreSQL

  • Update repository helpers

Packaging & automation

  • Build with Go 1.23
  • Update direct and transitive dependencies

v0.1.1 - 2024-01-26

Initial release

Added

Bookmarks

  • Create and manage users of the application
  • Create and manage bookmarks to Web pages (links)
  • Display bookmarks and bookmark tags
  • Import existing bookmarks
  • Search bookmarks by keywords (full-text search)

Command-line & configuration

  • Add a sparklemuffin root command to handle common program configuration
  • Add a createadmin subcommand to create users with administrator privileges
  • Add a migrate subcommand to manage database migrations
  • Add a run subcommand to start all services
  • Add a version subcommand to display the running version (featuring Git version information)
  • Allow configuring services via:
    • application defaults
    • configuration file
    • command-line flags
    • environment variables

Observability

  • Setup structured logging (formats: console, JSON)
  • Expose Prometheus metrics:
    • Go runtime
    • HTTP requests
    • Application build and version information

Packaging & automation

  • Package the application as a Docker container
  • Provide Docker Compose configuration for:
    • local development
    • example usage
  • Add mdBook documentation
  • Add Make targets to:
    • run static analysis tools (linters)
    • run unitary tests
    • run integration tests
    • generate coverage reports
    • run live-reload development servers
    • build HTML documentation
  • Add GitHub Actions workflows:
    • CI: build the application, run linters, run tests
    • Copywrite: ensure license headers are present in source files
    • Docker: build and publish Docker image to the GitHub Container Registry (GHCR)

Roadmap

All high-level goals and planned work for this project will be documented in this file.

The roadmap is based on the Now / Next / Later format to communicate current focus, upcoming work and longer-term ideas.

Now

  • www: Review OWASP Top 10 checklist

Next

  • Feed: Add entry tags, with auto-tagging rules
  • Feed: Bookmark entry
  • Internal: Rework error flow (logging, metadata)
    • www: Improve error messages

Later

Content & Features

  • Bookmark: Sanitize URLs to remove tracking parameters
  • Bookmark: Detect link rot
  • Bookmark, Feed: Store site favicon
  • Bookmark, Feed: Store site domain
  • Feed: Adapt fetch frequency to entry publication frequency
  • Feed: Improve duplicate entry detection
  • Search: Query language
  • Taxonomy: Tag hierarchy

Users

  • Authentication: Password reset
  • Authentication: OAuth2/OpenID
  • Authentication: Two-factor authentication
  • Documentation: Add a user guide with screenshots
  • Users: Audit log

www

  • www: Display curated content on the home page
  • www: Responsive design
  • www: Internationalization (i18n)
  • www: Dark mode / theme switching

Command-line

  • Database: Review connection pool transaction and timeout usage

API

  • API: OpenAPI or gRPC?
  • API: Authentication flow

Integrations

  • Integration: Browser extension
  • Integration: Archive.org
  • Integration: Self-hosted archive
  • Integration: News (HN, Lobste.rs)
  • Integration: Forges (Github, Gitlab, Gitea/Forgejo)

User Guide

How-to Guides

Practical step-by-step guides to help you achieve a specific goal. Most useful when you’re trying to get something done.

Run SparkleMuffin locally

Goals

This guide shows how to run SparkleMuffin locally on your computer, in development mode, with debug logging enabled.

This setup is only fit for testing purposes, and should not be used as-is in production (or a public-facing server).

Prerequisites

If you are not familiar with Docker or Docker Compose, take a look at the quickstart guides:

Run services with Docker Compose

Create a new directory and download the example Docker Compose configuration:

$ mkdir -p sparklemuffin
$ cd sparklemuffin
$ wget https://raw.githubusercontent.com/virtualtam/sparklemuffin/main/docker-compose.yml

Pull the Docker images for PostgreSQL and SparkleMuffin:

$ docker compose pull

Run PostgreSQL and SparkleMuffin:

$ docker compose up -d

Apply database migrations

Use the sparklemuffin migrate command to apply database migrations:

$ docker compose exec sparklemuffin sparklemuffin migrate

Create a first administrator user

Use the sparklemuffin createadmin command to create the admin@local.dev user; a strong password will be generated automatically for you:

$ docker compose exec sparklemuffin \
    sparklemuffin createadmin \
		--displayname Admin \
		--email admin@dev.local \
		--nickname admin

Log in to the SparkleMuffin

You are now ready to access SparkleMuffin’s Web interface by opening http://localhost:8080 in your Web browser, and log in using the credentials from the previous step:

  • login: admin@dev.local;
  • password: use the password generated by the sparklemuffin createadmin command.

Cleanup - When you are done using SparkleMuffin

Stop the services

Stop running containers with:

$ docker compose stop

Stop the services, remove containers and PostgreSQL data volume

$ docker compose down -v

Reference Guides

Features

Bookmarks

SparkleMuffin allows you to:

  • save, tag and search your Web bookmarks;
  • import your existing bookmarks from a Web browser or another bookmarking application, using the Netscape Bookmark File Format;
  • share your public bookmarks so they can be accessed by a Web browser or as an Atom feed.

Feeds

SparkleMuffin allows you to:

  • subscribe to Atom and RSS feeds;
  • import your existing feed subscriptions using the OPML File Format.

Web interface

SparkleMuffin aims at providing a Web interface that is:

  • simple to use;
  • minimalist and clutter-free;
  • accessible on Web browsers for both desktop and mobile devices;
  • light on Javascript.

Database

SparkleMuffin is backed by a PostgreSQL database, which allows multiple users to log in to the same instance, and eases administration, backup and maintenance.

Roadmap

See the changelog for past releases, and the milestones for upcoming features.

Configuration

Configuration formats

SparkleMuffin can be configured by:

  • using a configuration file;
  • setting POSIX flags, e.g. --log-level debug;
  • setting environment variables, e.g. SPARKLEMUFFIN_LOG_LEVEL=debug.

Configuration variable precedence

Configuration variables are evaluated in this order:

  • program defaults (lowest precedence);
  • configuration file;
  • command-line flags;
  • environment variables (highest precedence).

Naming convention for configuration variables

All configuration variables are specified as program flags (see command-line flags), from which environment variables names and configuration file keys are derived:

Command-line flagEnvironment VariableConfiguration File
--exampleSPARKLEMUFFIN_EXAMPLEexample: true
--log-level debugSPARKLEMUFFIN_LOG_LEVEL=debuglog-level: debug

Configuration file

  • TODO: add CLI flag to specify a configuration file
  • TODO: add CLI command to generate a configuration file with default values
  • TODO: specify configuration file format (TOML?)
  • TODO: add commented configuration file to SCM
  • TODO: add commented configuration file to docs (section on this page)

Command-line flags

SparkleMuffin is provided as a single-binary command-line application, that provides commands to:

  • run the Web server
  • create administrator users
  • apply database migrations
  • display information about how the program was built
  • etc.

To see which commands and global flags are available, run the sparklemuffin --help command:

$ sparklemuffin --help

SparkleMuffin - Web Bookmark Manager

Usage:
  sparklemuffin [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  createadmin Create a user with administration privileges
  help        Help about any command
  migrate     Initialize database and run migrations
  run         Start the HTTP server
  version     Display the prorgam version

Flags:
      --db-addr string       Database address (host:port) (default "localhost:15432")
      --db-name string       Database name (default "sparklemuffin")
      --db-password string   Database password (default "sparklemuffin")
      --db-sslmode string    Database sslmode (default "disable")
      --db-user string       Database user (default "sparklemuffin")
  -h, --help                 help for sparklemuffin
      --hmac-key string      Secret key for HMAC session token hashing (default "hmac-secret-key")
      --log-level string     Log level (trace, debug, info, warn, error, fatal, panic) (default "info")

Use "sparklemuffin [command] --help" for more information about a command.

To get more information about a specific command, run sparklemuffin <command> --help:

$ sparklemuffin run --help

Start the HTTP server

Usage:
  sparklemuffin run [flags]

Flags:
  -h, --help                         help for run
      --listen-addr string           Listen to this address (host:port) (default "0.0.0.0:8080")
      --monitoring-listen-addr string   Listen to this address for Prometheus monitoring (host:port) (default "127.0.0.1:8090")
      --public-addr string           Public HTTP address (if behind a proxy) (default "http://localhost:8080")

Global Flags:
      --db-addr string       Database address (host:port) (default "localhost:15432")
      --db-name string       Database name (default "sparklemuffin")
      --db-password string   Database password (default "sparklemuffin")
      --db-sslmode string    Database sslmode (default "disable")
      --db-user string       Database user (default "sparklemuffin")
      --hmac-key string      Secret key for HMAC session token hashing (default "hmac-secret-key")
      --log-level string     Log level (trace, debug, info, warn, error, fatal, panic) (default "info")

Observability

Structured Logs

SparkleMuffin logs information on the standard error (stderr) stream, using a structured log message format (JSON or logfmt).

The log format and level can be specified via configuration.

Example logs: program startup

Format: console

2024-01-20T17:03:52+01:00 INF configuration: no file found config_paths=["/etc","/home/dev/.config","."]
2024-01-20T17:03:53+01:00 INF database: successfully created connection pool database_addr=localhost:15432 database_driver=pgx database_name=sparklemuffin
2024-01-20T17:03:53+01:00 INF global: setting up services log_level=info version=devel
2024-01-20T17:03:53+01:00 INF monitoring: listening for HTTP requests metrics_addr=127.0.0.1:8090
2024-01-20T17:03:53+01:00 INF sparklemuffin: listening for HTTP requests http_addr=0.0.0.0:8080
2024-01-20T17:04:44+01:00 INF handle request duration_ms=0.750875 host=localhost:8080 method=GET path=/ remote_addr=127.0.0.1:51440 request_id=localhost.local/96bRV2ceWt-000001 size=1187 status=200
2024-01-20T17:04:44+01:00 INF handle request duration_ms=4.369792 host=localhost:8080 method=GET path=/static/awesomplete.css remote_addr=127.0.0.1:51441 request_id=localhost.local/96bRV2ceWt-000004 size=167 status=200
2024-01-20T17:04:44+01:00 INF handle request duration_ms=5.682958 host=localhost:8080 method=GET path=/static/easymde.css remote_addr=127.0.0.1:51442 request_id=localhost.local/96bRV2ceWt-000003 size=931 status=200
2024-01-20T17:04:44+01:00 INF handle request duration_ms=7.072792 host=localhost:8080 method=GET path=/static/www.css remote_addr=127.0.0.1:51440 request_id=localhost.local/96bRV2ceWt-000002 size=5402 status=200

Format: json

{"level":"info","config_paths":["/etc","/home/dev/.config","."],"time":"2024-01-22T22:56:30+01:00","message":"configuration: no file found"}
{"level":"info","database_driver":"pgx","database_addr":"localhost:15432","database_name":"sparklemuffin","time":"2024-01-22T22:56:30+01:00","message":"database: successfully created connection pool"}
{"level":"info","log_level":"info","version":"devel","time":"2024-01-22T22:56:30+01:00","message":"global: setting up services"}
{"level":"info","metrics_addr":"127.0.0.1:8090","time":"2024-01-22T22:56:30+01:00","message":"monitoring: listening for HTTP requests"}
{"level":"info","http_addr":"0.0.0.0:8080","time":"2024-01-22T22:56:30+01:00","message":"sparklemuffin: listening for HTTP requests"}
{"level":"info","duration_ms":69.373458,"host":"localhost:8080","method":"GET","path":"/bookmarks","remote_addr":"127.0.0.1:50857","request_id":"localhost.local/dqPgOolnvF-000001","size":26808,"status":200,"time":"2024-01-22T22:57:12+01:00","message":"handle request"}
{"level":"info","duration_ms":5.552375,"host":"localhost:8080","method":"GET","path":"/static/awesomplete.css","remote_addr":"127.0.0.1:50857","request_id":"localhost.local/dqPgOolnvF-000002","size":236,"status":200,"time":"2024-01-22T22:57:12+01:00","message":"handle request"}
{"level":"info","duration_ms":12.052417,"host":"localhost:8080","method":"GET","path":"/static/easymde.css","remote_addr":"127.0.0.1:50860","request_id":"localhost.local/dqPgOolnvF-000003","size":1000,"status":200,"time":"2024-01-22T22:57:12+01:00","message":"handle request"}

Prometheus Metrics

SparkleMuffin exposes Prometheus metrics, providing useful information that can be used for monitoring and alerting.

These metrics are exposed by default on http://0.0.0.0:8090/metrics; the host and port can be specified via configuration.

Available Metrics

Developer Guide

How-to Guides

Practical step-by-step guides to help you achieve a specific goal. Most useful when you’re trying to get something done.

Getting the source code

Clone the Git repository. Enter the new directory:

$ git clone git@github.com:virtualtam/sparklemuffin.git
$ cd sparklemuffin

Static Analysis

Dependencies

Install development utilities

Install the Go linter, the vulnerability scanner, and the license checker:

$ make dev-install-tools

Install SQLFluff:

$ make dev-install-sqlfluff

This uses uv to create a virtual environment under internal/repository/.venv/. The files internal/repository/pyproject.toml and internal/repository/uv.lock set the exact package versions.

Run linters

Go

Check Go sources with golangci-lint:

$ make lint

Check Go source headers with copywrite:

$ make copywrite

Check Go sources and go.mod for vulnerabilities:

$ make vulncheck

SQL Migrations

Check SQL files with SQLFluff:

$ make lint-sql

Format SQL files with SQLFluff:

$ make format-sql

Applied migrations must keep their statements unchanged. A newer SQLFluff release can add a rule that an applied migration no longer passes. In this case, add a -- noqa: <RULE> comment on the failing line. Do not change the statement. New migrations must pass the current rule set.

Running tests

Dependencies

Run tests

Run unit and integration tests:

$ make test

Run unit and integration tests with race detection enabled:

$ make race

Code coverage reports

Run unit and integration tests with code coverage enabled:

$ make cover

Generate an HTML report. Open it in your Web browser:

$ make coverhtml

Compiling

Build the application with:

$ make build

The binary is at build/sparklemuffin:

$ ls -lah build/

total 32M
drwxr-xr-x  2 dev dev 4.0K Nov  4 19:25 .
drwxr-xr-x 10 dev dev 4.0K Nov  5 00:07 ..
-rwxr-xr-x  1 dev dev  32M Nov  4 19:25 sparklemuffin

Live Development Server

Prerequisites

Run a local development server

The Make targets in this section start a local development server. The server uses:

  • a PostgreSQL database, running as a Docker container;
  • the SparkleMuffin application, running locally with go run.

Watchexec reloads the application server when a source file changes on disk.

These files configure and start the services:

Run a local development server:

$ make live

== Downloading frontend assets
cd internal/http/www/assets && npm ci

added 13 packages, and audited 14 packages in 623ms

found 0 vulnerabilities
== Building frontend assets
cd internal/http/www/assets && go run main.go

  ../static/www.min.css                          762.4kb
  ../static/fa-solid-900-PJNKLK6W.ttf            416.1kb
  ../static/FiraCode-VF-AEJJ5BCX.ttf             279.6kb
  ../static/Exo2-VariableFont_wght-Q2QZZPLQ.ttf  276.9kb
  ../static/fa-brands-400-R2XQZCET.ttf           205.9kb
  ../static/fa-solid-900-5ZUYHGA7.woff2          154.5kb
  ../static/fa-brands-400-Q3XCMWHQ.woff2         115.9kb
  ../static/fa-regular-400-XUOPSR7E.ttf           66.5kb
  ../static/fa-regular-400-QSNYFYRT.woff2         24.9kb
  ../static/fa-v4compatibility-YY67RJWG.ttf       10.6kb
  ../static/fa-v4compatibility-LFEHZI6Y.woff2      4.7kb

⚡ Done in 38ms
2025/06/01 21:06:18 copied node_modules/awesomplete/awesomplete.min.js to ../static/awesomplete.min.js
2025/06/01 21:06:18 copied node_modules/easymde/dist/easymde.min.js to ../static/easymde.min.js
2025/06/01 21:06:18 copied favicons/android-chrome-192x192.png to ../static/android-chrome-192x192.png
2025/06/01 21:06:18 copied favicons/android-chrome-512x512.png to ../static/android-chrome-512x512.png
2025/06/01 21:06:18 copied favicons/apple-touch-icon.png to ../static/apple-touch-icon.png
2025/06/01 21:06:18 copied favicons/favicon-16x16.png to ../static/favicon-16x16.png
2025/06/01 21:06:18 copied favicons/favicon-32x32.png to ../static/favicon-32x32.png
2025/06/01 21:06:18 copied favicons/favicon.ico to ../static/favicon.ico
2025/06/01 21:06:18 copied favicons/site.webmanifest to ../static/site.webmanifest
== Starting database
docker compose -f docker-compose.dev.yml up --remove-orphans -d
[+] Running 2/2
 ✔ Network sparklemuffin_default       Created                                                                                                                                                                                                            0.0s
 ✔ Container sparklemuffin-postgres-1  Started                                                                                                                                                                                                            0.3s
== Watching for changes... (hit Ctrl+C when done)
[Running: go run ./cmd/sparklemuffin/ run]
2025-06-01T21:06:19+02:00 INF configuration: no file found config_paths=["/etc","/home/dev/.config","."]
2025-06-01T21:06:19+02:00 INF database: successfully created connection pool database_addr=localhost:15432 database_driver=pgx database_name=sparklemuffin
2025-06-01T21:06:19+02:00 INF global: setting up services log_level=info version=devel
2025-06-01T21:06:19+02:00 INF feeds: synchronization scheduler started interval_seconds=3600000
2025-06-01T21:06:19+02:00 INF monitoring: listening for HTTP requests metrics_addr=127.0.0.1:8090
2025-06-01T21:06:19+02:00 INF sparklemuffin: listening for HTTP requests http_addr=0.0.0.0:8080

Run a local development server with race detection enabled:

$ make live-race

== Starting database
docker compose -f docker-compose.dev.yml up --remove-orphans -d
[+] Building 0.0s (0/0)                                                                                                                                                                               docker:default
[+] Running 1/0
 ✔ Container sparklemuffin-postgres-1  Running                                                                                                                                                                  0.0s
== Watching for changes... (hit Ctrl+C when done)
2023-11-03T10:27:38+01:00 INF configuration: no file found config_paths=["/etc","/home/dev/.config","."]
2023-11-03T10:27:38+01:00 INF database: successfully created connection pool database_addr=localhost:15432 database_driver=pgx database_name=sparklemuffin
2023-11-03T10:27:38+01:00 INF global: setting up services log_level=info version=devel
2023-11-03T10:27:38+01:00 INF monitoring: listening for HTTP requests metrics_addr=127.0.0.1:8090
2023-11-03T10:27:38+01:00 INF sparklemuffin: listening for HTTP requests http_addr=0.0.0.0:8080

Run database migrations

$ make dev-migrate

go run ./cmd/sparklemuffin migrate --db-password sparklemuffin --hmac-key insecure-hmac-key
2023-11-03T10:31:53+01:00 INF configuration: no file found config_paths=["/etc","/home/dev/.config","."]
2023-11-03T10:31:53+01:00 INF database: successfully created connection pool database_addr=localhost:15432 database_driver=pgx database_name=sparklemuffin
2023-11-03T10:31:53+01:00 INF successfully opened database connection database_addr=localhost:15432 database_driver=pgx database_name=sparklemuffin
2023-11-03T10:31:53+01:00 INF migrate: the database schema is up to date database_addr=localhost:15432 database_driver=pgx

Create a first administrator user

Create the user account with:

$ make dev-admin

go run ./cmd/sparklemuffin createadmin \
        --db-password sparklemuffin \
        --hmac-key insecure-hmac-key \
        --displayname Admin \
        --email admin@dev.local \
        --nickname admin
2023-11-03T10:34:50+01:00 INF configuration: no file found config_paths=["/etc","/home/dev/.config","."]
2023-11-03T10:34:50+01:00 INF database: successfully created connection pool database_addr=localhost:15432 database_driver=pgx database_name=sparklemuffin
2023-11-03T10:34:50+01:00 INF admin user successfully created email=admin@dev.local nickname=admin
Generated password: Qj3Qkeq4GpmEOrzjRv36VqVPQVymztbE4nlQ9u8KhjE=

Open the application in your Web browser:

  • go to http://localhost:8080;
  • log in with the generated credentials:
    • Email address: admin@dev.local
    • Password: find it in the make dev-admin output

Stop local services

$ docker compose stop

[+] Stopping 1/1
 ✔ Container sparklemuffin-postgres-1  Stopped

Remove containers and application data

Stop and remove application containers. Keep the data volumes:

$ docker compose down

Stop and remove application containers and their data volumes:

$ docker compose down -v

Upgrading the major version of the local PostgreSQL server

This guide shows the simplest way to upgrade the major version of a PostgreSQL server that runs as a Docker container. Follow these steps:

  1. Start the server
  2. Dump the database to a local file using pg_dump
  3. Stop the server. Delete the Docker volume that holds its data
  4. Bump the PostgreSQL version
  5. Start the server
  6. Restore the database from the local dump file using pg_restore

Start the PostgreSQL database server

$ docker compose up -d postgres

[+] Running 3/3
 ✔ Network sparklemuffin_default         Created
 ✔ Volume "sparklemuffin_postgres-data"  Created
 ✔ Container sparklemuffin-postgres-1    Started

Dump the database

$ make pgdump

# mkdir -p dump
# docker compose exec postgres pg_dump -U sparklemuffin sparklemuffin --format custom --compress zstd > dump/sparklemuffin.sql.zst

Stop the PostgreSQL server and delete its Docker volume

$ docker compose down -v

[+] Running 3/3
 ✔ Container sparklemuffin-postgres-1  Removed
 ✔ Volume sparklemuffin_postgres-data  Removed
 ✔ Network sparklemuffin_default       Removed

Update the PostgreSQL server version

Edit docker-compose.yml and docker-compose.dev.yml. Set the new PostgreSQL version:

services:
  postgres:
    image: postgres:17
    # [...]

Start the PostgreSQL database server

$ docker compose up -d postgres

[+] Running 3/3
 ✔ Network sparklemuffin_default         Created
 ✔ Volume "sparklemuffin_postgres-data"  Created
 ✔ Container sparklemuffin-postgres-1    Started

Restore the PostgreSQL database

$ make pgrestore

# docker compose exec -T postgres pg_restore -U sparklemuffin --dbname sparklemuffin < dump/sparklemuffin.sql.zst

$ make pgreindex

# docker compose exec postgres psql -U sparklemuffin -d sparklemuffin -c "REINDEX DATABASE sparklemuffin;"
REINDEX

# docker compose exec postgres psql -U sparklemuffin -d sparklemuffin -c "ALTER DATABASE sparklemuffin REFRESH COLLATION VERSION;"
ALTER DATABASE

Verification

$ make psql

# docker compose exec postgres psql -U sparklemuffin

psql (17.5 (Debian 17.5-1.pgdg120+1))
Type "help" for help.

sparklemuffin=# SELECT COUNT(*) FROM bookmarks;

 count
-------
  5126
(1 row)

Reference

PostgreSQL documentation

  • pg_dump - Extract a PostgreSQL database into a script file or other archive file
  • pg_restore - Restore a PostgreSQL database from an archive file created by pg_dump
  • psql - PostgreSQL interactive terminal
  • PostgreSQL 16 Release Notes - PostgreSQL 16 adds LZ4 and Zstandard compression to pg_dump

Sparklemuffin database

Articles

Generating the HTML documentation

Prerequisites

  • Install mdBook;
  • Install lychee to check documentation links.

HTML Documentation

Build the HTML documentation with:

$ make docs

mdbook build docs
2023-11-05 16:19:04 [INFO] (mdbook::book): Book building has started
2023-11-05 16:19:04 [INFO] (mdbook::book): Running the html backend

The generated website is at docs/book.

Check the generated website for broken links with:

$ make lint-docs

lychee --offline --no-progress --root-dir docs/book docs/book

This only checks links between local pages. It skips links to external websites.

Live-reload server

Start mdBook’s live-reload server with:

$ make live-docs

mdbook serve docs
2023-11-05 16:19:25 [INFO] (mdbook::book): Book building has started
2023-11-05 16:19:25 [INFO] (mdbook::book): Running the html backend
2023-11-05 16:19:25 [INFO] (mdbook::cmd::serve): Serving on: http://localhost:3000
2023-11-05 16:19:25 [INFO] (mdbook::cmd::watch): Listening for changes...
2023-11-05 16:19:25 [INFO] (warp::server): Server::run; addr=[::1]:3000
2023-11-05 16:19:25 [INFO] (warp::server): listening on http://[::1]:3000
  • The generated website is at docs/book;
  • Open http://localhost:3000 in a Web browser to view the live server.

Reference

Reference Guides

Detailed technical descriptions of how SparkleMuffin works. Most useful when you need details about SparkleMuffin’s implementation, database structure, or APIs.

Contributing to SparkleMuffin

SparkleMuffin is in an early stage of development. You may find missing features and incomplete parts.

The project experiments with a Web-based tool for Web bookmarks. The tool saves, tags, and retrieves bookmarks. Its interface stays simple, free of clutter, and responsive on mobile devices.

Read on if this project interests you.

Providing feedback

Issues and improvements

Use the GitHub issue tracker to give feedback about the project and its documentation.

Here are some good ways to start contributing:

  • report bugs in the application;
  • report missing or outdated documentation;
  • suggest improvements to accessibility;
  • suggest improvements to the User Interface (UI) and User Experience (UX).
Check the existing issues (open and closed) for similar reports, known issues, and planned work before you submit a new issue.

Attach information that helps fix the issue, for example:

  • application logs;
  • application configuration (redact secrets and other sensitive information first);
  • screenshots;
  • links to relevant documentation, articles, or previous issues.

Feature requests

Open a new issue on the GitHub issue tracker to discuss improvements to existing features, or to request new features.

Security issues

  • TODO: email address for security reports

Submitting patches

Documentation

To contribute to the documentation, set up a local development environment:

You can then build the documentation locally, and test your changes before you submit a Pull Request.

Bug Fixes

  • TODO: contribution workflow

Improvements and new features

Open a new issue on the GitHub issue tracker to discuss improvements to existing features, or requests for new features, before you submit a patch.

This confirms that:

  • the improvement or feature fits the project’s goals;
  • the contribution matches the quality of the existing codebase.

Project Structure

Overview

The Go source code is broken down into several top-level packages:

.
├── cmd       # Command-line application
├── internal  # Private packages and test helpers
└── pkg       # Domain packages

cmd - Command-line application

cmd
└── sparklemuffin
    ├── command    # Command-line application commands and sub-commands (CLI parser)
    ├── config     # Configuration utilities
    └── main.go    # Command-line entrypoint

internal - Application-specific and private packages

internal
├── hash            # Cryptographically secure hash helpers
├── http            # HTTP servers: monitoring, Web application
├── paginate        # Pagination utilities
├── rand            # Cryptographically secure pseudo-random helpers
├── repository
│   └── postgresql  # PostgreSQL database persistence layer (repository)
├── test            # Helpers for unit and integration tests
├── textkit         # Text processing utilities
└── version         # Version detection utilities

pkg - Domain packages

pkg
├── bookmark  # Web bookmark management
├── feed      # Feed subscription management
├── session   # User session persistence
└── user      # User and permission management

Documentation Structure

Markdown sources

mdBook generates the documentation as a static Website from Markdown files.

The SparkleMuffin repository stores the documentation resources under the docs/ directory:

docs/
├── book       # Generated Website (not tracked in Git)
├── book.toml  # mdBook configuration
└── src        # Markdown source files

Sections and page categories

The documentation has two main sections:

  • a User Guide that shows SparkleMuffin’s features and how to use them;
  • a Developer Guide that explains how SparkleMuffin works, and how to contribute to the project.

The Diátaxis approach organizes pages into four categories:

  • Tutorials: learning-oriented lessons that take you through a series of steps to use a feature;
  • How-to Guides: practical step-by-step guides to help you achieve a specific goal;
  • Reference Guides: details about how SparkleMuffin works;
  • Concept Guides: thoughts and reflections about why things work the way they do.

Reference

Development Tools

Git

SparkleMuffin uses Git to track source code changes. The code is available on GitHub at github.com/virtualtam/sparklemuffin.

To start using Git and GitHub:

Go

SparkleMuffin is written mainly in the Go programming language.

See go.mod for the minimum Go version that SparkleMuffin needs.

Linux

Install Go with your Linux distribution’s package manager.

macOS

Install Go with the Homebrew community packages:

$ brew install go

Windows

Install Go with winget:

$ winget install --id=GoLang.Go

Manual installation (advanced users)

To install a specific version of Go, see:

Node.js

SparkleMuffin uses the Node.js runtime to build its frontend assets.

Install the current Long-Term Support (LTS) version of Node.js.

Linux

Install Node.js with your Linux distribution’s package manager.

macOS

Install Node.js with the Homebrew community packages:

$ brew install node

Windows

Install Node.js with winget:

$ winget install --id=OpenJS.NodeJS

Docker

SparkleMuffin uses Docker to:

  • Package the application as easy-to-run Docker images;
  • Run database integration tests with Testcontainers;
  • Spin a local development environment with Docker Compose

Building Docker images locally needs a recent version of Docker. Local builds use:

Python and uv

SparkleMuffin uses SQLFluff to lint and format SQL migration files. SQLFluff is a Python tool, managed with uv.

See internal/repository/pyproject.toml for the pinned SQLFluff version.

Linux and macOS

Install uv with the standalone installer:

$ curl -LsSf https://astral.sh/uv/install.sh | sh

Windows

Install uv with winget:

$ winget install --id=astral-sh.uv

GNU Make

A Makefile provides targets to run tests and linters, generate documentation, and start local development environments.

lychee

SparkleMuffin uses lychee to check the generated HTML documentation for broken links.

mdBook

SparkleMuffin uses mdBook to generate a static HTML documentation from Markdown files.

Watchexec

SparkleMuffin uses watchexec to live-reload the development server when source files change on disk.

Continuous Integration

GitHub Actions Workflows

Each Action pin uses a full commit SHA. A trailing comment shows the released version, for example actions/checkout@<sha> # v7.0.1. Someone can move a tag to point at different code. A commit SHA cannot change. SHA-pinning protects workflows against a compromised or mistakenly re-tagged Action release.

CI Workflow

This workflow runs when:

  • new commits are pushed to the main Git branch;
  • new Git tags are pushed;
  • Pull Requests are created or updated.

It runs three jobs. The Build job compiles SparkleMuffin. The Lint job checks Go and SQL sources. The Test job runs the Go unit and integration tests.

See Compiling, Running Static Analysis, and Running Tests.

Copywrite Workflow

This workflow runs when:

  • new commits are pushed to the main Git branch;
  • new Git tags are pushed;
  • Pull Requests are created or updated.

It checks that Go source files have a valid license header. It uses copywrite.

See Running Static Analysis.

Vulnerabilities Workflow

This workflow runs when:

  • new commits are pushed to the main Git branch;
  • new Git tags are pushed;
  • Pull Requests are created or updated.

It checks Go sources and go.mod for known vulnerabilities. It uses govulncheck.

See Running Static Analysis.

Documentation Workflow

This workflow runs when:

  • new commits are pushed to the main Git branch;
  • new Git tags are pushed;
  • Pull Requests are created or updated.

It generates the HTML documentation with mdBook. Then it checks the documentation for broken links with lychee.

When new commits reach the main Git branch and the CI workflow succeeds, this workflow uploads the documentation to GitHub Pages. Find it at SparkleMuffin Documentation.

See Generating Documentation.

Docker Workflow

This workflow runs when:

  • new commits are pushed to the main Git branch;
  • new Git tags are pushed.

It builds and tags the SparkleMuffin production Docker images. It pushes them to the GitHub Container Registry (GHCR) at ghcr.io/virtualtam/sparklemuffin.

Dependabot

Dependabot opens a Pull Request once a month to update:

  • GitHub Actions, grouped into a single Pull Request;
  • Go modules;
  • Node.js packages, for the frontend asset pipeline;
  • SQLFluff, pinned as a uv dependency in internal/repository/pyproject.toml.

Content Security Policy

Content Security Policy (CSP) is a security mechanism. It helps prevent cross-site scripting (XSS), clickjacking, and other code-injection attacks. CSP controls which resources a Web page can load and run.

SparkleMuffin uses CSP headers to restrict inline scripts, styles, and external resources to trusted sources. This makes the application more secure.

Specifications and Resources

Database

  • TODO: PostgreSQL database
  • TODO: migrations

Netscape Bookmark Parser

The Netscape Bookmark File Format is a format commonly used by Web browsers and Web bookmarking applications to export and import bookmarks.

It has a loose specification. It has no DTD and no XSL Stylesheet. You can treat it as XML with some quirks:

  • some elements have an opening tag, but no closing tag:
    • <DT> items;
    • <DD> item descriptions;
  • some elements have an opening and closing tag:
    • <A>...</A> bookmarks;
    • <H1>...</H1> export title;
    • <H3>...</H3> folder name;
  • some elements have unusual opening and closing tags:
    • <DL><p>...</DL><p> item lists;
  • depending on the implementation:
    • elements may (or may not) be capitalized;
    • some elements may (or may not) be nested;
    • some attributes may (or may not) be present.

Example Netscape Bookmark Document

<!DOCTYPE NETSCAPE-Bookmark-file-1>
<!-- This is an automatically generated file.
     It will be read and overwritten.
     DO NOT EDIT! -->
<TITLE>Bookmarks</TITLE>
<H1>Bookmarks</H1>
<DL><p>
    <DT><H3 ADD_DATE="1622567473" LAST_MODIFIED="1627855786">Favorites</H3>
    <DD>Add bookmarks here
    <DL><p>
        <DT><A HREF="https://domain.tld" ADD_DATE="1641057073" PRIVATE="1">Test Domain</A>
        <DT><A HREF="https://test.domain.tld" ADD_DATE="1641057073" LAST_MODIFIED="1646172586" PRIVATE="1">Test Domain II</A>
        <DD>Second test
    </DL><p>
</DL><p>

Go Parser

SparkleMuffin uses virtualtam/netscape-go to parse (unmarshal) and export (marshal) bookmarks using the Netscape Bookmark File Format.

Users can import or sync their existing bookmarks into SparkleMuffin. They can also export bookmarks for use with another bookmarking service.

virtualtam/netscape-go is also available as a standalone library for other projects to use.

It uses:

OPML Feed Subscription Parser

Outline Processor Markup Language (OPML) is a format commonly used by feed aggregators and feed readers to export and import subscriptions to Atom and RSS feeds.

It has a permissive specification, and each feed aggregator or reader may:

  • specify extra attributes;
  • use non-standard attributes or attribute formats (e.g. to format dates and time);
  • use a nested structure to represent subscriptions, categories and directories.

Specifications

Example OPML Document

<?xml version="1.0" encoding="UTF-8"?>

<opml version="1.0">
    <head>
        <title>My subscriptions in feedly Cloud</title>
    </head>
    <body>
        <outline text="Programming" title="Programming">
            <outline type="rss" text="Elixir Lang" title="Elixir Lang" xmlUrl="https://feeds.feedburner.com/ElixirLang" htmlUrl="https://elixir-lang.org"/>
            <outline type="rss" text="Python Insider" title="Python Insider" xmlUrl="https://feeds.feedburner.com/PythonInsider" htmlUrl="https://pythoninsider.blogspot.com/"/>
        </outline>
        <outline text="Games" title="Games">
            <outline type="rss" text="Vintage Story" title="Vintage Story" xmlUrl="https://www.vintagestory.at/blog.html/?rss=1" htmlUrl="https://www.vintagestory.at/blog.html/"/>
        </outline>
    </body>
</opml>

Go Parser

SparkleMuffin uses virtualtam/opml-go to parse (unmarshal) and export (marshal) feed subscriptions using the OPML file format.

Users can import or sync their existing subscriptions into SparkleMuffin. They can also export subscriptions for use with another feed aggregator or feed reader.

virtualtam/opml-go is also available as a standalone library for other projects to use.

Conceptual Guides

Explanations of SparkleMuffin’s high-level concepts. Most useful when you want to understand a topic in depth.

Feed polling and caching

SparkleMuffin periodically makes HTTP requests to update Atom and RSS feeds. This creates two requirements:

  • do not put unnecessary load on the remote servers;
  • do not perform unnecessary database updates when the remote content has not changed.

SparkleMuffin uses HTTP caching features from the HTTP specification. It also performs extra checks on the feed content.

HTTP Conditional Requests

When responding to an HTTP request, a remote server may set the following headers:

  • ETag: the current entity tag for the selected representation (usually a hash of the feed data);
  • Last-Modified: the date and time the origin server last modified the selected representation.

When present, SparkleMuffin stores these values in the database. It uses them to set the following headers in later requests:

  • If-None-Match: the value of the ETag header from the previous response;
  • If-Modified-Since: the value of the Last-Modified header from the previous response.

The remote server responds in one of two ways:

  • 200 OK: the content changed. SparkleMuffin updates the feed and its entries;
  • 304 Not Modified: nothing changed. SparkleMuffin only updates the feed’s ETag and Last-Modified headers.

Feed content hash

A remote server can send a different ETag or Last-Modified value even when the feed content has not changed. It can also send neither header. To handle this, SparkleMuffin:

  • computes and stores a hash of the feed data, using the xxHash non-cryptographic hash function;
  • compares the hash of the feed data with the value stored in the database;
  • returns early if the hashes match, to avoid unnecessary database updates.

Reference

Feed caching

RFCs

HTTP Conditional Requests

Non-cryptographic hash functions