Espial

Espial is an open-source, web-based bookmarking server.

It allows mutiple accounts, but currently intended for self-host scenarios.

The bookmarks are stored in a sqlite3 database, for ease of deployment & maintenence.

The easist way for logged-in users to add bookmarks, is with the "bookmarklet", found on the Settings page.

Demo Server

Log in with:

• username: demo
• password: demo

https://espdemo.ae8.org/u:demo

Installation

Docker Setup

See:

https://github.com/jonschoning/espial-docker

Server Setup (From Source)

1. Install the Stack executable here:
   • https://tech.fpcomplete.com/haskell/get-started
2. Build executables:

stack build

3. Create the database:

stack exec migration -- createdb

4. Create a user:

stack exec migration -- createuser --userName myusername --userPassword myuserpassword

5. Import a pinboard bookmark file for a user (optional):

stack exec migration -- importbookmarks --userName myusername --bookmarkFile sample-bookmarks.json

6. Import a firefox bookmark file for a user (optional):

stack exec migration -- importfirefoxbookmarks --userName myusername --bookmarkFile firefox-bookmarks.json

7. Start a production server:

stack exec espial

Configuration

See config/settings.yml for changing default run-time parameters & environment variables.

• config/settings.yml is embedded into the app executable when compiled and also read once when the app starts. Current settings in config/settings.yml will override the embedded compile-time settings.
• config/settings.yml values formatted like _env:ENV_VAR_NAME:default_value can be overridden by the specified environment variable.
• Example:
  • _env:PORT:3000
  • environment variable PORT
  • default app http port: 3000

Request IP Logging

Espial supports the IP_FROM_HEADER environment variable for request logging.

• IP_FROM_HEADER=true: log the client IP from the X-Real-IP or X-Forwarded-For header when present, and fall back to the peer address if neither header is available.
• IP_FROM_HEADER=false: log the peer address from the HTTP connection.

Only set IP_FROM_HEADER=true if your application is safely positioned behind a trusted reverse proxy.

SSL / Reverse Proxy

Espial does not terminate TLS itself. Run it behind a reverse proxy that handles HTTPS and forwards traffic to Espial over HTTP.

For container-based deployment examples, including production-oriented layouts, see the espial-docker repository:

• https://github.com/jonschoning/espial-docker

Minimal Caddy example:

Localhost without a real domain:

https://localhost:3050 {
    reverse_proxy localhost:3000
}

or with a domain:

espial.example.com {
  reverse_proxy 127.0.0.1:3000
}

With the domain setup:

• Caddy terminates TLS for espial.example.com.
• Espial continues listening on HTTP, locally on 127.0.0.1:3000
  • If using Docker Compose, it would like like espial:3000
• Set IP_FROM_HEADER=true only when Espial is reachable solely through that trusted proxy.

If you are using Cloudflare:

• Prefer Cloudflare SSL mode Full (strict).
• use header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}
• If traffic can reach Espial directly without passing through your trusted proxy, do not enable IP_FROM_HEADER=true, because client IP headers can be spoofed.

Archive Backends

Espial supports configurable archive backends for saving bookmark snapshots.

Set the backend with archive-backend in config/settings.yml:

• disabled: archiving is turned off (default).

• wayback-machine: enables submission to the Internet Archive Wayback Machine.

  Wayback Machine support requires the following settings:

  • wayback-machine-access-key

  • wayback-machine-secret-key

    Create these by signing in to your Internet Archive account and generating S3-style API credentials at https://archive.org/account/s3.php.
    If wayback-machine is selected but the access key or secret key is missing, archiving is disabled at runtime.

• archivebox07: queues the URL in a local ArchiveBox 0.7 instance and stores an ArchiveBox link on the bookmark.

  IMPORTANT - ArchiveBox stores all archive data in a single global index space, so this arcive-backend is best suited to single-user Espial instances.

  Recommended setup is to use Docker Compose to run the ArchiveBox instance

  • Simple example, running on localhost: docker-compose.archivebox07.yml
  • See https://github.com/jonschoning/espial-docker for more examples intended for deployment
  • In all examples, you must change the ARCHIVEBOX_PASSWORD from it's default value.

  ArchiveBox support requires the following settings:

  • archivebox-url

    archivebox-url is the URL espial uses to sign in to ArchiveBox and submit URLs through the web UI. In Docker Compose this is typically http://archivebox:8000.

  • archivebox-public-url (optional)

    Public ArchiveBox URL stored on bookmarks.

  • archivebox-username plus archivebox-password

    Espial signs in to the ArchiveBox web UI with these credentials before submitting URLs.

  • archivebox-tag (optional)

    A tag Espial adds to submissions (example: espial).

  • archivebox-plugins (optional)

    Comma-separated list of ArchiveBox methods (plugins) to request when submitting URLs, e.g. title,favicon,singlefile,screenshot.

  Set the ArchiveBox admin credentials in the override path by supplying:

  • ARCHIVEBOX_USERNAME=...
  • ARCHIVEBOX_PASSWORD=...

  The Makefile includes the following helpers:

  • docker-compose-up-archivebox07
  • docker-compose-up-d-archivebox07
  • docker-compose-exec-archivebox07

  Or start the instance manually via docker compose, example:

  docker compose -f docker-compose.archivebox07.yml up
  

  Configure the enrivonment variable ARCHIVE_METHODS to control which archive methods ArchiveBox uses:

  environment:
    - ARCHIVE_METHODS=title,favicon,singlefile,screenshot
  

  Available ARCHIVE_METHODS plugins:

  • archive_org, dom, favicon, git, headers, htmltotext, media, mercury, pdf, readability, screenshot, singlefile, title, wget

  If ARCHIVE_METHODS is unset/not-present, ArchiveBox will uses all plugins.

  For additional information and configuration, refer to the ArchiveBox repository

Optional proxy settings for archive requests:

• archive-socks-proxy-host
• archive-socks-proxy-port

Related Projects

Also, see the android app for adding bookmarks via an Android Share intent:

https://github.com/jonschoning/espial-share-android

Development

Frontend

• See frontend/ folder

CLI

Migration commands are run via:

stack exec migration -- <command> [options]

All commands take an optional --conn parameter for the database location; if omitted, the database location is loaded from config/settings.yml or environment variable SQLITE_DATABASE

Commands

importbookmarks Command Notes:

See sample-bookmarks.json, which contains a JSON array, each line containing a FileBookmark object.

Example:

[
  {
    "href": "http://raganwald.com/2018/02/23/forde.html",
    "description": "Forde's Tenth Rule, or, \"How I Learned to Stop Worrying and \u2764\ufe0f the State Machine\"",
    "extended": "",
    "time": "2018-02-26T22:57:20Z",
    "shared": "yes",
    "toread": "yes",
    "tags": "raganwald"
  },
  ,
  {
    "href": "http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/flags.html",
    "description": "7.6. Flag reference \u2014 Glasgow Haskell Compiler 8.2.2 User's Guide",
    "extended": "-fprint-expanded-synonyms",
    "time": "2018-02-26T21:52:02Z",
    "shared": "yes",
    "toread": "no",
    "tags": "ghc haskell"
  }
]