mfmts: Implements multiformats specification

mfmts

This project implements the specifications described by Protocol Lab's multiformats written in Haskell, and split across several directories described below.

cid - Content identifiers for hashed data

Provides functionality for the multiformats CID binary format. These modules provide parsing CIDs from multibase prefixed digit strings, extracting CID data from byte strings and serializing CIDs stored in memory into its binary format.

Multiformats CID GitHub README example in GHCI:

λ> {-# LANGUAGE OverloadedStrings #-}
λ> import MultiFormats.CID (CID)
λ> import MultiFormats.CID.Parser qualified as CID
λ> import MultiFormats.CID.Errors qualified as CID
λ> CID.parser "zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA" :: CID.Okay CID
Right CIDv1(DataCodec=MultiCodec(raw=0x55), HashCodec=MultiHash(sha2-256=0x12), HashDigest[32]=0x6e6ff7950a36187a801613426e858dce686cd7d7e3c0fc42ee0330072d245c95)

multibase - Multibase prefix data transcoding

Implements multibase encoders and decoders for numeric basesystems identified by a specified prefix symbol. It uses Transcoders from the xcodec library, allowing for different types of bytestring data to be specified as type parameters.

Multibase GitHub README example in GHCI:

λ> {-# LANGUAGE OverloadedStrings #-}
λ> import MultiFormats.MultiBase.DigitTranscoder (decodeFromPrefix)
λ> import Data.ByteString.Short (ShortByteString)
λ> import Data.ByteString.Lazy (LazyByteString)
λ> type Sbs = ShortByteString
λ> type Lbs = LazyByteString
λ> -- uppercase hexadecimal (F)
λ> decodeFromPrefix "F4D756C74696261736520697320617765736F6D6521205C6F2F" :: Maybe Sbs
Just "Multibase is awesome! \\o/"
λ> -- base32 no padding (B)
λ> decodeFromPrefix "BJV2WY5DJMJQXGZJANFZSAYLXMVZW63LFEEQFY3ZP" :: Maybe Lbs
Just "Multibase is awesome! \\o/"
λ> -- bitcoin base58 (z)
λ> decodeFromPrefix "zYAjKoNbau5KiqmHPmSxYCvn66dA1vLmwbt" :: Maybe Sbs
Just "Multibase is awesome! \\o/"
λ> -- base64 with padding (M)
λ> decodeFromPrefix "MTXVsdGliYXNlIGlzIGF3ZXNvbWUhIFxvLw==" :: Maybe Lbs
Just "Multibase is awesome! \\o/"

multicodec - Magic number specifications

Provides lookup table and query functions for codecs defined from multiformats multicodec. These are magic number identifiers that represent certain data encoded within CIDs, mulithash and other related projects. This module is generated by a Python script in the ipfshs repo by using the CSV file provided from the multicodec GitHub.

Example of looking up codecs:

λ> import MultiFormats.MultiCodec qualified as MC
λ> -- Resolving from codec values:
λ> MC.resolveCodecName $ MC.MultiCodec 0x1
Just "cidv1"
λ> -- Resolving from names:
λ> MC.resolveCodecValue "cidv1"
Just MultiCodec(0x1, "cidv1")
λ> MC.resolveCodecValue "cidv0"
Nothing

multihash - Multicodec hash function identifiers

Multihashes are a type of multicodec for identifying hash functions. This module provides a lookup table for hash functions from the multicodec module, a way to get hash digests from specific functions over bytestrings, and to verify multicodec values as valid multihash codecs.

Example of looking up hash functions and applying them:

λ> import MultiFormats.MultiCodec qualified as MC
λ> import MultiFormats.MultiHash qualified as MH
λ> -- Here's an example of how we can verify names and codec values as valid
λ> -- hash functions.
λ> MH.resolveHashCodec $ MC.MultiCodec 0x12
Just MultiHash(0x12, "sha2-256")
λ> MH.resolveHashName "sha2-256"
Just MultiHash(0x12, "sha2-256")
λ> -- Building a function call that hashes the string "Hello World!" using the
λ> -- sha2-256 algorithm, then outputs the digest as hexadecimal:
λ> import Data.BaseSystem.Lazy (encoder, base16lower)
λ> import Data.ByteString.Builder qualified as Builder
λ> import Data.Maybe (fromJust)
λ> let sha256 = MH.multiHashFunction $ fromJust (MH.resolveHashName "sha2-256")
λ> let input = Builder.string8 "Hello World!"
λ> encoder base16lower $ sha256 input
"7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069"

varint - Unsigned variable length integers

Provides parsing and serialization for unsigned binary varints, with the caveat from the multiformats specification that requires them always to be minimally encoded.

Varint GitHub README example in GHCI using packValue from the xcodec library to transfer constants into bytestrings. We can then use extractVarInt to get the varint representation from the bytes:

λ> import Data.ByteString.Short (ShortByteString)
λ> import MultiFormats.VarInt (VarInt, extractVarInt)
λ> import MultiFormats.VarInt.Errors qualified as VarInt
λ> import XCodec.Transcoder (packValue)
λ> -- Alias to suppress type warnings
λ> let packInteger x = packValue (x :: Integer)
λ> -- 7-bit values fit into 1 byte:
λ> extractVarInt (packInteger 0x01) :: VarInt.Okay (VarInt, ShortByteString)
Right (1,"")
λ> extractVarInt (packInteger 0x7f) :: VarInt.Okay (VarInt, ShortByteString)
Right (127,"")
λ> -- 8-bit and larger values have MSB=1 until a byte with MSB=0 is found:
λ> extractVarInt (packInteger 0x8001) :: VarInt.Okay (VarInt, ShortByteString)
Right (128,"")
λ> extractVarInt (packInteger 0xff01) :: VarInt.Okay (VarInt, ShortByteString)
Right (255,"")
λ> extractVarInt (packInteger 0xac02) :: VarInt.Okay (VarInt, ShortByteString)
Right (300,"")
λ> extractVarInt (packInteger 0x808001) :: VarInt.Okay (VarInt, ShortByteString)
Right (16384,"")

Development

This project is part of ipfshs; unit tests are provided on the main page and bugs can be reported on its ticket tracker. Patches and pull requests can be submitted with git send-email. To build and test this project against all of ipfshs read this section on setting up an ipfshs development environment.

This project can also be built as a standalone library with cabal.

$ git clone https://git.sr.ht/~z0/mfmts
$ cd mfmts
$ cabal build

Licensing

The mfmts project and its modules are free software and licensed under the AGPLv3 license. See LICENSE.txt.

Copyright © 2026 Zoey McBride | zoeymcbride@mailbox.org