doctest-extract: Alternative doctest implementation that extracts comments to modules

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

doctest-extract lets you write test examples and QuickCheck properties in Haddock comments and extracts them to test modules. It means that the user sees your tests in the documentation and knows that the examples and properties are machine-tested, or at least, she can run the tests herself.

I found the barrier to write tests much lower when I do not need to write new test modules but just add some lines to the Haddock comments. I do not need to think of test names or filling test data structures. The test identifier is the module name and the line number and if a test fails I can easily jump to the failing code.

Differences to the original GHCi-based implementation of doctest:

Pros:

Package versions for tests are consistent with tested library
Tests run much faster, especially QuickCheck property tests
No dependency on GHCi or GHC-as-library
The tested package need not be ready for compilation. Our simple parser requires only clearly recognizable Haskell comments.
QuickCheck properties do not cause confusing type error messages when actually only identifiers are missing.
You can inspect extracted modules
doctest collects tests from the transitive hull of imports of the specified modules. This might help you to keep the list of modules short. doctest-extract only processes the specified modules and thus allows you to focus on a module for development of tests.
With option --verbose test source path and line number are formatted such that Emacs allows you to click and jump to the test definition.
Report success only of real tests. doctest reports successful imports and definition of helper types and functions as successful tests. This makes it hard to monitor the number of real tests, e.g. whether some tests have been dropped by accident.

Cons:

Cannot test for output of IO functions or error messages from partial functions.
All free variables in QuickCheck properties must be all-quantified using lambda. (Could be even seen as an advantage for the reader of your doctests.)
No support for a single-line let (as in a do-block) as an example.
The Test module does not automatically import modules that the tested module imports. Thus, you usually have to add a setup section with required imports or use the --import-tested option.
You need tools additional to Cabal, e.g. make and a Makefile or just and a Justfile, in order generate test modules. You may want to use grep to extract the doctest module list from a Cabal package description.

See packages utility-ht, apportionment or pathtype for packages with working setups of doctest-extract.

Alternatives: cabal-docspec, cabal-doctest

[Skip to Readme]

Properties

Versions	0.1, 0.1.0.1, 0.1.1, 0.1.1.1, 0.1.2, 0.1.2.1
Change log	None available
Dependencies	base (>=4.5 && <5), doctest-lib (>=0.0 && <0.2), non-empty (>=0.3.3 && <0.4), optparse-applicative (>=0.11 && <0.20), pathtype (>=0.8 && <0.9), semigroups (>=0.18.5 && <0.21), transformers (>=0.5.6 && <0.7), utility-ht (>=0.0.16 && <0.1) [details]
License	BSD-3-Clause
Author	Henning Thielemann <haskell@henning-thielemann.de>
Maintainer	Henning Thielemann <haskell@henning-thielemann.de>
Category	Testing
Home page	https://hub.darcs.net/thielema/doctest-extract/
Source repo	this: darcs get https://hub.darcs.net/thielema/doctest-extract/ --tag 0.1.2.1 head: darcs get https://hub.darcs.net/thielema/doctest-extract/
Uploaded	by HenningThielemann at 2026-08-20T17:33:57Z

Downloads

doctest-extract-0.1.2.1.tar.gz [browse] (Cabal source package)
Package description (as included in the package)

Maintainer's Corner

Package maintainers

HenningThielemann

For package maintainers and hackage trustees

edit package information

Readme for doctest-extract-0.1.2.1

[back to package description]

Known Issues

For compatibility with original doctest parser you cannot write
```
{-# LANGUAGE MyPreferredExtension #-}
```
Instead you must write
```
:set -XMyPreferredExtension
```
In Literal Haskell files only \\begin{code} ... \\end{code} blocks are scanned, but not bird style code blocks.
prop> supports multi-line code, but both original doctest and haddock do not support it.
IO tests are not supported as doctest examples, so far. We need a syntactic distinction for IO tests, because doctest-extract does not employ a type-checker. We could mark IO tests with a specific id function, as in ioTest $ runMyTest or a type annotation, as in runMyTest :: IO ().

Tipps and Tricks

Interaction with editor

You may extract your doctests using the --verbose option. This emits the tested expression before each test and it formats the source location in a way that is recognized by Emacs et.al. , i.e. you can click on the source location and thus jump to the according doctest.

Synchronize test module list between Cabal and doctest-extract

We recommend maintaining the list of test modules with unique name prefixes in the Cabal package description and extract this list using grep.

E.g.:

  Other-Modules:
    DocTest.MyProject.ABC
    DocTest.MyProject.DEF
    DocTest.MyProject.GHJ

Your grep expression should only accept lines that start with spaces exclusively. This way, modules are skipped if they are outcommented in the Cabal file.

How to disable selected tests?

For focussing on certain tests it can be useful to disable other ones. We have not implemented a mechanism to disable parts of the test suite in doctest-extract, because this would require to implement a way to identify tests. You can still disable some of the tests without explicit support by doctest-extract.

If you want to disable whole modules, you may make a copy of the auto-generated Test/Main.hs and remove the modules that you want to skip. If you grep the module list from the Cabal package description, then outcomment the according modules there.
For disabling all tests on a function you may turn a Haddock comment into a plain comment by removing the bar after the opening of the comment.
For disabling individual tests you may prefix >>> and prop> with an asterisk or the like.

These tricks work best in conjunction with a revision control systen, such that it always reminds you that there are tests disabled temporarily.

If you have a doctest property like

prop> let x = ... in \y -> p x y

then x will be re-evaluated for every of the 100 randomized QuickCheck tests. If you want to precompute a value once and re-use it in all QuickCheck runs, you may abuse QuickCheck.forAllBlind like so:

prop> forAllBlind (return ...) $ \x y -> p x y

Tests in the IO monad

As said above, tests in the IO monad are currently not directly supported. However, for properties you may use QuickCheck.ioProperty or QuickCheck.idempotentIOProperty. For Doctest examples you may roll your own unsafePerformIO hacks. Maybe we officially provide some in the future by ourselves.