Documentation

Looks up the tokenizer from Skylighting preset library by the short name of the language. Picks the first match.

Attempt to tokenize input, returns Nothing if unsuccessful, so the processor can just pass input further when tokenizer fails.

Fix Skylighting's incorrect tokenization of backslash operators. Skylighting incorrectly splits operators like +, >, /, etc. This function merges them back together to match Haskell tokenizer behavior.