Hackage :: [Package]

shibuya-kafka-adapter: Kafka adapter for the Shibuya queue processing framework

[ concurrency, library, mit, streaming ] [ Propose Tags ] [ Report a vulnerability ]

A Shibuya adapter that integrates with Apache Kafka via kafka-effectful and hw-kafka-client. Provides polling, offset commit semantics, partition awareness, and graceful shutdown.

[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.1.0.0, 0.2.0.0, 0.4.0.0, 0.5.0.0, 0.5.0.1, 0.6.0.0
Change log CHANGELOG.md
Dependencies base (>=4.21.0.0 && <4.22), bytestring (>=0.12 && <0.13), containers (>=0.7 && <0.8), effectful-core (>=2.6.1.0 && <2.7), hw-kafka-client (>=5.3 && <6), hw-kafka-streamly (>=0.1 && <0.2), kafka-effectful (>=0.1 && <0.2), shibuya-core (>=0.1.0.0 && <0.2), stm (>=2.5 && <2.6), streamly (>=0.11 && <0.12), streamly-core (>=0.3 && <0.4), text (>=2.1 && <2.2), time (>=1.14 && <1.15) [details]
Tested with ghc ==9.12.2
License MIT
Copyright 2026 Nadeem Bitar
Author Nadeem Bitar
Maintainer nadeem@gmail.com
Uploaded by shinzui at 2026-04-18T15:15:03Z
Category Concurrency, Streaming
Home page https://github.com/shinzui/shibuya-kafka-adapter
Bug tracker https://github.com/shinzui/shibuya-kafka-adapter/issues
Source repo head: git clone https://github.com/shinzui/shibuya-kafka-adapter.git
Distributions
Downloads 19 total (12 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]

Readme for shibuya-kafka-adapter-0.1.0.0

[back to package description]

shibuya-kafka-adapter

Kafka adapter for the Shibuya queue-processing framework.

Integrates with Apache Kafka via kafka-effectful for the consumer effect (polling, offset store, partition pause) and hw-kafka-streamly for error classification (skipNonFatal), on top of hw-kafka-client. Provides polling, offset commit semantics, partition awareness, and graceful shutdown.

Packages

  • shibuya-kafka-adapter — the adapter library (Shibuya.Adapter.Kafka, .Config, .Convert).
  • shibuya-kafka-adapter-bench — micro-benchmarks for the conversion hot path (ConsumerRecord → Envelope, W3C header extraction, timestamps).
  • shibuya-kafka-adapter-jitsurei — runnable examples: BasicConsumer, MultiTopic, MultiPartition, OffsetManagement.

Building

The repo ships a Nix flake and direnv config for a reproducible toolchain.

direnv allow        # or: nix develop
cabal build all
cabal test shibuya-kafka-adapter

Benchmarks and examples:

cabal bench shibuya-kafka-adapter-bench
cabal run BasicConsumer

Layout

shibuya-kafka-adapter/            library sources and tests
shibuya-kafka-adapter-bench/      tasty-bench micro-benchmarks
shibuya-kafka-adapter-jitsurei/   runnable usage examples
docs/plans/                       execution plans
mori.dhall                        project manifest (mori registry)

License

MIT. See package cabal files for details.