effect-queues-background
Queue and PubSub patterns, background fibers, and graceful shutdown. Use for decoupling producers/consumers.
When & Why to Use This Skill
This Claude skill facilitates the implementation of robust asynchronous messaging and concurrency patterns using the Effect library. It provides expert guidance on utilizing bounded queues for backpressure management, PubSub for event-driven broadcasting, and background fiber orchestration to ensure system decoupling and graceful shutdowns in high-performance applications.
Use Cases
- Decoupling high-frequency producers from slow consumers using bounded queues to maintain system stability and prevent memory exhaustion.
- Implementing scalable event-driven architectures where messages need to be broadcasted to multiple independent subscribers via PubSub patterns.
- Managing long-running background tasks and concurrent loops with managed lifecycles, ensuring all fibers are safely interrupted during application shutdown.
- Building resilient task processing pipelines that require explicit resource safety, logging, and error handling in a functional programming environment.
| name | effect-queues-background |
|---|---|
| description | Queue and PubSub patterns, background fibers, and graceful shutdown. Use for decoupling producers/consumers. |
| allowed-tools | Read, Grep, Glob, Edit, Write |
Queues, PubSub & Background
When to use
- Decoupling producers/consumers with backpressure
- Broadcasting events to multiple subscribers
- Running background loops with graceful shutdown
Queue (bounded)
import { Queue } from "effect"
const q = yield* Queue.bounded<string>(32)
yield* Queue.offer(q, "job")
const job = yield* Queue.take(q)
PubSub (broadcast)
import { PubSub } from "effect"
const ps = yield* PubSub.bounded<string>(32)
yield* PubSub.publish(ps, "evt")
Background Fiber
const fiber = yield* Effect.fork(loop)
yield* Fiber.interrupt(fiber)
Guidance
- Prefer bounded queues to apply natural backpressure
- Use multiple workers by forking consumers
- Ensure background fibers are interrupted during shutdown
Pitfalls
- Unbounded queues lead to memory growth
- Silent background failures → add logging/metrics
Cross-links
- Concurrency for pools and interruption
- Time/Logging for observability of background tasks
Local Source Reference
CRITICAL: Search local Effect source before implementing
The full Effect source code is available at docs/effect-source/. Always search the actual implementation before writing Effect code.
Key Source Files
- Queue:
docs/effect-source/effect/src/Queue.ts - PubSub:
docs/effect-source/effect/src/PubSub.ts - Fiber:
docs/effect-source/effect/src/Fiber.ts
Example Searches
# Find Queue patterns
grep -F "bounded" docs/effect-source/effect/src/Queue.ts
grep -F "offer" docs/effect-source/effect/src/Queue.ts
grep -F "take" docs/effect-source/effect/src/Queue.ts
# Study PubSub operations
grep -F "publish" docs/effect-source/effect/src/PubSub.ts
grep -F "subscribe" docs/effect-source/effect/src/PubSub.ts
# Find background fiber patterns
grep -F "fork" docs/effect-source/effect/src/Fiber.ts
grep -F "interrupt" docs/effect-source/effect/src/Fiber.ts
# Look at Queue test examples
grep -F "Queue." docs/effect-source/effect/test/Queue.test.ts
Workflow
- Identify the Queue or PubSub API you need
- Search
docs/effect-source/effect/src/Queue.tsorPubSub.tsfor the implementation - Study the types and backpressure patterns
- Look at test files for usage examples
- Write your code based on real implementations
Real source code > documentation > assumptions
References
- Agent Skills overview: https://www.anthropic.com/news/skills
- Skills guide: https://docs.claude.com/en/docs/claude-code/skills