workflow-management
Create, debug, or modify QStash workflows for data updates and social media posting in the API service. Use when adding new automated jobs, fixing workflow errors, or updating scheduling logic.
When & Why to Use This Skill
This Claude skill facilitates the end-to-end management of QStash-based workflows, enabling developers to create, debug, and optimize automated data update pipelines and social media posting tasks. It ensures high reliability through standardized error handling, idempotency patterns, and seamless integration with serverless API architectures.
Use Cases
- Automating scheduled data synchronization: Creating workflows to fetch and update real-time data, such as car registration or COE bidding information, into the system.
- Multi-platform social media orchestration: Developing and modifying automated posting logic for Discord, LinkedIn, Telegram, and Twitter via API integrations.
- Workflow debugging and error recovery: Identifying execution failures using QStash logs and CloudWatch, and implementing robust retry logic to ensure task completion.
- Infrastructure-as-Code updates: Modifying cron schedules and workflow triggers within SST infrastructure or QStash console configurations.
- Pipeline expansion: Integrating new data sources or third-party services into existing automated update sequences with minimal friction.
| name | workflow-management |
|---|---|
| description | Create, debug, or modify QStash workflows for data updates and social media posting in the API service. Use when adding new automated jobs, fixing workflow errors, or updating scheduling logic. |
| allowed-tools | Read, Edit, Write, Bash, Grep, Glob |
Workflow Management Skill
This skill helps you work with QStash-based workflows in apps/api/src/lib/workflows/.
When to Use This Skill
- Adding new scheduled workflows for data fetching
- Debugging workflow execution errors
- Modifying existing workflow schedules or logic
- Integrating new data sources into the update pipeline
- Adding new social media posting workflows
Workflow Architecture
The project uses QStash workflows with the following structure:
apps/api/src/lib/workflows/
├── cars/ # Car registration data workflows
│ └── update.ts # Scheduled car data updates
├── coe/ # COE bidding data workflows
│ └── update.ts # Scheduled COE data updates
└── social/ # Social media posting workflows
├── discord.ts
├── linkedin.ts
├── telegram.ts
└── twitter.ts
Key Patterns
1. Workflow Definition
Workflows are defined using QStash SDK:
import { serve } from "@upstash/workflow";
export const POST = serve(async (context) => {
// Step 1: Fetch data
await context.run("fetch-data", async () => {
// Fetching logic
});
// Step 2: Process data
const processed = await context.run("process-data", async () => {
// Processing logic
});
// Step 3: Store results
await context.run("store-results", async () => {
// Storage logic
});
});
2. Scheduling Workflows
Workflows are triggered via cron schedules configured in:
- SST infrastructure (
infra/) - QStash console
- Manual API calls to workflow endpoints
3. Error Handling
Always include comprehensive error handling:
await context.run("step-name", async () => {
try {
// Logic here
} catch (error) {
console.error("Step failed:", error);
// Log to monitoring service
throw error; // Re-throw for workflow retry
}
});
Common Tasks
Adding a New Workflow
- Create workflow file in appropriate directory
- Define workflow steps using
context.run() - Add route handler in
apps/api/src/routes/ - Configure scheduling (if needed)
- Add tests for workflow logic
Debugging Workflow Failures
- Check QStash dashboard for execution logs
- Review CloudWatch logs for Lambda errors
- Verify environment variables are set correctly
- Test workflow locally using development server
- Check database connectivity and Redis availability
Modifying Existing Workflows
- Read existing workflow implementation
- Identify which step needs modification
- Update step logic while maintaining error handling
- Test changes locally
- Deploy and monitor execution
Environment Variables
Workflows typically need:
DATABASE_URL- PostgreSQL connectionUPSTASH_REDIS_REST_URL/UPSTASH_REDIS_REST_TOKEN- RedisQSTASH_TOKEN- QStash authentication- Service-specific tokens (Discord webhook, Twitter API, etc.)
Testing Workflows
Run workflow tests:
pnpm -F @sgcarstrends/api test -- src/lib/workflows
Test individual workflow locally:
# Start dev server
pnpm dev
# Trigger workflow via HTTP
curl -X POST http://localhost:3000/api/workflows/cars/update
References
- QStash Workflows: Check Context7 for Upstash QStash documentation
- Related files:
apps/api/src/routes/workflows.ts- Workflow route handlersapps/api/src/config/qstash.ts- QStash configurationapps/api/CLAUDE.md- API service documentation
Best Practices
- Idempotency: Ensure workflows can safely retry without duplicating data
- Step Granularity: Break workflows into small, focused steps
- Logging: Add comprehensive logging for debugging
- Timeouts: Configure appropriate timeouts for long-running operations
- Testing: Write unit tests for workflow logic
- Monitoring: Track workflow execution metrics