understanding-flow

elbwalker 的头像作者 elbwalker

适用于学习walkerOS架构、理解数据流或设计可组合事件管道时使用。涵盖源→收集器→目标模式关注点分离。

📖 中文导读:适用于学习walkerOS架构、理解数据流或设计可组合事件管道时使用。涵盖源→收集器→目标模式关注点分离。 以下正文为该技能的原始 SKILL.md 内容(英文),上方安装命令可直接使用。

name: understanding-flow description: Use when learning walkerOS architecture, understanding data flow, or designing composable event pipelines. Covers Source→Collector→Destination pattern and separation of concerns.

Understanding walkerOS Flow

Overview

walkerOS follows a Source → Collector → Destination(s) architecture for composable, modular event processing.

Core principle: Separation of concerns. Each component has one job. Components are composable and replaceable.

The Flow Pattern

Sources → [Pre-Transformers] → Collector → [Post-Transformers] → Destinations
(Capture)  (source.next)    (Processing) (dest.before)       (Delivery)

- Browser DOM              - Validation   - Validation        - Google Analytics
- DataLayer                - Enrichment   - Enrichment        - Meta Pixel
- Server HTTP              - Redaction    - Consent check     - Custom API
- Cloud Functions                         - Routing           - Data Warehouse

Key Concepts

Composability

A Flow combines components. You can:

  • Use multiple sources feeding one collector
  • Route events to multiple destinations
  • Swap components without changing others

The Flow Type

See packages/core/src/types/flow.ts for the canonical interface.

// Conceptual structure (see source for full type)
interface Flow {
  sources?: Record<string, Source>;
  collector: Collector;
  destinations?: Record<string, Destination>;
}

Universal Push Interface

All components communicate via push functions:

Component Push Signature Purpose
Source push(input) → events Capture external data
Collector push(event) → void Process and route
Destination push(event, context) → void Transform and deliver

The elb() function is an alias for collector.push - used for component wiring.

startFlow Helper

See packages/collector/src/flow.ts for the startFlow function.

import { startFlow } from '@walkeros/collector';

const { collector, elb } = await startFlow({
  sources: {
    /* ... */
  },
  destinations: {
    /* ... */
  },
});

Separation of Concerns

Concern Handled By NOT Handled By
Event capture Sources Collector, Destinations
Event structure Event model Components
Consent checking Collector Sources, Destinations
Transformation Mapping system Raw push calls
Delivery Destinations Sources, Collector

Transformer Chains

Transformers run at two points in the pipeline, configured via next and before:

Pre-Collector Chain

Runs after source captures event, before collector processing:

sources: {
  browser: {
    code: sourceBrowser,
    next: 'validate'  // First transformer in pre-chain
  }
},
transformers: {
  validate: {
    code: transformerValidator,
    config: { next: 'enrich' }  // Chain continues
  },
  enrich: {
    code: transformerEnrich
    // No next = chain ends, event goes to collector
  }
}

Post-Collector Chain

Runs after collector enrichment, before destination receives event:

destinations: {
  gtag: {
    code: destinationGtag,
    before: 'redact'  // First transformer in post-chain
  }
},
transformers: {
  redact: {
    code: transformerRedact
    // Event then goes to destination
  }
}

Chain Resolution

  • source.next → starts pre-collector chain
  • transformer.config.next → links transformers together
  • destination.before → starts post-collector chain per destination

Skills:

Package READMEs:

Source Files:

Documentation: