What is speech-recognition?

This Claude skill provides a specialized implementation and debugging guide for real-time iOS speech recognition using @react-native-voice/voice. It addresses critical technical challenges such as accumulated text tracking, silence detection, and robust session management to ensure stable, high-performance transcription in React Native applications.

When should I use speech-recognition?

speech-recognition is useful in the following scenarios: • Debugging transcription issues where previous text persists or duplicates in new speech sessions due to iOS Speech Framework behavior. • Implementing a custom silence detection mechanism (e.g., 2000ms timeout) to automatically finalize and segment real-time speech input. • Resolving application crashes caused by improper Voice.start() calls or race conditions during speech recognition restarts. • Developing a reliable text tracking logic that accurately extracts new speech from the accumulated result buffer provided by the native iOS API.

name	speech-recognition
description	iOS speech recognition implementation using @react-native-voice/voice. Use when debugging transcription issues, modifying session handling, or understanding the accumulated text tracking mechanism.

Speech Recognition Implementation

Overview

The useSpeechRecognition hook in mobile/src/hooks/useSpeechRecognition.ts handles real-time Japanese speech recognition on iOS.

Key Concepts

iOS Voice API Behavior

@react-native-voice/voice wraps iOS Speech Framework
Accumulated text: iOS returns ALL text since Voice.start(), not just new words
onSpeechEnd fires when iOS decides speech is complete (unpredictable timing)
No native "continuous" mode - must manually restart

Session Management

┌─────────────────────────────────────────────────────┐
│  Voice.start()                                      │
│       ↓                                             │
│  onSpeechPartialResults → Update current session    │
│       ↓                                             │
│  [2 sec silence] → Finalize session, track length   │
│       ↓                                             │
│  New speech → Extract only NEW text (subtract old)  │
│       ↓                                             │
│  onSpeechEnd → Reset counter, restart Voice         │
└─────────────────────────────────────────────────────┘

Critical Implementation Details

1. Accumulated Text Tracking

const lastFinalizedTextLengthRef = useRef(0);

// When session finalizes (silence detected)
lastFinalizedTextLengthRef.current += current.text.length;

// Extract new text from accumulated result
const extractNewText = (fullText: string): string => {
  if (lastFinalizedTextLengthRef.current === 0) return fullText;
  return fullText.substring(lastFinalizedTextLengthRef.current).trim();
};

2. Silence Detection

const SILENCE_TIMEOUT_MS = 2000;

silenceTimerRef.current = setTimeout(() => {
  // Finalize current session
  setTranscripts(prev => prev.map(t =>
    t.id === currentTranscriptIdRef.current
      ? { ...t, isFinal: true }
      : t
  ));
  currentTranscriptIdRef.current = null;
}, SILENCE_TIMEOUT_MS);

3. Auto-Restart on Speech End

const onSpeechEnd = useCallback(() => {
  // Reset counter when iOS naturally ends (buffer cleared)
  lastFinalizedTextLengthRef.current = 0;

  if (isListeningRef.current) {
    restartRecognition(0);
  }
}, [restartRecognition]);

Visual Indicators

In Transcription.tsx:

Blue border: Active session (!entry.isFinal)
"入力中" label: Speech in progress
Border disappears when session finalizes

Common Issues & Solutions

Issue	Cause	Solution
Previous text in new session	Length not accumulated	Use `+=` not `=` for lastFinalizedTextLengthRef
App crash on Voice restart	Double Voice.start()	Don't restart Voice in silence timer - use text tracking
"Speech recognition already started"	Multiple start calls	Check isListening before Voice.start()
Empty sessions created	Empty text not filtered	Add `if (!transcript.trim()) return`

Debug Logging

Key log messages to watch:

"Finalized text length (累積): X"  → Length accumulating correctly
"Session finalized, ready for new input" → Session closed
"Auto-restarting speech recognition..." → Voice restarting after onSpeechEnd

Important: Do NOT

Restart Voice during silence timer (causes crashes)
Replace lastFinalizedTextLengthRef (must accumulate)
Ignore empty text results (creates phantom sessions)

speech-recognition

When & Why to Use This Skill

Use Cases