Notebook-to-Module Refactoring

Experiment Overview

Item	Details
Date	2025-12-27
Goal	Extract long functions from notebook cells into reusable Python modules
Environment	KINTSUGI Jupyter notebooks, Python modules in notebooks/ directory
Status	Success (after fixing)

Context

When notebook cells contain both:

1. Function definitions (candidates for extraction)
2. Variable definitions (processing parameters, configuration)

Extracting only the functions to a module leaves the variables undefined, breaking downstream cells.

Verified Workflow

CORRECT: Preserve Variable Definitions

When refactoring a cell that contains both functions and variables:

# BEFORE (single notebook cell):
# =============================================================================
# PROCESSING PARAMETERS
# =============================================================================
n_rows = 13
n_cols = 9
start_cycle = 1
end_cycle = 9
n_workers = CPU_COUNT

# =============================================================================
# CHANNEL NAME FUNCTIONS (to be extracted)
# =============================================================================
def load_channel_names(meta_dir):
    ...

def make_channel_names_unique(channel_dict):
    ...

# Usage
channel_name_dict = load_channel_names(meta_dir)
rows = list(...)  # Uses n_rows, n_cols

# AFTER (two notebook cells + module):

# --- Cell 1: Processing Parameters (KEEP IN NOTEBOOK) ---
n_rows = 13
n_cols = 9
start_cycle = 1
end_cycle = 9
n_workers = CPU_COUNT

# --- Cell 2: Module Import + Usage ---
from Kio import load_channel_names, make_channel_names_unique

channel_name_dict = load_channel_names(meta_dir)
rows = list(...)  # Still works - variables defined in Cell 1

# --- Kio.py module (NEW FILE) ---
def load_channel_names(meta_dir):
    ...
def make_channel_names_unique(channel_dict):
    ...

Checklist Before Refactoring

1. Identify ALL variable definitions in the cell
2. Identify ALL function definitions to extract
3. Check what the remaining code (after function removal) depends on
4. Create module with ONLY the functions
5. Keep variable definitions in notebook cell
6. Add import statement to notebook
7. Test that all downstream cells still work

Failed Attempts (Critical)

Attempt	Why it Failed	Lesson Learned
Moved entire cell content to module	Variables like n_workers, n_rows became undefined	Only move function definitions, keep variables
Replaced cell with just import	Lost 26 variable definitions	Cell content besides functions must be preserved
Edited project folder directly	Changes overwritten on sync	Always edit main repo first (repo-project-sync-workflow)

Variables Commonly Left Behind

When extracting channel name functions, these variables were accidentally removed:

Category	Variables
Grid config	n_rows, n_cols, rows, cols
Processing range	start_cycle, end_cycle, start_channel, end_channel, n_zplanes
HPC settings	n_workers, CPU_COUNT, IO_WORKERS, ZPLANES_PER_GPU, max_cores
Stitching	pou, overlap_percentage, initial_ncc_threshold
BaSiC params	BASIC_IF_DARKFIELD, BASIC_MAX_ITERATIONS, etc.
Blend mode	BLEND_MODE, BLEND_SIGMA

Key Insights

• Notebook cells often mix configuration and implementation
• Function extraction should be surgical - only the def blocks
• Variable definitions are cell-level configuration, not module content
• Always verify imports work by running the refactored cells
• Use /advise before refactoring to check for related skills

Related Skills

• repo-project-sync-workflow - Edit main repo first, then sync
• channel-name-parsing - The functions that were extracted

References

• KINTSUGI notebooks/Kio.py - Channel name I/O module
• KINTSUGI notebooks/2_Cycle_Processing.ipynb - Cell 7 refactoring