implement-connector
Implement a Python connector that conforms to the LakeflowConnect interface for data ingestion.
When & Why to Use This Skill
This Claude skill streamlines the development of Python-based data connectors by providing a structured framework for the LakeflowConnect interface. It automates the implementation of complex data ingestion logic, including schema mapping, metadata management, and efficient incremental loading, ensuring robust and standardized data pipelines for modern data architectures.
Use Cases
- Developing custom Python connectors to ingest data from niche SaaS APIs or internal databases into a centralized data lakehouse.
- Standardizing connector development across data engineering teams to ensure consistent error handling and metadata tracking using the LakeflowConnect interface.
- Accelerating the integration of new data sources by generating boilerplate code for required methods and schema definitions.
- Implementing efficient incremental data synchronization logic to reduce processing costs and improve data freshness in production environments.
| name | implement-connector |
|---|---|
| description | Implement a Python connector that conforms to the LakeflowConnect interface for data ingestion. |
Implement the Connector
Description
This skill helps you implement a Python connector for a data source that conforms to the LakeflowConnect interface. It guides you through implementing all required methods, handling schemas, metadata, and incremental data reading based on the source API documentation.
Instructions
Read and follow prompts/implement_connector.md