81 lines
2.9 KiB
Markdown
81 lines
2.9 KiB
Markdown
# 🤖 Automation Workflow Design (n8n + Python)
|
|
|
|
This document outlines the architecture for the **TradingTwins Lead Engine** automation, combining **n8n** (Trigger/Flow Control) and **Python** (Business Logic/Sync).
|
|
|
|
## 1. High-Level Architecture
|
|
|
|
We use a **Hybrid Approach**:
|
|
* **n8n:** Handles connectivity (IMAP, Slack, CRM) and triggers.
|
|
* **Python Lead Engine:** Handles data processing, parsing, and syncing with Company Explorer.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Mail as 📧 Email Server (IMAP)
|
|
participant n8n as ⚡ n8n Workflow
|
|
participant API as 🐍 Lead Engine API
|
|
participant DB as 🗄️ SQLite (leads.db)
|
|
participant CE as 🏢 Company Explorer
|
|
|
|
Mail->>n8n: New Email (Subject: "Lead...")
|
|
n8n->>n8n: Extract Body & Subject
|
|
n8n->>API: POST /api/ingest { "body": "...", "subject": "..." }
|
|
|
|
rect rgb(240, 248, 255)
|
|
Note right of API: Python Async Process
|
|
API->>DB: Save Lead (Status: 'new')
|
|
API-->>n8n: 200 OK (Lead Queued)
|
|
|
|
API->>CE: Check Existence / Create
|
|
CE-->>API: Company ID
|
|
API->>CE: Trigger Discovery
|
|
API->>DB: Update Status: 'synced'
|
|
end
|
|
|
|
API->>n8n: (Optional) Webhook "Sync Complete"
|
|
n8n->>Slack: Notify Sales "New Lead Synced!"
|
|
```
|
|
|
|
## 2. n8n Workflow Specification
|
|
|
|
**Trigger:** `IMAP Read` Node
|
|
* **Filter:** Subject contains "TradingTwins" OR Sender is "notification@tradingtwins.com"
|
|
* **Interval:** Every 5 minutes (or 1 min).
|
|
|
|
**Action:** `HTTP Request` Node
|
|
* **Method:** POST
|
|
* **URL:** `http://lead-engine:8000/api/ingest` (Internal Docker Network)
|
|
* **Body:** JSON
|
|
```json
|
|
{
|
|
"source": "tradingtwins",
|
|
"subject": "New Lead: Musterfirma GmbH",
|
|
"raw_body": "..."
|
|
}
|
|
```
|
|
|
|
## 3. Python Lead Engine Changes
|
|
|
|
To support this, the current Streamlit-only application needs a **FastAPI backend** (or similar) running alongside or instead of the Streamlit loop for API access.
|
|
|
|
### Requirements:
|
|
1. **Split `app.py`:** Separate UI (Streamlit) from Logic/API.
|
|
2. **New `api.py`:** FastAPI instance exposing `POST /api/ingest`.
|
|
3. **Refactor `enrich.py`:** Ensure `run_sync` can be called programmatically for a specific lead ID.
|
|
|
|
### Proposed File Structure:
|
|
```
|
|
lead-engine/
|
|
├── api.py # FastAPI (New)
|
|
├── ui.py # Streamlit (Renamed from app.py)
|
|
├── core/
|
|
│ ├── db.py # Shared DB access
|
|
│ ├── enrich.py # Sync Logic
|
|
│ └── parser.py # Parsing Logic
|
|
├── Dockerfile # Needs to run both (e.g. via supervisord or entrypoint script)
|
|
```
|
|
|
|
## 4. Why this approach?
|
|
* **Reliability:** n8n is better at keeping IMAP connections alive than a custom Python loop.
|
|
* **Scalability:** We can easily add other lead sources (e.g., Typeform, LinkedIn) just by adding another n8n trigger pointing to the same Python API.
|
|
* **Focus:** Python code focuses purely on "What to do with the data", not "How to get the data".
|