[31388f42] Feature: Integrate Roboplanet Contact Forms into Lead Engine

This commit integrates the Roboplanet website contact form submissions into the Lead Engine, allowing them to be processed alongside TradingTwins leads.

Key changes:
- **Database Schema Update (db.py):** Added a new source column to the leads table for tracking lead origin (TradingTwins or Website-Formular). Includes a migration check to safely add the column.
- **Improved HTML Parsing (ingest.py):** Refined the `parse_roboplanet_form` function to accurately extract data from the specific HTML structure of Roboplanet contact form emails.
- **Enhanced Ingestion Logic (trading_twins_ingest.py):**
    - Renamed `fetch_tradingtwins_emails` to `fetch_new_leads_emails` and updated it to fetch emails from both lead sources.
    - Modified `process_leads` to dynamically select the correct parser based on email subject.
    - Ensured `source` field is correctly populated and `is_low_quality` checks are applied for both lead types.
- **UI Enhancement (app.py):** Updated the Streamlit UI to visually distinguish lead types with icons and improved the "Low Quality Lead" warning message.

This feature enables a unified processing pipeline for different lead sources and provides better visibility in the Lead Engine dashboard.

lead-engine/db.py

@@ -35,13 +35,18 @@ def init_db():
         )
     ''')
     
-    # Simple migration check: check if lead_metadata column exists
-    try:
-        c.execute('SELECT lead_metadata FROM leads LIMIT 1')
-    except sqlite3.OperationalError:
+    # Simple migration check: add 'lead_metadata' if not exists
+    c.execute("PRAGMA table_info(leads)")
+    columns = [row[1] for row in c.fetchall()]
+    
+    if 'lead_metadata' not in columns:
         print("Migrating DB: Adding lead_metadata column...")
         c.execute('ALTER TABLE leads ADD COLUMN lead_metadata TEXT')
         
+    if 'source' not in columns:
+        print("Migrating DB: Adding source column...")
+        c.execute('ALTER TABLE leads ADD COLUMN source TEXT')
+        
     conn.commit()
     conn.close()
 
@@ -71,8 +76,8 @@ def insert_lead(lead_data):
     c = conn.cursor()
     try:
         c.execute('''
-            INSERT INTO leads (source_id, received_at, company_name, contact_name, email, phone, raw_body, lead_metadata, status)
-            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+            INSERT INTO leads (source_id, received_at, company_name, contact_name, email, phone, raw_body, lead_metadata, status, source)
+            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
         ''', (
             lead_data.get('id'),
             received_at,
@@ -82,7 +87,8 @@ def insert_lead(lead_data):
             lead_data.get('phone'),
             lead_data.get('raw_body'),
             json.dumps(meta),
-            'new'
+            'new',
+            lead_data.get('source') # Added source
         ))
         conn.commit()
         return True