This commit introduces two key improvements to the Lead Engine:
1. **Persistent Email Drafts:**
- Adds a new function to .
- Modifies to save generated email replies directly to the column in the database, ensuring they persist across sessions.
- Removes the previous session-based state for drafts.
2. **Enhanced UI Visibility:**
- Adds a warning icon (⚠️) directly to the lead expander's title if a lead is flagged as low-quality, making it easier to spot.
This commit resolves an `ImportError` by centralizing all lead parsing logic into `ingest.py`.
- **Move `parse_tradingtwins_html`:** The function for parsing TradingTwins HTML emails has been moved from `trading_twins_ingest.py` to `ingest.py`.
- **Move `is_free_mail`:** The utility function `is_free_mail` was also moved from `trading_twins_ingest.py` to `ingest.py` to be shared by all parsers.
- **Update Imports:** The import statement in `trading_twins_ingest.py` is now corrected to import all necessary functions from the central `ingest.py` module.
This refactoring improves code structure, removes redundancy, and fixes the critical bug that prevented the ingest process from running.
This commit integrates the Roboplanet website contact form submissions into the Lead Engine, allowing them to be processed alongside TradingTwins leads.
Key changes:
- **Database Schema Update (db.py):** Added a new source column to the leads table for tracking lead origin (TradingTwins or Website-Formular). Includes a migration check to safely add the column.
- **Improved HTML Parsing (ingest.py):** Refined the `parse_roboplanet_form` function to accurately extract data from the specific HTML structure of Roboplanet contact form emails.
- **Enhanced Ingestion Logic (trading_twins_ingest.py):**
- Renamed `fetch_tradingtwins_emails` to `fetch_new_leads_emails` and updated it to fetch emails from both lead sources.
- Modified `process_leads` to dynamically select the correct parser based on email subject.
- Ensured `source` field is correctly populated and `is_low_quality` checks are applied for both lead types.
- **UI Enhancement (app.py):** Updated the Streamlit UI to visually distinguish lead types with icons and improved the "Low Quality Lead" warning message.
This feature enables a unified processing pipeline for different lead sources and provides better visibility in the Lead Engine dashboard.
- Aktualisierung der Pains/Gains für Energy, Retail und Tech in der lokalen DB.
- Ergänzung von Secondary Products für Energy & Tech.
- Aufbau eines permanenten Werkzeugkastens unter /app/devtools/ für DB- & Notion-Tasks.
- Verifikation der manuellen Notion-Änderungen mittels neuer Maintenance-Skripte.
✦ Das Skript 'dev_session.py' wurde grundlegend überarbeitet, um den '#fertig'-Workflow zu automatisieren.
✦ Die Zusammenfassung für Notion wird nun auf Basis des Chatverlaufs erstellt, nicht mehr durch manuelle Eingabe oder Git-Diffs.
✦ Der Notion-Status wird nicht mehr automatisch geändert; dies bleibt eine manuelle Aktion.
✦ Geänderte Dokumentationsdateien werden nun automatisch mit einer separaten Nachricht committet.
