- Updates the main features section in `lead-engine/README.md` to reflect multi-source ingestion, UI indicators, and persistent draft storage.
- Adds a new "Roadmap / Nächste Schritte" section to document open To-Dos and future enhancements.
- Adjusts the version to 1.1 to signify the new capabilities.
This commit finalizes the centralization of lead parsing logic.
- Moves from to .
- Moves from to .
- Ensures is imported in for generation.
- Corrects all necessary imports in to use functions from .
This addresses the and improves modularity.
This commit introduces two key improvements to the Lead Engine:
1. **Persistent Email Drafts:**
- Adds a new function to .
- Modifies to save generated email replies directly to the column in the database, ensuring they persist across sessions.
- Removes the previous session-based state for drafts.
2. **Enhanced UI Visibility:**
- Adds a warning icon (⚠️) directly to the lead expander's title if a lead is flagged as low-quality, making it easier to spot.
This commit resolves an `ImportError` by centralizing all lead parsing logic into `ingest.py`.
- **Move `parse_tradingtwins_html`:** The function for parsing TradingTwins HTML emails has been moved from `trading_twins_ingest.py` to `ingest.py`.
- **Move `is_free_mail`:** The utility function `is_free_mail` was also moved from `trading_twins_ingest.py` to `ingest.py` to be shared by all parsers.
- **Update Imports:** The import statement in `trading_twins_ingest.py` is now corrected to import all necessary functions from the central `ingest.py` module.
This refactoring improves code structure, removes redundancy, and fixes the critical bug that prevented the ingest process from running.
This commit integrates the Roboplanet website contact form submissions into the Lead Engine, allowing them to be processed alongside TradingTwins leads.
Key changes:
- **Database Schema Update (db.py):** Added a new source column to the leads table for tracking lead origin (TradingTwins or Website-Formular). Includes a migration check to safely add the column.
- **Improved HTML Parsing (ingest.py):** Refined the `parse_roboplanet_form` function to accurately extract data from the specific HTML structure of Roboplanet contact form emails.
- **Enhanced Ingestion Logic (trading_twins_ingest.py):**
- Renamed `fetch_tradingtwins_emails` to `fetch_new_leads_emails` and updated it to fetch emails from both lead sources.
- Modified `process_leads` to dynamically select the correct parser based on email subject.
- Ensured `source` field is correctly populated and `is_low_quality` checks are applied for both lead types.
- **UI Enhancement (app.py):** Updated the Streamlit UI to visually distinguish lead types with icons and improved the "Low Quality Lead" warning message.
This feature enables a unified processing pipeline for different lead sources and provides better visibility in the Lead Engine dashboard.
