- Implemented Impressum scraping with Root-URL fallback and enhanced keyword detection.
- Added 'clean_json_response' helper to strip Markdown from LLM outputs, preventing JSONDecodeErrors.
- Improved numeric extraction for German formatting (thousands separators vs decimals).
- Updated Inspector UI with Polling logic for auto-refresh and display of AI Dossier and Legal Data.
- Added Manual Override for Website URL.
- Ported robust Wikipedia extraction logic (categories, first paragraph) from legacy system.
- Implemented database-driven Robotics Category configuration with frontend settings UI.
- Updated Robotics Potential analysis to use Chain-of-Thought infrastructure reasoning.
- Added Manual Override features for Wikipedia URL (with locking) and Website URL (with re-scrape trigger).
- Enhanced Inspector UI with Wikipedia profile, category tags, and action buttons.
This commit introduces the foundational elements for the new "Company Explorer" web application, marking a significant step away from the legacy Google Sheets / CLI system.
Key changes include:
- Project Structure: A new directory with separate (FastAPI) and (React/Vite) components.
- Data Persistence: Migration from Google Sheets to a local SQLite database () using SQLAlchemy.
- Core Utilities: Extraction and cleanup of essential helper functions (LLM wrappers, text utilities) into .
- Backend Services: , , for AI-powered analysis, and logic.
- Frontend UI: Basic React application with company table, import wizard, and dynamic inspector sidebar.
- Docker Integration: Updated and for multi-stage builds and sideloading.
- Deployment & Access: Integrated into central Nginx proxy and dashboard, accessible via .
Lessons Learned & Fixed during development:
- Frontend Asset Loading: Addressed issues with Vite's path and FastAPI's .
- TypeScript Configuration: Added and .
- Database Schema Evolution: Solved errors by forcing a new database file and correcting override.
- Logging: Implemented robust file-based logging ().
This new foundation provides a powerful and maintainable platform for future B2B robotics lead generation.
- Added documentation for Notion setup and resources (notion_integration.md).
- Added scripts for authentication test, database creation, and product insertion.
- Successfully tested connection and data mapping for 'RoboPlanet Product Master'.
- Added section on 'Hardware / DSL-Leitung' explaining the detected 17m branch line (Stichleitung) causing signal reflections and timeouts.
- Clarified that 'IP Flapping' was caused by stale caching of the previous day's IP, exacerbated by incorrect DNS settings (8.8.8.8).
- Finalized troubleshooting steps for Synology and FritzBox configuration.
- Documented the critical finding of hardcoded DNS (8.8.8.8) in Synology Network Interface settings overriding global DSM settings.
- Added comprehensive checklist for Synology, FritzBox, and Client configuration to ensure DNS stability and NAT loopback functionality.
- Updated log analysis section with 'NETWORK_WAIT' and 'ALERT' scenarios.
- Updated 'duckdns_setup.md' with critical instructions for configuring FritzBox DNS Rebind Protection to enable internal access (NAT Loopback).
- Added steps for clearing client-side DNS cache (Windows).
- Included Cloudflare DNS switch in documentation reflecting recent monitor script changes.
- Added 'Log_from_docker/' to ignore runtime logs.
- Added 'diag_venv/' to ignore temporary diagnostic environments.
- Added '*.db' to exclude local SQLite databases from version control.
- Updated 'docker-compose.yml' with new DuckDNS token to lock out rogue updaters (FritzBox/HA).
- Enhanced 'dns-monitor/monitor.sh' to detect 'IP Flapping' (Zombie Updaters) by comparing Public vs. Global vs. Local IPs.
- Updated documentation 'duckdns_setup.md' with troubleshooting steps for IP flapping and zombie updaters.
- Changes the model ID from the generic 'gemini-1.5-flash' to the specific 'gemini-1.5-flash-001'.
- This resolves the '404 NOT_FOUND' error where the API could not locate the generic model alias in the v1beta API version.
- Rewrites and in helpers.py to use the new and syntax.
- Removes the deprecated calls which were causing AttributeError.
- Updates parameter handling to use .
- This change ensures full compatibility with the library version 1.0+.
- Reverts docker-compose.yml to use volume mount for gemini_api_key.txt, due to format constraints.
- Restores and refines Config.load_api_keys() in config.py.
- **Crucially, calls Config.load_api_keys() at the start of gtm_architect_orchestrator.py to ensure keys are loaded.**
- Adjusts _get_gemini_api_key in helpers.py to prioritize keys from Config.API_KEYS.
- This definitively addresses the 'API Key missing' error by guaranteeing key initialization.
- Modifies docker-compose.yml to use for injecting the Gemini API key, which is more robust than volume mounting.
- Updates helpers.py to prioritize reading the API key from the environment variable.
- Removes the now-redundant file-based key loading logic from config.py and the Dockerfile.
- This change completely bypasses the problematic file system interactions within the container, providing a definitive fix for the 'API Key missing' error.
- Adds a prominent warning section to the migration guide.
- Explains that local volume mounts (sideloading) will overwrite container code.
- Explicitly states that Already up to date. on the host is mandatory before rebuilding to ensure changes are applied. This addresses a recurring debugging issue.
- Adds a version and timestamp to the orchestrator's startup logs to verify code deployment.
- Introduces extensive debug logging in config.py and helpers.py to trace the API key loading process, including exact file paths and environment variable checks. This will help diagnose the persistent 'API Key missing' error.
- Changes the expected API key filename from 'api_key.txt' to 'gemini_api_key.txt' to match the project standard.
- Enhances logging to output the absolute path when an API key file is not found, simplifying future debugging.
- Updates requirements.txt to use the new 'google-genai' package.
- Updates import statements and error messages in helpers.py to use the new library.
- Ensures the gemini_api_key.txt is copied into the Docker container to resolve API key errors.
- Refactors the gtm-architect Dockerfile for a flat, more efficient build process.
- Implements robust web scraping via BeautifulSoup in helpers.py for URL analysis in phase1.
- Makes shared library imports (gspread, pandas, etc.) in helpers.py optional to prevent ModuleNotFoundErrors in microservices.
- Implements the main execution logic in the orchestrator to handle command-line arguments.
- Updates documentation to reflect the new architecture, scraping feature, and dependency handling.
