Floke/Brancheneinstufung2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(connector): [31e88f42] Document webhook de-duplication shield

Browse Source 
Document the newly implemented de-duplication logic in the SuperOffice Connector README. This explains the problem of duplicate 'contact.created' webhooks from SuperOffice and how the worker now skips redundant jobs.
This commit is contained in:
Floke
2026-03-09 08:42:22 +00:00
parent 78461b0e71
commit de576e2a9a
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
connector-superoffice/README.md
View File

@@ -32,6 +32,12 @@ This directory contains Python scripts designed to integrate with the SuperOffic
* **Solution:** **Late Name Resolution**. The worker persists the resolved Company Name and Associate Shortname to the SQLite database as soon as it fetches them from SuperOffice. * **Solution:** **Late Name Resolution**. The worker persists the resolved Company Name and Associate Shortname to the SQLite database as soon as it fetches them from SuperOffice.
* **Status Priority:** Success (`COMPLETED`) now "outshines" subsequent ignored echos (`SKIPPED`). Once an account is green, it stays green in the dashboard cluster for 15 minutes. * **Status Priority:** Success (`COMPLETED`) now "outshines" subsequent ignored echos (`SKIPPED`). Once an account is green, it stays green in the dashboard cluster for 15 minutes.
#### 🛡️ D. Webhook De-Duplication (Contact Creation)
* **Problem:** SuperOffice occasionally sends *multiple* `contact.created` webhooks for the *same* new contact within a very short timeframe. This leads to duplicate processing jobs for the same entity.
* **Solution:** A **de-duplication shield** has been implemented in `worker.py` (March 2026).
* Before extensive processing, the worker now checks `connector_queue.db` via `queue_manager.check_for_recent_duplicate()`.
* If a `contact.created` event for the same company name is already `PROCESSING` or has been `COMPLETED` within the last 5 minutes, the current job is immediately `SKIPPED` as a duplicate.
### 3. Advanced API Handling (Critical Fixes) ### 3. Advanced API Handling (Critical Fixes)
* **OData Pagination:** We implemented `odata.nextLink` support (Manuel Zierl's advice) to correctly handle large result sets (>1000 records). * **OData Pagination:** We implemented `odata.nextLink` support (Manuel Zierl's advice) to correctly handle large result sets (>1000 records).