Commit Graph

86 Commits

Author SHA1 Message Date
2947312236 data_processor.py aktualisiert 2025-07-20 12:41:31 +00:00
df52c9ab7e data_processor.py aktualisiert 2025-07-20 12:37:45 +00:00
9771cabf55 data_processor.py aktualisiert 2025-07-20 10:43:42 +00:00
8dfe7d23ec großes rework, vieles gelöscht
- Refactors the website scraping batch process to fix critical stability issues.
- Replaces multiple redundant and conflicting scraping functions (`_scrape_website_task`, `_scrape_raw_text_task`, `_scrape_and_summarize_task`) with a single, robust worker function: `_scrape_website_task_batch`.
- The new worker function now consistently returns a structured dictionary, resolving the `TypeError` that prevented results from being written to the sheet.
- The main batch function `process_website_scraping_batch` is updated to correctly handle this new dictionary structure, including error states.
- Functionality is now aligned with the single-row processing mode by also fetching meta-details in the batch process, not just raw text.
- The two large, duplicated, and now obsolete `process_website_scraping` functions have been removed to improve code clarity and maintainability.
2025-07-20 09:18:49 +00:00
155675d827 data_processor.py aktualisiert 2025-07-20 08:49:15 +00:00
b0a7b8893a data_processor.py aktualisiert 2025-07-20 08:47:54 +00:00
4037656029 data_processor.py aktualisiert 2025-07-20 08:33:21 +00:00
7dbd8a59f2 data_processor.py aktualisiert 2025-07-20 08:05:15 +00:00
b38fcaa7fd data_processor.py aktualisiert 2025-07-20 07:57:06 +00:00
7b76cc09ef data_processor.py aktualisiert 2025-07-20 07:56:31 +00:00
5cef0b0260 data_processor.py aktualisiert 2025-07-20 07:52:24 +00:00
071be8a410 data_processor.py aktualisiert 2025-07-20 07:48:03 +00:00
a09746dadd data_processor.py aktualisiert 2025-07-20 07:46:35 +00:00
e8c901a081 data_processor.py aktualisiert 2025-07-20 07:40:30 +00:00
444e93e1d9 data_processor.py aktualisiert 2025-07-20 07:21:01 +00:00
9afb7888bd data_processor.py aktualisiert 2025-07-20 06:04:56 +00:00
eb83ccdf48 data_processor.py aktualisiert 2025-07-20 06:03:11 +00:00
82ccd46c7f data_processor.py aktualisiert 2025-07-20 05:26:00 +00:00
aeb5bb5ac1 data_processor.py aktualisiert 2025-07-20 04:47:04 +00:00
c77dd6938d data_processor.py aktualisiert 2025-07-20 04:28:51 +00:00
b8c3416854 data_processor.py aktualisiert 2025-07-19 20:25:35 +00:00
585f84c3bd data_processor.py aktualisiert 2025-07-19 20:24:50 +00:00
3cc4ec7f9a data_processor.py aktualisiert 2025-07-19 20:23:56 +00:00
3a8bff2d27 data_processor.py aktualisiert 2025-07-19 20:03:28 +00:00
9bd38bbaea data_processor.py aktualisiert 2025-07-19 19:56:18 +00:00
22afa4b1d2 Regex anpassung
COLUMN_MAP\["([^"]+)"\]
get_col_idx("$1")
2025-07-19 19:45:01 +00:00
f9cb3488da data_processor.py aktualisiert 2025-07-19 19:14:16 +00:00
7b066d794e data_processor.py aktualisiert 2025-07-19 18:42:41 +00:00
81d83a48f9 Anpassung reeval 2025-07-19 15:38:56 +00:00
cefda9c7c0 Anpassung Verify Wiki 2025-07-19 15:25:34 +00:00
0f6ba56071 data_processor.py aktualisiert 2025-07-18 18:09:22 +00:00
43855e3737 data_processor.py aktualisiert 2025-07-18 16:45:22 +00:00
a3087a245d data_processor.py aktualisiert 2025-07-18 15:49:47 +00:00
54afe00a0c data_processor.py aktualisiert 2025-07-18 15:45:05 +00:00
689fcfc6fe data_processor.py aktualisiert 2025-07-18 14:55:21 +00:00
e42461fdf0 data_processor.py aktualisiert 2025-07-18 14:40:30 +00:00
609bb2dc89 data_processor.py aktualisiert 2025-07-18 14:00:29 +00:00
4f1bc3fb84 data_processor.py aktualisiert 2025-07-18 13:59:34 +00:00
09bfa1ae81 data_processor.py aktualisiert 2025-07-18 13:50:31 +00:00
f79b2c0490 data_processor.py aktualisiert 2025-07-18 13:40:01 +00:00
c773244d02 data_processor.py aktualisiert 2025-07-18 13:34:26 +00:00
da4ad029f2 data_processor.py aktualisiert 2025-07-18 12:47:46 +00:00
1b1bd3eeb8 data_processor.py aktualisiert 2025-07-18 12:09:14 +00:00
6e876f7dba data_processor.py aktualisiert 2025-07-18 12:01:41 +00:00
b29b546eed bugfix 2025-07-18 11:21:25 +00:00
427f87f753 Wiki Verify update 2025-07-18 11:18:48 +00:00
11ef5ad339 Wiki verify update 2025-07-18 11:08:10 +00:00
1c7945d047 Batch mode Wiki Verify 2025-07-18 09:49:25 +00:00
a364232fa0 data_processor.py aktualisiert 2025-07-18 09:43:02 +00:00
2b9cc07cc1 data_processor.py aktualisiert 2025-07-18 09:41:21 +00:00