Task #6123
openTask #6116: EPIC: Speed up the stats updater daemon (scripts/workflow/update_stats.sh)
Gather metrics for companies/partners/users with a bounded worker pool
80%
Description
Problem¶
The three metric commands iterate strictly serially:
-
InitialStatsGatheringCommand—for (metricTitle …) { for (company in companies) { … } } -
PartnersMetricsGatheringCommand—for (metricService …) { for (partner in partners) { … } } -
UsersMetricsGatheringCommand— same shape over users
Each inner iteration is a network-bound call chain, so the CPU is idle almost the whole time and the wall clock is the sum of every request.
Proposed change¶
Run the inner loop over a bounded pool (a fixed executor / coroutine Semaphore, size from config, default small — 4 to 8) so several companies are in flight at once.
Depends on the session-reuse task. Parallelising the current code would multiply login/logout churn and hit IIKO's session limit immediately; with one shared session the concurrency budget goes to actual report requests. Size the pool below IIKO's licensed concurrent-request limit and make it configurable so it can be tuned down if IIKO starts refusing.
Also set OkHttp's Dispatcher.maxRequestsPerHost on the shared client in NetworkRequestsService — it defaults to 5 and would otherwise silently cap the pool.
Per-item failures must stay isolated the way they are today (the existing try/catch … continue per company).
Acceptance¶
- Pool size configurable, default conservative.
- A failing company does not abort the run.
- Measured wall-clock improvement recorded on the issue.