Task #6116
openEPIC: Speed up the stats updater daemon (scripts/workflow/update_stats.sh)
79%
Description
Source: sdlc/0-vibes/raw/2026-08-26/slow_ daemon.md — the infinite bash loop in scripts/workflow/update_stats.sh that chains 10 CLI commands, each a separate JVM launch, forever.
Where the time actually goes¶
The round is dominated by IIKO OLAP traffic, and the traffic volume is a product of four independent multipliers, each of which is fixable on its own:
| Multiplier | Where | Factor |
|---|---|---|
| 3 HTTP calls per logical request (auth → olap → logout) | IikoOlapV2RequestsFetchingService.fetchOlapData:35,56 |
×3 |
| one OLAP request per day instead of one per range | IikoStatsGatheringService.getMetricValuesFromIikoDateDesc:117 |
×365 |
| full 365-day window re-downloaded every round, nothing checks what is already stored | same, plus all three metric commands | ×∞ (repeats forever) |
| strictly serial across companies / partners / users |
InitialStatsGatheringCommand, PartnersMetricsGatheringCommand, UsersMetricsGatheringCommand
|
×N |
Order of magnitude for iiko-stats-data-sync alone with N companies:
6 metrics × (1 network + N companies) × 365 days × 3 HTTP calls. At N=200 that is ~1.3M HTTP round-trips per round, all serialised, against an IIKO server that licenses a small number of concurrent API sessions.
On top of that, each of those requests writes its full request+response JSON to iiko_requests twice (INSERT then UPDATE) and printlns the whole payload to nohup.out.
Subtasks¶
Ordered by payoff. The first three are multiplicative — doing all three is what turns a multi-hour round into a short one.
The "IIKO is single-threaded" symptom is mostly self-inflicted: the code opens and closes an IIKO API session for every single request, so the session limit is hit constantly by our own login/logout churn rather than by concurrent work.