GLog — Implementation Plan (final_version_tech.md realization)

Derived from docs/final_version_tech.md (primary source of truth) reconciled with the existing Spring Boot backend (secondary). Strategy: evolve the existing service, same repo, incremental increments. This document is the authoritative build plan; keep it in sync as increments land.

1. Context

• Current glog = backend-only Spring Boot (Java 17, non-JHipster). Already implements: immutable SHA-256 hash-chain integrity, REST + AMQP (RabbitMQ) ingest, tenant guard, idempotency, Postgres + JSONB details, RFC7807 errors, JWT/Keycloak security, a forward-outbox dispatcher (WebhookSink → SIEM/OpenSearch, MLog sink stub).
• Ecosystem: GRegistry holds the InformationSystem catalog + typed SystemDependency graph and already names GLog as a consumer (“resolve source system code + metadata”). saas_gnotify (JHipster) is the mature GNotify (/api/v1/notify mTLS). gstyle is the shared GovStack Design System (@esempla/gds-angular).
• The doc adds (not yet built): service-aware event schema, per-service queues + worker pool, MinIO archive, Postgres full-text search + dashboards, standalone Angular UI, correlated cross-service search via the GRegistry graph, and a GNotify alerting rule engine.

2. Locked decisions

#	Decision	Choice
1	Build strategy	Evolve the existing backend, same repo, incremental
2	Correlated-search model	Centralized store; correlate inside GLog via correlation_id + GRegistry dependency graph
3	Frontend	Standalone Angular + GDS (@esempla/gds-angular); backend stays pure API
4	Search engine	Postgres-first (tsvector + GIN + B-tree) behind a SearchPort; OpenSearch deferred
5	Object storage	Batched compressed NDJSON.gz rollups → MinIO behind a StoragePort (GStorage-swappable)
6	Queues/workers	RabbitMQ topic exchange, per-service routing keys → per-service queues + worker pool, DLX kept
7	Integrity vs concurrency	Serialize per chain-partition with Postgres advisory lock; chain key = (tenant, service); chain stays mandatory
8	Event schema	Extend: add service, module, operation, actorType, resourceType, sessionId, status; keep existing fields + hash chain; service = GRegistry InformationSystem.code
9	Alerting	Admin-configurable rule engine → GNotify, delivered via existing forward-outbox
10	GRegistry access	Sync + cache catalog + dependency graph locally, periodic refresh (+ DEPENDENCY_CHANGE events)
11	GNotify target	saas_gnotify (JHipster) over mTLS /api/v1/notify
12	Sequencing	Backend foundation first, UI after a correct backend

3. Target architecture

Producers (SDKs/sidecars)                Admin/Operators
   │ REST  │ AMQP(topic: service.*)            │
   ▼       ▼                                    ▼
GLog API ──► topic exchange ──► q.<service> ──► worker pool
                                                 │  (advisory lock per (tenant,service))
                                                 ├─► IntegrityService (hash chain)
                                                 ├─► Postgres (queryable truth: new cols + tsvector/GIN)
                                                 ├─► Archive batcher ──► MinIO  (service/date/hour.ndjson.gz)
                                                 └─► Forward outbox ──► WebhookSink / MLog / GNotify
GRegistry ──(sync)──► local catalog+dep cache ──► correlated search expands related services
Rule engine ──(on ingest + windowed agg)──► outbox ──► saas_gnotify (mTLS)
Angular + GDS UI ──► /api/v1 (right-side sidenav, expandable rows, correlated search, dashboards)

Seams (swap without touching callers, mirrors existing ForwardSink)

• SearchPort — Postgres impl now; OpenSearch later.
• StoragePort — MinIO impl now; GStorage later.
• NotifierPort — saas_gnotify impl; alt providers later.

4. Increment plan

Inc 1 — Schema + service identity ✅ DONE (46/46 tests green, incl. Postgres/RabbitMQ Testcontainers)

• New append-only Liquibase changelog 004-event-service-fields.xml: add service, module, operation, actor_type, resource_type, session_id, status columns.
• Extend AuditEventIngestRequest / AuditEventResponse; service bound to the authenticated principal (JWT claim / registration), not a free client field.
• Validate service against synced GRegistry codes (soft in Inc 1, hard once Inc 7 lands).
• Repartition the hash chain to (tenant, service) + one-time reseal migration of existing rows; IntegrityService + verify updated accordingly.
• Backfill service for existing rows (from tenant→service mapping / default).

Inc 2 — Queues + worker pool ✅ DONE

• RabbitMQ topic exchange, routing key = service; per-service queues (+ # default); DLX retained; concurrent listener container (N consumers). Testcontainers coverage.

Inc 3 — Chain serialization ✅ DONE (concurrency proof: 200 parallel ingests, 0 chain breaks)

• pg_advisory_xact_lock(hash(tenant,service)) around read-tip → compute → insert.
• Parallel across services, serial within one. Concurrency test proving no chain forks.

Inc 4 — MinIO archive

• StoragePort + MinIO adapter; batcher rolls up per (service, hour) into NDJSON.gz at service/yyyy/MM/dd/HH.ndjson.gz; export/import endpoints. docker-compose gains MinIO.

Inc 5 — Search

• SearchPort + Postgres impl: tsvector on message/details, GIN on details, B-tree on (timestamp, service, severity, status). Filter API (keyword, date range, service, severity, status) + dashboard aggregation endpoints (counts by service/severity/error/IP over time).

Inc 6 — Angular + GDS UI

• Standalone SPA: right-side sidenav, expandable/tabbed table rows (view/edit), general search page, per-service/log search, dashboards (GDS chart/kpi-row), admin config zone. JWT → /api/v1.

Inc 7 — GRegistry sync + correlated search

• Local catalog + SystemDependency cache (scheduled pull + DEPENDENCY_CHANGE event).
• Correlated-search API: expand a service to related systems, return a stitched, correlation_id-grouped timeline; the “why don’t I see X” investigation page.

Inc 8 — GNotify rule engine

• Admin rule CRUD (condition: severity threshold / failure-rate window / action match → recipients + template); evaluator on ingest + windowed aggregation; delivery via existing outbox → saas_gnotify mTLS.

5. Risks

• Chain repartition (Inc 1) touches persisted data — needs a correct one-time reseal or verify breaks.
• service provenance must be authenticated (claim/registration), not client-set, or the partition/correlation is spoofable. Bind like tenantId today.
• mTLS to saas_gnotify needs PKI/cert material not in the repo — Inc 8 stubs transport until certs exist (same posture as the MLog sink).