OB-CAS Persistency App v2
About
The Persistency Application v2 extends the persister v1 capabilities by introducing metric ingestion and alarm lifecycle management based on metric evaluation. This version focuses on consuming real-time metric data and alarm events from Kafka and persisting them into appropriate backend data stores for monitoring and analysis.
System Overview
Persister v2 follows an event-driven architecture, where metric data and alarm events are consumed asynchronously from Kafka topics and processed independently. The application ensures reliable persistence, real-time alarm handling, and proper alarm state transitions.
The Persistency Application v2 uses an event-driven architecture to persist metrics and manage alarm lifecycles.
Network elements or streamers publish metric data to Kafka metric topics. A metric processor evaluates these metrics and publishes alarm and clear events to Kafka alarm topics. Kafka acts as a reliable, scalable messaging backbone.
Persister v2 consumes metrics and alarms independently. Metrics are validated and stored as time-series data in OpenTSDB. Alarm events indicating abnormal conditions are stored in OpenSearch active_alarms index. When the condition clears, the alarm is removed from active_alarms and persisted in the history_alarms index.
This design ensures real-time ingestion, clear alarm lifecycle management, scalability, and efficient monitoring and historical analysis. Metric Persistence
The Persister v2 subscribes to metric data topics in Kafka. Each incoming metric event contains time-series data reported by network elements. Metrics are consumed from Kafka, validated, and normalized before being persisted into OpenTSDB as time-series data. OpenTSDB acts as the primary backend for metric storage and historical analysis, enabling efficient querying, aggregation, and visualization of network metrics over time.
Alarm Processing and Persistence
Persister v2 subscribes to alarm events published to Kafka. These alarms are generated by upstream components based on metric evaluation or threshold violations.
When an alarm event indicating an abnormal or undesired condition is received, the alarm is persisted in the active_alarms index in OpenSearch. These alarms represent ongoing issues that require attention.
When subsequent metric data indicates that the condition has returned to normal, a corresponding alarm-clear event is consumed from Kafka. The alarm is removed from the active_alarms index and persisted in the history_alarms index in OpenSearch. This ensures a clear separation between currently active alarms and historical alarm records.
Alarm Lifecycle Summary
Metric anomaly detected results in an alarm being generated and published to Kafka. The Persister V2 consumes the alarm event and stores it in the active_alarms index. Once the metric returns to a normal state, the alarm is cleared and moved to the history_alarms index.
Key Benefits of Persister v2
The application provides real-time metric ingestion and storage, clear alarm lifecycle management, scalable event-driven processing using Kafka, efficient time-series storage with OpenTSDB, and improved observability with historical analysis.
{
"deviceRefId": "onu1",
"serialNumber": "ABCD12345678",
"aniRefId": "an1",
"olt": "olt1",
"channelTermination": "olt1.ct1",
"splitter1": "olt1.ct1.sp.sp1",
"splitter2": "olt1.ct1.sp",
"vendor": "alcl",
"vomci": "vomci1",
"powerDistributionArea": "area1"
}