Automating Pattern Detection using Machine Learning for Telecom

Abstract

Consistent asset health across various levels, from cell sites to regions, is crucial for telecom carriers to ensure ongoing operations. However, proactively detecting aberrant patterns such as equipment breakdown remains a significant problem. This article, based on Subex’s extensive implementation expertise, covers the key processes required to instal effective pattern detection solutions. It also looks at certain high-impact pattern identification use cases that have consistently generated value for telecom operators.

Introduction

High operational efficiency is a key distinction for telecom operators that want to build their brand name on consistency and service availability. Unusual consumption patterns in the call, Internet, and SMS services might reveal underlying concerns that could lead to greater ones. Consider how variations caused by power outages, technological problems, or competition growth might result in SLA breaches and revenue loss. To keep ahead of operational interruption, area managers must regularly monitor key performance indicators (KPIs) across many cell sites, clusters, and regions.

To detect abnormalities, most CSPs rely on manual and reactive methods. Any attempt to delve into the fundamental cause and take corrective action necessitates the usage of standard dashboards or the perusal of long performance analytics reports. These methods are prone to errors and frequently fail to account for shifting patterns, seasonality, and discrepancies in the data.

Automating pattern detection

Automated machine learning technologies can assist carriers in addressing the aforementioned difficulties. Data-centric frameworks are used to collect and cleanse historical data, which is then automatically fed into machine learning models that can learn from big datasets. This reveals tendencies and, as a result, forecasts daily consumption levels throughout each geographical area.

Consistent tracking is an important component of an automated system. Within a certain location, all deviations of real KPIs from predefined values are calculated. Significant deviations or abnormalities are immediately reported in a dashboard for prompt repair by the relevant area manager.

Implementation approach

For huge data, Apache Hadoop boasts industry-leading distributed processing capabilities. As a result, Hadoop is an effective tool for converting XDRs into trainable datasets that can be fed into powerful machine-learning algorithms to detect abnormalities. These anomalies in telecom pertain to spikes and decreases in everyday usage. The following are the major steps of a machine learning-driven pattern identification solution:

Step 1:Ingest the data — An automated process is configured to transmit data from numerous sources to Apache Hive (data warehouse) via real-time streaming systems that collect information such as session length, charges received, location, and so on. To build analytical datasets, this granular data must be analysed, wrangled, and organised.

Step 2:Select an algorithm — When developing the framework, it is critical to select the appropriate algorithm, one that scales well and solves data complexity. When comparing several forecasting approaches, some essential criteria to consider are:

• • Capability to identify outliers and missing data
  • Capability to learn at speed and scale as needed
  • Handling significant changes in time series
  • Integration and automation capabilities
  • Customizability and interpretability

Following a thorough examination of regression models such as autoregressive integrated moving averages (ARIMA), Holt-Winters, and others, it was discovered that Facebook’s Prophet algorithm meets the aforementioned requirements and can be implemented fast. It enables users to quickly personalise predictions. The model may also be supplied with domain information via human-interpretable parameters, enhancing forecast accuracy even more. Subex has exhaustively analysed the performance of Prophet after working with a massive quantity of real-world telecom data and discovered that it is 8-10% more accurate than traditional methodologies.

Step 3:Deploy the model – Once the Prophet algorithm has been developed in R, the input data streams must be set up using Hadoop to R connectors. To keep up with the newest developments and trends, the model should be updated daily with consumption statistics, data from each cell site, and so on.

The cell sites can be organised into clusters or regions and examined accordingly to establish the hyperparameters. Alternatively, hyperparameters for groups based on cell site category (2G, 3G, or 4G), K-means clustering, or other classification approaches can be defined. The data is then divided into training and test sets in a 4:1 ratio.

For example, the first 80 days are devoted to training sets, followed by 20 days of assessment. Once the hyperparameters for each group have been determined based on the trend analysis of the group, the Prophet model may be repeated for each cell site using training data. Any break of a threshold, whether a spike or a decrease, might be marked as an abnormality. The abnormal cell location will be marked, and the information will be put back into the Hive for viewing on the dashboard.

Step 4:Visualize the results — The Apache Hive table (created by R’s machine learning package) contains data on all cell locations that saw spikes or drops in a single day.

This data comprises the preset criteria, predicted values, and actual use metrics, as well as the amount of the difference between forecasted and actual metrics. The spatial hierarchy associated with anomalous places is also noted. The Hive tables are combined with a dashboard platform like Qlik Sense to allow for speedier decision-making and simpler visualisation.

Benefits of pattern detection

Pattern recognition based on machine learning assists telecom operators in transforming the time-consuming, manual, and reactive monitoring of multi-level operational assets into an end-to-end, touchless, and highly efficient process.

Subex assisted a large African communication service provider in implementing pattern detection to improve on-site assets and use monitoring. Among the primary benefits obtained were:

• Effort savings — Effective anomaly detection saves 20-30 man-hours each month by taking seasonality, trend, holidays, and change points into account.
• Increased productivity – By just monitoring data on the dashboard, area managers may determine the core causes of irregularities across cell sites.
• Site enhancements – As a result of the multi-country deployment, the telecom is now able to log abnormalities as they occur. They discovered roughly 30 unusual instances spread throughout 1600 locations, over 100 clusters, and 8 regions.
• Useful insights — They could correctly identify the causes of usage drop like power interruptions, airtime recharge issues, client relocation, competitor inflow, extreme pricing spikes, and so on.
• Quicker decision-making – The telecom may undertake site-specific remedial measures such as refill schedule revisions, tailored marketing campaigns, and site renovations in real-time.

Case studies for Subex: Pattern detection applications in telecommunications

Reduced consumption data prompt immediate response, increasing customer retention by 90%.

A telecom operator discovered various irregularities in their international call use across significant cell sites using Subex’s Analytics Center of Trust. A closer investigation found that many of its dual sim users had switched to a competitor’s profitable overseas package. In response, the telecom quickly launched an appealing counter-bundle, allowing it to retain 90% of at-risk subscribers. The telecom was able to save USD 250,000 in monthly losses as a result of this.

Anomaly detection assists telcos in detecting revenue leakage and underlying fraud.

A major telco’s critical 4G cell site was identified for unusually high usage of the airtime credit service (ACS). Root cause analysis revealed that 5 individuals were abusing one of the ACS channels, borrowing USD 10,000 in credit many times. Within 24 hours, the Subex system detected the scam and took prompt action by blocking the fraudsters’ accounts. Following that, the security weaknesses in the ACS channel were fixed, saving USD 120,000 in damages.

Using market forces to increase client loyalty

When a telecom operator saw a dramatic increase in data service demand in a certain area, he proceeded to investigate regional abnormalities. According to reports, many new consumers had joined as a result of a service outage in a competitor’s network. Customers’ proclivity to carry numerous SIM cards resulted in roughly 10,000 additional members. The operator quickly launched a campaign to boost consumer loyalty to its data services, resulting in considerably increased data income.

Minor network availability issues result in significant cost reductions.

The availability of network cell sites is a key measure of network health. However, many telcos are unaware of how low network availability affects their company. Subex built a pattern identification system that defined KPI levels and established automated monitoring processes for a telecom operator experiencing network availability concerns. Daily anomaly detection of even small network failures provided the operator with the necessary knowledge to act quickly, allowing them to save USD 1 million each month.

 Using pattern recognition to improve the customer experience

Faced with several customer complaints about poor network performance during late hours, a CSP decided to employ pattern recognition to determine the actual reason. According to the model’s findings, consumer complaints were 2.3 times greater than in other sectors. However, because use trends revealed only a few subscribers lived at the cell site, they could rule out network congestion as a potential cause.

Conclusion

Maintaining asset health through continuous monitoring is a critical competence for telecoms seeking to maintain a competitive advantage through robust service delivery. Pattern identification solutions based on automated and machine learning are developing as a beneficial technique to monitor use trends while utilising powerful analytics and sensible visualisation. CSPs should establish the correct business cases and schedule solution deployment to achieve strong returns on investment. Subex has extensive implementation expertise as well as industry-leading solutions to assist operators in automating pattern identification for increased income and efficiency.

Derive maximum ROI for your business from AI

Request Demo

Payal Paranjape

Payal is a Product Marketing Specialist at Subex, who covers Artificial Intelligence and its application around Generative AI. In her current role, she focuses on Telecom challenges with AI and its potential solutions to these challenges. She is a postgraduate in management from Symbiosis Institute of Digital and Telecom Management, with analytics as her majors, and has prior engineering experience in the Telecom industry. She enjoys reading and authoring content at the intersection of analytics and technology.