A proposed architecture for big data analytics systems for telecommunication industry

Abstract

The competitive nature of the telecommunication industry needs fast decisions, and proactive solutions based on complete and updated views of business, customers, and network. However, current telecommunication information systems and mainly Data Warehouses were designed to implement predefined analytics, and they are challenged by large-scale and fast growing telecom data, and the variety of its applications. The main aim of this research is to address the issues of high volume, velocity of data in telecommunication industry, and the variety of its usages, to achieve Big Data analytics system architecture. This architecture is proposed to facilitate developing heterogeneous telecom data analytics, and to support real and near real time use cases. Therefore, in this work, telecom analytics applications that require Big Data processing were investigated, and their requirements were analyzed. Then the architecture was designed based on the collected requirements. The proposed architecture uses Spark as a general purpose processing engine for implementing heterogeneous stream and batch analytics applications. Hadoop Distributed File System HDFS and Cassandra are the data stores, and Kafka is used for data streaming and pipelining. The architecture was then evaluated based on its functional completeness and its performance. The performance figures of the proposed architecture were calculated using performance figures of Spark, Cassandra, and Kafka. The results showed that the proposed architecture can support real-time and large-scale analytics applications with average 15 ms latency and 2-8min processing time respectively.