Opportunities ahead: Benchmark measurements in 5G networks

By: Zoran Lazarevic, Chief Technical Officer, Ericsson Middle East and Africa

The evolution of 4G into 5G has the potential to be a powerful economic growth engine, particularly as this advanced mobile technology is deployed looking into improving productivity in key vertical industrial sectors.

Regulators across the Gulf countries have done an excellent job in supporting Communication Service Providers (CSPs) with an efficient spectrum allocations process for 5G deployments. As a result of regulators' work on 5G spectrum allocations, some of the first 5G deployments globally were executed by CSPs in the Gulf countries and these CSPs are now global leaders in 5G deployments.

One year after the first 5G deployments, regulators in the Gulf countries are now starting the benchmarking of 5G coverage, to ensure that CSPs are meeting conditions for spectrum allocations. However, there are challenges in terms of how measurements are done. The methodology that is typically used for the benchmark measurements in 2G/3G/4G networks might not be suitable for 5G networks and might not provide accurate benchmark results.

So the question remains, what measurements can be used for benchmarking of the 5G networks? 

The answer to this question is not simple as it is not possible to conduct signal strength measurements in idle mode in the same way it was done for 2G/3G/4G networks. 

The preferred way for 5G networks benchmark measurements is to conduct the end-user performance driven benchmark measurements. These benchmark measurements should be based on the throughput as the main performance indicator that defines end-user experiences when using network services.

We conducted several benchmark measurements between 5G networks with different configurations for common channel and the results demonstrate that it is not accurate to compare 5G networks based on the common channel measurements. The benchmarking results presented the following insights:

1. 1. Benchmark measurements in Gulf Country A: Measurements are conducted for the two CSPs that use different configurations for the common channel in idle mode. CSP “A1” has a control channel configuration with beam sweeping while CSP “A2” has a control channel configuration with a wide beam. 
2. 2. Benchmark measurements in Gulf Country B: Similarly to the first example, the two CSPs have different configurations for the common channels in idle mode. Again, it can be seen that CSP that typically has lower signal strength as well as lower signal to interference ratio, CSP “B2”, has a better downlink and uplink throughput.

The throughput measurements are more complex to conduct then signal strength based measurement and the network traffic level at the time of measurement has an impact on the measurements. However, the throughput measurements will present an accurate snapshot of the real end-user experiences at the time of the measurement. If one needs to use the signal strength measurements for 5G networks benchmarking, then measurements need to be normalized to reflect different configurations for control channels.

There is no perfect solution to the current needs to conduct an objective 5G networks benchmark measurement with similar simplicity and efficiency as it was done for the previous generation of mobile networks.  

However, the preferred way to conduct benchmark measurements is end-user performance driven measurements. The downlink and uplink throughput based measurements will reflect actual end-user experiences in the measured network. 

The industry may need to come up with an alignment on how the signal strength measurements can be conducted for 5G networks in a meaningful way. Ericsson is recognizing an industry-wide challenge related to the methodology for 5G benchmark measurement and is currently working on a document for the 5G measurement methodology that will be made public in the near future.