The TRIANGLE Testbed provides a simple solution to a complex problem: benchmarking and testing applications and devices in realistic 5G network scenarios.
The TRIANGLE Testbed caters to many types of users with varying degrees of knowledge about mobile networks, ranging from app developers to network researchers. You do not need to be an expert to get useful results, but if you are you can dive in and tweak many parameters to run the benchmarks to your liking.
Here we introduce the capabilities of the TRIANGLE Testbed, starting from the point of view of a user that is not really familiar with the intricacies of mobile networks, and digging progressively into the individual components of the Testbed.
What can we do for you?
The TRIANGLE Testbed offers many benchmarking capabilities for many user profiles: application developers, device makers, mobile network operators, researchers, etc… Here we summarize some of the most valuable high-level capabilities for each user profile:
- Benchmark your app network performance in different realistic scenarios
- Measure your app power consumption in unexpected network scenarios
- Automate your tests using state-of-the-art tools
- Test your app on a set of popular phones
- Benchmark your device networking capabilities in a controlled environment
- Check the performance of your device against a set of popular reference apps
Mobile network operator
- Characterize the traffic profile of new mobile applications.
- Use our devices or bring your own for testing
- Perform experiments in a controlled environment that can simulate a wide range of scenarios
- Access to low level configuration parameters
- Powerful automation capabilities to create dynamic experiments
From a high-level perspective, the testbed is composed of five main elements:
- Device cloud: the testbed provides a set of mobile phones ready to be used.
- Network Emulation: from the physical channel to the network core. The system is a 4G/4.5G one, and will undergo upgrades following the 5G standardization.
- Mobile Device Monitoring: a set of capabilities which range from software to evaluate the QoE, to hardware probes that measure power consumption
- App Testing: automates the user flows in your app to be able to repeat the same user test under different conditions
- Automation & UI: the software that coordinates the experiment, providing a user interface with the appropriate level of details for each user profile
Physical deployment of the testbed
This section provides a more detailed description of some of these components, with links to useful resources if you want to know more about their capabilities and how to fully take advantage of them.
UXM RAN emulator
An essential component in the testbed is an instrument capable of emulating multiple cellular networks in a controlled manner. To that end, TRIANGLE uses the UXM Wireless Test Platform device by Keysight Technologies, which supports multiple radio access technologies (multi-RAT), including GSM/GPRS, UMTS and LTE-Advanced networks (i.e., 2G, 3G and 4G). The UXM features include intra-RAT and inter-RAT handovers, protocol debugging, IP end-to-end delay and throughput measurements, and RF conformance tests. Finally, it should be noted that the UXM also features an advanced fading engine with the main channel models defined by 3GPP.
Mobile Device Monitoring
5G test scenarios will require high resolution for reporting the targeted QoS KPIs. In order to extend the measurement capabilities of other tools like iperf, TRIANGLE provides up to OSI layer 7 SDU packet resolution in the computation of data performance KPIs thanks to the integration of the AT4 Wireless Performance Tool. This tool is composed of two components, Controller and Agents (data endpoints), and uses proprietary mechanisms to synchronize the Agents and provide accurate one-way measurements. This tool includes a built-in traffic generator with the capability of generating constant rates, ramps, loops and statistical traffic patterns which is something of utmost importance for setting up the desired environment in terms of varying traffic loads (e.g., for measuring LTE-U impact on Wi-Fi networks). Additionally, this tool has the ability to automate some mobile Apps on Android devices and measure relevant QoE KPI such as YouTube buffering occurrences.
TestelDroid Mobile Monitoring App
TestelDroid is a software tool that enables passive monitoring of radio parameters and data traffic in Android-based devices. Logging is implemented as an Android service that can be running in the background logging all the information while the application under test is being executed. This functionality enables monitoring of the traffic information generated by any application, which extends the testing to a very wide range of use cases. The parameters to be logged (network, neighbour cells, GPS, traffic) can be flexibly configured using the SCPI interface.
Power Consumption Analyzer
In order to provide a complete overview of the performance of the App or the Device under test, it is nowadays unthinkable not to include an analysis of the energy consumed by the mobile platform. For this reason TRIANGLE provides accurate measurement of the power consumption thanks to the Keysight N6705B Power Analyzer. The instrument allows the provision of DC voltage to the device and accurately measures the drawn current to verify how much power is required by each app and network operation over time.
Device action automation tool
TRIANGLE also uses Quamotion test automation tool which provides the ability to write and execute automated tests for Android and iOS applications. Execution can be performed without any manual interaction. Not only gestures and keyboard interactions but also the full lifecycle of the application can be automated, i.e. installing, launching, terminating. These set of actions are exposed to the overall test orchestrator such that the end-user actions can be synchronized with various network events, e.g. an unstable network connection in a train. Quamotion also has a recording capability to increase the efficiency in the process of writing test cases. Each step performed on the device is recorded and translated automatically into an action as part of a test script.
App Lab live environment
TRIANGLE will use App Lab, a high level platform developed by UCL for distributing mobile apps to a large- scale testbed for pre-commercial testing and validation services. App Lab provides a private app store offering in-the-wild rapid user field-testing to UCL staff and students. App Lab is comprised of a server portal for application upload, management and distribution and native mobile client apps for both Android and iOS devices. App Lab uses analytics tools to track how users interact with apps in the real world. By examining the results of app usage, test cases which mimic actual users’ behaviour can be derived and executed in a variety of network conditions in the TRIANGLE testbed as shown in Figure 3
Available in future open calls
Model Based Testing
Another alternative considered in the project is model-based testing. Instead of writing test scripts manually with sequences of user actions, e.g. pressing a button, entering some text, app developers can provide a high-level description of the intended workflow. Test cases could be automatically generated from this description, achieving a high coverage of realistic usage scenarios. These models may also be based on crowdsourced data, such as observed user interactions from live tests, including tests run using App Lab.
The Evolved Packet Core (EPC) is the core of LTE networks, and it will also play a central role in pre-5G deployments that include for instance LTE to Wi-Fi offloading. UMA lab brings Polaris Networks EPC to the project, which are a set of emulators that enable the deployment of multiple instances of each of the components of the network. The supported components currently are MME (Mobility Management Entity), PGW (Packet Data Network Gateway), SGW (Serving Gateway), PCRF (Policy and Charging Rules Function), HSS (Home Subscriber Server), ANDSF (Access Network Discovery and Selection Function) and ePDG (evolved Packet Data Gateway). The emulator offer carrier grade performance to up to 300 users, can be deployed dynamically and enable positive and negative testing of the behaviors of the EPC. The Project can support scenarios with commercial small cells (available in the testbed), which will provide real but non fully-controllable radio propagation conditions. The main scenario is based on the UXM emulator, which provides fully controllable radio propagation conditions as well as measurements of the full communication stack. Both scenarios are connected to the Polaris EPC through standard 3GPP interfaces.
Virtual Path Slicing Engine
Software Defined Networks (SDN), Network Function Virtualization (NFV) and OpenFlow are techniques that provide high flexibility and can cut costs in packet data networks. Virtual Path Slicing (VPS) is a method that has evolved from studies like on how to implement end to end QoS in packet networks. RedZinc has implemented a VPS engine that can steer traffic on the edge of the network based on the request of the application and coordinate this traffic steering with other autonomous systems. In traditional 3GPP EPC the VPS engine can provide services to the application layer and invoke them using the Rx interface in the Policy Charging Rules Function (PCRF) of the EPC where differentiated QoS Class Identifier can be used for different service levels.