New! The PerformNetworks tutorial is now available as a PDF: PerformNetwork_Tutorialv1.0.pdf
The purpose of this document is to serve as an introduction to the use of the PerformNetworks testbed located in the research laboratories of the University of Málaga. These instructions give new experimenters a general overview of the testbed and a step-by-step tutorial to run basic experiments with our equipment.
A detailed list of the measurement equipment available in the PerformNetworks testbed consist:
- An Agilent E2010 conformance tester. This equipment emulates a complete LTE eNodeB and can perform different measures in the signal quality of the UE connected to it. As part of the Fed4FIRE effort the University of Malaga has instrumented a Resource Controller (RC) compatible with the OMF/OML technologies to automatize experiments.
- Various Android devices with a RC also developed by us that allow the remote control and measures from the UE point of view.
- Two Pixie small cells by Athena Wireless and one experimental pico cell by Alcatel Lucent, which can be offered depending on its availability. All the equipments work in LTE band 7 (2.6GHz)
- A Core Network Emulation System by Polaris Networks, that provides full LTE core network functionality, it enables the definition of multiple core network elements, even with different PLMN, including MME, SGW, PGW, PCRF and HSS.
- The Keysight N3705B power analyzer by Keysight technologies, that can be used to extract measurements of certain UEs.
- Several UEs in different LTE bands (see table 1).
- Vodafone SIMS available for experimenters that wants to make testing on live commercial networks, which are deployed on band 7 and band 3.
|Samsung B3730||4||Samsung||Band 7|
|LG Motion MetroPCS||2||Qualcomm MSM8960||Band 4|
|LG VL-600||2||Qualcomm MSM6800A||Band 13|
|Huawei K5005 (Vodafone)||1||Qualcomm MDM9200||Bands 7 and 20|
|Samsung Galaxy Tab LTE||1||Band 17|
|Samsung Galaxy S4 LTE||4||Qualcomm APQ8064T||Bands 1,3,5,7,8 and 20|
|HTC One||1||Qualcomm APQ8064T||800,1800/2600 MHz|
|Sierra Aircard 313U||2||Qualcomm||700/1700/2100 MHz|
|Samsung Galaxy S3||1||Qualcomm||800/1800/2600 MHz|
Table 1: List of user equipment available at University of Málaga
The OMF framework uses another external library to perform and save the result of the measurements. In our testbed these results are stored in a PostgreSQL database for further extraction.Our infrastructure uses the OMF framework to connect all the devices (information and some examples can be found in http://mytestbed.net/projects/omf6/wiki/Wiki). To run experiments the framework defines an Experiment Description Language (OEDL http://mytestbed.net/projects/omf6/wiki/OEDLOMF6) that in reality is a subset of the Ruby programming language. During the experiment run the Experiment Controller of OMF translates these scripts to XMPP messages and route them to the appropriate Resource Controller that process them. The logical architecture of doing an experiment in our testbed is shown in the next diagram:
The instrumented E2010 Conformance Tester
The E2010 from Agilent (now Keysight) is a configurable eNodeB with measurement capabilities. In order to automatize tests it provides a Standard Commands for Programmable Instruments (SCPI) interface which is used by the RC to write to and read from the device. Almost every parameter of the E2010 can be configured using OEDL. A complete OEDL script with all the available parameters should be received along this document.
NOTE: in our validation tests the mentioned script takes up to two minute to complete. As the E2010 doesn’t change its configuration unless told so, we recommend to completely configure the device at the beginning of the experimentation session and then use smaller scripts with only the applicable parameter changed.
Another detail to take into account is that there are certain parameters that causes some others to change: for example changing the Radio Access Technology to FDD or TDD changes the LTE frequency band to its default value for the given RAT (band 1 and 33 respectively) and activate/deactivate various subframes. Thus we recommend to always configure the higher-level parameters first to avoid the lost of configuration from its dependants.
As our RC is taylored to the devices of our testbed we need some extra information in the OEDL script to signal various events. We address them using the following examples.
Performing an Experiment
We are experiencing some issues with the current version of the federation and reservation system, so its description is skipped in this version of the tutorial.
After the Fed4FIRE procedure for requesting a time slice has been accepted, you will have a SSH session to the experimentation server. That session will have already installed every OMF package and executables needed to run the scripts and get the results. Also note that while we use the IP 127.0.0.1 for the purposes of this tutorial, you will be given the correct IP of the Resource Controller, XMPP and Database server.
Scripts are executed by using the console executable omf_ec (short for “OMF Experiment Controller”). The program has a few configuration parameters related to the domain of the experiment, the network address of some components, etcetera.
$ omf_ec --help INFO OML4R Client 2.10.6 [OMSPv4; Ruby 1.9.3] Copyright 2009-2014, NICTA OMF Experiment Controller version '6.1.12' Usage: omf_ec [options] path_to_oedl_file [-- --experiment_property value] -u, --uri ADDRESS URI for communication layer [amqp://localhost] -c CONFIGFILE Configuration File --log_config CONFIGFILE Logging Configuration File -e ENVIRONMENT Environment (development, production ...) [development] --root_cert_dir DIRECTORY Directory containing root certificates --cert CERTIFICATE Your certificate --key KEY Your private key --name, --experiment EXPERIMENT_NAME Experiment name --slice SLICE_NAME Slice name [optional] --job-service URL URL to the JobService [optional] --job-url URL URL to the Job for this experiment trial [optional] --slice-service URL URL to the SliceService [optional] --oml_uri URI URI for the OML data collection of experiment applications --inst_oml_uri URI EC Instrumentation: OML URI to use --inst_oml_id ID EC Instrumentation: OML ID to use --inst_oml_domain DOMAIN EC Instrumentation: OML Domain to use -g, --show-graph Parse graph definition to construct graph information in log output -v, --version Show version -d, --debug Debug mode (Set logging level in Stdout to :debug) -h, --help Show this message --oml-id id Name to identify this app instance [undefined] --oml-domain domain Name of experimental domain [undefined] *EXPERIMENTAL* --oml-collect uri URI of server to send measurements to --oml-protocol p Protocol number  --oml-log-level l Log level used (info: 0 .. debug: 1) --oml-noop Do not collect measurements --oml-config file File holding OML configuration parameters --oml-exp-id domain Obsolescent equivalent to --oml-domain domain --oml-file localPath Obsolescent equivalent to --oml-collect file:localPath --oml-server uri Obsolescent equivalent to --oml-collect uri --oml-help Show this message
For this simple experiment we only are going to use the first one that sets the host address of the XMPP server. This server will be the one to pass the experiment messages to the corresponding recipient, be it the resource controller, the database server or us.
We now proceed to run the second one of the experimentation examples and view the results in the database:
After a successful run we can see how the data is organized in the database and how the values changed as the result of our petition for increase the power output of the UE.
For the sake of simplicity, for all experiment we assume a default configuration (i.e. the complete configuration script) has already been loaded to the device. Also, all measures will have a period of one measure every 1 second.
This is the most basic experiment, and we’ll use it to point to various peculiarities of the OEDL that our RC expects. The next experiment script will serve as a first run and we’ll explore it in detail:
defEvent :e2010_ready do |state| state.find_all do |v| v[:type] == 'umaresourcecontroller' && v[:hrn] == 'e2010' && !v[:membership].empty? && \ v[:device_ready] == true end.size > 0 end onEvent :ALL_UP do group('performNetworks') do |g| g.create_resource('resourcecontroller', type: 'e2010') onEvent :e2010_ready do after 3.second do g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0) end after 25.seconds do info ">>> Release e2010" g.resources[hrn: 'e2010'].release done! end end end end
Broadly this experiment do the following:
- Prepares itself to receive an OMF event that will signal the preconfiguration of the device
- Wait until all the OMF infrastructure is loaded and ready (the predefined event :ALL_UP)
- Creates an instance of the RC that controls the device.
- Wait until the RC connects to the device.
- Access the resource and change some parameter.
- End the experiment.
With a little more detail:
defEvent :e2010_ready do |state| state.find_all do |v| v[:type] == 'umaresourcecontroller' && v[:hrn] == 'e2010' && !v[:membership].empty? && \ v[:device_ready] == true end.size > 0 end
The first event “:e2010_ready” defined in the script is needed because internally the RC needs to establish a connection with the device and setup several parameters before it can be configured. When the experiment is running this event will signal that the device is ready.
onEvent :ALL_UP do
With the lines
group('performNetworks') do |g| g.create_resource('resourcecontroller', type: 'e2010')
we create a group to address the rest of the resources of our experiment. The only resource created in this experiment is the E2010 controller. We have to specify the exact type of the resource because our RC is extensible to other types of SCPI devices and it needs to know what (internal) configuration has to load.
Finally the constructs
after 3.second do g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0) end after 25.seconds do info ">>> Release e2010" g.resources[hrn: 'e2010'].release done! end
are the actual commands of the experiment, in this case we configure the downlink to allocate 10 MHz and to release the devices 20 seconds later. We are using here standard Ruby timers to execute the enclosed commands after the number of seconds defined. Strictly speaking the first timer is not really needed because in theory the first line is going to be executed just after the :e2010_ready event is received, but OMF relies internally in the XMPP protocol to dispatch messages and there is no guarantee that messages sent roughly at the same time arrived to their destination in order. As every log messages is also saved in the database, we find easier to debug an experiment if we give a couple of seconds after the experiment begins before start issuing commands.
The first experiment introduces some concepts needed to run scripts in our testbed but they don’t produce any information from the point of view of the experimenter. With this second script we present the the first real measurement performed in a real UE. We also use some other features of OEDL that can be used to ease the creation of new experiments.
This experiment will monitor the received channel power from the UE for 20 seconds. Halfway the experiment we instruct the UE to raise its output power and see how it affects the data we collect.
def_property('umaproxy', 'umaproxy', 'scpi_instr') defProperty('FDD_DL_FREQ', 3100, 'description') defEvent :e2010_ready do |state| state.find_all do |v| v[:type] == 'umaresourcecontroller' && v[:hrn] == 'e2010' && !v[:membership].empty? && v[:device_ready] == true end.size > 0 end defApplication('foo_bar') do |app| app.description = 'foo desc' end # Define a group externally. defGroup('performNetworks', prop.umaproxy) do |g| g.addApplication('foo_bar') end onEvent :ALL_UP do group('performNetworks') do |g| g.create_resource('resourcecontroller', type: 'e2010') onEvent :e2010_ready do after 2.seconds do info ">>> e2010 Configuration" g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0) g.resources[type: 'e2010'].FDD_DL_FREQ = Integer(property.FDD_DL_FREQ) g.resources[type: 'e2010'].channelpower_mp_enable = true g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 1 end after 20.seconds do info “>>> Starting measurements” g.startApplications end after 30.seconds do info “>>> Changing UE power” g.resources[type: 'e2010'].FDD_CNTRL_CONNECT_UL_TPCCTRL=1 end after 40.seconds do info “>>> Stopping measurements” allGroups.stopApplications g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 0 end after 45.seconds do info ">>> Release e2010" g.resources[hrn: 'e2010'].release done! end end end end
def_property('umaproxy', 'umaproxy', 'scpi_instr') defProperty('FDD_DL_FREQ', 3100, 'description')
With these lines we define the ‘umaproxy’ and ‘FDD_DL_FREQ’ to the values ‘umaproxy’ and 3100 respectively, and add some descriptions of what they are. By using them the experimenter can create generic experiments and change the values in the scripts or directly in the command line when the experiment is run (see last chapter of this tutorial). For example the value of FDD_DL_FREQ indicates the central frequency of the carrier that will be used in the experiment expressed in EARFCN, in this case we select the frequency of LTE band 7.
With the lines
defApplication('foo_bar') do |app| app.description = 'foo desc' end # Define a group externally. defGroup('performNetworks', prop.umaproxy) do |g| g.addApplication('foo_bar') end
we define a group of resources and an application that will be used to do the measurements. The names doesn’t have any repercussion during the experiment, but they will be used by the OML framework to create the tables that will hold the measurement data in the database. One use of this would be to set the application name to the date/time of the experiment, or a generic explanation for the experiment, so when you retrieve all the tables you’ll have a direct annotation in the database of what you were attempting to do.
As before, after setting up the :e2010 resource we wait for the :e2010_ready signal to start configuring the device:
after 2.seconds do info ">>> e2010 Configuration" g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0) g.resources[type: 'e2010'].FDD_DL_FREQ = Integer(property.FDD_DL_FREQ) g.resources[type: 'e2010'].channelpower_mp_enable = true g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 1 end after 20.seconds do info “>>> Starting measurements” g.startApplications end
We set a bandwidth of 10 MHz, a central frequency defined by the property FDD_DL_FREQ, enable the ‘channelpower’ measurement point and start broadcasting. After 20 seconds we start all the applications defined for the group. This will signal all the measurement threads to start requesting data from the E2010 and store them in the database. The 20 seconds delay depends on the type of the UE tested because they usually have an internal timer to check the carrier before starting the LTE attach procedure.
10 seconds after the experiment starts we make the eNodeB to signal the UE to use its maximum power output, while the measurement process is still running, and we should see a change in the collected data.
after 30.seconds do info “>>> Changing UE power” g.resources[type: 'e2010'].FDD_CNTRL_CONNECT_UL_TPCCTRL=1 end after 40.seconds do info “>>> Stopping measurements” allGroups.stopApplications g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 0 end
The measures are stopped when the corresponding application stops, and we then shut down the carrier of the eNodeB. The only step left is to release the experiment resources as we did in the first experiment.