Jump to Navigation

PerformNetworks

PerformNetworks tutorial


New! The PerformNetworks tutorial is now available as a PDF: PerformNetwork_Tutorialv1.0.pdf


Introduction

The purpose of this document is to serve as an introduction to the use of the PerformNetworks testbed located in the research laboratories of the University of Málaga. These instructions give new experimenters a general overview of the testbed and a step-by-step tutorial to run basic experiments with our equipment.

Resources

A detailed list of the measurement equipment available in the PerformNetworks testbed consist:

  • An Agilent E2010 conformance tester. This equipment emulates a complete LTE eNodeB and can perform different measures in the signal quality of the UE connected to it. As part of the Fed4FIRE effort the University of Malaga has instrumented a Resource Controller (RC) compatible with the OMF/OML technologies to automatize experiments.
  • Various Android devices with a RC also developed by us that allow the remote control and measures from the UE point of view.
  • Two Pixie small cells by Athena Wireless and one experimental pico cell by Alcatel Lucent, which can be offered depending on its availability. All the equipments work in LTE band 7 (2.6GHz)
  • A Core Network Emulation System by Polaris Networks, that provides full LTE core network functionality, it enables the definition of multiple core network elements, even with different PLMN, including MME, SGW, PGW, PCRF and HSS.
  • The Keysight N3705B power analyzer by Keysight technologies, that can be used to extract measurements of certain UEs.
  • Several UEs in different LTE bands (see table 1).
  • Vodafone SIMS available for experimenters that wants to make testing on live commercial networks, which are deployed on band 7 and band 3.
  Model   Units   Chipset   LTE Band(FDD)
  Samsung B3730  4     Samsung   Band 7
  LG Motion MetroPCS 2     Qualcomm MSM8960   Band 4
  LG VL-600 2     Qualcomm MSM6800A   Band 13
  Huawei K5005 (Vodafone) 1     Qualcomm MDM9200   Bands 7 and 20
  Samsung Galaxy Tab LTE 1       Band 17
  Samsung Galaxy S4 LTE 4     Qualcomm APQ8064T   Bands 1,3,5,7,8 and 20
  HTC One 1     Qualcomm APQ8064T   800,1800/2600 MHz
  Sierra Aircard 313U 2     Qualcomm   700/1700/2100 MHz
  Samsung Galaxy S3 1     Qualcomm   800/1800/2600 MHz 

Table 1: List of user equipment available at University of Málaga

Experimentation Framework
The OMF framework uses another external library to perform and save the result of the measurements. In our testbed these results are stored in a PostgreSQL database for further extraction.Our infrastructure uses the OMF framework to connect all the devices (information and some examples can be found in http://mytestbed.net/projects/omf6/wiki/Wiki). To run experiments the framework defines an Experiment Description Language (OEDL http://mytestbed.net/projects/omf6/wiki/OEDLOMF6) that in reality is a subset of the Ruby programming language. During the experiment run the Experiment Controller of OMF translates these scripts to XMPP messages and route them to the appropriate Resource Controller that process them. The logical architecture of doing an experiment in our testbed is shown in the next diagram:

 

The instrumented E2010 Conformance Tester

The E2010 from Agilent (now Keysight) is a configurable eNodeB with measurement capabilities. In order to automatize tests it provides a Standard Commands for Programmable Instruments (SCPI) interface which is used by the RC to write to and read from the device. Almost every parameter of the E2010 can be configured using OEDL. A complete OEDL script with all the available parameters should be received along this document.

NOTE: in our validation tests the mentioned script takes up to two minute to complete. As the E2010 doesn’t change its configuration unless told so, we recommend to completely configure the device at the beginning of the experimentation session and then use smaller scripts with only the applicable parameter changed.

Another detail to take into account is that there are certain parameters that causes some others to change: for example changing the Radio Access Technology to FDD or TDD changes the LTE frequency band to its default value for the given RAT (band 1 and 33 respectively) and activate/deactivate various subframes. Thus we recommend to always configure the higher-level parameters first to avoid the lost of configuration from its dependants.

As our RC is taylored to the devices of our testbed we need some extra information in the OEDL script to signal various events. We address them using the following examples.

Performing an Experiment

Step-by-step run of an Experiment 

We are experiencing some issues with the current version of the federation and reservation system, so its description is skipped in this version of the tutorial.

After the Fed4FIRE procedure for requesting a time slice has been accepted, you will have a SSH session to the experimentation server. That session will have already installed every OMF package and executables needed to run the scripts and get the results. Also note that while we use the IP 127.0.0.1 for the purposes of this tutorial, you will be given the correct IP of the Resource Controller, XMPP and Database server.

Scripts are executed by using the console executable omf_ec (short for “OMF Experiment Controller”). The program has a few configuration parameters related to the domain of the experiment, the network address of some components, etcetera.

$ omf_ec --help

INFO    OML4R Client 2.10.6 [OMSPv4; Ruby 1.9.3] Copyright 2009-2014, NICTA
OMF Experiment Controller version '6.1.12'

Usage: omf_ec [options] path_to_oedl_file [-- --experiment_property value]
    -u, --uri ADDRESS                URI for communication layer [amqp://localhost]
    -c CONFIGFILE                    Configuration File
    --log_config CONFIGFILE      Logging Configuration File
    -e ENVIRONMENT                   Environment (development, production ...) [development]
    --root_cert_dir DIRECTORY    Directory containing root certificates
    --cert CERTIFICATE           Your certificate
    --key KEY                    Your private key
    --name, --experiment EXPERIMENT_NAME   Experiment name
    --slice SLICE_NAME           Slice name [optional]
    --job-service URL            URL to the JobService [optional]
    --job-url URL                URL to the Job for this experiment trial [optional]
    --slice-service URL          URL to the SliceService [optional]
    --oml_uri URI                URI for the OML data collection of experiment applications
    --inst_oml_uri URI           EC Instrumentation: OML URI to use
    --inst_oml_id ID             EC Instrumentation: OML ID to use
    --inst_oml_domain DOMAIN     EC Instrumentation: OML Domain to use
    -g, --show-graph                 Parse graph definition to construct graph information in log output
    -v, --version                    Show version
    -d, --debug                      Debug mode (Set logging level in Stdout to :debug)
    -h, --help                       Show this message
    --oml-id id                  Name to identify this app instance [undefined]
    --oml-domain domain          Name of experimental domain [undefined] *EXPERIMENTAL*
    --oml-collect uri            URI of server to send measurements to
    --oml-protocol p             Protocol number [4]
    --oml-log-level l            Log level used (info: 0 .. debug: 1)
    --oml-noop                   Do not collect measurements
    --oml-config file            File holding OML configuration parameters
    --oml-exp-id domain          Obsolescent equivalent to --oml-domain domain
    --oml-file localPath         Obsolescent equivalent to --oml-collect file:localPath
    --oml-server uri             Obsolescent equivalent to --oml-collect uri
    --oml-help                   Show this message
 

For this simple experiment we only are going to use the first one that sets the host address of the XMPP server. This server will be the one to pass the experiment messages to the corresponding recipient, be it the resource controller, the database server or us.

We now proceed to run the second one of the experimentation examples and view the results in the database:

$ omf_ec -u xmpp://localhost tutorial2.oedl
INFO    OML4R Client 2.10.6 [OMSPv4; Ruby 1.9.3] Copyright 2009-2014, NICTA
Warning: OML4R: Missing values for parameter :domain (--oml-domain, OML_DOMAIN)! to instrument, so it will 
run without instrumentation. (see --oml-help)
19:10:29  INFO XMPP::Communicator: Connecting to 'localhost' ...
19:10:30  WARN XMPP::Communicator: Disconnected... Last known state: initializing
19:10:30  INFO XMPP::Communicator: Connecting to 'localhost' ...
19:10:31  INFO XMPP::Communicator: Connected
19:10:31  INFO OmfEc::Runner: OMF Experiment Controller 6.1.12 - Start
19:10:31  INFO OmfEc::Runner: Connected using {:proto=>:xmpp, :user=>"debian7-45454", :domain=>"debian7"}
19:10:31  INFO OmfEc::Runner: Execute: /home/bob/tutorial.oedl
19:10:31  INFO OmfEc::Runner: Properties: {}
19:10:31  INFO OmfEc::ExperimentProperty: umaproxy = "umaproxy" (String)
19:10:31  INFO OmfEc::ExperimentProperty: FDD_DL_FREQ = 3100 (Fixnum)
19:10:31  INFO OmfEc::Experiment: Experiment: 2015-01-21T18:10:28.957Z starts
19:10:31  INFO OmfEc::Experiment: Configure 'umaproxy' to join 'performNetworks'
19:10:32  INFO OmfEc::Experiment: Newly discovered resource >> xmpp://umaproxy@debian7
19:10:32  INFO OmfEc::Experiment: Config xmpp://umaproxy@debian7 to join performNetworks
19:10:32  INFO OmfEc::Experiment: Event triggered: 'ALL_NODES_UP, ALL_UP'
19:11:07  INFO OmfEc::Experiment: Event triggered: 'e2010_ready'
19:11:09  INFO Object: >>> e2010 Configuration
19:11:27  INFO Object: >>> Starting measurements
19:11:37  INFO Object: >>> Changing UE power
19:12:17  INFO Object: >>> Stopping measurements
19:12:22  INFO Object: >>> Release e2010
19:12:22  INFO OmfEc::Experiment: Experiment: 2015-01-21T18:10:28.957Z finished
19:12:22  INFO OmfEc::Experiment: Release applications and network interfaces
19:12:22  INFO OmfEc::Experiment: Exit in 15 seconds...
19:12:36  INFO OmfEc::Experiment: OMF Experiment Controller 6.1.12 - Exit.
19:12:37  INFO XMPP::Communicator: Disconnecting...
 

After a successful run we can see how the data is organized in the database and how the values changed as the result of our petition for increase the power output of the UE.

Experiment examples

For the sake of simplicity, for all experiment we assume a default configuration (i.e. the complete configuration script) has already been loaded to the device. Also, all measures will have a period of one measure every 1 second.

Experiment 1

This is the most basic experiment, and we’ll use it to point to various peculiarities of the OEDL that our RC expects. The next experiment script will serve as a first run and we’ll explore it in detail:

defEvent :e2010_ready do |state|
  state.find_all do |v|
  v[:type] == 'umaresourcecontroller' && v[:hrn] == 'e2010' && !v[:membership].empty? && \
v[:device_ready] == true
  end.size > 0
end

onEvent :ALL_UP do
  group('performNetworks') do |g|
    g.create_resource('resourcecontroller', type: 'e2010')
    onEvent :e2010_ready do
      after 3.second do
        g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0)
      end

      after 25.seconds do
          info ">>> Release e2010"
          g.resources[hrn: 'e2010'].release
          done!
      end
    end
  end
end
 

Broadly this experiment do the following:

  • Prepares  itself to receive an OMF event that will signal the preconfiguration of the device
  • Wait until all the OMF infrastructure is loaded and ready (the predefined event :ALL_UP)
  • Creates an instance of the RC that controls the device.
  • Wait until the RC connects to the device.
  • Access the resource and change some parameter.
  • End the experiment.

With a little more detail:

defEvent :e2010_ready do |state|
  state.find_all do |v|
  v[:type] == 'umaresourcecontroller' && v[:hrn] == 'e2010' && !v[:membership].empty? && \
v[:device_ready] == true
  end.size > 0
end
 

The first event “:e2010_ready” defined in the script is needed because internally the RC needs to establish a connection with the device and setup several parameters before it can be configured. When the experiment is running this event will signal that the device is ready.

The line 

onEvent :ALL_UP do
 
is one of the predefined OMF events to signal the experiment runner. This is triggered when the framework has been correctly initialized and is ready to process further tasks. The complete list of predefined events can be found in http://mytestbed.net/projects/omf6/wiki/OEDLOMF6#onEvent

With the lines

group('performNetworks') do |g|
    g.create_resource('resourcecontroller', type: 'e2010')
 

we create a group to address the rest of the resources of our experiment. The only resource created in this experiment is the E2010 controller. We have to specify the exact type of the resource because our RC is extensible to other types of SCPI devices and it needs to know what (internal) configuration has to load.

Finally the constructs

after 3.second do
        g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0)
      end
      after 25.seconds do
          info ">>> Release e2010"
          g.resources[hrn: 'e2010'].release
          done!
      end
 

are the actual commands of the experiment, in this case we configure the downlink to allocate 10 MHz and to release the devices 20 seconds later. We are using here standard Ruby timers to execute the enclosed commands after the number of seconds defined. Strictly speaking the first timer is not really needed because in theory the first line is going to be executed just after the :e2010_ready event is received, but OMF relies internally in the XMPP protocol to dispatch messages and there is no guarantee that messages sent roughly at the same time arrived to their destination in order. As every log messages is also saved in the database, we find easier to debug an experiment if we give a couple of seconds after the experiment begins before start issuing commands.

Experiment 2

The first experiment introduces some concepts needed to run scripts in our testbed but they don’t produce any information from the point of view of the experimenter. With this second script we present the the first real measurement performed in a real UE. We also use some other features of OEDL that can be used to ease the creation of new experiments.

This experiment will monitor the received channel power from the UE for 20 seconds. Halfway the experiment we instruct the UE to raise its output power and see how it affects the data we collect.

def_property('umaproxy', 'umaproxy', 'scpi_instr')
defProperty('FDD_DL_FREQ', 3100, 'description')
defEvent :e2010_ready do |state|
  state.find_all do |v|
    v[:type] == 'umaresourcecontroller' && v[:hrn] == 'e2010' && !v[:membership].empty? 
      && v[:device_ready] == true
  end.size > 0
end

defApplication('foo_bar') do |app|
  app.description = 'foo desc'
end

# Define a group externally.
defGroup('performNetworks', prop.umaproxy) do |g|
    g.addApplication('foo_bar')
end

onEvent :ALL_UP do
  group('performNetworks') do |g|
    g.create_resource('resourcecontroller', type: 'e2010')
    onEvent :e2010_ready do
      after 2.seconds do
        info ">>> e2010 Configuration"
        g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0)
        g.resources[type: 'e2010'].FDD_DL_FREQ = Integer(property.FDD_DL_FREQ)
        g.resources[type: 'e2010'].channelpower_mp_enable = true
        g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 1
      end     

      after 20.seconds do
        info “>>> Starting measurements”
        g.startApplications
      end

      after 30.seconds do
        info “>>> Changing UE power”
        g.resources[type: 'e2010'].FDD_CNTRL_CONNECT_UL_TPCCTRL=1
      end
      after 40.seconds do
        info “>>> Stopping measurements”
        allGroups.stopApplications
        g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 0
      end

      after 45.seconds do
        info ">>> Release e2010"
        g.resources[hrn: 'e2010'].release
            done!
      end
    end
   end
end
 
Aside the :e2010_ready event already discussed in the first experiment, we use OEDL capabilities to define properties and groups and use them in the actual experiment.
def_property('umaproxy', 'umaproxy', 'scpi_instr')
defProperty('FDD_DL_FREQ', 3100, 'description')
 

With these lines we define the ‘umaproxy’ and ‘FDD_DL_FREQ’ to the values ‘umaproxy’ and 3100 respectively, and add some descriptions of what they are. By using them the experimenter can  create generic experiments and change the values in the scripts or directly in the command line when the experiment is run (see last chapter of this tutorial). For example the value of FDD_DL_FREQ indicates the central frequency of the carrier that will be used in the experiment expressed in EARFCN, in this case we select the frequency of LTE band 7.

With the lines 

defApplication('foo_bar') do |app|
  app.description = 'foo desc'
end

# Define a group externally.
defGroup('performNetworks', prop.umaproxy) do |g|
    g.addApplication('foo_bar')
end
 

we define a group of resources and an application that will be used to do the measurements. The names doesn’t have any repercussion during the experiment, but they will be used by the OML framework to create the tables that will hold the measurement data in the database. One use of this would be to set the application name to the date/time of the experiment, or a generic explanation for the experiment, so when you retrieve all the tables you’ll have a direct annotation in the database of what you were attempting to do.

As before, after setting up the :e2010 resource we wait for the :e2010_ready signal to start configuring the device:

      after 2.seconds do
        info ">>> e2010 Configuration"
        g.resources[type: 'e2010'].FDD_DL_BW=Float(10.0)
        g.resources[type: 'e2010'].FDD_DL_FREQ = Integer(property.FDD_DL_FREQ)
        g.resources[type: 'e2010'].channelpower_mp_enable = true
        g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 1
      end

      after 20.seconds do
        info “>>> Starting measurements”
        g.startApplications
      end
 

We set a bandwidth of 10 MHz, a central frequency defined by the property FDD_DL_FREQ, enable the ‘channelpower’ measurement point and start broadcasting. After 20 seconds we start all the applications defined for the group. This will signal all the measurement threads to start requesting data from the E2010 and store them in the database. The 20 seconds delay depends on the type of the UE tested because they usually have an internal timer to check the carrier before starting the LTE attach procedure.

10 seconds after the experiment starts we make the eNodeB to signal the UE to use its maximum power output, while the measurement process is still running, and we should see a change in the collected data.

      after 30.seconds do
        info “>>> Changing UE power”
        g.resources[type: 'e2010'].FDD_CNTRL_CONNECT_UL_TPCCTRL=1
      end
      after 40.seconds do
        info “>>> Stopping measurements”
        allGroups.stopApplications
        g.resources[type: 'e2010'].FDD_CNTRL_BCAST_STAT = 0
      end

 

The measures are stopped when the corresponding application stops, and we then shut down the carrier of the eNodeB. The only step left is to release the experiment resources as we did in the first experiment.



Main menu 2

Page | by Dr. Radut