---

1. Scope and objectives
2. Measurement process
   1. High level description
   2. Measurement script
3. Setup
   1. Hardware setup
   2. Software setup
      1. Controller PC
      2. SUT-Back
      3. SUT-Front
      4. Router
      5. Meter
4. Performing the measurement
   1. Discovering the IP addresses
   2. Launching PowerGoblin
   3. Launching the back-end service
   4. Launching the measurement script
      1. Approach 1: Local user
      2. Approach 2: SSH
      3. Approach 3: SSH + Screen
      4. Approach 4: System service
      5. Approach 5: Modified service
5. Analyzing the results
6. External links

---

This project documents and implements a procedure for performing hardware based power/energy measurement of the back-end machine in a full-stack web application (see this example for a measurement setup for the front-end).

Hardware based here means that we have a dedicated measurement hardware for measuring the accumulated total consumption.

While power consumption quite likely correlates with the efficiency of the code (in terms of computational performance) and its execution speed, it is a distinct concept -- especially in the age of multicore processors. In practice, the consumption metrics in general cannot be derived from these other properties.

---

Scope and objectives

Full-stack applications typically consist of a front-end and a back-end. Even typical traditional web pages include both a front-end and a back-end, but the processing in the front-end may be limited to just rendering the page document without executing any application code. This emphasizes the role of the back-end as a contributor to overall consumption.

The measurement will determine the amount of power consumed by the back-end, without including the front-end's consumption. Since all kinds of services have a back-end, the measurement is not only suitable for (static) web pages and application that involve a browser and human user, but also (dynamic) API calls invoked by other applications, IoT applications, etc.

Typical focus points of measurements are the consumption of a page load, calls to one or more back-end endpoint, or some use case coordinated by a script (either a script orchestrating the front-end's execution or making direct calls to the endpoints). When measuring larger scenarios, we may want to measure the entire user session, which includes session state management either on the front-end, on the back-end, or both.

There is a significant and growing number of full-stack software deployment models and implementation technologies. Going through them here is too laborious and would require constant maintenance, which places such a survey outside the scope of our example. Instead, the goal is to demonstrate the process of augmenting an existing project implemented in Java & Quarkus with the code required for doing a measurement.

The implementation covered in this example demonstrates the measurement of back-end code with two main approaches: white-box test code triggers the measurement with code added to the actual endpoint code of the back-end. Black-box test code initiates the measurement and works with unmodified back-end code.

---

Measurement process

We study the structure of the measurement process in this section. This is a typical measurement performed by an automated testing script. It has the following important properties which basically makes it suitable as a basis for scientific research:

• Fully documented process
• Reproducible procedure
• Open data
• Generated results usable for quantitative research

The whole measurement session can be fully automated if needed. Our example uses a hybrid approach where the user is still needed to initiate the measurement process, but the actual measurement runs are done automatically. Tools can greatly help when aggregating the result data, especially when performing a large number of measurements.

High level description

On high level we identify the following units required for this measurement: PC (controller PC, PowerGoblin instance), SUT-Back (executes the back-end service), and SUT-Front (executes the measurement script).

We perform the following tasks in a sequence:

Unit	Task
All	Set up the hardware and software
PC	Launch PowerGoblin
SUT-Back	Launch the back-end service
SUT-Front	Launch the measurement script
-	Wait for the measurement to finish..
PC	Analyze the logs & plots

The measurement session is fully deterministic without exceptions. The first three phases are not time-critical, and they are performed manually. The execution of the measurement script is automatic.

Measurement script

The following operational structure is extracted from the test-backend.sh script:

• MSG: start measurement session (line 11)
• MSG: upload the template for a measurement report (line 12)
• MSG: start the "blackbox" measurement and rename it (lines 14-15)
• for run in { 1, 2, 3, 4, 5 }
  • MSG: start run (line 18)
  • perform GET request http://sut-back/hash
  • MSG: stop run (line 20)
• MSG: stop the measurement (line 22)
• perform the GET request http://sut-back/mhash (initiates the "whitebox" measurement)
• MSG: close the measurement session (lines 26-27)

In this abstract description, the messages are sent over HTTP/JSON, as PowerGoblin provides REST like endpoints for such messages. The terminology used by PowerGoblin is explained in the usage section of its manual.

---

Setup

Hardware setup

The hardware configuration diagram for this demonstration is presented here.

The graph is divided into four clusters, starting from the left:

• PC (controller, PowerGoblin instance)
• Power meters (Meter, Meter #2, PSUs, USB Hub)
• SUT-Front (front-end system under test)
• SUT-Back (back-end system under test)
• Network router

Relevant hardware interfaces are indicated by the labeled edges in the graph. All the power meters are connected to the controller PC via USB, all the SUT and network devices are connected to the power meters with DC cables, and all the SUT, network, and controller devices also communicate using the ethernet network provided by the network router.

Although this hardware configuration enables the power measurement of both the front-end and the back-end, in this example we focus on the measurement of the back-end devices and software.

Software setup

On the devices we use the following software setup:

• Controller PC: PowerGoblin + Arch Linux (any supported operating system should work)
• SUT-Front: Arch Linux (any PC compatible operating system should work)
• SUT-Back: Arch Linux (any PC compatible operating system should work)
• Router: OpenWRT (official factory firmware should also work)
• Meter: latest official firmware

Controller PC

See the PowerGoblin's documentation on downloading and deploying the software. In this example we download the "fat" jar distribution, which should work on any platform:

$ wget https://tech.utugit.fi/soft/tools/power/powergoblin/powergoblin.jar

The file can be downloaded with any browser as well.

In addition, a Java distribution is required on the machine. The manual describes this as well.

SUT-Back

We have installed Arch Linux using archinstall.

We use the following configuration for the installer:

• Locales: us / en_US / UTF-8
• Disk configuration: best effort default / ext4 or btrfs
• Swap: on zram
• Bootloader: systemd-boot
• Hostname: sut-back
• Audio: no
• Profile: Minimal (others will install unnecessary software and require a lot more disk space.)

We also need to install the following extra software:

• openssh: for connecting from the controller PC
• docker: for setting up the server application with containers

$ sudo pacman -Suy
$ sudo pacman -S docker openssh

Enable ssh to make the machine accessible over the network and Docker to set up containers:

$ sudo systemctl --now enable sshd docker

Reboot.

The back-end service for this example is provided as a source package. Building the back-end requires curl, tar, OpenJDK 21+, Apache Maven, and Docker:

$ sudo pacman -S jdk-openjdk curl tar maven

The example can be built by invoking the following commands:

$ curl https://tech.utugit.fi/soft/tools/power/doc/fullstack-example.tar.gz | tar xfzv -
$ cd fullstack-example
$ mvn package
$ docker build . -t fullstack-example

At this point we have a Docker image available for deploying the Docker container(s) for the back-end service(s). Other third party services could be deployed in a similar fashion. The Docker and Docker Compose tutorials provide starting points for building a service with Docker technologies.

SUT-Front

We have installed Arch Linux using archinstall.

We use the following configuration for the installer:

• Locales: us / en_US / UTF-8
• Disk configuration: best effort default / ext4 or btrfs
• Swap: on zram
• Bootloader: systemd-boot
• Hostname: sut-front
• Audio: no
• Profile: Minimal (others will install unnecessary software and require a lot more disk space.)

We also need to install the following extra software:

• openssh: for connecting from the controller PC
• python: measurement scripts
• python-requests: measurement scripts
• python-dotenv: measurement scripts
• screen: background session

First, install the extra software:

$ sudo pacman -Suy
$ sudo pacman -S openssh python python-requests python-dotenv screen

Enable ssh to make the machine accessible over the network:

$ sudo systemctl --now enable sshd

From the example distribution, we will only use the Python scripts. The scripts can be extracted by invoking the following commands:

$ cd /
$ curl https://tech.utugit.fi/soft/tools/power/doc/fullstack-example.tar.gz | sudo tar xfzv -
$ cd fullstack-example

The extracted test-backend.py script will be started in the next step.

In this example, we have suggested running the measurement script either on the back-end or front-end. Due to this, the same core software configuration (Arch, systemd, ssh) applies to the front-end as well. If one wants to run the scripts on the sut-back instead, the Python dependencies need to be installed there as well.

Router

Standard OpenWRT installation should work with default settings. The default settings will set up a bridge, NAT, DHCP, DNS, and firewall. The machines should be connected to the LAN ports. The OpenWRT's web interface can be used for discovering the IP addresses of the devices.

Meter

The PowerGoblin manual explain the process of updating the power meter's firmware.

---

Performing the measurement

Discovering the IP addresses

First, go to the router's web user interface (e.g. 192.168.1.1 should be the OpenWRT's default setting). If you cannot locate this page, find out the PC's IP address and try another one where the fourth octet (number) is 1. The page should list the addresses of the controller PC and the SUT if both are connected.

In this example we assume the following assignment of addresses:

Device	Host name	IP address	Service port
Router	router	192.168.1.1	-
Controller	controller	192.168.1.10	8888
SUT-Front	sut-front	192.168.1.20	-
SUT-Back	sut-back	192.168.1.30	8080

Launching PowerGoblin

On the PC, we assume that the PowerGoblin has been installed in the current working directory and Java has also been installed. You can start the application by running:

$ java -jar powergoblin.jar httpPort=8888 -cors

When launching the application, it should display the IP addresses it is listening to, e.g.

Web [Simple HTTP @ 8888] listening at [ http://172.18.0.1:8888 ]
Web [Simple HTTP @ 8888] listening at [ http://172.17.0.1:8888 ]
Web [Simple HTTP @ 8888] listening at [ http://192.168.1.10:8888 ]

Here http://192.168.1.10:8888 is the address we are going to use later. We use the service port 8888 instead of the default, 8080, to avoid service conflicts in configurations where the back-end service runs on the same machine.

Launching the back-end service

Assuming the Docker image for the back-end service has been build or pulled on the SUT-Back, we can run:

$ docker run -e POWERGOBLIN=controller:8888 --network host fullstack-example:latest

The back-end service does not need any further configuration and is now ready for doing measurements.

Launching the measurement script

When measuring the back-end, there are multiple possible configurations for setting up the measurement. If we want to simulate a normal interaction by a client, we could run an external, simulated script on the front-end machine. if the script has a negligible effect on the overall consumption, we can also run the script in the back-end machine. Sometimes it is not possible to use a separate script, but instead inject the code to be executed in the back-end service.

The external measurement script looks like this:

#!/bin/python

from powergoblin import PowerGoblin
import time
import requests
import os

pg = PowerGoblin(os.getenv('POWERGOBLIN'))
web = os.getenv('BACKEND')

pg.get("cmd/startSession")
pg.post_file("session/latest/import/template", "README.md")

pg.get("session/latest/measurement/start")
pg.get("session/latest/measurement/rename/Blackbox")

for i in range(5):
    pg.get("session/latest/run/start")
    requests.get(f"http://{web}/hash")
    pg.get("session/latest/run/stop")

pg.get("session/latest/measurement/stop")

requests.get(f"http://{web}/mhash")

pg.get("session/latest/close")
pg.get("session/latest/remove")

The addresses of the other services are provided to the script via environment variables BACKEND and POWERGOBLIN.

There are now several options to start the measurement:

1. Run the measurement script directly on the front-end / back-end machine by logging in locally.
2. Connect to the back-end machine via SSH and run the measurement script.
3. Connect to the back-end machine via SSH, launch the screen and run the measurement script.
4. Run the measurement script as a systemd system service or in a Docker container as a service launched by the service at boot time of the device.
5. Edit the source code of the back-end service (or configure it to load extra plugins if supported by the framework) and inject the necessary commands to initiate and finalize the measurement.

Approach 1: Local user

This is the most straightforward approach for coordinating the measurement. The most significant downside is that the measurement now include both the consumption of the back-end service and the measurement script. Another issue is that local access to the back-end system is needed. It cannot be a headless machine.

login: abc
password: abc
$ export BACKEND=sut-back:8080
$ export POWERGOBLIN=controller:8888
$ python test-backend.py

Approach 2: SSH

Controlling the back-end via SSH is almost as straightforward as the previous approach, but requires a secure configuration of SSH. The most significant downside is that the measurement now include both the consumption of the back-end service and the measurement script. The network connection is also active during the measurement, and can thus affect the results.

$ ssh user@sut-front
$ export BACKEND=sut-back:8080
$ export POWERGOBLIN=controller:8888
$ python test-backend.py

Approach 3: SSH + Screen

Controlling the back-end via SSH inside a GNU screen session is almost as straightforward as the previous approach, but requires starting of the GNU screen and detaching after the startup. The most significant downside is that the measurement now include both the consumption of the back-end service and the measurement script.

Example:

$ ssh user@sut-front
$ screen
$ export BACKEND=sut-back:8080
$ export POWERGOBLIN=controller:8888
$ python test-backend.py

To avoid extra network traffic, one can type Ctrl-A, D, Ctrl-D after starting the script. This could be combined with a small delay in the beginning of the script to isolate this from the actual measurement session.

Approach 4: System service

The idea of this approach is to launch the measurement script right after booting the machine. Setting up the service is possible via SSH if the system is headless.

First we need to create the following file in /etc/systemd/system/mscript.service:

[Unit]
Description=Measurement script service
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/usr/bin/bash /fullstack-example/mscript.sh

[Install]
WantedBy=multi-user.target

The service can be activated with the following commands:

$ systemctl daemon-reload
$ systemctl enable mscript.service

The script should now launch after rebooting the system, right after the network configuration has been set up.

Approach 5: Modified service

In this approach the idea is to modify the code of the service directly. Some services also support loading of modified code via some plugin or extension API. In our example, the distribution of the example back-end service already contains the injected measurement code in org/acme/MeasuredHashResource.java:

public class MeasuredHashResource {
  private static GoblinClient pg =
      new GoblinClient(System.getenv("POWERGOBLIN"));

  ...

  public String get() {
    pg.get("session/latest/measurement/start");
    pg.get("session/latest/measurement/rename/Whitebox");

    for (int j = 0; j < 10; j++) {
      pg.get("session/latest/run/start");

      for (int i = 0; i < 50; i++)
        readIpsum();

      pg.get("session/latest/run/stop");
    }

    pg.get("session/latest/measurement/stop");

    var result = readIpsum();

    return result;
  }
}

The added Java code assumes that the measurement session has been started and closed outside the call to the endpoint, because the measurement session likely has a longer duration than the measurement. In practice, the lines 14--22 of our measurement script are unnecessary for this case, but the other lines are needed for coordinating the measurement.

---

Analyzing the results

The PowerGoblin UI (available at http://192.168.1.10:8888 in our setup) contains visualizations and listings for preliminary analysis in the 'Measure' tab (on the top, third from the left). The resulting measurement logs ('Logs' tab) are stored in the logs folder and can also be downloaded via the UI.

Further analysis can be performed by analyzing the measurement logs with the help of analysis tools like Python / R.

---

External links

• Project repository in UTU GitLab: https://gitlab.utu.fi/tech/soft/mitvidi/software-measurement