UTU Logo EU Logo ELY Logo EU Logo

  1. Measurement plan description
    1. Metadata block
    2. Report related fields
    3. Agent related fields
    4. Calibration related fields
    5. Markdown document
  2. Example
  3. Next step: Setting up a measurement plan

This section documents the structure of the measurement plans. The plan is defined by a plan description file (measurement.md). The file along with the other files required by the measurement must be either manually added to the node or downloaded with the agent's download command.

The plan then describes how the measurement should be performed. PowerGoblin has a layered structure. The first iteration of PowerGoblin only had support for a single session per execution. Later, support for concurrent sessions was added. As more and more features were added to describe and configure a session, instead of defining new API endpoints for each new feature, a plan description file was developed. This file basically

The purpose of the agent is to manage the execution of measurements on the nodes. The agent can also perform preparatory tasks such as enumeration of measurement plans and calibration of the meters.

Measurement plan description

When launching the agent, by default it will browse the subdirectories and attempts to parse all files with the name rootPath/*/measurement.md.

The file has the following structure:

  • A YAML metadata block
  • Markdown document that

Metadata block

The metadata block uses the YAML syntax:

---
key: value
another-key:
  - value-list item 1
  - value-list item 2
---

The metadata block recognizes the following special fields:

  • name: renames the session / used for listing measurements
    • shown in the UI and report
    • no special requirements
    • should not be too long
    • API: also available via API
  • author: used for listing measurements
    • shown in the UI and report
    • no special requirements
    • currently only a name, no email address
    • API: also available via API
  • description: used for listing measurements
    • shown in the UI and report
    • no special requirements
    • maximum length ~one paragraph
    • API: also available via API
  • active-channels: used in the measurements
    • format: meter/channel=friendly_name Description that can have spaces
      • special meter/channels: RAPL/node-core, RAPL/node-package-0
    • when starting a measurement, affects the selected meters:
      • initialization-strategy=IncludeNone
        • do not include any channels / nodes
      • initialization-strategy=IncludeAllFree
        • include all available channels / nodes
      • initialization-strategy=IncludeActive
        • includes only active channels / nodes, if available
      • initialization-strategy=RequireActive
        • include the active channels / nodes, fail to initialize if something is missing
        • does not consider RAPL meters when initializing
    • in the report view, the total energy readings from active meters/channels are shown by default
    • API: No specific endpoints, available via the variables API (import/variables)
  • active-resources: used in the measurements
    • format: node|unit/resource=friendly_name Description that can have spaces
    • TODO: do we convert : to / now?
    • API: No specific endpoints, available via the variables API (import/variables)
  • active-nodes: used in the measurements
    • format: list of node names
    • API: No specific endpoints, available via the variables API (import/variables)
  • initialization-strategy:
    • format: one of
      • IncludeNone | IncludeAllFree | IncludeActive | RequireActive
    • API: No specific endpoints, available via the variables API (import/variables)
  • resource-filters: used in the measurements
    • format: list of substrings used for matching (union)
    • the idea is to filter the resources before logging because some systems produce excessive amounts of resource data
    • 'counter' is always needed for RAPL/NVML meters
    • API: also available via API
  • report-modes: The names of the modes
  • report-palette: The palette to use for the report UI.
  • visible-sections: Sections visible in different modes. A comma separated list entry should be provided for each mode and the number of modes should match.
  • expanded-sections: Sections expanded in different modes. A comma separated list entry should be provided for each mode and the number of modes should match.
  • plan-id: unique UUID code of the plan
    • API: no, the concept is only relevant in the context of agents.
  • collector: used in the measurements
    • format: list of collector instructions for agents
      • agentId=list,of,collector,flags
    • sets up the collector for the 'agentId'. The flags control features such as RAPL
    • can configure multiple agents (multiple items)
      • the agent id must match the node name
      • currently unimplemented for other collectors than the agent itself
    • API: no, the concept is only relevant in the context of agents.
  • script: scripts to be called when measuring
    • format: list of shell command invocations
    • each item can contain space separated arguments
    • if the first item ends with .sh, it is considered a Bash script and opened with bash
    • if the first item ends with .py, it is considered a Python script and opened with python
    • other scripts must be started with the interpreter, not directly (a full path to the executable must be provided as the first item)
    • the command line will be appended with the extra arguments:
      • server host name
      • session id
      • run id
    • the scripts are responsible for starting measurements & runs. The agent takes care of starting and configuring the session, starting the collectd collectors, and synchronizing the time stamps.
    • API: no, the concept is only relevant in the context of agents.
  • runs: number of runs
    • common for all measurements
    • can be non-numeric - e.g. comma separated numbers
    • measurement script
    • API: no, the concept is only relevant in the context of agents.
  • timeout: maximum time to run the measurement
    • unit: seconds, a long integer
    • the same timeout is used for all scripts and the collector
    • API: no, the concept is only relevant in the context of agents.

The following undocumented fields are used for the calibration task:

  • calibration: calibration target
  • calibration-threshold: threshold value for determining calibration targets
  • calibration-delta: delta value for determining calibration targets
  • node: name of the node associated with the calibration
  • units: units that are potential targets for the calibration
  • exclude-units: units that should be excluded for calibration

Markdown document

The Markdown document is used as a source template for the final measurement report. It also comes with a couple of special features.

The document consists of sections. Each section is separated with a line that only contains --- and nothing else. The sections are associated with an index, starting with 0, 1, 2, ... These indices can be referred to by the special fields visible-sections and expanded-sections.

The Markdown parser supports the following special commands:

  • [BOXPLOT]: Add a box plot widget
  • [SUMMARYTABLE:boxplot-1]: Add a summary table, associated with the first box plot. Note: the 1 here is the id of the widget. All widgets have an id.
  • [SCATTERPLOT]: Add a scatter plot widget
  • [NODEDATA]: Add a node data widgets for all nodes
  • [NODEDATA:name]: Add a node data widget for node 'name'

Example

---
name: System utility benchmarks
author: Captain Planet
description: Runs CPU benchmarks using the cryptsetup utility.
active-channels:
  - RAPL/sut-package-0=sut_cpu CPU package power via RAPL
  - RAPL/sut-core=sut_core CPU core power via RAPL
active-resources:
  - sut|memory/memory-free=free_ram Free RAM memory
  - sut|memory/memory-used=used_ram Used RAM memory
active-nodes:
  - sut
resource-filters:
  - memory-used
  - memory-free
  - memory-cached
  - memory-buffered
  - if_octets
  - cpu-user
  - cpu-system
  - cpu-idle
  - counter
initialization-strategy:
  - RequireActive
sections:
  - intro
  - another
collector:
  - sut=ACTIVE,HTTP,RAPL
script: 
  - algorithm1.sh
  - algorithm2.sh
  - algorithm3.sh
runs: 4
timeout: 600
report-modes:
  - Technical Report
  - High-Level Report
report-palette:
  - #0b84a5
  - #f6c85f
  - #6f4e7c
  - #9b5de5
  - #ff6b6b
  - #3ac569
  - #ffa600
visible-sections:
  - 0,1,2,3,4
  - 0,1
expanded-sections:
  - 0,1,2
  - 0,1
---


This is an example.

![](images/image.png)

---

## Summary

[BOXPLOT]

[SUMMARYTABLE:boxplot-1]

---

## Overview of the runs

[SCATTERPLOT]

---

## Nodes used in the measurement

[NODEDATA]

Next step: Setting up a measurement plan

The API for controlling the agent is described here as well.