Performing measurements
Analyzing the data
  1. Locating the session data
  2. Session summary
    1. JSON data schema
  3. Event logs
    1. JSON data schema
    2. CSV data
    3. ODS data
  4. Comparison data
  5. Measurement report
  6. Plots

Locating the session data

By default, PowerGoblin stores the past session data in the logs/ directory inside the application's work directory. We can study the contents of the directory vie the web user interface:

The different files have the following meaning:

  • comparison.csv: Comparison of cumulative energy consumption in (processed) CSV format. (can be used for reconstructing the events of the previous session)
  • comparison.csv-comp-*.PNG: box plots comparing the measurements)
  • events.csv: Logged events of the measurements in (processed) CSV format.
  • events.csv-*.PNG: scatter plots of stacked runs
  • events.json: Logged events of the measurements in serialized JSON format.
  • events.ods: Logged events of the measurements in (processed) OpenDocument format.
  • plot.R: R script for drawing scatter plots from events.csv
  • plot2.R: R script for drawing box plots from comparison.csv
  • report.html: A HTML/JS report of the measurement session. Since the meter / measurement configurations can be quite complex for some sessions, the report has interactive functionality for switching between different visualizations. The file is self-container, i.e. all JS, CSS and image resources are included in the HTML file.
  • resources-UNITNAME.csv: Logged resource data from unit UNITNAME in (processed) CSV format.
  • resources-UNITNAME.json: Logged resource data from unit UNITNAME in JSON format.
  • resourcedump-UNITNAME.zip: An archived copy of the latest received resource dump file.
  • session.json: Session summary data. (can be used for reconstructing the previous session)
  • session.zip: This is an archive of the whole session that contains all the previously mentioned files (except the resource dump zip) in zip format. It's easy to just download this if you need to analyze the results elsewhere.
  • template.md: An archived copy of the template file for creating the energy report.

The user interface will label a session as corrupted if it is missing the session.json file or the file cannot be parsed correctly.

Session summary

The session summary file contains necessary information to reconstruct the previously organized measurement session.

JSON data schema

The JSON data is generated from the data classes located in the fi.utu.tech.powergoblin.model package. If you wish to extend the functionality of the application or build compatible tools, please refer to the source code in this package.

The strongly typed and well documented Kotlin representations work as schemas. See the unit test SerializationTest for examples of serialization and deserialization using these schemas.

The appName and appVersion fields in the InstanceSpecs (in InstanceSummary) and version in SessionSummary represent the schema version. In the future, the UI is expected to inform about incompatible JSON schemas.

Event logs

The event logs contain serialized data about the past events in the session. Since the logs can grow relatively large, the files are written incrementally, one line at a time. This is especially important to note when parsing the JSON files as the events are not stored as a single array.

The ODS spreadsheet, however, is written in a non-incremental way. This basically means that it shouldn't be used if the session can grow too large to fit in memory.

JSON data schema

The JSON data is generated from the data classes implementing the Event interface, located in the fi.utu.tech.powergoblin.model package. See the unit test SerializationTest for examples of serialization and deserialization using these schemas.

CSV data

The event data in CSV format looks like this:

default;2025-05-11 12:00:00;user
Measurement,Run,Timediff,TimediffRun,Meter,Channel,FriendlyName,MonotonicTime,Unixtime,Metertime,Voltage,Current,Power,Energy,Online
TESTSCENARIO1,1,14,0,TRIGGER,RUN,TRIGGER,192804700304783,1747021042224,NA,NA,NA,NA,NA,NA
TESTSCENARIO1,1,16,2,SP3-7A4B3DD6B19C5649C943DCBA0AA751D3,OUT1,SP3-7A OUT1,192804702540807,1747021042226,709414612,0,1,0,NA,FALSE
TESTSCENARIO1,1,16,2,SP3-7A4B3DD6B19C5649C943DCBA0AA751D3,OUT2,SP3-7A OUT2,192804702541327,1747021042226,709414612,0,1,0,NA,FALSE

The CSV fields have the following meaning:

Field Description
Measurement Measurement id (integer; 1, 2, ...)
Run Run id (integer; implicit id = 0) or 'NA' (no active run)
Timediff Time diff between the beginning of the session and current event
TimediffRun Time diff between the beginning of the measurement and current event or NA (no active run)
Meter Machine generated name of the meter or 'NA' (trigger event)
Channel Name of the meter's channel (driver dependent) or 'NA' (trigger event)
FriendlyName A new name mapping for the meter & channel
MonotonicTime High precision time stamp (may wrap around)
Unixtime The Unix time stamp of the PowerGoblin instance
Metertime Time stamp extracted from the meter (driver dependent format) or 'NA'
Voltage Voltage value in millivolts or 'NA'
Current Current value in milliamps or 'NA'
Power Power value in milliwatts or 'NA'
Energy Cumulative energy value or 'NA' (not measured)
Online Indicates whether the meter's channel and/or meter is active or 'NA' (trigger event)

In the future, the CSV schema may change without further notice. We are planning to implement functionality that can regenerate the CSV outputs from the JSON files using the most up-to-date schemas when loading and re-saving past sessions.

ODS data

The ODS file has two sheets. The first contains the name, date, and other summary data describing the session. The second sheet contains the same CSV data as the CSV file split in 15 columns for easier processing.

Comparison data

The comparison data in CSV format looks like this:

default;2025-05-11 12:00:00;user
Meter,MeterShort,Channel,MeasurementId,Measurement,Run,Energy
SP3-7A4B3DD6B19C5649C943DCBA0AA751D3,SP3-7A,OUT1,1,TestScenario1,0,0
SP3-7A4B3DD6B19C5649C943DCBA0AA751D3,SP3-7A,OUT1,1,TestScenario1,1,0
SP3-7A4B3DD6B19C5649C943DCBA0AA751D3,SP3-7A,OUT1,1,TestScenario1,2,0

The CSV fields have the following meaning:

Field Description
Meter Machine generated name of the meter
MeterShort Short name of the meter, if any
Channel Name of the meter's channel
MeasurementId Measurement id (1, 2, ...)
Measurement Measurement name (M-1, M-2, ...)
Run Run id (implicit id = 0)
Energy Cumulative sum of consumed energy during the run

Measurement report

The report consists of the following sections:

  • Summary
    • Data summarizing the measurement session
  • Measurements
    • Contains summary of the energy consumption of each meter in each measurement
    • For each measurement, contains the min/max/avg values for cumulative time and energy
    • For each measurement, contains the average voltage, current, and power
    • For each measurement, contains the scatter plot of the energy readings, grouped by runs
  • Meters
    • Contains specifications of each meter associated with the session
  • Resources
    • Contains information about the resource logs associated with the session
  • Runs
    • Contains the length of the run in seconds
    • Contains the energy readings for each meter in each run
    • Contains detailed per-meter min/max/avg voltage / current / power readings
  • Detailed specifications
    • Contains detailed information about the session
    • Contains detailed information about the instance configuration
    • Contains detailed information about the meters

Plots

If R and/or Docker has been set up properly and plot generation has been enabled, the log directory should also contain .png files for each measurement (scatter plots of stacked runs) and for each meter (box plots comparing the measurements).

The source files (*.R) for generating the plots are also stored in the session directory. These are provided as a useful starting point for generating plots for publications.