Most of the software components in telekyb3
are able to produce data for
post-flight analysis. This logging is usually activated via a service named
log
(for instance, check the
rotorcraft log
service documentation). The logging can usually be stopped by a service called
log_stop
.
Log files produced by the log services are text files that contain one record per line. Each record contains multiple data points, separated by white space and sampled simultaneously.
Records beginning with #
are comments that contain generic information about
the log file or the context in which it was recorded.
Records with no characters other than blanks and a newline designate discontinuities in the log; it means that at least one record could not be logged and was skipped.
The first entry of a log file that is not a comment and not a blank record is a text field, and represent the key title of the corresponding data point found in the rest of the file.
Data points are most of the time numeric. They can also be the string nan
,
which corresponds to an invalid value or the character -
, which corresponds
to a missing value (or a value not changed since the previous data point).
Most of the time, records contain a ts
column which represents the sampling
time of the record in seconds since the Unix epoch. This is convenient to plot
synchronized data coming from different log files and components.
gnuplot
The log files are directly compatible with the
gnuplot
plotting tool. In particular, comments are
properly ignored, column titles are recognized and blank records are plotted
without a line joining them to indicate possible missing data. The nan
values
are ignored and the treatment -
special value can be configured properly.
This section cannot fully describe the gnuplot
tool: check the documentation
for details. However, simple examples are provided to get you started.
For instance, plotting the 'imu_az' column from a log file from rotorcraft
can be as simple as:
$ gnuplot
gnuplot> plot 'rotorcraft.log' using 'imu_az' with lines
More advanced usage include plotting multiple columns, adjusting the scale and adding some legend. Note how the columns are retrieved by name, making the plot robust to future column order changes in the log files.
gnuplot> set xlabel 'sample #'
gnuplot> set ylabel 'acceleration in m/s^2'
gnuplot> plot [][8.5:11] 'rotorcraft.log' using 'raw_az' with dots title 'raw imu a_z', 'rotorcraft.log' using 'imu_az' with lines title 'filtered imu a_z'
Or with the shorter syntax:
gnuplot> plot [][8.5:11] 'rotorcraft.log' u 'raw_az' w d t 'raw imu a_z', '' u 'imu_az' w l t 'filtered imu a_z'
Plotting versus time is usually more interesting than plotting versus the
sample number, especially when plotting multiple log files with different
sampling rates. As already mentionned, log files normally contain a ts
column
that represents the sample acquisition time in seconds since January
1st, 1970. As this number of seconds is difficult to interpret, it is usually
better to plot against the time in seconds since the beginning of the log.
This can be achieved by noting the first timestamp of a log file by visual
inspection, and using a variable in gnuplot
to compute the difference. This
example sets the t0
variable to the first sample time and uses the
column('name')
expression to retrieve a column by name inside a matematical
expression:
$ more rotorcraft.log
$ # note the first timestamp
$ gnuplot
gnuplot> set xlabel 'time in s'
gnuplot> set ylabel 'acceleration in m/s^2'
gnuplot> t0=1670524455.187710000
gnuplot> plot 'rotorcraft.log' u (column('ts')-t0):'imu_az' w l t 'imu a_z'
As a final remark, it’s worth noting that gnuplot
can use a special file in
your $HOME
directory called .gnuplot
. This file can be useful to define
your most often used commands or shortcuts (check the gnuplot "Substitution and
Command line macros" documentation). For instance, since it can be cumbersome
to first get the origin timestamp from multiple log files and then plot data,
the `$HOME/.gnuplot file can be used to define a macro doing the job for you:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# gnuplot startup file
#
set datafile missing '-'
set macros
set grid
t0 = `awk 2>/dev/null \
'BEGIN { ts = systime() } \
FNR == 1 { col = 0 } \
col && col < NF { if (ts > $col) ts = $col; nextfile } \
/^[^#]/ { for (i=1;i<=NF;i++) if ($i == "ts") { col=i; next } } \
END { print ts }' *.log || echo 0`
ts = "(column('ts')-t0)"
Line 3 defines the special string -
in input data files to denote a missing
data entry. Gnuplot makes a distinction between missing data and invalid data
(e.g. "NaN"). For example invalid data causes a gap in a line drawn through
sequential data points; missing data does not.
Lines 4-5 define common settings that are usually wanted.
Lines 7-11 define a variable t0
whose content is the result of an awk
script. This script simply checks all files ending in .log
in the current
working directory and extracts the minimum timestamp value found in them, so
that t0
eventually represents the first timestamp among all log files. This
script is of course not very robust and could easily be trapped. But it can
still be helpful in basic scenarios.
Finally, line 13 defines a macro that substracts t0
from the current value
of the column ts
, as in the previous example.
This sample .gnuplot
file can be copied to your $HOME
and tuned to match
your needs. With it, you can simply redo the previous plot like this, without
even manually checking the log file:
gnuplot> plot 'rotorcraft.log' u @ts:'imu_az' w l t 'imu a_z'