Visualize real-time data streams with Gnuplot
For the last couple of years, I've been working on European Space Agency (ESA) projects - writing rather complex code generators. In the ESA project I am currently working on, I am also the technical lead; and I recently faced the need to (quickly) provide real-time plotting of streaming data. Being a firm believer in open-source, after a little Googling I found Gnuplot. From my (somewhat limited) viewpoint, Gnuplot appears to be the LaTEX equivalent in the world of graphs: amazing functionality that is also easily accessible. Equally important, Gnuplot follows the powerful paradigm that UNIX established: it comes with an easy to use scripting language, thus allowing its users to prescribe actions and "glue" Gnuplot together with other applications - and form powerful combinations.
To that end, I humbly submit a little creation of mine: a Perl script that spawns instances of Gnuplot and plots streaming data in real-time.
| Update, April 6, 2009: At the request
of my friends from ESA, I created another version
of the script, that works with a different kind of input. Details further down... Update, November 30, 2009: Andreas Bernauer has improved the script further, allowing multiple streams to be plotted in the same window. His work is here. Update, December 20, 2009: Dima Kogan has done his own version, which detects the number of streams dynamically. He placed his code on GitHub. |
Plotting data in real-time
Interfacing over standard input
My coding experience has taught me to strive for minimal and complete interfaces: the script plots a number of streams, whose data will arrive over the script's standard input, one sample per line. The samples are just numbers (integers or floating point numbers, must work with both), and each plot window will be configured to display a specific number of samples.The resulting script is relatively simple - and easy to use:
Usage: ./driveGnuPlots.pl <options> where mandatory options are (in order): NumberOfStreams How many streams to plot (windows) Stream1_WindowSampleSize this many samples for stream1 window <Stream2_WindowSampleSize> ...for stream2 window... ... <StreamN_WindowSampleSize> ...for streamN... Stream1_YRangeMin Stream1_YRangeMax Min and Max values for stream 1 <Stream1_YRangeMin Stream1_YRangeMax> Min and Max values for stream 2 ... <StreamN_YRangeMin StreamN_YRangeMax> Min and Max values for stream N |
An example usage scenario: plotting sine and cosine
Let's say that for whatever reason, we want to see a sine and a cosine run at real-time - and also watch them run at different "zoom" levels. The following code will print our test samples:|
#!/usr/bin/perl -w
use strict;
# First, set the standard output to auto-flush
select((select(STDOUT), $| = 1)[0]);
# And loop 5000 times, printing values...
my $offset = 0.0;
while(1) {
print sin($offset)."\n";
print cos($offset)."\n";
$offset += 0.1;
if ($offset > 500) {
last;
}
system("sleep 0.02") == 0
or last;
}
|
This is how to test the plotting script (assuming we saved the sample code above in sinuses.pl):
bash$ ./sinuses.pl | ./driveGnuPlots.pl 2 50 500 -1.1 1.1 -1.1 1.1 |
- 2 is the number of streams
- The window for the first stream (sine) will be 50 samples wide
- The window for the second stream (cosine) will be 500 samples wide (hence the different "zoom" factor)
- The y-range for the first window will be [-1.1, 1.1]
- The y-range for the second window will be [-1.1, 1.1]
Executive summary: plotting streaming data is now as simple as selecting them out from your "producer" program (filtering its standard output through any means you wish: grep, sed, awk, etc), and outputing them, one number per line. Just make sure you flush your standard output, e.g.
programName | grep --line-buffered 'pattern' | awk -F: '{print $2; fflush();}'
|
P.S. UNIX power in all its glory: it took me 30min to code this, and another 30 to debug it. Using pipes to spawned copies of gnuplots, we are able to do something that would require one or maybe two orders of magnitude more effort in any conventional programming language (yes, even accounting for custom graph libraries - you do have to learn their API and do your windows/interface handling...)
Update, April 6, 2009: At the request
of my friends from ESA, I created another version
of the script, that works with a different kind of input. It turns out
that the output you have to work with, might include the inputs
you want in random order and at different rates (i.e.
the frequency of reports for signal A are much more common than the
reports for signal B). Additionally, when preparing for a demonstration,
you want to label the outputs, and you also want to pre-arrange
the gnuplot windows' positions. To support these new features, the updated
version of the script expects these command line arguments:
Combined with the provisioning of titles (simple strings) and window placement information (in the form: WIDTHxHEIGHT+XOFF+YOFF, in pixels), this version of the script is perfect for demonstrations. |
| Back to homepage | Last update on: Sun Dec 20 14:23:45 2009 (Valid HTML, Valid CSS) |