Staff Manual

Purpose

What is Staff?

Staff stands for Solar Timelines viewer for AFFECTS.
AFFECTS (Advanced Forecast For Ensuring Communications Through Space) is a project funded by the FP7 program of the European union.
Staff has been designed to show different types of data to the user in the form of 1D (one-dimensional) timelines.
It is capable of showing multiple sets on one plot, or alternatively you can open multiple pages at once.

Staff combines the data of different sources into one easy-to-use source.
It can be used by professionals and amateurs alike for various reasons; be it Space Weather Forecasting or using the plots for an article.

Note that currently STAFF consists of three applications:

• The STAFF Viewer.
  This is the main application and the primary usage of STAFF data.
   
• The StaffBox
  With very simple programming techniques the plots can be grafted into your own website, without any bells and whistles.
  Check the tab "The StaffBox" for more information.
   
• The STAFF Web Services
  With some programming knowledge, the STAFF data and objects are available for usage in code. Only data and metadata, no plots.
  Good for usage in models, calculations or plugging in your own plotting tool.
  Check the tab "Staff Web Services" for more information.
   

If you find an error in a dataset, feel free to let us know and though we strive for quality by putting multiple data checks are in place, STAFF will not take responsibility for errors from the source of the data.

Navigating (how to)

There are a few ways to navigate through Staff.

• Menu
The menu allows more advanced (and less frequently used) options.
• File
• Import Plot Parameters
Import (previously exported) parameters to regenerate a plot.
• Export Plot Parameters
Export parameters to regenerate a plot later (via the Import Plot Parameter menu).
• Export as file
Export the current data and save as a datafile. (xls, csv, ...)
• Export as image
Export the current plot as image. To save, right click the image in the popup and select save as... in the option menu.
• Advanced
• Plot Options...
• Carrington Rotation View...
Opens a new page and shows three plots.
1. current plot as shown in the main window.
2. A plot with the same timeframe as the main window, but with a time shift of 1 Carrington Rotation.
3. A plot with the same timeframe as the main window, but with a time shift of 2 Carrington Rotations.
• Split graph View...
Shows the data as selected in the main window, but with a different plot area for each series.
• Statistics...
Shows a page with statistical data for example the timewindow of the available data for each plot.
• Reset
Resets STAFF to the default situation (GOES X-Ray selected)
• About
• Dataset Information
Gives information about the available datasets. Where they come from and links to the source.
• Contact
Provides a form to send questions, provide feedback or just say hello.
• References
Provides interesting links to sites related to Staff.
• Help
Opens the dialog showing this info among many other things.
• About
Quick info about who, what, where, when.
• Accordeon principle
The site has parts that open or close by sliding, effectively showing/hiding various parts.
By using this technique the major part of the page is used for showing what really matters: The plot.
There are two accordeon parts in this page: Dataset Selection and Timerange Selection

Dataset Selection

Select the datasets you would like to see.
Datasets (eg. GOES X-Ray 0.1-0.8nm) are grouped into classes (eg. X-Ray) in different tabs.
If a class has a set selected the tab title will turn red so you can easily see where the selected sets are.
Each class has an icon:

•  Sun
•  Heliosphere
•  Earth
•  Image based
•  Quality information

A marks a selected plot, a an unselected plot.
It is advised not to select too many plots at the same time, because you can lose the overview very quickly.

each category has a subcategory (which is sort of a quality label).

• quicklook,
• science,
• predictions 1 (p1),
• predictions 2 (p2),
• predictions 3 (p3)

Science data is mostly validated, calibrated data and therefore the most qualitative.
quicklook is (more or less) raw data.
Predictions is (near future) forecast data that is generated using some model. They are not actual measured data. Several predictions can exist for some plots.
Use the draw (, ) button to plot the selected data if you're happy with the selected time period, else click on "Timerange Selection" for further options. This avoids unnecessary postbacks to the server (slowing the site down).

Timerange Selection

The plot control allows the user to quickly select a time range for the selected plots.
In the statistics page you can find the available timeranges for all datasets (menu "Advanced - Statistics").
There are three ways to select a time range:

1. Start - End time
   You can enter the datetime with the keyboard or select the values from the datetime selector that pops up.
2. Start time with duration
   You can select any start datetime, but this option will force the start/end time to a close range. eg. 21/03/2013 23:00:00 - 3 Days will result in a selected time range from 21/03/2013 00:00:00 to 24/03/2014 00:00:00
3. quick menu - Last [timerange]
   This option will show the selected period including the current datetime.

Additionally you can use the and arrows to navigate forwards or backwards in the selected timewindow.

Time is shifted forward/backward with the difference of the start/end time values.

In other words, if start/end has a one day difference, the back and forth buttons will shift time with one day,
if the difference is one year, the back and forth buttons will shift with one year, etc...
The Start/End radio option is automatically set when using these buttons.

Use the draw (, ) button to plot the selected data after you finished making the selections. This avoids unnecessary postbacks to the server (slowing the site down).

All times in STAFF are in UTC.

Some selected time frames containing a daylightsavings change can come out with a distorted X-axis. This is due to a limitation in the library and might be in combination of the selected time zone on your browser and/or computer. The data that you export from it however, is correct.

The Plot Area

The plot area is always visible. This is where the plot will show.

In addition to the plot, there is a toolbar with some basic functionality that is also available in the options.

• Clear Selection:
Clears the dataset selection, so you don't have to do that manually. The draw button disappears until you selected at least one timeline.
• Auto Refresh time out value:
Value in minutes before an auto refresh of the page occurs (if auto refresh is "on")
• Auto Refresh time out on/off:
Toggles the auto refresh on/off. When "on" an indication appears.
Auto refresh is "on" and pressing it, will switch it "off".
Auto refresh is "off" and pressing it, will switch it "on". Auto refresh needs a redraw to take effect.
• Maximum Downsample threshold value:
Increases the resolution of the plot. Value is limited to 20000. Changing the resolution needs a redraw to take effect.
• Draw button:
Redraws the plot
Draws the plot, but except for updated data, nothing will change.
"dirty state", the plot will redraw with the changes you made in dataset selection, time range selection or in options.

Zooming

You can zoom into the plot by dragging a square inside the plotarea.
You can zoom out again one level at the time by double clicking and it is possible to zoom in several times and to zoom out several times effectively allowing drill-downs into to the data.
if you drag a square on the plot, a postback is sent to the server and new data is sent back within the time range of your selected zooming.
To zoom back out you can double-click the plot. However, after setting a datetime manually via the plot control, the memory containing the timewindows while zooming in, is cleaned. Therefore the zoomout function will not work after the manual timewindow has been set.

Plot markers

Occasionally one or more markers will be visible. A small description is shown if you hover over the marker.

• vertical
• Current date: Marked by a dashed red line on the current datetime.
Present on: All plots containing current date.
• horizontal
• X Flare: Marked by a full red line on value 1x10⁴
Present on: GOES X-Ray plots.
• M Flare: Marked by a full purple line on value 1x10⁵
Present on: GOES X-Ray plots.
• C Flare: Marked by a full blue line on value 1x10⁶
Present on: GOES X-Ray plots.
• B Flare: Marked by a full green line on value 1x10⁷
Present on: GOES X-Ray plots.
• 10.0 mark:Marked by dashed black line
Present on: GOES particles and Ace Solar Wind (density).
• 1000.0 mark:Marked by dashed black line
Present on: GOES particles.
• 0.0 mark:Marked by red line
Present on: Ace Interplanetary Magnetic Filed (IMF).
• 1.0 mark:Marked by dashed black line
Present on: Ace Solar Wind (density)
• 100 000.0 mark:Marked by dashed black line
Present on: Ace Solar Wind (temperature)

Plot data

The data is usually presented by a a series of dots connected with a line, though some data is represented by bars and it is possible to change some options in the options menu.
Hovering over a data point will show a small tooltip with the associated values.
The data can be downsampled data, it is therefore prudent to interpret the data correctly as this process can "flatten" peaks.
The downsample level is shown in the legend.

If no data is available, the plot area will remain blank.
For now, colors are assigned automatically by the plot library and are not related to the category.

Plot Options

Allows the user limited control over the plots.

Category	Option	Type	Definition
Filtering	Show all datasets	On/Off	Allows the user to show all datasets or just the default ones.
	Class On/Of	On/Off	Allows a user to completely switch on/off an entire class. (Sun, Heliosphere, Earth, Image, Quality)
Auto Refresh	Auto Refresh timeout	integer	If the Auto Refresh option is On, the page will refresh this amount of minutes. minimum value: 1 maximum value: 10
	Auto Refresh	On/Off	If On, the page will refresh after each amount of minutes, as set in the Auto Refresh Timeout, has past.
Plot Options	Plot Area Size	integer	Determines the width and height of the plot. Especially useful on wide screens or for higher resolution export to image.
	Adapt to screen	On/Off	If "on" the plot will try to follow the width/height of the browser, instead of taking a fixed width/height. (The fixed width/height is particularly useful if you want to export to an image.)
	Plot Background Color	Option	Default/Black. Allows the user to change the background color of the plot. That is, the area including the title, legend and entire plot.
	Grid Background Color	Option	Default/Black/White. Allows the user to change the background color of the grid. That is, the area within the axes.
	Grid Lines Color	Option	Default/Black/White. Allows the user to change the line color of the grid.
	Axis Labels Color	Option	Default/Black/White. Allows the user to change the color of the labels of the axis.
	Axis Label Font Size	integer	Change the font size of the axis labels.
	Connect Dots	On/Off	if On, connects the datapoints with a line.
	Tick Size	integer	Determines the tick size of the datapoints in the plots. A smaller value allows a finer plot, but makes it harder to hover on the point for more details. minimum value: 1 maximum value: 10
	Line Width	integer	Sets the thickness of the connecting line.
	Downsample Treshold	integer	Sets the amount of datapoints allowed before downsampling. Maximum value is 20000. The higher the value, the longer it takes to generate the plot.
	Legend Information	On/Off	Allows a user to set parts of the legend information on or off.
	Carrington Rotation X-Axis	On/Off	if On, shows a secondary X-axis with the Carrington Rotation number.
	Copyright	On/Off	If off, removes the copyright indication on the exported image. Please only use, if you mention the source somewhere else.

Export/Import data

Import Plot Parameters

To import a plot with previously exported parameters, "Select File - Import Plot Parameters". A small dialog appears. In the textbox paste the xml generated with an export and press the OK () button. The page will reload and the plot generated based on the parameters. (Cancel () to cancel the operation.)

Export Plot Parameters

To export a plot for later research eg, choose "Select File - Export Plot Parameters". A small dialog appears. Click the "Create" button. In the textbox copy the generated xml. You can then copy it to a text editor or mail it, ... for later use. The XML can be used in the Import Plot Parameters option. Close the dialog with the cancel () button.

As image

To export the data as image, "Select File - Export as image". A preview is shown in a dialog. To save the image, right click the preview and select "Save as..."

Please do not remove the copyright notice.

As data

To export the data as file, "Select File - Export as file". A preview is shown where data is presented in a table in a dialog. Here, several options are available:

• copy/paste the table

In this case, select the desired data and copy to the operating system's clipboard.
(generally this is ctrl-c or "right-click - copy")
Then the text can be copied to another application as-is.
(generally this is ctrl-v or "right-click - paste")

 

• export as file

Each series needs to be downloaded seperately by using the export button ().
A popup appears allowing to choose the file format and if the data should be exported as raw data (highest cadence available) or as shown on the plot.


Caution! The raw data size can become quite extensive !

The file can be opened in another window, and/or be opened in a default application for that extension on your operating system.
Often you are prompted to save the file to disk.

Downsampling algorithm

From milliseconds up until hours

Downsampling from these levels is done by selecting the average of all values, over the interval where the resulting datetimestamp is exactly in the middle of average interval. The resulting datetimestamp is logically chosen.

eg. 1)
for 5 minute data, the resulting datetimestamps are (from 9 to 10 o'clock):
09:00, 09:05, 09:10, 09:15, 09:20, 09:25, 09:30, 09:35, 09:40, 09:45, 09:50, 09:55.
This means that the values chosen to calculate the 09:00 downsample value are the values:
[08:58, 08:59, 09:00, 09:01, 09:02]

eg. 2)
for hourly data the resulting datetimestamps are (from 8 to 20 o'clock):
08:00, 09:00, 10:00, 11:00, 12:00, 13:00, 14:00, 15:00, 16:00, 17:00, 18:00, 19:00, 20:00.
This means that the values chosen to calculate the 09:00 downsample value are the values:
[08:30, 08:31, 08:32, ... 09:28, 09:29, 09:30[

From days up until years

For these downsample values the interval was achieved by using a group by clause of the source values.
This means that daily averages are calculated from the interval 00:00 until 23:59.

Additional notes

• Above explanation is based on 1 minute cadence data. In case of higher or lower cadence the details change, but the principle remains the same.
• All downsample values based on datasets with cadence 1 minute (or lower) are derived from the raw data. Only datasets with a higher cadence than 1 minute, downsample the values based on the previous downsample level.
• Only averages of more or equal than 50% complete intervals are taken (eg. 3 x 1 minute values for a 5 minute average, 30 x 1 minute values for a 1 hour average, ...), but although invalid values are counted for the amount of necessary values, they are not included in the average itself.
  This means that invalid values are only present in the raw dataset. These values are also excluded in the graph.

Levels

With the exception of 5 seconds and 5 minutes, the downsample levels are kept on time based units. In case of non timebased levels (eg. 3 hours, 6 hours) the raw data is put on the nearest higher cadence level.

code	level	explanation
ds	deciseconds	1 averaged value for each decisecond
ss	seconds	1 averaged value for each second
5s	5 seconds	1 averaged value for each 5 seconds starting at a whole interval (00, 05, 10, ...)
mi	minutes	1 averaged value for each minute
5m	5 minutes	1 averaged value for each 5 minutes starting at a whole interval (00, 05, 10, ...)
hh	hours	1 averaged value for each hour
dd	days	1 averaged value for each day
ca	carrington rotations	1 averaged value for each carrington rotation (only available on database level)
mo	months	1 averaged value for each month
yy	years	1 averaged value for each year

The StaffBox

The Staffbox is a way to include the power of Staff in your own webpage. You can show this in an iframe tag eg.

The StaffBox is a stripped version of Staff that only shows the graph.
By doing a post to this page you can create any graph that the normal Staff is capable of. The necessary parameters are:

Use the get parameter "notshowxaxislabels=true" to remove the x-axis labels. This allows you to position multiple plots closer together over a similar time range. eg. : http://www.staff.oma.be/StaffBox.jsp?notshowxaxislabels=true

Staff Web Services

WSDL

The WSDL can be found at: http://www.staff.oma.be/Staff/StaffWS?wsdl

Usage

To use the webservice you need to create an instance of the StaffWSClient object.

C#

StaffWSClient staff = new StaffWSClient();

Java

StaffWS_Service service = new StaffWS_Service(); StaffWS staffservice = service.getStaffWSPort();

When the instance is created you have three methods at your disposal:

Method name	Description	Return	Arguments
Step1_getAvailablePlots	Gets the available plots for STAFF	availablePlot[]
Step2_getPlotCategories	Gets the available categories for the plotnames passed via the argument	plotCategory[]	string[]
Step3_getDataPoints	Gets the datapoints for the given arguments (starttime, endtime, categoryname, precision). Datetimestamps are in format: yyyy-MM-dd HH:mm:ss Precision is any of the following: sc, ql, p1, p2, p3	dataPoint[]	string,string,string,string

You can also lookup the plotcategorynames and available precision (necessary for Step3_getDataPoints) on the tab "The StaffBox".

Class Descriptions

Sets if the category has science dataSets if the category has quicklook dataSets if the category has predictions 1 data Sets if the category has predictions 2 data Sets if the category has predictions 3 data stringstring

Class	Method name	Description	Return	Arguments
availablePlot	setPlotName	sets the name of the plot		string
	setDataLocation	sets the datalocation of the plot: sun, heliosphere, earth, image.		string
	setDescription	sets the description of the plot		string
	getPlotName	gets the name of the plot	string
	getDataLocation	gets the datalocation of the plot: sun, heliosphere, earth, image.	string
	getDescription	gets the description of the plot	string
plotCategory	setCategoryName	Sets the name of the category		string
	setDescription	Sets the description of the category		string
	setHasScience			boolean
	setHasQuicklook			boolean
	setHasPredictions1			boolean
	setHasPredictions2			boolean
	setHasPredictions3			boolean
	setInfoScience	Sets information about the science data		string
	setInfoQuicklook	Sets information about the quicklook data		string
	setInfoPredictions1	Sets information about the predictions 1 data		string
	setInfoPredictions2	Sets information about the predictions 2 data		string
	setInfoPredictions3	Sets information about the predictions 3 data		string
	getCategoryName	Gets the name of the category	string
	getDescription	Gets the description of the category	string
	getHasScience	Gets if the category has science data	boolean
	getHasQuicklook	Gets if the category has quicklook data	boolean
	getHasPredictions1	Gets if the category has predictions 1 data	boolean
	getHasPredictions2	Gets if the category has predictions 2 data	boolean
	getHasPredictions3	Gets if the category has predictions 3 data	boolean
	getInfoScience	Gets information about the science data
	getInfoQuicklook	Gets information about the quicklook data
	getInfoPredictions1	Gets information about the predictions 1 data	string
	getInfoPredictions2	Gets information about the predictions 2 data	string
	getInfoPredictions3	Gets information about the predictions 3 data	string
dataPoint	setDts	Sets the datetimestamp as string. Format: yyyy-MM-dd HH:mm:ss		string
	setDataPointValue	Sets the datapoint value		string
	setSampleLevel	Sets the downsamplelevel		string
	setUnit	Sets the unit of the value		string
	getDts	Gets the datetimestamp as string. Format: yyyy-MM-dd HH:mm:ss	string
	getDataPointValue	Gets the datapoint value	string
	getSampleLevel	Gets the downsamplelevel	string
	getUnit	Gets the unit of the value	string

The set functions are normally not necessary. You could use them however to create objects with your own data in the same structure as STAFF data, which makes it easier to plot them together or find corrolations.

The webservice uses the same sourcecode as the STAFF website, but augments the resolution to 30 000 points before downsampling.

Sample Code

C#

//Returns the available plots in STAFF availablePlot [] plots = staff.Step1_getAvailablePlots(); string [] plot = { plots[1].getPlotName() }; //Returns the categories available for the plots passed. plotCategory [] plotcategories = stafflocal.Step2_getPlotCategories(plot); //Returns the datapoints between the passed dates for that category and precision. //For the correctprecision check the HasXXX functions on the plotCategory object. //precision should be passed as sc, ql, p1, p2, p3 string [] arg = new string[4]; arg[0] = "2013-06-02 13:00:00"; arg[1] = "2013-06-07 16:00:00"; arg[2] = plotcategories[0].getCategoryName(); arg[3] = "ql"; dataPoint [] datapoints = stafflocal.Step3_getDataPoints(arg[0], arg[1], arg[2], arg[3]);

Java

//Returns the available plots in STAFF List<AvailablePlot> plots = staffservice.step1GetAvailablePlots(); ArrayList<String> plot_arr = new ArrayList<String>(); plot_arr.add(plots[1].getPlotName()); //Returns the categories available for the plots passed. List<PlotCategory> pc = staffservice.step2GetPlotCategories(plot_arr); //Returns the datapoints between the passed dates for that category and precision. //For the correctprecision check the HasXXX functions on the plotCategory object. //precision should be passed as sc, ql, p1, p2, p3 String [] arg = new String[4]; arg[0] = "2013-06-02 13:00:00"; arg[1] = "2013-06-07 16:00:00"; arg[2] = pc.get(0).getCategoryName(); arg[3] = "ql"; List<DataPoint> points = staffservice.step3GetDataPoints(arg[0], arg[1], arg[2], arg[3]);

Each coding language has its own way to connect to webservices and your actual code can therefore differ from the examples here. However, the function names do not change. For more information about how to connect to web services via a coding language, please refer to the documentation or forums for that coding language.

FAQ (Frequently Asked Questions)

Q: Something went terribly wrong and I receive weird error messages.
A: Try refreshing the page or opening a new instance. If all else fails send a mail through the contact form or to .
But do try to be as specific as possible so we can track down the exact issue.

Q: This is a really cool site, I have suggestions.
A: Thank you! Feel free to use the contact form and we'll gladly look into it (no promises though)

Q: How big are the datasets?
A: That highly depends on the source.
For GOES X-Ray there are about 1440 records a day. For the Ri number there is only one per day and for Lyra EUV anywhere between 1.5x10⁶ to 8x10⁶ !
And this is only counting the raw data (not the downsampling data). More info can be found under the menu "Advanced - Statistics".

Q: Sometimes I see many points, other times I see less points and it doesn't seem logical?
A: This phenomenon is due to the downsampling. An algorithm is in place to determin the correct downsampling based on the amount of datapoints expected to be retreived.
Simply put, A limit of about 2000 datapoints is allowed for one plot, if the number of datapoints gets higher than this treshold, the data is downsampled to the next level.

Q: I can only zoom too a certain point, it doesn't go further.
A: If you think there are still more points available on a deeper level (check the menu Advanced - Statistics), try setting the timewindow manually.
The plotting library is limited in its zooming levels.

Q: I have a great dataset that could be used in STAFF.
A: Feel free to contact us using the contact form. Explain the following:

• Is the dataset somehow solar related and how?
• Is the dataset free for use? (with perhaps a source reference)
• Give a short description about the dataset and of yourself (including contact details).

We will then contact you personally for further details.

Q: What's up with the bar plots?
A: You probably noticed that the bar plots don't take up the 'period of time' like they normally should. The reason for this is that if you do that, it becomes very complex, very quickly to plot "multiple" bar plots at once. In order to keep STAFF simple (and fast!) we decided to make the bars smaller so the values can be shown one next to the other. If you hover over a bar, the exact value of the datapoint with timestamp is shown in a tooltip. The placement of the bars is always next to the tick, with the first bar on the left, the second bar on the right, etc...

Q: The time stamps for the timelines of monthly and yearly sunspot numbers do not correspond exactly to STAFF’s timestamps when plotting the (automatically downsampled) daily sunspot numbers over the same time interval. Why is this?
A: Contrary to the STAFF downsamplings, the time stamps provided by silso for the monthly and yearly sunspot numbers do not correspond exactly to the middle of the month or the middle of the year.

Q: Sometimes the X-Axis comes out distorted.
A: There is a limitation in the library that messes up the X-Axis (in certain timeranges) when it contains a daylightsavings change.
This can possibly be fixed by setting your browser and/or computer in UTC time. In any case is the data always in UTC and you can still export the data with correct (UTC) timestamps.