IDL statistics routines

IDL routines are available to read .stat and .statn files for manipulation and plotting of statistics data. These routines are still under developement and are subject to change (whenever I feel like it, so there!). Watch this space.

The routines are currently located in the files:

/people/ets/idl/Cutlass/statistics/core_routines.pro

/people/ets/idl/Cutlass/statistics/ps_routines.pro

/people/ets/idl/Cutlass/statistics/statistics.pro

core_routines.pro contains idl procedures to read files and aid in plotting data. ps_routines.pro contains procedures to aid in creating postscript output of statistics plots. statistics.pro contains procedures demonstrating strategies for manipulating and plotting statistics data.

At present it is suggested that these files are copied to the users own area and messed about with to see how they work. Only the procedures for reading statistics files are 'fixed in stone' and it is envisaged that the user will tailor the other routines to their own needs. Hopefully the procedures are sufficiently commented for their ingenious and cleverly crafted mechanisms to be disected by the casual observer. If not ask me... (Steve Milan, ets@ion.le.ac.uk).

core_routines.pro

read_file,'file' [,INT=n]

Read a 2 hour statistics file where file is in the format 95022816f (n.b. quotes are necessary). The optional n indicates that an n-scan post-integrated statistics file should be read i.e. with a file extension .statn. The pathname of the directory in which files are searched for is set by the variable path, which is initially set to /cutlass/statistics/finland/95. This can be reset by the user from the command line (e.g. path='/cutlass/data/ets').

The file is read into two structures within IDL, a parameter block called params containing parameters relating to the whole two hour period, and stats containing the data for each scan or integration period (the structure is identical for non-integrated and integrated statistics).

The parameter structure has the format:

params =	{file		: '                ', $
		stat_rev_no	: 0,		$
		rev		: 0,		$
		st_id		: INTARR(2),	$
		tfreq		: 0,		$
		txstat		: 0,		$
		stat_res	: INTARR(3),	$
		glat		: INTARR(16),	$
		glon		: INTARR(16),	$
		year		: 0,		$
		month		: 0,		$
		day		: 0,		$
		start_hour	: 0,		$
		no_bins		: 0,		$
		bin_scans	: 0}

params.start_hour indicates the UT of the start of the two hour file.

params.bin_scans indicates the number of scans integrated into each integration period or bin (params.bin_scans=1 for non-integrated data).

params.no_bins indicates the number of integration bins (or scans for non-integrated data) contained in the file. Usually params.no_bins=72 / params.bin_scans, though this will be fewer if data gaps are present.

params.glat() and params.glon() contain the geographic locations of the sixteen summary points.

The data structure has the format:

stats=REPLICATE({hour		: -1.0,		$
		no_scans	: 0,		$
		gipercent	: INTARR(2),	$
		pts		: INTARR(2,16),	$
		elevpts		: INTARR(2,16),	$
		noise		: LONARR(2,16),	$
		noise_lev	: LONARR(2,16),	$
		noise_lag0	: LONARR(2,16),	$
		noise_vel	: LONARR(2,16),	$
		pwr_lag0	: LONARR(2,16),	$
		pwr_l		: LONARR(2,16),	$
		pwr_s		: LONARR(2,16),	$
		vel		: LONARR(2,16),	$
		width_l		: LONARR(2,16),	$
		width_s		: LONARR(2,16),	$
		elev		: INTARR(2,16)},73)

stats.hour indicates the UT of the start of the scan or integration bin.

stats.no_scans indicates the number of scans in the integration bin.

stats.gipercent(scat) indicates the f-o-v % ground (scat=0) and ionospheric (scat=1) backscatter.

stats.pts(scat,point_id) and stats.elevpts(scat,point_id) indicate the number of points which were integrated to determine the parameter values and elevation angles for ground and ionospheric backscatter at summary point number point_id.

stats.parameter(scat,point_id) indicate the parameter values for summary point number point_id for ground/ionospheric backscatter scat.

Note that stats is an array of a structure and consequently stats(2).pwr_l(1,6) is pwr_l of ionospheric scatter at summary point 6 during the third (counted from 0) scan or integration bin in the 2 hour period. The subscript for stats allows 73 scans to be held; the usual number is 72 and only in exceptional circumstances are there 73 scans in a file. If data gaps or integrated statistics are read then the stats array will only be partially full - in the subscript range stats(0:params.no_bins-1).

Data points for periods when no backscatter was observed contain the value 10000 to allow easy detection of missing data.

stat_files,'search_pattern'

A function which returns a list of statistics files (pathname stripped) which match the search pattern.

e.g. PRINT,stat_files('950228*f')

prints a list of files available for 28-02-95.

day_no [,day,month,year]

A function which returns the day number of day,month,year. If a date is not specified then the day number of the currently loaded statistics file is returned.

info

Print general information concerning the currently loaded statistics file etc.

plot_axes,yrange [,xrange=[,],ytitle='',title='']

Generate UT axes with the desired yrange (e.g. [-10,30]). If xrange is not specified then the two hour period encompassing the currently loaded statistics file is employed.

plot_par, par [,psym=psym,line=line,color=color]

Plot the data contained in par on axes previously generated by plot_axes.

plot_density, density_array(,)

Superimpose a colour coded two dimensional array onto previously defined axes.

Notes about colours

The colours red, green, blue, yellow, black and white have been defined and can be refered to by name (due to the common block colours)

e.g. PLOT,stats.gipercent(1),COLOR=red

The colours foreground and background (white and black respectively on the screen and black and white on paper) have also been defined. Use of these ensures proper swapping of black and white when postscript printing.

Also, a sixteen colour blue-to-red spectrum has been defined. The spectrum is defined for colour numbers spectrum to spectrum+15, where the variable spectrum is held in the colour common block.

ps_routines.pro

Several routines are provided for the control of postscript output. Use of these routines ensures correct swapping of black and white for printing purposes.

ps_start [,file]

Open a postscript file for output. If file is not specified then the default piccy.ps is used. The specified file name is stored and assumed for all following postscript activities, unless explicitly reset.

Colour postscript and landscape orientation are set by ps_start.

ps_close

Close the currently open postscript file.

ps_print [,file]

Print a postscript file. If file is not specified then the last created file is printed. At present, files are sent automatically to the DeskJet; this will change in future versions. To print on a laser printer, use lp -dLaser file from a unix shell.

ps_landscape and ps_portrait

Swap between landscape and portrait orientations.

ps_info

Print out information on the current status of the postscript output. This information is also available from info.

statistics.pro

This file contains various routines for plotting statistics data. These routines can be used as templates for the users own statistics requirements. A partial list of the routines is:

plot_pwr_vel [,/density | /pwr_diff]

Plot scatter or density plot of power against velocity. If /pwr_diff is specified then then a scatter plot is produced, colour coded with the difference between pwr_l and pwr_lag0 (it's a long story...). Useful for determining how to plot scatter and density plots.

plot_occ_vel [,/all,max_vel]

Plot histograms of the occurrence of different velocities. Useful for determining how to produce occurrence statistics.

plot_mean_gipercent

Plot the diurnal mean variation in the f-o-v % ground and ionospheric backscatter.

plot_daily_gipercent

Plot the daily variation in the mean ground and ionospheric backscatter f-o-v % coverage.

plot_daily_frequency

Plot the daily variation in the operating frequency of the radar.

Using the statistics routines

These routines are designed to be used in a command line environment, and all variables, values, parameters, etc. can be printed, plotted, etc. from the command line or from within users own routines.

After entering IDL, type:

IDL> .run core_routines
IDL> .run ps_routines
IDL> .run statistics

Then load a file, for instance:

IDL> read_file,'95022816f'

To plot the variation in gipercent as a function of time during the two hour interval, first create some UT axes:

IDL> plot_axes,[0,30]

And then to plot the occurrence of ground scatter:

IDL> oplot,stats(0:params.no_bins-1).hour,stats.gipercent(0),max_value=99999,color=red

And ionospheric scatter:

IDL> oplot,stats(0:params.no_bins-1).hour,stats.gipercent(1),max_value=99999,color=blue

For a scatter plot of pwr_l against vel (ionospheric backscatter only, at summary point 8) type:

IDL> plot,stats.vel(1,7),stats.pwr_l(1,7),xrange=[-4000,4000],max_value=9999,color=foreground,psym=1

For a scatter plot of pwr_l against vel (ground scatter only, all 16 summary points), type:

IDL> plot,stats.vel(0,*),stats.pwr_l(0,*),xrange=[-4000,4000],max_value=9999,color=foreground,psym=1

Unsurprisingly all the velocities are near 0 m/s. Ho hum...

Well, there's a taster for you. Any enquiries to be directed to me, Steve Milan (ets@ion.le.ac.uk).

See also...

Statistics overview, stat-make, and stat-int.