MAP_POTENTIAL


Introduction

Getting started

A brief tutorial

The map_potential widget

The map_potential model


Introduction

Map_potential is a tool developed by JHU/APL to derive large-scale global convection maps from line-of-sight velocities observed by the SuperDARN radars. The model has been explained by Ruohoniemi and Baker (1998). A brief description of the model is presented below.

The SuperDARN radars are arranged such that observations in common-volume area are bidirectional and the 2D ExB velocity can be resolved unambigiously. By merging measurements from all the radars in the Northern or Southern hemisphere, direct mapping of the convection can be extended to almost 12 hours of magnetic local time. However, although the two-dimensional velocity at a point can only be unambigiously resolved if two or more radars make simultaneous measurements, all the available line-of-sight velocity data serves to constrain the possibilities for the large-scale convection velocity. The convection pattern that is most consistent with the measuremets can be determined by mathematical fitting. This is the basis of the map_potential model, described by Ruohoniemi and Baker (1998) as "..a method of filtering, combining and reducing data from the SuperDARN radars which optimizes the mapping of large- scale convection pattern."


Getting started

The map-potential technique can be used to image flows in the Northern and Southern hemispheres. Identical steps should be followed to image flows in either hemisphere. You do not need to specify which hemisphere you are imaging, but never combing data from Northern and Southern hemipshere radars in the same file.

To run the map_potential model all you need to do is to initialize the .rad_env and .radlogin files.

In the Leicester group this is done by going to the /people/cutlass/new_pot_map directory and simply typing :

. map_setup

The model requires that make_gridmake_grid , combine_grid and trim_grid are run from the directory which contains your SD fit data files. The map_potential model is run from the same directory which contains the radvec file.

N.B. - New Shortcut There is now a unix script called make_radvec whic will grid and combine files for all given radars given the starting time of the files. To use, simply go into area where the fit files are kept and type :

make_radvec -c (if files are concatenated) yyyymmddhh

But be careful, this method makes it more difficult to identify problems with any individual radars. More importantly, if you have data from both Northern and Southern hemisphere radars for the same time interval in the same directory, make_radvec will attempt to combine it all into one file - creating a big mess.


A brief tutorial

Below is a example of how to use the technique to image flows in the Northern hemisphere on the day 1997/12/01 from 20 to 22 UT.

1)

Go into your fit data area

2) Making Grid Files using make_grid

The first step is to grid the data into a global grid, while spatially and temporally averaging the data. Be warned that the map-potential technique was initially written for normal resolution data. It is possible to grid high-time resolution data (see below), but care should be taken, especially if using concatenated files that might be too large to handle.

Grid files are created using make_grid in the following way :

make_grid filename.fit > filename.grid

e.g. For Finland and Iceland East for 1 Dec 1997, 2000 UT.

make_grid 97120120f.fit > 971201f.grid
make_grid 97120120e.fit > 971201e.grid

The grid file does not need to have this filename.

Normally this would be done for all the available Northern or Southern hemisphere data, remembering that make_grid can be used on concatenated data files.

There are the following options that can be used with make_grid

-st hr:mn set start time

-et hr:mn set end time

-ex hr_mn set extent of time

-i sec set number of seconds (sec) in sampling interval

-tl sec ignoring scan flag (if problem with special modes) and set pseudo scantime (sec)

-vb verbose mode (details data being processed)

-e max maximum vel. error aaccepted (default 200 m/s)

-a abs amsolute minimum magnitude of vel accepted (default 35 m/s)

-r disable temporal averaging

-t disable spatial averaging (presently disabled!)

-rv1 range1 range2 vel1 vel2 - apply range velocity filter (range2 > range1, vel2 > vel1> to remove data

-rv2 apply range/velocity filter (as above)

-rv3 apply range/velocity filter (as above)

Some examples using make_grid

High-time resolution data :
To change the scan period used type make_grid -i (sec), where (sec) is the number of seconds you want per scan.

Some SuperDARN non-common time modes do not always use the scan flag correctly. If this is a problem, you need to type make_grid -tl (sec) What this does is to ignore the scan flag and instead look at the time stamps for individual records in a fit file to decide which time period they lie in. So to produce a grid file contaning records for each minute of the day use : make_grid -tl 60 -i 60

Beware the make_grid -tl command should not be used on normal resolution data. If this is done the resulting grid file would contain records from alternate parts of the array.

Temporal averaging
When the data is gridded using make_grid, it is both spatially ane temporally averaged. To switch of the temporal averaging type : make_grid -r

To create 4 min sampling file for a 12 min period on 8th May at Halley, and see the details of the processing make_grid -st 14:50 -et 15:02 -vb 98050814h.fit > 98050814h.grid

3) Creating master files using combine_grid

Once all the appropriate data has been gridded using make make_grid, it is necessary to combine all the data files into one file. This is done using the combine_grid routine.

combine_grid filename1.grid filename2.grid filename3.grid .... > radvec.dat

E.g.

combine_grid 971201f.grid 971201k.grid ..... > radvec.dat

Any number of grid files can be combine to produce the masterm file. Generally, one grif file for each radar. Grid files from radars in opposite hemisphere should not be combined. The radvec.dat filename is flexible.

There are the following options that can be used with combine_grid

-vb verbose mode (details data being processed)

-r replacement mode, overwrites data in master file.

4) Altering master files using trim_grid

Master files can ne altered using trim_grid in the following way.

trim_grid options radvec.dat > new_radvec.dat

If no options are choosen then the output file is equal to the input file

There are the options that can be used with trim_grid

-st hr:mn set start time

-et hr:mn set end time

-ex hr:mn set extent of time

-vb verbose mode (details data being processed)

-# excludes data from radar with identifier letter #

Example If instead of studying the day 1997/12/01 from 20 to 22 UT, you wanted to study it from 20 to 21 UT and to exclude Goose Bay data.

trim_grid -st 20:00 -et 21:00 -g radvec.dat > radvec.dat

5)

When the master file is ready, the potential maps can be created using the map widget. Type
map_start and the map_potential widget should appear.

The map_potential widget

This is the map_potential widget which is used when deriving and studying the two-dimensional velocity patterns. The widget reads data from the master file, combines it with the APL model average convection pattern that mathces the IMF conditions and creates an estiamte of the potential pattern for the interval concerned. This potential pattern can be plotted and overplotted with different forms of the velocity vectors in the radar fields-of-view. I have attempted to annotate the widget so that it is fairly easy to understand, the points are explained in more detail below.



Input file : This is where the name of the master file is entered. After entering the filename remember to press return, or the software will not register the change (this is the case for all the entry fields on the widget).

Output file: This is the name of the output file. The output file contains all the information in the plot. It will only be output if return is pressed in the entry field.

Latitude limit: This is the estimated low latitude limit of the convection. If it is constant then the value can be entered into the fixed entry field. If the value varies across the interval, then the values must be placed into a file, the filename of which is entered into the filename entry field. Each line of the file contains a time and a latitude and takes the format hr mn lat, eg 20 00 68 (68 degress at 20:00 UT). A good estimate of the latitude limit is generally the equatorward limit of he radar backscatter. A wrong positioning of the latitude limit can lead to very unrealistic potential patterns. Getting it right can sometimes be a matter of trial and error.

Shift pole This is the shift in degrees of the centre of the potential pattern away from the magnetic pole in the direction of 0 MLT. Values are entered in the same way as for the latitude limit. A nonzero pole shift will affect the low-latitude convection limit away from magentic local noon. For example, if you have a low-latitude limit of 65 degrees and a pole shift of 4 degrees, the model will consider scatter down to 65 degrees at noon and 57 (65 -4x2) degrees at midnight.

IMF data Here you can choose whether to use or not to use measured IMF conditions to drive the background model. If IMF data is selected, there is a choice between the ACE and WIND spacecrafts. The software will automatically look for the corresponding CDF file containing the IMF data in the data archive. If it does not exist in this archive the filename can be entered by hand (the CDF file for a particular interval of data can be downloaded from CDAWEB). The slider sets the time delay (in minutes) between the IMF observations at the satellite and when it impinges on the subsolar magnetopause. The scale corresponds to the size of the vector in IMF annotation.

Model : There are three options here (Fixed, From IMF, None)

Fixed - The same APL potential model is used for the whole interval. This requires the selection of the model (IMF clock angle sector & field magnitude) from the pulldown menus.

From IMF - This selects the APL potential model by looking at the (shifted) IMF data for each interval.

None - No APL potential model is used.

Model Weighting : This can either be flat or modified.

Flat - Data points from the statistical model are assigned a fixed weight equal to the average weight of the data points. At high order of fit, where more data pointsare ised from the statistical mode, this leads to an increased dependence on the model.

Modified - The statistical model are de-weighted with increase in the fitting order so that the contribution to the fitting from the model is roughly constant for oders above 4.

Order This represents the potential of the spherical harmonic expansion that will be fitted to the data. Relatively low order patterns (~4) resolve the global scale of the patterns. Higher orders are often needed to resolve mesoscale structure in the data (see Ruohoniemi and Baker, 1996)

Output Device There are three choices here, X-windows,Postscript (B&W),Postscript (colour), Output files are idl.ps

Plot Limits : These represent the maximum and minimum latitudes and longitudes for the plots (can be absolute values, the master file signifies the hemisphere). Default values are automatically entered. The latitude limit will automatically go from below the low-latitude convection limit to the pole, whereas the longitude limit will go from -180 to 180.

Plot Options : This details which data is presented on each plot

Contour - plots contour of constant electric potential.

Fit Vec - plot velocity vectors fitted to the measured and modelled velocity information.

Model vec - plot velocity vectors from the statistical model.

Residual - plot the filtered, gridded and combined line-of-sight vectors.

True Vec - plot the vectors obtained by combining the measured line-of-soght velocity measurement with the transverse component of the fitted line-of-sight velocity vectors.

Merge Vec - plot merged velocity vectors calculated from overlapping filtered and gridded line-of-sight measurements from different radars.

Operational buttons : To finally perform any operations

Preview - plots a quick preview of all the line-of-sight data.

Execute - produces the potential maps as requested.

Quit - exits from the map widget.


The map_potential model

Preprocesing

The velocity analysis begins with a sequence of scan data from the HF radars. Velocity with error estimates greater than 200 m/s are ignored, as is groundscatter. Using the make_grid program, data from each radar is processed for a selected time interval and integration period, The data is first filtered using a boxcar filter involving both spatial and temporal filtering. The filter is descibed in some detail in Ruohoniemi and Baker (1998). Basically for a scan collected at time t(k), the filtering samples includes the scans performed at t(k-1) and t(k+1), where the subscript indicates the scan order. The spatial sampling is performed over a 3x3 beam/gate filter centered on the cell of interest. After the data has been filtered it is written to a holding file

The next step is to combine the data from the individual radar holding files into a single file. This is done using the combine_grid program. A global grid is defined for spatial averaging and a time step for temporal averaging period. The grid is defined by the spatial scale of 1 degree of latitude (or abour 111 km projected to the surface of the Earth). For each integral interval of 1 degree in co-latitude, the number of grid cells distributed in longitude is set by the requirement that the step in longitude projected onto the Earth's surface be as close to 111 km as possible. Ruohonimie and Baker (1998) stated that "it is preferable to average the radar velocity radar than work directly with the individual velocity values because the latter would lead to oversampling of the near-radar regions relative to the far-radar regions". When combining data an average velocity and its uncertainly were obtained for a given grid cell for a given time if at least 25% of the radar sampling within that cell returned a velocity value. Once all the above steps have been done we have produced what are refered to by Ruohoniemi and Baker as average line-of-sight velocity measurements. These velocities will be used to derieve two-dimensional convection patterns.

Imaging large-scale convection

Merging common-volume pairs of line-of-sight radar velocity measurements is the standard method of producing two-dimensional velocity measurements. However, all the available line-of-sight velocity data serve to constrain the possiblity for the large scale convection pattern. The pattern that is most consistent with all the average line-of-sight velocity measurements can be determined by mathematical fitting procedures. The electrostatoc potential is expanded in terms of spherical harmonic functions according to e.g. Jackson(1962). The order and degree of the expansion determine the spatial filtering performed on the velocity data. The global-scale character of the pattern is resolved by expansion of relatively low order and degree. Often both the order and degree of the expansion are equal to four, the spatial scales associated with this fitting are 6 deg. in latitude and 45 deg. in longitude.

The fitting procedure can be applied directly to the preprocessed data points. In principle it should then be possible to calculate the electrostatic potential over the entire global grid. However this amounts to extrapolation of the fitting solution obtained over the limited area covered by the measurements to the uncovered portions of the high-latitude zone, and these results may be unphysical over the areas that are distant from the input velocity data.

In order to obtain a global picture of the convection, the map-potential model fold in velocity information from the statistical model of Ruohoniemi and Baker (1996). Briefly the statistical model is sampled at the least number of grid positions that is required to bound each term in the spherical harmonic expansion of the electrostatic potential. For a given expansion order a set of sampling points can be determined. If fewer points are taken, there is instability in the model related to the effect of unconstrained extrapolation. If more points are taken the global solution owes more of its character to the input from the statistical model than is necessary to achieve stabilisation.

For more detailed information on the map_potential model, read Ruohoniemi and Baker, 1998.