grey layout spacer                                                 NEAR REAL-TIME       NOTIFICATIONS       PRODUCTS      
print page
 

Specification of the ASCII data file format -- SCIAMACHY data

The daily data at coordinates of the measurements by the satellite instrument are delivered in the form of ASCII files, one for each orbit file with spectral data. Measurements of SCIAMACHY come in principle as one file per orbit, but it happens quite often that files span only a part of an orbit, or even the end of one orbit and the beginning of the next. The data files are delivered via the website as zip-files containing all orbit files of a given day.
See the main Data format specification page
for some general notes on the data formats.
For a given day, the orbits treated are those orbits that have a start time during that day; this is the time mentioned in the name of the data file. The name of the SO2 data files contains the orbit date YYYYMMDD and the start time HHMMSS, both taken from the name of the original measurement file:
   so2cdYYYYMMDD_HHMMSS.dat
where so2cd stands for "SO2 column density". These files contain the SO2 slant column data (SCD) and -- if available -- the SO2 vertical column density (VCD) derived with an appropriate air-mass factor, and the cloud cover fraction.

The ASCII data file can be read easily with, for example, a Fortran program (example page) for further processing. The ASCII file can also be read with many plot programs, such as IDL (example page) and MatLab.
 

File header

Each data file has a header with comment lines (starting with the # mark), giving orbit and analysis information, as well as a list of the data columns. Also given is for format of each data fields, and the full data format for reading the data lines. A typical data file header for data files looks as follows. Some remarks regarding the entries in the file header are given below the file header.

# SO2 column density for TEMIS / PROMOTE
# --------------------------------------
#
# Product status  : archive data
# Process version : 1.0.3
#
# Instrument      : SCIAMACHY
# Orbit date/time : 20070320_120511
# Orbit number    : 26416
#
# Analysis date   : 2007/08/13
# Cloud cover data: FRESCO (SC-v5)
# AMF & VCD values: yes
#
# Nr plume heights:  3
# Nr data columns : 47
#
# Data columns                                      [format]
#
# --- Ground pixel data
#     1 = measurement date as YYYYMMDD              [a8]
#     2 = measurement time as HHMMSS.SSS            [1x,a10]
#     3 = pixel id: 0=forward, 3=backscan           [i4]
#   4-7 = pixel corner latitudes                    [4f9.3]
#     8 = pixel center latitude                     [f9.3]
#  9-12 = pixel corner longitudes                   [4f9.3]
#    13 = pixel center longitude                    [f9.3]
#    14 = solar    zenith  angle (SZA) at TOA       [f9.3]
#    15 = viewing  zenith  angle (VZA) at TOA       [f9.3]
#    16 = relative azimuth angle (RAA) at TOA       [f9.3]
#
# --- Slant column data
#    17 = SO2 slant column density SCD  (in DU)     [f9.3]
#         with background correction
#    18 = retrieval error on the SCD    (in DU)     [f9.3]
#    19 = chi^2 of the slant column fit (x 1e-6)    [f9.3]
#    20 = slant column value index (SVI)            [i4]
#         0 : SO2 SCD <= 1.5 DU
#         1 : SO2 SCD >  1.5 DU, no notification issued
#         2 : SO2 SCD >  1.5 DU, notification issued for state
#
# --- Vertical column data
#    21 = AMF quality index (AQI)                   [i4]
#         = -1 : no AMF calculation selected
#         =  0 : successful AMF calculation
#         = +1 : no cloud cover data available
#         > +1 : error computing AMF
#    22 = AMF profile shape number (1 or 2)         [i4]
#     --- using plume height #1 =  2.0 km above surface
#    23 = SO2 vertical column density VCD  (in DU)  [f9.3]
#    24 = Error on VCD from column 18      (in DU)  [f9.3]
#    25 = Total air-mass factor  (AMFtot)           [f9.3]
#    26 = AMF for clear-sky part (AMFclr)           [f9.3]
#    27 = AMF for cloudy part    (AMFcld)           [f9.3]
#     --- using plume height #2 =  6.0 km *
#    28 = SO2 vertical column density VCD  (in DU)  [f9.3]
#    29 = Error on VCD from column 18      (in DU)  [f9.3]
#    30 = Total air-mass factor  (AMFtot)           [f9.3]
#    31 = AMF for clear-sky part (AMFclr)           [f9.3]
#    32 = AMF for cloudy part    (AMFcld)           [f9.3]
#     --- using plume height #3 = 14.0 km *
#    33 = SO2 vertical column density VCD  (in DU)  [f9.3]
#    34 = Error on VCD from column 18      (in DU)  [f9.3]
#    35 = Total air-mass factor  (AMFtot)           [f9.3]
#    36 = AMF for clear-sky part (AMFclr)           [f9.3]
#    37 = AMF for cloudy part    (AMFcld)           [f9.3]
#     *) or 1 km above surface if that is higher
#
# --- Cloud and Surface data
#    38 = cloud cover index (CCI)                   [i4]
#         = 0 : no cloud cover data
#         = 1 : clear sky mode
#         = 2 : normal FRESCO mode
#         = 3 : snow/ice FRESCO mode 
#         = 4 : missing or invalid FRESCO data
#    39 = cloud fraction                            [f9.3]
#    40 = cloud top pressure (in hPa)               [f9.3]
#    41 = cloud top height   (in km)                [f9.3]
#    42 = cloud top albedo                          [f9.3]
#    43 = surface pressure   (in hPa)               [f9.3]
#    44 = surface elevation  (in km)                [f9.3]
#    45 = surface albedo                            [f9.3]
#
# --- Additional gound pixel data
#    46 = sciamachy state_index                     [i4]
#    47 = sciamachy state_id                        [i4]
#
#
# Full data format: (a8,1x,a10,i4,16f9.3,3i4,15f9.3,i4,7f9.3,2i4)
#
#

Which is followed by two lines spanning all columns and giving a header to each column. Then follow the data lines themselves. The very end of the file is marked by:

#
# --- end of file.

The first part of the comments gives some general information on the orbit and retrieval. The number of data columns in the file depends on the number of plume heights used for computing the AMF and VCD, hence both these numbers are given here. The first 22 columns are always present. Then follow sets times five lines, one set for each plume height. And the last 9 columns are always present.

If no vertical column densities (VCDs) have been computed, then:

  • the  AMF & VCD values  comment line in this part of the file says  no ;
  • the AMF quality index (AQI) in column 21 is set to -1;
  • all data entries concerning the AMF and VCD are given the "no data" value (-99.0), except where possible the clear-sky AMF (see below).
If cloud cover information is included, as in the example, the source of the cloud data is mentioned. If no cloud cover information has been included, then:
  • the  Cloud cover data  comment line in this part of the file says  none ;
  • the cloud cover index (CCI) in column 38 in the above example is set to 0;
  • all data entries concerning the cloud cover, the AMF and VCD are given the "no data" value (-99.0).
and no AMFs or VCDs can be computed.
 

Remarks

Remarks regarding some of the entries in the above file header:
  • On the right of the list of data columns and below the list is the format with which the data can be read in the notation used, for example, in Fortran and IDL.

  • Column 3 gives the type of ground pixel. For SCIAMACHY data there is only a distiction between forward (0) and backward (3) scan pixels.

  • The zenith and azimuth angles (columns 14, 15 and 16) are given at the top-of-atmosphere (TOA) for the centre of the ground pixel.

  • The error value given in column 18 is the error following from the DOAS slant column retrieval. Its value is based on the RMS of the fit between the measured and the fitted spectrum. The chi-square in column 19 is also a measure of the quality of the fit. In fact, these three quantities are all interlinked: chi-square = RMS * RMS, and the relationship between the slant column error and the RMS is assumed to be linear. To assist data users in their analysis, both the slant column error and the chi-square are written to the data file.

  • The "slant column value index" (SVI) in column 20 is meant to give the user a quick look facility on the SO2 slant column value. For more information, see the chapter on the criteria for the notifications on exceptional SO2 concentrations.

  • If an AMF and VCD have been computed, the "AMF quality index" (AQI) in column 21 indicates success of the AMF computation, and if successful the VCD was computed. The relevant AQI values are listed in the file header; more details are given on a separate page.

  • Column 22 gives the number of the profile used for the AMF computation, which depends on the SO2 concentration: '1' for low concentrations, '2' for high concentrations; more details are given on the Air-mass factor using look-up tables info page.

  • In order to compute the AMF, a height must be given of the SO2 plume. As this height is not known automatically, a number of assumed heights are used, in this case three. For each assumed plume hieght, the data file gives five quantities: the SO2 vertical column, an estimate of the error on it (based on the error given in column 18), and the three components of the AMF; more details are given on the Air-mass factor using look-up tables info page.

  • If cloud cover data is included, the "cloud cover index (CCI)" in column 38 in the above example indicates success of this inclusion. The relevant CCI values are listed in the file header; some more details are given on a separate page.

  • Data fields for which no value was (successfully) computed are given the "no data" value -99.
 
Link naar de website van het Federaal Wetenschapsbeleid      grey layout spacer Link naar de Federale Portaalsite
 
TEMIS Home Page  PROMOTE Home Page  O3M SAF Home Page Last update on 14 Mar 2013