Eqpac Moorings: Processing
INTRODUCTION:
This file describes the processing of the incoming STORDAT data from the equatorial Pacific. The incoming STORDAT data from M3 is processed by separate software written by Bob Herlien. For information regarding the deployments and recoveries, including serial numbers and calibration files, please refer to the file "stordatdep.txt".
DATA FLOW, PROCESSING AND FILE ARRANGEMENT:
The raw incoming ARGOS data are stored in TSUNAMI|oasis|eqpac|satpac, in files named according to the PTT ID of the system. These data arrive in the directory from an email message that is sent to MBARI from NOAA/PMEL. It is perhaps useful to note that the processing of these incoming mail messages is not strictly a cron job - it is a script (called argosprc?) that is triggered by the arrival of the message. These files are completely regenerated every day, because it is faster to do it that way, but there is a potential problem if the raw archive is corrupted, which has happened once before. For this reason, the STORDAT processing software essentially appends to the end of the existing processed data.
Note that the directory TSUNAMI|oasis|eqpac|satpac is readable and writable to "the world".
The original processing of the STORDAT ARGOS data was performed by some Matlab scripts (which were stored in TSUNAMI|oasis|eqpac|sat_files) written by Jeff Scrutton at Satlantic, but it was difficult to automate the combination of Matlab and asciicon (Satlantic's data conversion software). For this reason, Brian Schlining modified Jeff Scruton's Matlab scripts and ported asciicon to Matlab. These functions convert the argos messages into a SeaBASS-compatible format, derives some quantities (Lw and Chl), and places the resulting data files in TSUNAMI|oasis|eqpac|pete|ftp_nasa ready to be transferred via FTP to NASA's SeaBASS database. The SeaBASS format files are named according to the PTT ID with the extension '.ftp' added
So, as described above, the raw data reside in TSUNAMI|oasis|eqpac|satpac. The Matlab functions are located in OASIS|eqpac|brian. They are run daily as an AT job on the WindowsNT machine, Sunfish. All files located in the satpac directory are reprocessed daily
The order of function execution is as follows:
SATLAN_PRC %Shell function specific to our directory setup. It calls the following functions:
SATLAN2OUT % Converts Argos messages to asciicon like ascii output files. It requires the following functions
SATLAN1 % Creates a file of good argos messages
SATLAN2 % Turns argos messages into a binary format usable by asciicon
SATLAN3 % Finds the correct calibration file and converts the binary data to text (This function calls the matlab version of ASCIICON)
SATLAN_LOC % Parses the location messages which in turn are used to specify 'BAD' data (i.e. When the instruments are not at the deployment buoy's location)
SATLAN_WRITE % Converts the asciicon style output to SeaBASS style output
The processing requires access to the appropriate satlantic calibration files. No modification of the calibration files should be nesscessary.
EXAMPLE HEADER FILE FOR SEABASS:
/begin_header
/affiliations=Monterey_Bay_Aquarium_Research_Institute
/experiment=TAO_Moorings
/cruise=TAO_98
/investigators=Francisco_Chavez,Pete_Strutton,Brian_Schlining
/contact=chfr@mbari.org,stpe@mbari.org,brian@mbari.org
/parameters=Lu,Ed,Lw,Nlw,Wt,Chl
/data_type=mooring
/original_file_name=7037
/delimiter=space
/missing=-999
/north_latitude=2.580[DEG]
/south_latitude=-7.967[DEG]
/east_longitude=220.008[DEG]
/west_longitude=249.843[DEG]
/start_date=19990425
/end_date=19990510
/start_time=10:44:05[GMT]
/end_time=03:59:35[GMT]
/documents=mbari_mooring_readme.doc
/calibration_files=dat0025e_.cal
! Comments: Last column is quality flag; -1=bad, 0=unchecked, 1=good
! Median Longitude = 2.456889e+002, Median Latitude = 1.517820e+000
/fields=Date1,Time,Date2,Temperature[1.5m],Flourescense[1.5m],Ed[3m+]412,Ed[3m+]443,Ed[3m+]490,Ed[3m+]510,Ed[3m+]555,Ed[3m+]656,Ed[3m+]670,Ed[3m+]683,Ed[3m+]PAR,Lu[1.5m]412,Lu[1.5m]443,Lu[1.5m]490,Lu[1.5m]510,Lu[1.5m]555,Lu[1.5m]670,Lu[1.5m]683,Lw[0m+]412,Lw[0m+]443,Lw[0m+]490,Lw[0m+]510,Lw[0m+]555,Lw[0m+]670,Lw[0m+]683,Nlw[0m+]412,Nlw[0m+]443,Nlw[0m+]490,Nlw[0m+]510,Nlw[0m+]555,Nlw[0m+]670,Nlw[0m+]683,OC2V2Chl[0m],Lu443/Lu555Chl[0m],CZCSChl[0m],Lu683/Lu555[0m],QCflag
/units=yyyymmdd,hh:mm:ss,mm/dd/yyyy,C,unitless,uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),uW/(cm2.nm.sr),ug/L,ug/L,ug/L,unitless,u
/end_header@
------------------
END EXAMPLE HEADER
QUALITY CONTROL:
Some primitive quality control is implemented, as described below:
-
If the data location of a record is more than 0.03 degrees from the median location of all data records in the files the record is flaggeg 'bad'
-
Negative radiances and irradiances are also flagged 'bad'. This primarily marks nighttime data as being unusable
OUTPUT FILE FORMAT & DERIVED QUANTITIES:
The file header is as described above, a description of the fields follows:
The first four columns are date and time, in several formats for ease of interpretation, depending on user's preferences. Decimal day takes into account both time and date, and the rest are self-explanatory. The upwelling radiance data (Lu412 to Lu683) are as recorded by the ocr-100 at ~1.5m depth. The Ed490 is as recorded by the ED-20 at the top of the buoy tower (3m above the water surface), and the remaining Ed data are derived from Ed490 using an empirical relationship. This relationship was derived from the combined ep1 and ep2 Dec 1996 to May/Jun 1997 deployments, and the equations are:
Ed412 = 0.804*Ed490 + 5.229
Ed443 = 0.926*Ed490 + 2.366
Ed510 = 1.013*Ed490 - 1.599
Ed555 = 0.996*Ed490 - 2.873
Ed656 = 0.813*Ed490 - 8.455
EdPAR = 0.00126*Ed490 - 0.0048
where EdPAR is in uE/(cm^2.s).
The Lw data are derived from an empirical relationship between Lu(1.5m) and Lw, which was arrived from analysis of the GP3-98-KA Satlantic profiling data. The equations are:
412nm: Lw = 0.5574*Lu(1.5m) + 0.014
443nm: Lw = 0.5588*Lu(1.5m) + 0.012
490nm: Lw = 0.5632*Lu(1.5m) + 0.003
510nm: Lw = 0.5778*Lu(1.5m) + 0.003
555nm: Lw = 0.6005*Lu(1.5m) + 0.002
670nm: Lw = 0.6713*Lu(1.5m) + 0.001
683nm: Lw = 0.5904*Lu(1.5m) + 0.001
Fluorescence from the surface Seatech fluorometer is in arbitrary units - perhaps millivolts, with a maximum of 4095. SST and battery voltage are as reported by the instrument package. OC2V2 chlorophyll is calculated from the current version of the SeaWiFS algorithm:
Chl = 10^(0.2974 - 2.2429*r + 0.8358*r^2 - 0.0077*r^3) - 0.0929
where r = Lu490/Lu555
This could perhaps be changed in the future to use the derived Lw data instead.
The Lu443/Lu555 Chl is derived from an empirical relationship between in situ chlorophyll measurements and the Satlantic profiling data from GP7-97-KA and GP3-98-KA. the equation is:
Chl = 0.7469*r^-1.2797, where r = Lu443/Lu555.
The data quality flag is set to 0 for unchecked data, -1 for bad data, and 1 for good data.
