INFO 2025-06-26 05:32:57,148 process.py process_mission():429 [1] commandline = src/data/process_dorado.py --start_year 2003 --end_year 2014 --num_cores 1 -v --noinput --clobber
INFO 2025-06-26 05:32:57,148 process.py process_mission():434 [1] dorado_info[mission]["comment"] = Portaled files exist in archive, but not processed by reprocess_surveys.m - ctdToUse = ctd1 
INFO 2025-06-26 05:32:57,149 process.py process_mission():449 [1] Processing AUVCTD mission 2003.191.00 by user docker_user on host 671049448ece
INFO 2025-06-26 05:32:57,149 process.py download_process():165 [1] Download and processing steps for 2003.191.00
INFO 2025-06-26 05:32:57,149 logs2netcdfs.py download_process_logs():739 [1] src_dir = /Volumes/AUVCTD/missionlogs/2003/2003191/2003.191.00
INFO 2025-06-26 05:32:57,149 logs2netcdfs.py download_process_logs():768 [1] Rsyncing /Volumes/AUVCTD/missionlogs/2003/2003191/2003.191.00 to /data/auv_data/dorado/missionlogs/2003.191.00
INFO 2025-06-26 05:32:57,523 logs2netcdfs.py download_process_logs():779 [1] Processing mission: dorado 2003.191.00
INFO 2025-06-26 05:32:57,523 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/ctdDriver.log (46507 bytes)
INFO 2025-06-26 05:32:57,534 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/ctdDriver2.log (46061 bytes)
INFO 2025-06-26 05:32:57,543 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/gps.log (0 bytes)
WARNING 2025-06-26 05:32:57,543 logs2netcdfs.py download_process_logs():790 [1] /data/auv_data/dorado/missionlogs/2003.191.00/gps.log is empty
INFO 2025-06-26 05:32:57,544 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/hydroscatlog.log (38354 bytes)
INFO 2025-06-26 05:32:57,555 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/navigation.log (135598 bytes)
INFO 2025-06-26 05:32:57,574 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/isuslog.log (1749 bytes)
INFO 2025-06-26 05:32:57,578 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/parosci.log (38198 bytes)
INFO 2025-06-26 05:32:57,585 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.191.00/tailCone.log (37915 bytes)
WARNING 2025-06-26 05:32:57,585 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.191.00/tailCone.log
INFO 2025-06-26 05:32:57,585 logs2netcdfs.py download_process_logs():807 [1] Time to process: 0.06 seconds
WARNING 2025-06-26 05:32:57,585 process.py download_process():196 [1] No lopc.bin file for 2003.191.00
INFO 2025-06-26 05:32:57,585 process.py calibrate():227 [1] Calibration steps for 2003.191.00
INFO 2025-06-26 05:32:57,621 calibrate.py _navigation_process():1524 [1] Converting from Easting/Northing to lat/lon for mission 2003.191.00
WARNING 2025-06-26 05:32:57,623 calibrate.py _navigation_process():1588 [1] No positive depths found in 2003.191.00/navigation.nc
ERROR 2025-06-26 05:32:57,624 calibrate.py process_logs():3542 [1] Error processing gps: gps has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.191.00
WARNING 2025-06-26 05:32:57,626 calibrate.py _depth_process():2193 [1] Expected sample_rate to be 10 Hz, instead it's 5.00 Hz
INFO 2025-06-26 05:32:57,640 calibrate.py _hs2_process():2400 [1] Number of bad hs2 points: 2 of 624
INFO 2025-06-26 05:32:57,648 calibrate.py _geometric_depth_correction():3423 [1] hs2: d_beg_time_diff: 1 seconds, d_end_time_diff: -1 seconds
ERROR 2025-06-26 05:32:57,663 calibrate.py process_logs():3548 [1] Error processing ctd1: missing variable 'ctd'
INFO 2025-06-26 05:32:57,680 calibrate.py _geometric_depth_correction():3423 [1] ctd2: d_beg_time_diff: 28 seconds, d_end_time_diff: -1 seconds
INFO 2025-06-26 05:32:57,685 calibrate.py _geometric_depth_correction():3423 [1] ctd2: d_beg_time_diff: 28 seconds, d_end_time_diff: -1 seconds
INFO 2025-06-26 05:32:57,686 calibrate.py _ctd_process():2919 [1] Performing range checking of ['ctd2_salinity'] in 2003.191.00/ctd2.nc
WARNING 2025-06-26 05:32:57,687 calibrate.py _range_qc_combined_nc():890 [1] More than 500 (516) ctd2_salinity values found outside of range. This may indicate a problem with the ctd2 data.
INFO 2025-06-26 05:32:57,688 calibrate.py _range_qc_combined_nc():898 [1] Setting 516 ctd2_salinity values to NaN
INFO 2025-06-26 05:32:57,688 calibrate.py _range_qc_combined_nc():910 [1] Checked for data outside of these variables and ranges: [('ctd2_salinity', Range(min=30, max=40))]
INFO 2025-06-26 05:32:57,688 calibrate.py _range_qc_combined_nc():936 [1] Done range checking ctd2
ERROR 2025-06-26 05:32:57,693 calibrate.py process_logs():3542 [1] Error processing biolume: biolume has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.191.00
ERROR 2025-06-26 05:32:57,693 calibrate.py process_logs():3542 [1] Error processing lopc: lopc has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.191.00
ERROR 2025-06-26 05:32:57,693 calibrate.py process_logs():3542 [1] Error processing ecopuck: ecopuck has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.191.00
ERROR 2025-06-26 05:32:57,694 calibrate.py process_logs():3542 [1] Error processing tailcone: tailcone has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.191.00
INFO 2025-06-26 05:32:57,694 calibrate.py write_netcdf():3508 [1] Writing calibrated instrument data to /data/auv_data/dorado/missionnetcdfs/2003.191.00/dorado_2003.191.00_cal.nc
INFO 2025-06-26 05:32:57,720 calibrate.py write_netcdf():3512 [1] Data variables written: ctd1_conductivity, ctd1_conductivity_onboard, ctd1_dissolvedO2, ctd1_salinity, ctd1_temperature, ctd1_temperature_onboard, ctd1_time, ctd2_conductivity, ctd2_conductivity_onboard, ctd2_depth, ctd2_flow2, ctd2_salinity, ctd2_temperature, ctd2_temperature_onboard, ctd2_time, depth_filtdepth, depth_filtpres, depth_time, hs2_bbp470, hs2_bbp676, hs2_depth, hs2_fl676, hs2_time, isus_nitrate, isus_quality, isus_temp, isus_time, navigation_depth, navigation_latitude, navigation_longitude, navigation_mWaterSpeed, navigation_pitch, navigation_posx, navigation_posy, navigation_roll, navigation_time, navigation_yaw
INFO 2025-06-26 05:32:57,723 process.py align():251 [1] Alignment steps for 2003.191.00
INFO 2025-06-26 05:32:57,737 align.py process_cal():139 [1] Processing dorado_2003.191.00_cal.nc from /data/auv_data/dorado/missionnetcdfs/2003.191.00
ERROR 2025-06-26 05:32:57,738 process.py process_mission_exception_wrapper():537 [1] An exception occurred
Traceback (most recent call last):
  File "/src/data/process.py", line 529, in process_mission_exception_wrapper
    self.process_mission(mission, src_dir=src_dir)
  File "/src/data/process.py", line 485, in process_mission
    self.align(mission)
  File "/src/data/process.py", line 263, in align
    netcdf_dir = align_netcdf.process_cal()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/data/align.py", line 182, in process_cal
    raise InvalidCalFile(error_message) from None
align.InvalidCalFile: No nudged_latitude data in dorado_2003.191.00_cal.nc
ERROR 2025-06-26 05:32:57,739 process.py process_mission_exception_wrapper():538 [1] Failed to process to completion: 2003.191.00
INFO 2025-06-26 05:32:57,740 process.py archive():337 [1] Archiving steps for 2003.191.00
INFO 2025-06-26 05:32:57,740 process.py archive():347 [1] nc_file_base = /data/auv_data/dorado/missionnetcdfs/2003.191.00/dorado_2003.191.00, BASE_PATH = /data/auv_data
INFO 2025-06-26 05:32:57,740 archive.py copy_to_AUVTCD():60 [1] Archiving /data/auv_data/dorado/missionnetcdfs/2003.191.00/dorado_2003.191.00 files to /Volumes/AUVCTD/surveys/2003/netcdf
INFO 2025-06-26 05:32:57,741 archive.py copy_to_AUVTCD():78 [1] Removing /Volumes/AUVCTD/surveys/2003/netcdf/dorado_2003.191.00_cal.nc
INFO 2025-06-26 05:32:57,747 archive.py copy_to_AUVTCD():82 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/dorado_2003.191.00_cal.nc /Volumes/AUVCTD/surveys/2003/netcdf done.
INFO 2025-06-26 05:32:57,754 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/ctdDriver.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003191/2003.191.00 done.
INFO 2025-06-26 05:32:57,758 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/ctdDriver2.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003191/2003.191.00 done.
INFO 2025-06-26 05:32:57,762 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/hydroscatlog.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003191/2003.191.00 done.
INFO 2025-06-26 05:32:57,767 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/navigation.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003191/2003.191.00 done.
INFO 2025-06-26 05:32:57,771 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/isuslog.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003191/2003.191.00 done.
INFO 2025-06-26 05:32:57,776 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/parosci.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003191/2003.191.00 done.
INFO 2025-06-26 05:32:57,778 archive.py copy_to_AUVTCD():116 [1] Archiving product files
INFO 2025-06-26 05:32:57,779 archive.py copy_to_AUVTCD():161 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.191.00/dorado_2003.191.00_processing.log /Volumes/AUVCTD/surveys/2003/netcdf