INFO 2025-06-26 05:46:25,963 process.py process_mission():429 [1] commandline = src/data/process_dorado.py --start_year 2003 --end_year 2014 --num_cores 1 -v --noinput --clobber
INFO 2025-06-26 05:46:25,963 process.py process_mission():434 [1] dorado_info[mission]["comment"] = 2004.029.03: parosci interp1 problem - multigenerateNetcdf failed to create navigation.nc file because of '#' problem # begin #<binary_data> Will need to rerun the portal with the fix to set byteOffset. Re-portaled & processed 7/26/05 According to GPS data AUV is in Mexico beginning: 29 Jan 2004 19:37:36 GMT	1.0754050561900759E9	0.4733113223544617 *** Need to reprocess with just the .03 mission and not include the data equal to and after this time *** Had bad nav point - processed 8/10/05 - ctdToUse = ctd1 
INFO 2025-06-26 05:46:25,963 process.py process_mission():449 [1] Processing AUVCTD mission 2004.029.03 by user docker_user on host 671049448ece
INFO 2025-06-26 05:46:25,963 process.py download_process():165 [1] Download and processing steps for 2004.029.03
INFO 2025-06-26 05:46:25,963 logs2netcdfs.py download_process_logs():739 [1] src_dir = /Volumes/AUVCTD/missionlogs/2004/2004029/2004.029.03
INFO 2025-06-26 05:46:25,963 logs2netcdfs.py download_process_logs():768 [1] Rsyncing /Volumes/AUVCTD/missionlogs/2004/2004029/2004.029.03 to /data/auv_data/dorado/missionlogs/2004.029.03
INFO 2025-06-26 05:46:27,740 logs2netcdfs.py download_process_logs():779 [1] Processing mission: dorado 2004.029.03
INFO 2025-06-26 05:46:27,741 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/ctdDriver.log (3357507 bytes)
INFO 2025-06-26 05:46:27,995 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/ctdDriver2.log (3357413 bytes)
INFO 2025-06-26 05:46:28,237 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/gps.log (24260 bytes)
INFO 2025-06-26 05:46:28,248 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/hydroscatlog.log (2292434 bytes)
INFO 2025-06-26 05:46:28,522 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/navigation.log (6674965 bytes)
INFO 2025-06-26 05:46:29,028 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/isuslog.log (183349 bytes)
INFO 2025-06-26 05:46:29,051 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/parosci.log (2293574 bytes)
INFO 2025-06-26 05:46:29,245 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/tailCone.log (2293531 bytes)
INFO 2025-06-26 05:46:29,529 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2004.029.03/biolume.log (0 bytes)
WARNING 2025-06-26 05:46:29,529 logs2netcdfs.py download_process_logs():790 [1] /data/auv_data/dorado/missionlogs/2004.029.03/biolume.log is empty
INFO 2025-06-26 05:46:29,530 logs2netcdfs.py download_process_logs():807 [1] Time to process: 1.79 seconds
WARNING 2025-06-26 05:46:29,530 process.py download_process():196 [1] No lopc.bin file for 2004.029.03
INFO 2025-06-26 05:46:29,530 process.py calibrate():227 [1] Calibration steps for 2004.029.03
INFO 2025-06-26 05:46:29,696 calibrate.py _navigation_process():1524 [1] Converting from Easting/Northing to lat/lon for mission 2004.029.03
INFO 2025-06-26 05:46:29,709 calibrate.py _nudge_pos():1680 [1] seg#   end_sec_diff end_lon_diff end_lat_diff len(segi)   seg_min u_drift (cm/s) v_drift (cm/s)     start datetime of segment
INFO 2025-06-26 05:46:29,711 calibrate.py _nudge_pos():1711 [1]                   -            -            -         0      0.00              -              -                             -
INFO 2025-06-26 05:46:29,814 calibrate.py _nudge_pos():1806 [1]    28:        0.150    0.0000104    0.0000244        25      0.08         19.245         56.534 2004-01-29T17:38:57.017772672
INFO 2025-06-26 05:46:31,137 calibrate.py _nudge_pos():1806 [1]   169:        0.160    0.0061324    0.0026579     11320     37.73         24.137         13.053 2004-01-29T18:19:01.013087744
INFO 2025-06-26 05:46:31,576 calibrate.py _nudge_pos():1806 [1]   216:        0.160   -0.0000095   -0.0000030        25      0.08        -17.556         -6.834 2004-01-29T18:19:52.009594240
INFO 2025-06-26 05:46:31,803 calibrate.py _nudge_pos():1806 [1]   240:        0.170    0.0051947    0.0013874     11540     38.46         20.055          6.684 2004-01-29T18:58:43.009909632
INFO 2025-06-26 05:46:32,289 calibrate.py _nudge_pos():1765 [1]   291:        0.180   14.9009034   -9.6271121     11415      0.01      64594.231     -46887.389 2004-01-29T19:37:36.010088192
ERROR 2025-06-26 05:46:32,289 calibrate.py _nudge_pos():1770 [1] End of underwater segment dead reckoned position is too different from GPS fix: abs(end_lon_diff) (14.900903378900608) > 1 or abs(end_lat_diff) (-9.627112065516677) > 1
INFO 2025-06-26 05:46:32,289 calibrate.py _nudge_pos():1776 [1] Fix this error by calling _range_qc_combined_nc() in _navigation_process() and/or _gps_process() for dorado 2004.029.03
ERROR 2025-06-26 05:46:32,289 calibrate.py process_logs():3546 [1] Error processing gps
Traceback (most recent call last):
  File "/src/data/calibrate.py", line 3535, in process_logs
    self._process(sensor, logs_dir, netcdfs_dir)
  File "/src/data/calibrate.py", line 3480, in _process
    self._gps_process(sensor)
  File "/src/data/calibrate.py", line 2085, in _gps_process
    nudged_longitude, nudged_latitude = self._nudge_pos()
                                        ^^^^^^^^^^^^^^^^^
  File "/src/data/calibrate.py", line 1786, in _nudge_pos
    raise ValueError(error_message)
ValueError: abs(end_lon_diff) (14.900903378900608) > 1 or abs(end_lat_diff) (-9.627112065516677) > 1
WARNING 2025-06-26 05:46:32,328 calibrate.py _depth_process():2193 [1] Expected sample_rate to be 10 Hz, instead it's 5.00 Hz
INFO 2025-06-26 05:46:32,690 calibrate.py _geometric_depth_correction():3423 [1] hs2: d_beg_time_diff: 3 seconds, d_end_time_diff: 0 seconds
INFO 2025-06-26 05:46:33,048 calibrate.py _geometric_depth_correction():3423 [1] ctd1: d_beg_time_diff: 19 seconds, d_end_time_diff: -1 seconds
INFO 2025-06-26 05:46:33,338 calibrate.py _geometric_depth_correction():3423 [1] ctd1: d_beg_time_diff: 19 seconds, d_end_time_diff: -1 seconds
INFO 2025-06-26 05:46:33,340 calibrate.py _ctd_process():2919 [1] Performing range checking of ['ctd1_salinity'] in 2004.029.03/ctd1.nc
INFO 2025-06-26 05:46:33,341 calibrate.py _range_qc_combined_nc():898 [1] Setting 0 ctd1_salinity values to NaN
INFO 2025-06-26 05:46:33,341 calibrate.py _range_qc_combined_nc():910 [1] Checked for data outside of these variables and ranges: [('ctd1_salinity', Range(min=30, max=40))]
INFO 2025-06-26 05:46:33,341 calibrate.py _range_qc_combined_nc():936 [1] Done range checking ctd1
INFO 2025-06-26 05:46:33,698 calibrate.py _geometric_depth_correction():3423 [1] ctd2: d_beg_time_diff: 19 seconds, d_end_time_diff: -1 seconds
INFO 2025-06-26 05:46:33,980 calibrate.py _geometric_depth_correction():3423 [1] ctd2: d_beg_time_diff: 19 seconds, d_end_time_diff: -1 seconds
INFO 2025-06-26 05:46:33,982 calibrate.py _ctd_process():2919 [1] Performing range checking of ['ctd2_salinity'] in 2004.029.03/ctd2.nc
INFO 2025-06-26 05:46:33,983 calibrate.py _range_qc_combined_nc():898 [1] Setting 1 ctd2_salinity values to NaN
INFO 2025-06-26 05:46:33,984 calibrate.py _range_qc_combined_nc():910 [1] Checked for data outside of these variables and ranges: [('ctd2_salinity', Range(min=30, max=40))]
INFO 2025-06-26 05:46:33,984 calibrate.py _range_qc_combined_nc():936 [1] Done range checking ctd2
ERROR 2025-06-26 05:46:33,994 calibrate.py process_logs():3542 [1] Error processing biolume: biolume has no orig_data - likely a missing or zero-sized .log file in missionlogs/2004.029.03
ERROR 2025-06-26 05:46:33,995 calibrate.py process_logs():3542 [1] Error processing lopc: lopc has no orig_data - likely a missing or zero-sized .log file in missionlogs/2004.029.03
ERROR 2025-06-26 05:46:33,995 calibrate.py process_logs():3542 [1] Error processing ecopuck: ecopuck has no orig_data - likely a missing or zero-sized .log file in missionlogs/2004.029.03
INFO 2025-06-26 05:46:34,034 calibrate.py write_netcdf():3508 [1] Writing calibrated instrument data to /data/auv_data/dorado/missionnetcdfs/2004.029.03/dorado_2004.029.03_cal.nc
INFO 2025-06-26 05:46:34,091 calibrate.py write_netcdf():3512 [1] Data variables written: ctd1_conductivity, ctd1_conductivity_onboard, ctd1_depth, ctd1_dissolvedO2, ctd1_flow1, ctd1_oxygen_mll, ctd1_oxygen_umolkg, ctd1_salinity, ctd1_temperature, ctd1_temperature_onboard, ctd1_time, ctd2_conductivity, ctd2_conductivity_onboard, ctd2_depth, ctd2_flow2, ctd2_salinity, ctd2_temperature, ctd2_temperature_onboard, ctd2_time, depth_filtdepth, depth_filtpres, depth_time, gps_latitude, gps_longitude, gps_time, hs2_bbp470, hs2_bbp676, hs2_depth, hs2_fl676, hs2_time, isus_nitrate, isus_quality, isus_temp, isus_time, navigation_depth, navigation_latitude, navigation_longitude, navigation_mWaterSpeed, navigation_pitch, navigation_posx, navigation_posy, navigation_roll, navigation_time, navigation_yaw, tailcone_propRpm, tailcone_time
INFO 2025-06-26 05:46:34,094 process.py align():251 [1] Alignment steps for 2004.029.03
INFO 2025-06-26 05:46:34,116 align.py process_cal():139 [1] Processing dorado_2004.029.03_cal.nc from /data/auv_data/dorado/missionnetcdfs/2004.029.03
ERROR 2025-06-26 05:46:34,117 process.py process_mission_exception_wrapper():537 [1] An exception occurred
Traceback (most recent call last):
  File "/src/data/process.py", line 529, in process_mission_exception_wrapper
    self.process_mission(mission, src_dir=src_dir)
  File "/src/data/process.py", line 485, in process_mission
    self.align(mission)
  File "/src/data/process.py", line 263, in align
    netcdf_dir = align_netcdf.process_cal()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/data/align.py", line 182, in process_cal
    raise InvalidCalFile(error_message) from None
align.InvalidCalFile: No nudged_latitude data in dorado_2004.029.03_cal.nc
ERROR 2025-06-26 05:46:34,118 process.py process_mission_exception_wrapper():538 [1] Failed to process to completion: 2004.029.03
INFO 2025-06-26 05:46:34,119 process.py archive():337 [1] Archiving steps for 2004.029.03
INFO 2025-06-26 05:46:34,119 process.py archive():347 [1] nc_file_base = /data/auv_data/dorado/missionnetcdfs/2004.029.03/dorado_2004.029.03, BASE_PATH = /data/auv_data
INFO 2025-06-26 05:46:34,119 archive.py copy_to_AUVTCD():60 [1] Archiving /data/auv_data/dorado/missionnetcdfs/2004.029.03/dorado_2004.029.03 files to /Volumes/AUVCTD/surveys/2004/netcdf
INFO 2025-06-26 05:46:34,122 archive.py copy_to_AUVTCD():78 [1] Removing /Volumes/AUVCTD/surveys/2004/netcdf/dorado_2004.029.03_cal.nc
INFO 2025-06-26 05:46:34,159 archive.py copy_to_AUVTCD():82 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/dorado_2004.029.03_cal.nc /Volumes/AUVCTD/surveys/2004/netcdf done.
INFO 2025-06-26 05:46:34,177 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/ctdDriver.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,193 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/ctdDriver2.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,197 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/gps.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,210 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/hydroscatlog.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,234 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/navigation.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,241 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/isuslog.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,255 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/parosci.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,272 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/tailCone.nc /Volumes/AUVCTD/missionnetcdfs/2004/2004029/2004.029.03 done.
INFO 2025-06-26 05:46:34,272 archive.py copy_to_AUVTCD():116 [1] Archiving product files
INFO 2025-06-26 05:46:34,273 archive.py copy_to_AUVTCD():161 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2004.029.03/dorado_2004.029.03_processing.log /Volumes/AUVCTD/surveys/2004/netcdf