INFO 2025-06-26 05:31:08,460 process.py process_mission():429 [1] commandline = src/data/process_dorado.py --start_year 2003 --end_year 2014 --num_cores 1 -v --noinput --clobber INFO 2025-06-26 05:31:08,460 process.py process_mission():434 [1] dorado_info[mission]["comment"] = OCCO2003169: missions {04,05,06} - ctdToUse = ctd1 INFO 2025-06-26 05:31:08,460 process.py process_mission():449 [1] Processing OCCO mission 2003.169.06 by user docker_user on host 671049448ece INFO 2025-06-26 05:31:08,460 process.py download_process():165 [1] Download and processing steps for 2003.169.06 INFO 2025-06-26 05:31:08,460 logs2netcdfs.py download_process_logs():739 [1] src_dir = /Volumes/AUVCTD/missionlogs/2003/2003169/2003.169.06 INFO 2025-06-26 05:31:08,460 logs2netcdfs.py download_process_logs():768 [1] Rsyncing /Volumes/AUVCTD/missionlogs/2003/2003169/2003.169.06 to /data/auv_data/dorado/missionlogs/2003.169.06 INFO 2025-06-26 05:31:10,604 logs2netcdfs.py download_process_logs():779 [1] Processing mission: dorado 2003.169.06 INFO 2025-06-26 05:31:10,604 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/ctdDriver.log (962198 bytes) INFO 2025-06-26 05:31:10,678 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/ctdDriver2.log (962280 bytes) INFO 2025-06-26 05:31:10,753 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/gps.log (11126 bytes) INFO 2025-06-26 05:31:10,760 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/hydroscatlog.log (657937 bytes) INFO 2025-06-26 05:31:10,842 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/navigation.log (2668506 bytes) INFO 2025-06-26 05:31:11,045 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/isuslog.log (50766 bytes) INFO 2025-06-26 05:31:11,054 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/parosci.log (658460 bytes) INFO 2025-06-26 05:31:11,113 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.169.06/tailCone.log (658123 bytes) WARNING 2025-06-26 05:31:11,113 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.169.06/tailCone.log INFO 2025-06-26 05:31:11,113 logs2netcdfs.py download_process_logs():807 [1] Time to process: 0.51 seconds WARNING 2025-06-26 05:31:11,114 process.py download_process():196 [1] No lopc.bin file for 2003.169.06 INFO 2025-06-26 05:31:11,114 process.py calibrate():227 [1] Calibration steps for 2003.169.06 INFO 2025-06-26 05:31:11,168 calibrate.py _navigation_process():1524 [1] Converting from Easting/Northing to lat/lon for mission 2003.169.06 INFO 2025-06-26 05:31:11,176 calibrate.py _nudge_pos():1680 [1] seg# end_sec_diff end_lon_diff end_lat_diff len(segi) seg_min u_drift (cm/s) v_drift (cm/s) start datetime of segment INFO 2025-06-26 05:31:11,177 calibrate.py _nudge_pos():1711 [1] - - - 0 0.00 - - - INFO 2025-06-26 05:31:12,615 calibrate.py _nudge_pos():1806 [1] 173: 0.150 -0.0265562 0.0242347 12846 42.82 -92.005 104.882 2003-06-18T19:33:15.579152768 INFO 2025-06-26 05:31:12,640 calibrate.py _nudge_pos():1862 [1] 154: - - - 2 0.00 - - INFO 2025-06-26 05:31:12,640 calibrate.py _nudge_pos():1868 [1] Points in final series = 13608 WARNING 2025-06-26 05:31:12,655 calibrate.py _depth_process():2193 [1] Expected sample_rate to be 10 Hz, instead it's 5.00 Hz WARNING 2025-06-26 05:31:12,669 calibrate.py _hs2_process():2289 [1] Cannot process HS2 data: [Errno 2] No such file or directory: '/data/auv_data/dorado/missionlogs/2003.169.06/hs2Calibration.dat' ERROR 2025-06-26 05:31:12,700 calibrate.py process_logs():3548 [1] Error processing ctd1: missing variable 'ctd' INFO 2025-06-26 05:31:12,805 calibrate.py _geometric_depth_correction():3423 [1] ctd2: d_beg_time_diff: 10 seconds, d_end_time_diff: 0 seconds INFO 2025-06-26 05:31:12,889 calibrate.py _geometric_depth_correction():3423 [1] ctd2: d_beg_time_diff: 10 seconds, d_end_time_diff: 0 seconds INFO 2025-06-26 05:31:12,891 calibrate.py _ctd_process():2919 [1] Performing range checking of ['ctd2_salinity'] in 2003.169.06/ctd2.nc INFO 2025-06-26 05:31:12,892 calibrate.py _range_qc_combined_nc():898 [1] Setting 1 ctd2_salinity values to NaN INFO 2025-06-26 05:31:12,892 calibrate.py _range_qc_combined_nc():910 [1] Checked for data outside of these variables and ranges: [('ctd2_salinity', Range(min=30, max=40))] INFO 2025-06-26 05:31:12,892 calibrate.py _range_qc_combined_nc():936 [1] Done range checking ctd2 ERROR 2025-06-26 05:31:12,899 calibrate.py process_logs():3542 [1] Error processing biolume: biolume has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.169.06 ERROR 2025-06-26 05:31:12,899 calibrate.py process_logs():3542 [1] Error processing lopc: lopc has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.169.06 ERROR 2025-06-26 05:31:12,899 calibrate.py process_logs():3542 [1] Error processing ecopuck: ecopuck has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.169.06 ERROR 2025-06-26 05:31:12,899 calibrate.py process_logs():3542 [1] Error processing tailcone: tailcone has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.169.06 INFO 2025-06-26 05:31:12,900 calibrate.py write_netcdf():3508 [1] Writing calibrated instrument data to /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_cal.nc INFO 2025-06-26 05:31:12,932 calibrate.py write_netcdf():3512 [1] Data variables written: ctd1_conductivity, ctd1_conductivity_onboard, ctd1_dissolvedO2, ctd1_salinity, ctd1_temperature, ctd1_temperature_onboard, ctd1_time, ctd2_conductivity, ctd2_conductivity_onboard, ctd2_depth, ctd2_flow2, ctd2_salinity, ctd2_temperature, ctd2_temperature_onboard, ctd2_time, depth_filtdepth, depth_filtpres, depth_time, gps_latitude, gps_longitude, gps_time, isus_nitrate, isus_quality, isus_temp, isus_time, navigation_depth, navigation_latitude, navigation_longitude, navigation_mWaterSpeed, navigation_pitch, navigation_posx, navigation_posy, navigation_roll, navigation_time, navigation_yaw, nudged_latitude, nudged_longitude, time INFO 2025-06-26 05:31:12,934 process.py align():251 [1] Alignment steps for 2003.169.06 INFO 2025-06-26 05:31:12,952 align.py process_cal():139 [1] Processing dorado_2003.169.06_cal.nc from /data/auv_data/dorado/missionnetcdfs/2003.169.06 INFO 2025-06-26 05:31:12,960 align.py process_cal():206 [1] Using pitch corrected navigation_depth: mDepth from navigation.nc INFO 2025-06-26 05:31:12,962 align.py process_cal():249 [1] navigation_roll: instrument_sample_rate_hz = 5.00 INFO 2025-06-26 05:31:12,979 align.py process_cal():206 [1] Using pitch corrected navigation_depth: mDepth from navigation.nc INFO 2025-06-26 05:31:12,981 align.py process_cal():249 [1] navigation_pitch: instrument_sample_rate_hz = 5.00 INFO 2025-06-26 05:31:12,998 align.py process_cal():206 [1] Using pitch corrected navigation_depth: mDepth from navigation.nc INFO 2025-06-26 05:31:13,000 align.py process_cal():249 [1] navigation_yaw: instrument_sample_rate_hz = 5.00 INFO 2025-06-26 05:31:13,010 align.py process_cal():161 [1] Skipping navigation_posx INFO 2025-06-26 05:31:13,010 align.py process_cal():161 [1] Skipping navigation_posy INFO 2025-06-26 05:31:13,016 align.py process_cal():206 [1] Using pitch corrected navigation_depth: mDepth from navigation.nc INFO 2025-06-26 05:31:13,018 align.py process_cal():249 [1] navigation_mWaterSpeed: instrument_sample_rate_hz = 5.00 INFO 2025-06-26 05:31:13,037 align.py process_cal():249 [1] ctd1_temperature: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,055 align.py process_cal():249 [1] ctd1_conductivity: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,073 align.py process_cal():249 [1] ctd1_salinity: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,091 align.py process_cal():249 [1] ctd1_temperature_onboard: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,108 align.py process_cal():249 [1] ctd1_conductivity_onboard: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,126 align.py process_cal():249 [1] ctd1_dissolvedO2: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,142 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,144 align.py process_cal():249 [1] ctd2_temperature: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,161 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,163 align.py process_cal():249 [1] ctd2_conductivity: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,179 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,181 align.py process_cal():249 [1] ctd2_salinity: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,197 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,199 align.py process_cal():249 [1] ctd2_temperature_onboard: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,215 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,217 align.py process_cal():249 [1] ctd2_conductivity_onboard: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,233 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,235 align.py process_cal():249 [1] ctd2_flow2: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,251 align.py process_cal():206 [1] Using pitch corrected ctd2_depth: Variable depth_filtdepth from dorado_2003.169.06_cal.nc linearly interpolated to ctd2_time and corrected for pitch using SensorOffset(x=1.003, y=0.0001) INFO 2025-06-26 05:31:13,252 align.py process_cal():249 [1] ctd2_depth: instrument_sample_rate_hz = 4.00 INFO 2025-06-26 05:31:13,270 align.py process_cal():249 [1] isus_nitrate: instrument_sample_rate_hz = 0.59 INFO 2025-06-26 05:31:13,285 align.py process_cal():249 [1] isus_temp: instrument_sample_rate_hz = 0.59 INFO 2025-06-26 05:31:13,300 align.py process_cal():249 [1] isus_quality: instrument_sample_rate_hz = 0.59 INFO 2025-06-26 05:31:13,308 align.py write_netcdf():337 [1] Writing aligned data to /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_align.nc INFO 2025-06-26 05:31:13,338 align.py write_netcdf():342 [1] Data variables written: ctd1_conductivity, ctd1_conductivity_onboard, ctd1_depth, ctd1_dissolvedO2, ctd1_latitude, ctd1_longitude, ctd1_salinity, ctd1_temperature, ctd1_temperature_onboard, ctd1_time, ctd2_conductivity, ctd2_conductivity_onboard, ctd2_depth, ctd2_flow2, ctd2_latitude, ctd2_longitude, ctd2_salinity, ctd2_temperature, ctd2_temperature_onboard, ctd2_time, isus_depth, isus_latitude, isus_longitude, isus_nitrate, isus_quality, isus_temp, isus_time, navigation_depth, navigation_latitude, navigation_longitude, navigation_mWaterSpeed, navigation_pitch, navigation_roll, navigation_time, navigation_yaw INFO 2025-06-26 05:31:13,340 process.py resample():273 [1] Resampling steps for 2003.169.06 INFO 2025-06-26 05:31:13,341 resample.py resample_mission():1185 [1] Using depth_threshold = 10.00 m INFO 2025-06-26 05:31:13,355 resample.py instruments_variables():235 [1] Reading variables from /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_align.nc mission netCDF file INFO 2025-06-26 05:31:13,358 resample.py instruments_variables():235 [1] Reading variables from /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_align.nc mission netCDF file INFO 2025-06-26 05:31:13,359 resample.py get_mission_start_end():1145 [1] navigation: 2003-06-18T18:47:56.385430272 to 2003-06-18T19:33:17.978988416 (0 days 00:00:00.039997184 before mission_end) INFO 2025-06-26 05:31:13,361 resample.py get_mission_start_end():1145 [1] ctd1 : 2003-06-18T18:47:46.796087296 to 2003-06-18T19:33:17.908993152 (0 days 00:00:00.109992448 before mission_end) INFO 2025-06-26 05:31:13,362 resample.py get_mission_start_end():1145 [1] ctd2 : 2003-06-18T18:47:46.666096128 to 2003-06-18T19:33:18.018985600 (0 days 00:00:00 before mission_end) INFO 2025-06-26 05:31:13,364 resample.py get_mission_start_end():1145 [1] isus : 2003-06-18T18:48:51.311667584 to 2003-06-18T19:33:17.789001344 (0 days 00:00:00.229984256 before mission_end) INFO 2025-06-26 05:31:13,365 resample.py instruments_variables():235 [1] Reading variables from /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_align.nc mission netCDF file INFO 2025-06-26 05:31:13,365 resample.py resample_coordinates():249 [1] Resampling coordinates depth, latitude and longitude with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,420 resample.py resample_variable():1031 [1] Resampling navigation_roll with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,432 resample.py resample_variable():1031 [1] Resampling navigation_pitch with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,444 resample.py resample_variable():1031 [1] Resampling navigation_yaw with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,457 resample.py resample_variable():1031 [1] Resampling navigation_mWaterSpeed with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,468 resample.py resample_variable():1031 [1] Resampling ctd1_temperature with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,479 resample.py resample_variable():1031 [1] Resampling ctd1_conductivity with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,489 resample.py resample_variable():1031 [1] Resampling ctd1_salinity with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,499 resample.py resample_variable():1031 [1] Resampling ctd1_temperature_onboard with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,509 resample.py resample_variable():1031 [1] Resampling ctd1_conductivity_onboard with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,522 resample.py resample_variable():1031 [1] Resampling ctd1_dissolvedO2 with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,535 resample.py resample_variable():1031 [1] Resampling ctd2_temperature with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,549 resample.py resample_variable():1031 [1] Resampling ctd2_conductivity with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,565 resample.py resample_variable():1031 [1] Resampling ctd2_salinity with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,581 resample.py resample_variable():1031 [1] Resampling ctd2_temperature_onboard with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,594 resample.py resample_variable():1031 [1] Resampling ctd2_conductivity_onboard with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,605 resample.py resample_variable():1031 [1] Resampling ctd2_flow2 with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,614 resample.py resample_variable():1031 [1] Resampling isus_nitrate with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,623 resample.py resample_variable():1031 [1] Resampling isus_temp with frequency 1S following 3 point median filter INFO 2025-06-26 05:31:13,640 resample.py resample_variable():1031 [1] Resampling isus_quality with frequency 1S following 3 point median filter WARNING 2025-06-26 05:31:13,710 resample.py dorado_global_metadata():177 [1] No comment for pitch correction in ctd1_depth for mission 2003.169.06 INFO 2025-06-26 05:31:13,729 resample.py resample_mission():1297 [1] Saved resampled mission to /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_1S.nc INFO 2025-06-26 05:31:14,774 create_products.py gulper_odv():401 [1] No gulper times found for 2003.169.06 INFO 2025-06-26 05:31:14,774 process.py archive():337 [1] Archiving steps for 2003.169.06 INFO 2025-06-26 05:31:14,775 process.py archive():347 [1] nc_file_base = /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06, BASE_PATH = /data/auv_data INFO 2025-06-26 05:31:14,775 archive.py copy_to_AUVTCD():60 [1] Archiving /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06 files to /Volumes/AUVCTD/surveys/2003/netcdf INFO 2025-06-26 05:31:14,776 archive.py copy_to_AUVTCD():78 [1] Removing /Volumes/AUVCTD/surveys/2003/netcdf/dorado_2003.169.06_1S.nc INFO 2025-06-26 05:31:14,785 archive.py copy_to_AUVTCD():82 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_1S.nc /Volumes/AUVCTD/surveys/2003/netcdf done. INFO 2025-06-26 05:31:14,786 archive.py copy_to_AUVTCD():78 [1] Removing /Volumes/AUVCTD/surveys/2003/netcdf/dorado_2003.169.06_cal.nc INFO 2025-06-26 05:31:14,800 archive.py copy_to_AUVTCD():82 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_cal.nc /Volumes/AUVCTD/surveys/2003/netcdf done. INFO 2025-06-26 05:31:14,801 archive.py copy_to_AUVTCD():78 [1] Removing /Volumes/AUVCTD/surveys/2003/netcdf/dorado_2003.169.06_align.nc INFO 2025-06-26 05:31:14,815 archive.py copy_to_AUVTCD():82 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_align.nc /Volumes/AUVCTD/surveys/2003/netcdf done. INFO 2025-06-26 05:31:14,825 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/ctdDriver.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,834 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/ctdDriver2.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,838 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/gps.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,846 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/hydroscatlog.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,860 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/navigation.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,864 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/isuslog.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,871 archive.py copy_to_AUVTCD():108 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/parosci.nc /Volumes/AUVCTD/missionnetcdfs/2003/2003169/2003.169.06 done. INFO 2025-06-26 05:31:14,871 archive.py copy_to_AUVTCD():116 [1] Archiving product files INFO 2025-06-26 05:31:14,872 archive.py copy_to_AUVTCD():161 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.169.06/dorado_2003.169.06_processing.log /Volumes/AUVCTD/surveys/2003/netcdf