INFO 2025-06-26 05:29:50,372 process.py process_mission():429 [1] commandline = src/data/process_dorado.py --start_year 2003 --end_year 2014 --num_cores 1 -v --noinput --clobber INFO 2025-06-26 05:29:50,372 process.py process_mission():434 [1] dorado_info[mission]["comment"] = OCCO2003115: missions {02,03} - Missing log files - are they on an AUVCTD project computer? - (JR found them on old RH laptop & copied to my tempbox - I'll put them in place) - does not process with legacy Matlab code - ctdToUse = ctd1 INFO 2025-06-26 05:29:50,372 process.py process_mission():449 [1] Processing OCCO mission 2003.115.03 by user docker_user on host 671049448ece INFO 2025-06-26 05:29:50,372 process.py download_process():165 [1] Download and processing steps for 2003.115.03 INFO 2025-06-26 05:29:50,372 logs2netcdfs.py download_process_logs():739 [1] src_dir = /Volumes/AUVCTD/missionlogs/2003/2003115/2003.115.03 INFO 2025-06-26 05:29:50,372 logs2netcdfs.py download_process_logs():768 [1] Rsyncing /Volumes/AUVCTD/missionlogs/2003/2003115/2003.115.03 to /data/auv_data/dorado/missionlogs/2003.115.03 INFO 2025-06-26 05:29:50,464 logs2netcdfs.py download_process_logs():779 [1] Processing mission: dorado 2003.115.03 INFO 2025-06-26 05:29:50,464 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/ctdDriver.log (1736014 bytes) WARNING 2025-06-26 05:29:50,464 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/ctdDriver.log INFO 2025-06-26 05:29:50,464 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/ctdDriver2.log (1736448 bytes) WARNING 2025-06-26 05:29:50,464 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/ctdDriver2.log INFO 2025-06-26 05:29:50,464 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/gps.log (15336 bytes) WARNING 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/gps.log INFO 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/hydroscatlog.log (1187929 bytes) WARNING 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/hydroscatlog.log INFO 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/navigation.log (4835369 bytes) WARNING 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/navigation.log INFO 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/isuslog.log (93276 bytes) WARNING 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/isuslog.log INFO 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/parosci.log (1189225 bytes) WARNING 2025-06-26 05:29:50,465 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/parosci.log INFO 2025-06-26 05:29:50,466 logs2netcdfs.py download_process_logs():788 [1] Processing file /data/auv_data/dorado/missionlogs/2003.115.03/tailCone.log (1189291 bytes) WARNING 2025-06-26 05:29:50,466 logs2netcdfs.py download_process_logs():795 [1] Failed to parse long_name & units from header in file /data/auv_data/dorado/missionlogs/2003.115.03/tailCone.log INFO 2025-06-26 05:29:50,466 logs2netcdfs.py download_process_logs():807 [1] Time to process: 0.00 seconds WARNING 2025-06-26 05:29:50,466 process.py download_process():196 [1] No lopc.bin file for 2003.115.03 INFO 2025-06-26 05:29:50,466 process.py calibrate():227 [1] Calibration steps for 2003.115.03 ERROR 2025-06-26 05:29:50,468 calibrate.py process_logs():3542 [1] Error processing navigation: navigation has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,468 calibrate.py process_logs():3542 [1] Error processing gps: gps has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,469 calibrate.py process_logs():3542 [1] Error processing isus: isus has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,469 calibrate.py process_logs():3542 [1] Error processing biolume: biolume has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,469 calibrate.py process_logs():3542 [1] Error processing lopc: lopc has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,469 calibrate.py process_logs():3542 [1] Error processing ecopuck: ecopuck has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,470 calibrate.py process_logs():3542 [1] Error processing tailcone: tailcone has no orig_data - likely a missing or zero-sized .log file in missionlogs/2003.115.03 ERROR 2025-06-26 05:29:50,470 process.py calibrate():247 [1] 2003.115.03 No depth_time variable in combined_nc INFO 2025-06-26 05:29:50,470 process.py align():251 [1] Alignment steps for 2003.115.03 ERROR 2025-06-26 05:29:50,470 process.py align():266 [1] 2003.115.03 [Errno 2] No such file or directory: '/data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03_cal.nc' ERROR 2025-06-26 05:29:50,470 process.py process_mission_exception_wrapper():537 [1] An exception occurred Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info file = self._cache[self._key] ~~~~~~~~~~~^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__ value = self._cache[key] ~~~~~~~~~~~^^^^^ KeyError: [, ('/data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03_cal.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), '9f7c644a-3a1f-4ec1-9833-4a33dc45ace8'] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/src/data/process.py", line 263, in align netcdf_dir = align_netcdf.process_cal() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/data/align.py", line 136, in process_cal self.calibrated_nc = xr.open_dataset(Path(netcdfs_dir, in_fn)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/api.py", line 687, in open_dataset backend_ds = backend.open_dataset( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 666, in open_dataset store = NetCDF4DataStore.open( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 452, in open return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 393, in __init__ self.format = self.ds.data_model ^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 461, in ds return self._acquire() ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 455, in _acquire with self._manager.acquire_context(needs_lock) as root: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 199, in acquire_context file, cached = self._acquire_with_cache_info(needs_lock) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info file = self._opener(*self._args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "src/netCDF4/_netCDF4.pyx", line 2521, in netCDF4._netCDF4.Dataset.__init__ File "src/netCDF4/_netCDF4.pyx", line 2158, in netCDF4._netCDF4._ensure_nc_success FileNotFoundError: [Errno 2] No such file or directory: '/data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03_cal.nc' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/src/data/process.py", line 529, in process_mission_exception_wrapper self.process_mission(mission, src_dir=src_dir) File "/src/data/process.py", line 485, in process_mission self.align(mission) File "/src/data/process.py", line 268, in align raise InvalidCalFile(error_message) from e align.InvalidCalFile: 2003.115.03 [Errno 2] No such file or directory: '/data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03_cal.nc' ERROR 2025-06-26 05:29:50,472 process.py process_mission_exception_wrapper():538 [1] Failed to process to completion: 2003.115.03 INFO 2025-06-26 05:29:50,472 process.py archive():337 [1] Archiving steps for 2003.115.03 INFO 2025-06-26 05:29:50,472 process.py archive():347 [1] nc_file_base = /data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03, BASE_PATH = /data/auv_data INFO 2025-06-26 05:29:50,473 archive.py copy_to_AUVTCD():60 [1] Archiving /data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03 files to /Volumes/AUVCTD/surveys/2003/netcdf INFO 2025-06-26 05:29:50,473 archive.py copy_to_AUVTCD():116 [1] Archiving product files INFO 2025-06-26 05:29:50,474 archive.py copy_to_AUVTCD():161 [1] copyfile /data/auv_data/dorado/missionnetcdfs/2003.115.03/dorado_2003.115.03_processing.log /Volumes/AUVCTD/surveys/2003/netcdf