2023-04-14T04:46:04.793Z,1681447564.793 [Supervisor](IMPORTANT): Reading configuration overrides from Data/persisted.cfg 2023-04-14T04:46:08.944Z,1681447568.944 [StratificationFrontDetector](IMPORTANT): thresholdVTHI set to: 0.100000 degC 2023-04-14T04:46:10.016Z,1681447570.016 [SBIT](IMPORTANT): git: 2023-04-14 2023-04-14T04:46:10.017Z,1681447570.017 [SBIT](IMPORTANT): Kernel Release: 2.6.27.8 2023-04-14T04:48:37.528Z,1681447717.528 [SBIT](IMPORTANT): Beginning Startup BIT 2023-04-14T04:48:37.532Z,1681447717.532 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-14T04:48:48.460Z,1681447728.460 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005406 CHAN A1 (24V): -0.001034 CHAN A2 (12V): -0.000381 CHAN A3 (5V): -0.000611 CHAN B0 (3.3V): -0.000641 CHAN B1 (3.15aV): -0.000031 CHAN B2 (3.15bV): 0.000315 CHAN B3 (GND): -0.000092 OPEN: 0.003042 Full Scale: +/- 1 mA 2023-04-14T04:48:50.888Z,1681447730.888 [SBIT](FAULT): Mass: EXPECTED:0.032000 ACTUAL:0.031759 2023-04-14T04:49:31.682Z,1681447771.682 [SBIT](CRITICAL): SBIT FAILED 2023-04-14T04:49:31.683Z,1681447771.683 [SBIT](IMPORTANT): Listing configuration overrides from Data/persisted.cfg 2023-04-14T04:49:31.683Z,1681447771.683 [SBIT](IMPORTANT): BPC1.batteryMissingStickThreshold=12 count; 2023-04-14T04:49:31.684Z,1681447771.684 [SBIT](IMPORTANT): DAT.sbdAddress=-1 enum; 2023-04-14T04:49:31.684Z,1681447771.684 [SBIT](IMPORTANT): Depth_Keller.offset=3.6 decibar; 2023-04-14T04:49:31.684Z,1681447771.684 [SBIT](IMPORTANT): DropWeight.loadAtStartup=0 bool; 2023-04-14T04:49:31.684Z,1681447771.684 [SBIT](IMPORTANT): Express none CBIT.ampHoursUsed; 2023-04-14T04:49:31.684Z,1681447771.684 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_salinity_homogeneity_index 0.050000 practical_salinity_unit; 2023-04-14T04:49:31.684Z,1681447771.684 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_temperature_homogeneity_index 0.050000 celsius; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): Express none WetLabsBB2FL.bin_mean_mass_concentration_of_chlorophyll_in_sea_water; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): Express linearApproximation height_above_sea_floor 5.000000 meter; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): StratificationFrontDetector.loadAtStartup=1 bool; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): StratificationFrontDetector.threshold_VTHI=0.1 celsius; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): StratificationFrontDetector.verbosity=1 count; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): VerticalControl.buoyancyNeutral=73.007802 cubic_centimeter; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): VerticalControl.massDefault=31.756082 millimeter; 2023-04-14T04:49:31.685Z,1681447771.685 [SBIT](IMPORTANT): VerticalControl.massPositionLimitAft=-28 millimeter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalControl.massPositionLimitFwd=32 millimeter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalControl.surfaceThreshold=1.5 meter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth1=3 meter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth2=5 meter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth3=10 meter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth4=15 meter; 2023-04-14T04:49:31.686Z,1681447771.686 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.loadAtStartup=1 bool; 2023-04-14T04:49:31.688Z,1681447771.688 [CommandExec](FAULT): Scheduling is paused 2023-04-14T04:49:32.068Z,1681447772.068 [MissionManager](IMPORTANT): Started mission Startup 2023-04-14T04:50:51.559Z,1681447851.559 [MissionManager](IMPORTANT): Started mission Default 2023-04-14T04:51:34.565Z,1681447894.565 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T045132 2023-04-14T04:51:43.091Z,1681447903.091 [CommandExec](IMPORTANT): got command schedule clear 2023-04-14T04:51:43.091Z,1681447903.091 [CommandExec](IMPORTANT): Cleared 0 scheduled commands. 2023-04-14T04:51:43.092Z,1681447903.092 [CommandExec](IMPORTANT): got command schedule resume 2023-04-14T04:51:43.092Z,1681447903.092 [CommandExec](IMPORTANT): Scheduling is resumed 2023-04-14T04:53:30.437Z,1681448010.437 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T045329 2023-04-14T04:53:38.936Z,1681448018.936 [CommandExec](IMPORTANT): got command load ./Missions/Science/sci2.tl 2023-04-14T04:53:38.937Z,1681448018.937 [MissionManager](IMPORTANT): Loading Compiled TethysL script from file ./Missions/Science/sci2.tx 2023-04-14T04:53:41.551Z,1681448021.551 [CommandExec](IMPORTANT): Loaded ./Missions/Science/sci2.tl 2023-04-14T04:53:49.639Z,1681448029.639 [CommandExec](IMPORTANT): got command set sci2.MissionTimeout 8 hour 2023-04-14T04:53:49.640Z,1681448029.640 [CommandExec](IMPORTANT): got command set sci2.Lat1 36.835 degree 2023-04-14T04:53:49.641Z,1681448029.641 [CommandExec](IMPORTANT): got command set sci2.Lon1 -121.855 degree 2023-04-14T04:53:49.642Z,1681448029.642 [CommandExec](IMPORTANT): got command set sci2.YoYoMaxDepth 50 meter 2023-04-14T04:53:49.642Z,1681448029.642 [CommandExec](IMPORTANT): got command set sci2.MinAltitude 6 meter 2023-04-14T04:53:49.643Z,1681448029.643 [CommandExec](IMPORTANT): got command set sci2.MaxDepth 60 meter 2023-04-14T04:53:49.643Z,1681448029.643 [CommandExec](IMPORTANT): got command run 2023-04-14T04:53:49.646Z,1681448029.646 [CommandExec](IMPORTANT): Running 2023-04-14T04:53:49.756Z,1681448029.756 [MissionManager](IMPORTANT): Started mission sci2 2023-04-14T04:54:50.584Z,1681448090.584 [sci2:Lap:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.834999,-121.855003 2023-04-14T04:56:27.440Z,1681448187.440 [StratificationFrontDetector](IMPORTANT): Initial vertical homogeneity condition (0 = stratified, 1 = homogeneous): 1 2023-04-14T05:25:03.709Z,1681449903.709 [StratificationFrontDetector](IMPORTANT): Front detection is now enabled. Present vertical homogeneity condition (0 = stratified, 1 = homogeneous), VTHI, VSHI, countSameWaterType_, countOppositeWaterType_, frontDetectionEnabled_ = 0, 0.502039, nan, 0, 5, 1 2023-04-14T05:46:07.033Z,1681451167.033 [sci2:Lap:Wpt1:A.Waypoint](IMPORTANT): Reached waypoint: 36.834999,-121.855003 2023-04-14T05:46:09.870Z,1681451169.870 [MissionManager](IMPORTANT): Started mission Default 2023-04-14T05:55:29.988Z,1681451729.988 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 9.328379 min 2023-04-14T06:01:42.906Z,1681452102.906 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T060140 2023-04-14T06:01:48.322Z,1681452108.322 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 15.636406 min 2023-04-14T06:01:51.712Z,1681452111.712 [CommandExec](IMPORTANT): got command schedule asap "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 3 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool" 35fes 1 3.000000 2023-04-14T06:01:51.713Z,1681452111.713 [CommandExec](IMPORTANT): Scheduling command #1 of 3 with id=35fes 2023-04-14T06:01:51.714Z,1681452111.714 [CommandExec](IMPORTANT): Scheduled #1 (#1 of 3 with id='35fes'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 3 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool", ASAP 2023-04-14T06:01:54.591Z,1681452114.591 [DataOverHttps](IMPORTANT): SBD MOMSN=18220115, MTMSN=20230414T060152 2023-04-14T06:02:18.656Z,1681452138.656 [CommandExec](IMPORTANT): got command schedule asap "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m" 35fes 2 3.000000 2023-04-14T06:02:18.657Z,1681452138.657 [CommandExec](IMPORTANT): Scheduling command #2 of 3 with id=35fes 2023-04-14T06:02:18.658Z,1681452138.658 [CommandExec](IMPORTANT): Scheduled #2 (#2 of 3 with id='35fes'): "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m", ASAP 2023-04-14T06:02:19.946Z,1681452139.946 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T060217 2023-04-14T06:02:30.143Z,1681452150.143 [CommandExec](IMPORTANT): got command schedule asap "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run" 35fes 3 3.000000 2023-04-14T06:02:30.144Z,1681452150.144 [CommandExec](IMPORTANT): Scheduling command #3 of 3 with id=35fes 2023-04-14T06:02:30.145Z,1681452150.145 [CommandExec](IMPORTANT): Scheduled #3 (#3 of 3 with id='35fes'): "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run", ASAP 2023-04-14T06:02:30.280Z,1681452150.280 [CommandExec](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml 2023-04-14T06:02:59.038Z,1681452179.038 [CommandExec](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml 2023-04-14T06:03:26.157Z,1681452206.157 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 3 hour 2023-04-14T06:03:26.157Z,1681452206.157 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 1 hour 2023-04-14T06:03:26.158Z,1681452206.158 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendSampleStatusAndData 0 bool 2023-04-14T06:03:52.126Z,1681452232.126 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendDataToMultipleModems 0 bool 2023-04-14T06:03:52.127Z,1681452232.127 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1 bool 2023-04-14T06:03:52.128Z,1681452232.128 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 3 meter 2023-04-14T06:03:52.129Z,1681452232.129 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DeepBoundChl 20 meter 2023-04-14T06:04:04.773Z,1681452244.773 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 meter 2023-04-14T06:04:04.774Z,1681452244.774 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 5 minute 2023-04-14T06:04:04.774Z,1681452244.774 [CommandExec](IMPORTANT): got command run 2023-04-14T06:04:04.786Z,1681452244.786 [CommandExec](IMPORTANT): Running 2023-04-14T06:04:04.945Z,1681452244.945 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling 2023-04-14T06:04:05.031Z,1681452245.031 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 240.00 hours. 2023-04-14T06:09:48.455Z,1681452588.455 [IsothermDepthSampling:StayAwayFromWallsAndShore:FarEnoughFromWallsAndShore:A](IMPORTANT): Far enough from walls and shore. Latitude = 36.833864 arcdeg , longitude = -121.855010 arcdeg . Distance from shore = 4258.615234 m . OffshoreDistanceTurnBack (MinOffshore + MarginMinOffshore) = 2000.000000 m + 100.000000 m . Shortest distance nan m to side. MinDistFromPolygonSides = 200.000000 m . 2023-04-14T06:09:50.453Z,1681452590.453 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.196757 m 2023-04-14T06:10:59.964Z,1681452659.964 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2023-04-14T06:13:02.382Z,1681452782.382 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 23.108704 m 2023-04-14T06:18:07.827Z,1681453087.827 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 17.909584 m , 0.290000 ug/l , 5.168397 m , 11.338850 degC , 5.823200 ug/l 2023-04-14T06:18:08.198Z,1681453088.198 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Target depth = 5.168397 m . 2023-04-14T06:18:08.614Z,1681453088.614 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2023-04-14T06:22:09.006Z,1681453329.006 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 5.133131 m , settling for 5.000000 min 2023-04-14T06:27:11.555Z,1681453631.555 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::|summary:' 2023-04-14T06:27:21.705Z,1681453641.705 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-14T06:27:22.034Z,1681453642.034 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-14T06:28:14.317Z,1681453694.317 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-14T06:28:15.133Z,1681453695.133 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -6 2023-04-14T06:28:15.134Z,1681453695.134 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-14T06:28:15.587Z,1681453695.587 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.744605 m in 11.367182 degC water with 5.737360 ug/l chlorophyll fluorescence. 2023-04-14T06:28:16.338Z,1681453696.338 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-14T06:28:16.742Z,1681453696.742 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-14T06:28:18.362Z,1681453698.362 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-14T06:28:50.295Z,1681453730.295 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PRIMING (@57,archive_bac) 2023-04-14T06:28:53.117Z,1681453733.117 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PRIMING (@100.0) 2023-04-14T06:28:53.119Z,1681453733.119 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-14T06:28:53.529Z,1681453733.529 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-14T06:31:36.342Z,1681453896.342 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_FILTERING (@0.0) 2023-04-14T06:31:36.744Z,1681453896.744 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-14T06:56:40.991Z,1681455400.991 [ThrusterHE](IMPORTANT): Status lock Detected. 2023-04-14T06:56:41.395Z,1681455401.395 [ThrusterHE](IMPORTANT): Status lock Detected. 2023-04-14T06:56:41.812Z,1681455401.812 [ThrusterHE](IMPORTANT): Status lock Detected. 2023-04-14T06:56:42.244Z,1681455402.244 [ThrusterHE](IMPORTANT): Status lock Detected. 2023-04-14T07:08:48.922Z,1681456128.922 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PAUSED (@618.4) 2023-04-14T07:08:49.329Z,1681456129.329 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PROCESSING (@618.4) 2023-04-14T07:08:51.349Z,1681456131.349 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-14T07:08:51.751Z,1681456131.751 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PROCESSED (@57,archive_bac) 2023-04-14T07:20:55.325Z,1681456855.325 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING (@57,archive_bac) 2023-04-14T07:21:05.427Z,1681456865.427 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-14T07:21:05.838Z,1681456865.838 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2023-04-14T07:26:05.607Z,1681457165.607 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence completed normally. 2023-04-14T07:26:06.504Z,1681457166.504 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed. 2023-04-14T07:26:09.594Z,1681457169.594 [MissionManager](IMPORTANT): Started mission Default 2023-04-14T07:29:11.746Z,1681457351.746 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15395, MT Status=1, MTMSN=1113 2023-04-14T07:29:12.511Z,1681457352.511 [CommandExec](IMPORTANT): got command schedule "load Science/front_sampling.xml;set FrontSampling.NumSamplers 4 count;set FrontSampling.MissionTimeout 24 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.MaxWaitNoFiring 1 h;set FrontSampling.FrontAutonomousDetection 0 bool" 35fu1 1 6.000000 2023-04-14T07:29:12.516Z,1681457352.516 [CommandExec](IMPORTANT): Scheduling command #1 of 6 with id=35fu1 2023-04-14T07:29:12.517Z,1681457352.517 [CommandExec](IMPORTANT): Scheduled #4 (#1 of 6 with id='35fu1'): "load Science/front_sampling.xml;set FrontSampling.NumSamplers 4 count;set FrontSampling.MissionTimeout 24 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.MaxWaitNoFiring 1 h;set FrontSampling.FrontAutonomousDetection 0 bool", AFTER MISSION 2023-04-14T07:34:50.814Z,1681457690.814 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 8.680390 min 2023-04-14T07:39:55.468Z,1681457995.468 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T07:40:20.990Z,1681458020.990 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 14.183110 min 2023-04-14T07:48:45.260Z,1681458525.260 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 22.587858 min 2023-04-14T07:53:49.500Z,1681458829.500 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T07:54:15.351Z,1681458855.351 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 28.089360 min 2023-04-14T07:59:53.272Z,1681459193.272 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 33.721240 min 2023-04-14T08:04:57.250Z,1681459497.250 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T08:05:17.766Z,1681459517.766 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T080516 2023-04-14T08:05:25.253Z,1681459525.253 [CommandExec](IMPORTANT): got command schedule asap "load Science/front_sampling.xml;set FrontSampling.NumSamplers 4 count;set FrontSampling.MissionTimeout 24 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.MaxWaitNoFiring 1 h;set FrontSampling.FrontAutonomousDetection 0 bool" 35l47 1 6.000000 2023-04-14T08:05:25.253Z,1681459525.253 [CommandExec](IMPORTANT): Scheduling command #1 of 6 with id=35l47 2023-04-14T08:05:25.254Z,1681459525.254 [CommandExec](IMPORTANT): Scheduled #5 (#1 of 6 with id='35l47'): "load Science/front_sampling.xml;set FrontSampling.NumSamplers 4 count;set FrontSampling.MissionTimeout 24 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.MaxWaitNoFiring 1 h;set FrontSampling.FrontAutonomousDetection 0 bool", ASAP 2023-04-14T08:05:27.162Z,1681459527.162 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T080525 2023-04-14T08:05:27.988Z,1681459527.988 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 39.299520 min 2023-04-14T08:05:34.228Z,1681459534.228 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonFrontMinDesignated -121.9376 degree;set FrontSampling.LonFrontMaxDesignated -121.9375 degree;set FrontSampling.Lat1 36.835 degree;set FrontSampling.Lon1 -121.855 degree;set FrontSampling.Lat2 36.7555 degree" 35l47 2 6.000000 2023-04-14T08:05:34.229Z,1681459534.229 [CommandExec](IMPORTANT): Scheduling command #2 of 6 with id=35l47 2023-04-14T08:05:34.230Z,1681459534.230 [CommandExec](IMPORTANT): Scheduled #6 (#2 of 6 with id='35l47'): "set FrontSampling.LonFrontMinDesignated -121.9376 degree;set FrontSampling.LonFrontMaxDesignated -121.9375 degree;set FrontSampling.Lat1 36.835 degree;set FrontSampling.Lon1 -121.855 degree;set FrontSampling.Lat2 36.7555 degree", ASAP 2023-04-14T08:05:35.165Z,1681459535.165 [DataOverHttps](IMPORTANT): SBD MOMSN=18220365, MTMSN=20230414T080533 2023-04-14T08:05:42.372Z,1681459542.372 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.Lon2 -122.03 degree;set FrontSampling.LatUpwellingEnd 36.7555 degree;set FrontSampling.LonUpwellingEnd -122.03 degree;set FrontSampling.DeltaLatUpwellingSide1 -0.0375 degree" 35l47 3 6.000000 2023-04-14T08:05:42.373Z,1681459542.373 [CommandExec](IMPORTANT): Scheduling command #3 of 6 with id=35l47 2023-04-14T08:05:42.374Z,1681459542.374 [CommandExec](IMPORTANT): Scheduled #7 (#3 of 6 with id='35l47'): "set FrontSampling.Lon2 -122.03 degree;set FrontSampling.LatUpwellingEnd 36.7555 degree;set FrontSampling.LonUpwellingEnd -122.03 degree;set FrontSampling.DeltaLatUpwellingSide1 -0.0375 degree", ASAP 2023-04-14T08:05:43.529Z,1681459543.529 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T080542 2023-04-14T08:05:51.016Z,1681459551.016 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonUpwellingSide1 -0.0825 degree;set FrontSampling.LatStratifiedEnd 36.8395 degree;set FrontSampling.LonStratifiedEnd -121.845 degree;set FrontSampling.DeltaLatStratifiedSide1 0.0375 degree" 35l47 4 6.000000 2023-04-14T08:05:51.017Z,1681459551.017 [CommandExec](IMPORTANT): Scheduling command #4 of 6 with id=35l47 2023-04-14T08:05:51.018Z,1681459551.018 [CommandExec](IMPORTANT): Scheduled #8 (#4 of 6 with id='35l47'): "set FrontSampling.DeltaLonUpwellingSide1 -0.0825 degree;set FrontSampling.LatStratifiedEnd 36.8395 degree;set FrontSampling.LonStratifiedEnd -121.845 degree;set FrontSampling.DeltaLatStratifiedSide1 0.0375 degree", ASAP 2023-04-14T08:05:54.019Z,1681459554.019 [DataOverHttps](IMPORTANT): SBD MOMSN=18220370, MTMSN=20230414T080552 2023-04-14T08:06:01.573Z,1681459561.573 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonStratifiedSide1 0.0825 degree;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m;set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m" 35l47 5 6.000000 2023-04-14T08:06:01.574Z,1681459561.574 [CommandExec](IMPORTANT): Scheduling command #5 of 6 with id=35l47 2023-04-14T08:06:01.575Z,1681459561.575 [CommandExec](IMPORTANT): Scheduled #9 (#5 of 6 with id='35l47'): "set FrontSampling.DeltaLonStratifiedSide1 0.0825 degree;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m;set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m", ASAP 2023-04-14T08:06:03.569Z,1681459563.569 [DataOverHttps](IMPORTANT): SBD MTMSN=20230414T080601 2023-04-14T08:06:13.867Z,1681459573.867 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.WaitDepthUndulation 5 min;run" 35l47 6 6.000000 2023-04-14T08:06:13.867Z,1681459573.867 [CommandExec](IMPORTANT): Scheduling command #6 of 6 with id=35l47 2023-04-14T08:06:13.868Z,1681459573.868 [CommandExec](IMPORTANT): Scheduled #10 (#6 of 6 with id='35l47'): "set FrontSampling.WaitDepthUndulation 5 min;run", ASAP 2023-04-14T08:06:13.000Z,1681459574.000 [CommandExec](IMPORTANT): got command load ./Missions/Science/front_sampling.xml 2023-04-14T08:06:37.019Z,1681459597.019 [CommandExec](IMPORTANT): Loaded ./Missions/Science/front_sampling.xml 2023-04-14T08:07:13.713Z,1681459633.713 [CommandExec](IMPORTANT): got command set FrontSampling.NumSamplers 4 count 2023-04-14T08:07:13.714Z,1681459633.714 [CommandExec](IMPORTANT): got command set FrontSampling.MissionTimeout 24 hour 2023-04-14T08:07:13.714Z,1681459633.714 [CommandExec](IMPORTANT): got command set FrontSampling.MaxDepth 60 meter 2023-04-14T08:07:13.715Z,1681459633.715 [CommandExec](IMPORTANT): got command set FrontSampling.MaxWaitNoFiring 1 hour 2023-04-14T08:07:13.716Z,1681459633.716 [CommandExec](IMPORTANT): got command set FrontSampling.FrontAutonomousDetection 0 bool 2023-04-14T08:07:42.020Z,1681459662.020 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMinDesignated -121.9376 degree 2023-04-14T08:07:42.022Z,1681459662.022 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMaxDesignated -121.9375 degree 2023-04-14T08:07:42.026Z,1681459662.026 [CommandExec](IMPORTANT): got command set FrontSampling.Lat1 36.835 degree 2023-04-14T08:07:42.028Z,1681459662.028 [CommandExec](IMPORTANT): got command set FrontSampling.Lon1 -121.855 degree 2023-04-14T08:07:42.029Z,1681459662.029 [CommandExec](IMPORTANT): got command set FrontSampling.Lat2 36.7555 degree 2023-04-14T08:08:11.056Z,1681459691.056 [CommandExec](IMPORTANT): got command set FrontSampling.Lon2 -122.03 degree 2023-04-14T08:08:11.057Z,1681459691.057 [CommandExec](IMPORTANT): got command set FrontSampling.LatUpwellingEnd 36.7555 degree 2023-04-14T08:08:11.058Z,1681459691.058 [CommandExec](IMPORTANT): got command set FrontSampling.LonUpwellingEnd -122.03 degree 2023-04-14T08:08:11.058Z,1681459691.058 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide1 -0.0375 degree 2023-04-14T08:08:53.608Z,1681459733.608 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide1 -0.0825 degree 2023-04-14T08:08:53.609Z,1681459733.609 [CommandExec](IMPORTANT): got command set FrontSampling.LatStratifiedEnd 36.8395 degree 2023-04-14T08:08:53.609Z,1681459733.609 [CommandExec](IMPORTANT): got command set FrontSampling.LonStratifiedEnd -121.845 degree 2023-04-14T08:08:53.610Z,1681459733.610 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatStratifiedSide1 0.0375 degree 2023-04-14T08:09:17.547Z,1681459757.547 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonStratifiedSide1 0.0825 degree 2023-04-14T08:09:17.548Z,1681459757.548 [CommandExec](IMPORTANT): got command set FrontSampling.ShallowBoundChl 3 meter 2023-04-14T08:09:17.548Z,1681459757.548 [CommandExec](IMPORTANT): got command set FrontSampling.DeepBoundChl 20 meter 2023-04-14T08:09:17.549Z,1681459757.549 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMinDepth 2 meter 2023-04-14T08:09:17.550Z,1681459757.550 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMaxDepth 50 meter 2023-04-14T08:09:23.121Z,1681459763.121 [CommandExec](IMPORTANT): got command set FrontSampling.WaitDepthUndulation 5 minute 2023-04-14T08:09:23.122Z,1681459763.122 [CommandExec](IMPORTANT): got command run 2023-04-14T08:09:23.207Z,1681459763.207 [CommandExec](IMPORTANT): Running 2023-04-14T08:09:23.357Z,1681459763.357 [MissionManager](IMPORTANT): Started mission FrontSampling 2023-04-14T08:10:32.421Z,1681459832.421 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T08:13:05.696Z,1681459985.696 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.834999,-121.855003 2023-04-14T08:18:07.096Z,1681460287.096 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T08:23:50.118Z,1681460630.118 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Reached waypoint: 36.834999,-121.855003 2023-04-14T08:23:50.122Z,1681460630.122 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt2:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.755501,-122.029999 2023-04-14T09:14:36.308Z,1681463676.308 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15396, MT Status=1, MTMSN=1114 2023-04-14T09:14:37.354Z,1681463677.354 [CommandExec](IMPORTANT): got command schedule "set FrontSampling.LonFrontMinDesignated -121.9376 degree;set FrontSampling.LonFrontMaxDesignated -121.9375 degree;set FrontSampling.Lat1 36.835 degree;set FrontSampling.Lon1 -121.855 degree;set FrontSampling.Lat2 36.7555 degree" 35fu1 2 6.000000 2023-04-14T09:14:37.355Z,1681463677.355 [CommandExec](IMPORTANT): Scheduling command #2 of 6 with id=35fu1 2023-04-14T09:14:37.360Z,1681463677.360 [CommandExec](IMPORTANT): Scheduled #11 (#2 of 6 with id='35fu1'): "set FrontSampling.LonFrontMinDesignated -121.9376 degree;set FrontSampling.LonFrontMaxDesignated -121.9375 degree;set FrontSampling.Lat1 36.835 degree;set FrontSampling.Lon1 -121.855 degree;set FrontSampling.Lat2 36.7555 degree", AFTER MISSION 2023-04-14T09:16:03.633Z,1681463763.633 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15397, MT Status=1, MTMSN=1115 2023-04-14T09:16:04.532Z,1681463764.532 [CommandExec](IMPORTANT): got command schedule "set FrontSampling.Lon2 -122.03 degree;set FrontSampling.LatUpwellingEnd 36.7555 degree;set FrontSampling.LonUpwellingEnd -122.03 degree;set FrontSampling.DeltaLatUpwellingSide1 -0.0375 degree" 35fu1 3 6.000000 2023-04-14T09:16:04.534Z,1681463764.534 [CommandExec](IMPORTANT): Scheduling command #3 of 6 with id=35fu1 2023-04-14T09:16:04.535Z,1681463764.535 [CommandExec](IMPORTANT): Scheduled #12 (#3 of 6 with id='35fu1'): "set FrontSampling.Lon2 -122.03 degree;set FrontSampling.LatUpwellingEnd 36.7555 degree;set FrontSampling.LonUpwellingEnd -122.03 degree;set FrontSampling.DeltaLatUpwellingSide1 -0.0375 degree", AFTER MISSION 2023-04-14T09:24:23.484Z,1681464263.484 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-14T10:20:14.031Z,1681467614.031 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15398, MT Status=1, MTMSN=1116 2023-04-14T10:20:14.889Z,1681467614.889 [CommandExec](IMPORTANT): got command schedule "set FrontSampling.DeltaLonUpwellingSide1 -0.0825 degree;set FrontSampling.LatStratifiedEnd 36.8395 degree;set FrontSampling.LonStratifiedEnd -121.845 degree;set FrontSampling.DeltaLatStratifiedSide1 0.0375 degree" 35fu1 4 6.000000 2023-04-14T10:20:14.890Z,1681467614.890 [CommandExec](IMPORTANT): Scheduling command #4 of 6 with id=35fu1 2023-04-14T10:20:14.891Z,1681467614.891 [CommandExec](IMPORTANT): Scheduled #13 (#4 of 6 with id='35fu1'): "set FrontSampling.DeltaLonUpwellingSide1 -0.0825 degree;set FrontSampling.LatStratifiedEnd 36.8395 degree;set FrontSampling.LonStratifiedEnd -121.845 degree;set FrontSampling.DeltaLatStratifiedSide1 0.0375 degree", AFTER MISSION 2023-04-14T10:48:48.615Z,1681469328.615 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-14T10:48:59.635Z,1681469339.635 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003399 CHAN A1 (24V): -0.001021 CHAN A2 (12V): -0.000755 CHAN A3 (5V): 0.000166 CHAN B0 (3.3V): 0.000086 CHAN B1 (3.15aV): -0.000235 CHAN B2 (3.15bV): 0.000351 CHAN B3 (GND): 0.000513 OPEN: 0.007354 Full Scale: +/- 1 mA 2023-04-14T11:23:58.234Z,1681471438.234 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15399, MT Status=1, MTMSN=1117 2023-04-14T11:23:59.034Z,1681471439.034 [CommandExec](IMPORTANT): got command schedule "set FrontSampling.DeltaLonStratifiedSide1 0.0825 degree;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m;set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m" 35fu1 5 6.000000 2023-04-14T11:23:59.034Z,1681471439.034 [CommandExec](IMPORTANT): Scheduling command #5 of 6 with id=35fu1 2023-04-14T11:23:59.035Z,1681471439.035 [CommandExec](IMPORTANT): Scheduled #14 (#5 of 6 with id='35fu1'): "set FrontSampling.DeltaLonStratifiedSide1 0.0825 degree;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m;set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m", AFTER MISSION 2023-04-14T11:26:09.353Z,1681471569.353 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15400, MT Status=1, MTMSN=1118 2023-04-14T11:26:10.012Z,1681471570.012 [CommandExec](IMPORTANT): got command schedule "set FrontSampling.WaitDepthUndulation 5 min;run" 35fu1 6 6.000000 2023-04-14T11:26:10.013Z,1681471570.013 [CommandExec](IMPORTANT): Scheduling command #6 of 6 with id=35fu1 2023-04-14T11:26:10.014Z,1681471570.014 [CommandExec](IMPORTANT): Scheduled #15 (#6 of 6 with id='35fu1'): "set FrontSampling.WaitDepthUndulation 5 min;run", AFTER MISSION 2023-04-14T11:26:48.541Z,1681471608.541 [FrontSampling:CrossFrontFromStratifiedToUpwelling:P](IMPORTANT): Crossed the front from stratified to upwelling. VTHI = 0.022913 degC . VSHI = 0.006547 psu . Front latitude, longitude = 36.797110 arcdeg , -121.939312 arcdeg . Lat1, Lon1 = 36.759610 arcdeg , -122.021812 arcdeg . Lat2, Lon2 = nan arcdeg , nan arcdeg . Lat3, Lon3 = nan arcdeg , nan arcdeg . Lat4, Lon4 = nan arcdeg , nan arcdeg . Lat5, Lon5 = 36.755501 arcdeg , -122.029999 arcdeg . Lat6, Lon6 = 36.839500 arcdeg , -121.845001 arcdeg . Sample count = 1.000000 count . 2023-04-14T11:26:51.734Z,1681471611.734 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.755501,-122.029999 2023-04-14T11:26:52.947Z,1681471612.947 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.759610,-122.021812 2023-04-14T11:26:53.350Z,1681471613.350 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.755501,-122.029999 2023-04-14T11:26:53.761Z,1681471613.761 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.759610,-122.021812 2023-04-14T11:26:55.384Z,1681471615.384 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.301779 m 2023-04-14T11:28:46.193Z,1681471726.193 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.166254 m 2023-04-14T11:29:28.210Z,1681471768.210 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 20.150331 m , 1.693600 ug/l , 3.718102 m , 2.306676 ug/l 2023-04-14T11:29:31.827Z,1681471771.827 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Latitude, longitude = 36.797134 arcdeg , -121.939737 arcdeg . Target depth = 3.718102 m . 2023-04-14T11:29:32.239Z,1681471772.239 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-14T11:33:37.482Z,1681472017.482 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 3.705661 m , settling for 5.000000 min 2023-04-14T11:38:50.268Z,1681472330.268 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-14T11:38:50.421Z,1681472330.421 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-14T11:39:42.339Z,1681472382.339 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-14T11:39:43.154Z,1681472383.154 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -6 2023-04-14T11:39:43.154Z,1681472383.154 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-14T11:39:43.610Z,1681472383.610 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.786479 m in 10.226191 degC water with 2.373360 ug/l chlorophyll fluorescence. 2023-04-14T11:39:43.954Z,1681472383.954 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-14T11:39:44.358Z,1681472384.358 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-14T11:39:46.374Z,1681472386.374 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-14T11:40:18.298Z,1681472418.298 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PRIMING (@56,archive_bac) 2023-04-14T11:40:20.315Z,1681472420.315 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PRIMING (@100.0) 2023-04-14T11:40:20.737Z,1681472420.737 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-14T11:40:21.118Z,1681472421.118 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-14T11:43:04.334Z,1681472584.334 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_FILTERING (@0.0) 2023-04-14T11:43:04.737Z,1681472584.737 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-14T12:37:20.232Z,1681475840.232 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PAUSED (@980.8) 2023-04-14T12:37:20.649Z,1681475840.649 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PROCESSING (@980.8) 2023-04-14T12:37:22.672Z,1681475842.672 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-14T12:37:23.068Z,1681475843.068 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PROCESSED (@56,archive_bac) 2023-04-14T12:49:26.650Z,1681476566.650 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPING (@56,archive_bac) 2023-04-14T12:49:36.744Z,1681476576.744 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-14T12:49:37.193Z,1681476577.193 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPED 2023-04-14T12:54:36.912Z,1681476876.912 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence completed normally. 2023-04-14T12:54:37.740Z,1681476877.740 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed. 2023-04-14T12:54:38.955Z,1681476878.955 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:NoFiringForTooLong:B](IMPORTANT): No firing for too long. Stopping mission. ElapsedTime, MaxWaitNoFiring = 87.739510 min , 60.000000 min 2023-04-14T12:54:39.390Z,1681476879.390 [CommandExec](IMPORTANT): got command stop 2023-04-14T12:54:39.392Z,1681476879.392 [CommandExec](IMPORTANT): Scheduling is paused for 6 commands 2023-04-14T12:54:40.329Z,1681476880.329 [MissionManager](IMPORTANT): Started mission Default 2023-04-14T13:00:00.081Z,1681477200.081 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T13:05:26.165Z,1681477526.165 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.757054 min 2023-04-14T13:05:27.769Z,1681477527.769 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-04-14T13:15:08.873Z,1681478108.873 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 20.468685 min 2023-04-14T13:21:45.343Z,1681478505.343 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15404, MT Status=1, MTMSN=1119 2023-04-14T13:21:45.847Z,1681478505.847 [CommandExec](IMPORTANT): got command schedule clear 2023-04-14T13:21:45.848Z,1681478505.848 [CommandExec](IMPORTANT): Cleared 6 scheduled commands. 2023-04-14T13:21:45.849Z,1681478505.849 [CommandExec](IMPORTANT): got command schedule resume 2023-04-14T13:21:45.849Z,1681478505.849 [CommandExec](IMPORTANT): Scheduling is resumed 2023-04-14T13:29:20.798Z,1681478960.798 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 34.667712 min 2023-04-14T13:31:36.451Z,1681479096.451 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15409, MT Status=1, MTMSN=1120 2023-04-14T13:31:37.241Z,1681479097.241 [CommandExec](IMPORTANT): got command schedule "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 8 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool" 3600p 1 4.000000 2023-04-14T13:31:37.241Z,1681479097.241 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=3600p 2023-04-14T13:31:37.242Z,1681479097.242 [CommandExec](IMPORTANT): Scheduled #16 (#1 of 4 with id='3600p'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 8 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool", AFTER MISSION 2023-04-14T13:32:47.227Z,1681479167.227 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15410, MT Status=1, MTMSN=1121 2023-04-14T13:32:48.071Z,1681479168.071 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.MaxDepth 60 m;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m" 3600p 2 4.000000 2023-04-14T13:32:48.072Z,1681479168.072 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=3600p 2023-04-14T13:32:48.073Z,1681479168.073 [CommandExec](IMPORTANT): Scheduled #17 (#2 of 4 with id='3600p'): "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.MaxDepth 60 m;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m", AFTER MISSION 2023-04-14T13:34:54.500Z,1681479294.500 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15411, MT Status=1, MTMSN=1122 2023-04-14T13:34:55.205Z,1681479295.205 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.TransitYoYoMinDepth 2 m;set IsothermDepthSampling.TransitYoYoMaxDepth 50 m;set IsothermDepthSampling.Lat1 36.76 degree;set IsothermDepthSampling.Lon1 -122.02 degree" 3600p 3 4.000000 2023-04-14T13:34:55.205Z,1681479295.205 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=3600p 2023-04-14T13:34:55.206Z,1681479295.206 [CommandExec](IMPORTANT): Scheduled #18 (#3 of 4 with id='3600p'): "set IsothermDepthSampling.TransitYoYoMinDepth 2 m;set IsothermDepthSampling.TransitYoYoMaxDepth 50 m;set IsothermDepthSampling.Lat1 36.76 degree;set IsothermDepthSampling.Lon1 -122.02 degree", AFTER MISSION 2023-04-14T13:36:13.280Z,1681479373.280 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15412, MT Status=1, MTMSN=1123 2023-04-14T13:36:13.948Z,1681479373.948 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run" 3600p 4 4.000000 2023-04-14T13:36:13.949Z,1681479373.949 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=3600p 2023-04-14T13:36:13.956Z,1681479373.956 [CommandExec](IMPORTANT): Scheduled #19 (#4 of 4 with id='3600p'): "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run", AFTER MISSION 2023-04-14T13:36:14.093Z,1681479374.093 [CommandExec](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml 2023-04-14T13:36:30.286Z,1681479390.286 [CommandExec](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml 2023-04-14T13:36:50.671Z,1681479410.671 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 8 hour 2023-04-14T13:36:50.672Z,1681479410.672 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 1 hour 2023-04-14T13:36:50.679Z,1681479410.679 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendSampleStatusAndData 0 bool 2023-04-14T13:37:22.079Z,1681479442.079 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendDataToMultipleModems 0 bool 2023-04-14T13:37:22.079Z,1681479442.079 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1 bool 2023-04-14T13:37:22.081Z,1681479442.081 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxDepth 60 meter 2023-04-14T13:37:22.081Z,1681479442.081 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 3 meter 2023-04-14T13:37:22.082Z,1681479442.082 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DeepBoundChl 20 meter 2023-04-14T13:37:44.884Z,1681479464.884 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.TransitYoYoMinDepth 2 meter 2023-04-14T13:37:44.885Z,1681479464.885 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.TransitYoYoMaxDepth 50 meter 2023-04-14T13:37:44.886Z,1681479464.886 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.Lat1 36.76 degree 2023-04-14T13:37:44.886Z,1681479464.886 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.Lon1 -122.02 degree 2023-04-14T13:37:59.446Z,1681479479.446 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 meter 2023-04-14T13:37:59.447Z,1681479479.447 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 5 minute 2023-04-14T13:37:59.447Z,1681479479.447 [CommandExec](IMPORTANT): got command run 2023-04-14T13:37:59.451Z,1681479479.451 [CommandExec](IMPORTANT): Running 2023-04-14T13:37:59.670Z,1681479479.670 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling 2023-04-14T13:37:59.748Z,1681479479.748 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 240.00 hours. 2023-04-14T13:42:36.047Z,1681479756.047 [IsothermDepthSampling:TransitToSamplingLocationNoCommsOrNoTransit:Wpt1NoComms:Wpt1:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.759998,-122.019997 2023-04-14T16:00:45.617Z,1681488045.617 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-14T16:12:03.583Z,1681488723.583 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15437, MT Status=1, MTMSN=1124 2023-04-14T16:12:04.085Z,1681488724.085 [CommandExec](IMPORTANT): got command stop 2023-04-14T16:12:04.085Z,1681488724.085 [CommandExec](IMPORTANT): Scheduling is paused 2023-04-14T16:12:05.187Z,1681488725.187 [MissionManager](IMPORTANT): Started mission Default 2023-04-14T16:21:53.558Z,1681489313.558 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 9.799430 min 2023-04-14T16:23:04.383Z,1681489384.383 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15447, MT Status=1, MTMSN=1125 2023-04-14T16:23:04.911Z,1681489384.911 [CommandExec](IMPORTANT): got command schedule clear 2023-04-14T16:23:04.911Z,1681489384.911 [CommandExec](IMPORTANT): Cleared 0 scheduled commands. 2023-04-14T16:23:04.911Z,1681489384.911 [CommandExec](IMPORTANT): got command schedule resume 2023-04-14T16:23:04.911Z,1681489384.911 [CommandExec](IMPORTANT): Scheduling is resumed 2023-04-14T16:24:03.923Z,1681489443.923 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15451, MT Status=1, MTMSN=1126 2023-04-14T16:24:04.697Z,1681489444.697 [CommandExec](IMPORTANT): got command schedule "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 8 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool" 3687q 1 4.000000 2023-04-14T16:24:04.697Z,1681489444.697 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=3687q 2023-04-14T16:24:04.698Z,1681489444.698 [CommandExec](IMPORTANT): Scheduled #20 (#1 of 4 with id='3687q'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 8 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool", AFTER MISSION 2023-04-14T16:24:50.442Z,1681489490.442 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15453, MT Status=1, MTMSN=1127 2023-04-14T16:24:51.207Z,1681489491.207 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.MaxDepth 60 m;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m" 3687q 2 4.000000 2023-04-14T16:24:51.208Z,1681489491.208 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=3687q 2023-04-14T16:24:51.213Z,1681489491.213 [CommandExec](IMPORTANT): Scheduled #21 (#2 of 4 with id='3687q'): "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.MaxDepth 60 m;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m", AFTER MISSION 2023-04-14T16:26:06.951Z,1681489566.951 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=15454, MT Status=1, MTMSN=1128 2023-04-14T16:26:07.719Z,1681489567.719 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.TransitYoYoMinDepth 2 m;set IsothermDepthSampling.TransitYoYoMaxDepth 50 m;set IsothermDepthSampling.Lat1 36.868 degree;set IsothermDepthSampling.Lon1 -121.967 degree" 3687q 3 4.000000 2023-04-14T16:26:07.720Z,1681489567.720 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=3687q 2023-04-14T16:26:07.721Z,1681489567.721 [CommandExec](IMPORTANT): Scheduled #22 (#3 of 4 with id='3687q'): "set IsothermDepthSampling.TransitYoYoMinDepth 2 m;set IsothermDepthSampling.TransitYoYoMaxDepth 50 m;set IsothermDepthSampling.Lat1 36.868 degree;set IsothermDepthSampling.Lon1 -121.967 degree", AFTER MISSION 2023-04-14T16:26:20.681Z,1681489580.681 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15455, MT Status=1, MTMSN=1129 2023-04-14T16:26:21.368Z,1681489581.368 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run" 3687q 4 4.000000 2023-04-14T16:26:21.369Z,1681489581.369 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=3687q 2023-04-14T16:26:21.370Z,1681489581.370 [CommandExec](IMPORTANT): Scheduled #23 (#4 of 4 with id='3687q'): "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run", AFTER MISSION 2023-04-14T16:26:21.579Z,1681489581.579 [CommandExec](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml 2023-04-14T16:26:37.641Z,1681489597.641 [CommandExec](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml 2023-04-14T16:26:57.568Z,1681489617.568 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 8 hour 2023-04-14T16:26:57.568Z,1681489617.568 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 1 hour 2023-04-14T16:26:57.569Z,1681489617.569 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendSampleStatusAndData 0 bool 2023-04-14T16:27:28.889Z,1681489648.889 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendDataToMultipleModems 0 bool 2023-04-14T16:27:28.890Z,1681489648.890 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1 bool 2023-04-14T16:27:28.891Z,1681489648.891 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxDepth 60 meter 2023-04-14T16:27:28.891Z,1681489648.891 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 3 meter 2023-04-14T16:27:28.893Z,1681489648.893 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DeepBoundChl 20 meter 2023-04-14T16:27:51.289Z,1681489671.289 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.TransitYoYoMinDepth 2 meter 2023-04-14T16:27:51.290Z,1681489671.290 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.TransitYoYoMaxDepth 50 meter 2023-04-14T16:27:51.290Z,1681489671.290 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.Lat1 36.868 degree 2023-04-14T16:27:51.291Z,1681489671.291 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.Lon1 -121.967 degree 2023-04-14T16:28:05.768Z,1681489685.768 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 meter 2023-04-14T16:28:05.769Z,1681489685.769 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 5 minute 2023-04-14T16:28:05.769Z,1681489685.769 [CommandExec](IMPORTANT): got command run 2023-04-14T16:28:05.771Z,1681489685.771 [CommandExec](IMPORTANT): Running 2023-04-14T16:28:05.847Z,1681489685.847 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling 2023-04-14T16:28:05.906Z,1681489685.906 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 240.00 hours. 2023-04-14T16:30:24.411Z,1681489824.411 [IsothermDepthSampling:TransitToSamplingLocationNoCommsOrNoTransit:Wpt1NoComms:Wpt1:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.868000,-121.967003 2023-04-14T16:34:07.783Z,1681490047.783 [DVL_micro](FAULT): Failed to initialize within timeout. 2023-04-14T16:48:59.000Z,1681490940.000 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-14T16:49:10.930Z,1681490950.930 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003368 CHAN A1 (24V): -0.000941 CHAN A2 (12V): 0.000134 CHAN A3 (5V): -0.001467 CHAN B0 (3.3V): 0.000075 CHAN B1 (3.15aV): 0.000253 CHAN B2 (3.15bV): 0.000356 CHAN B3 (GND): 0.000367 OPEN: 0.007648 Full Scale: +/- 1 mA 2023-04-14T18:12:01.204Z,1681495921.204 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2023-04-14T18:44:27.018Z,1681497867.018 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15476, MT Status=1, MTMSN=1130