2022-05-21T23:48:43.038Z,1653176923.038 [CommandExec](IMPORTANT): got command restart logs 2022-05-21T23:52:45.794Z,1653177165.794 [DataOverHttps](IMPORTANT): SBD MTMSN=20220521T235242 2022-05-21T23:52:46.723Z,1653177166.723 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 11.978093 min 2022-05-21T23:52:48.330Z,1653177168.330 [NAL9602](FAULT): GPS failed to acquire within timeout. 2022-05-21T23:52:53.488Z,1653177173.488 [CommandExec](IMPORTANT): got command schedule asap "load Science/trackPatch_yoyo.xml;set trackPatch_yoyo.MissionTimeout 2 h;set trackPatch_yoyo.MaxDepth 25 m;set trackPatch_yoyo.MinOffshore 4000 m;set trackPatch_yoyo.NeedCommsTimePatchTracking 60 min" 1qybi 1 2.000000 2022-05-21T23:52:53.488Z,1653177173.488 [CommandExec](IMPORTANT): Scheduling command #1 of 2 with id=1qybi 2022-05-21T23:52:53.489Z,1653177173.489 [CommandExec](IMPORTANT): Scheduled #31 (#1 of 2 with id='1qybi'): "load Science/trackPatch_yoyo.xml;set trackPatch_yoyo.MissionTimeout 2 h;set trackPatch_yoyo.MaxDepth 25 m;set trackPatch_yoyo.MinOffshore 4000 m;set trackPatch_yoyo.NeedCommsTimePatchTracking 60 min", ASAP 2022-05-21T23:52:54.462Z,1653177174.462 [DataOverHttps](IMPORTANT): SBD MTMSN=20220521T235250 2022-05-21T23:53:02.051Z,1653177182.051 [CommandExec](IMPORTANT): got command schedule asap "set trackPatch_yoyo.PatchTrackingYoYoMaxDepth 20 m;set trackPatch_yoyo.PeakShallowBnd 2 m;run" 1qybi 2 2.000000 2022-05-21T23:53:02.052Z,1653177182.052 [CommandExec](IMPORTANT): Scheduling command #2 of 2 with id=1qybi 2022-05-21T23:53:02.053Z,1653177182.053 [CommandExec](IMPORTANT): Scheduled #32 (#2 of 2 with id='1qybi'): "set trackPatch_yoyo.PatchTrackingYoYoMaxDepth 20 m;set trackPatch_yoyo.PeakShallowBnd 2 m;run", ASAP 2022-05-21T23:53:02.178Z,1653177182.178 [CommandExec](IMPORTANT): got command load ./Missions/Science/trackPatch_yoyo.xml 2022-05-21T23:53:05.400Z,1653177185.400 [CommandExec](IMPORTANT): Loaded ./Missions/Science/trackPatch_yoyo.xml 2022-05-21T23:53:23.270Z,1653177203.270 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.MissionTimeout 2 hour 2022-05-21T23:53:23.271Z,1653177203.271 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.MaxDepth 25 meter 2022-05-21T23:53:23.272Z,1653177203.272 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.MinOffshore 4000 meter 2022-05-21T23:53:23.272Z,1653177203.272 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.NeedCommsTimePatchTracking 60 minute 2022-05-21T23:53:37.866Z,1653177217.866 [DataOverHttps](IMPORTANT): SBD MTMSN=20220521T235334 2022-05-21T23:53:39.571Z,1653177219.571 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.PatchTrackingYoYoMaxDepth 20 meter 2022-05-21T23:53:39.572Z,1653177219.572 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.PeakShallowBnd 2 meter 2022-05-21T23:53:39.572Z,1653177219.572 [CommandExec](IMPORTANT): got command run 2022-05-21T23:53:39.574Z,1653177219.574 [CommandExec](IMPORTANT): Running 2022-05-21T23:53:39.738Z,1653177219.738 [MissionManager](IMPORTANT): Started mission trackPatch_yoyo 2022-05-21T23:53:50.346Z,1653177230.346 [CommandExec](IMPORTANT): got command schedule "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 4 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.MaxWaitNotAchievingDepth 0.5 h" 1qycj 1 4.000000 2022-05-21T23:53:50.346Z,1653177230.346 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=1qycj 2022-05-21T23:53:50.359Z,1653177230.359 [CommandExec](IMPORTANT): Scheduled #33 (#1 of 4 with id='1qycj'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 4 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.MaxWaitNotAchievingDepth 0.5 h", AFTER MISSION 2022-05-21T23:53:51.263Z,1653177231.263 [DataOverHttps](IMPORTANT): SBD MTMSN=20220521T235347 2022-05-21T23:53:59.425Z,1653177239.425 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.SendSampleStatusAndData 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.InitialDiveDuration 1 min;set IsothermDepthSampling.HeadingLeavingSide1 210 degree" 1qycj 2 4.000000 2022-05-21T23:53:59.425Z,1653177239.425 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=1qycj 2022-05-21T23:53:59.427Z,1653177239.427 [CommandExec](IMPORTANT): Scheduled #34 (#2 of 4 with id='1qycj'): "set IsothermDepthSampling.SendSampleStatusAndData 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.InitialDiveDuration 1 min;set IsothermDepthSampling.HeadingLeavingSide1 210 degree", AFTER MISSION 2022-05-21T23:54:00.354Z,1653177240.354 [DataOverHttps](IMPORTANT): SBD MTMSN=20220521T235356 2022-05-21T23:54:08.574Z,1653177248.574 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.HeadingLeavingSide2 270 degree;set IsothermDepthSampling.HeadingLeavingSide3 30 degree;set IsothermDepthSampling.HeadingLeavingSide4 120 degree;set IsothermDepthSampling.MaxDepth 30 m" 1qycj 3 4.000000 2022-05-21T23:54:08.575Z,1653177248.575 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=1qycj 2022-05-21T23:54:08.577Z,1653177248.577 [CommandExec](IMPORTANT): Scheduled #35 (#3 of 4 with id='1qycj'): "set IsothermDepthSampling.HeadingLeavingSide2 270 degree;set IsothermDepthSampling.HeadingLeavingSide3 30 degree;set IsothermDepthSampling.HeadingLeavingSide4 120 degree;set IsothermDepthSampling.MaxDepth 30 m", AFTER MISSION 2022-05-21T23:54:09.534Z,1653177249.534 [DataOverHttps](IMPORTANT): SBD MTMSN=20220521T235405 2022-05-21T23:54:17.766Z,1653177257.766 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.MinOffshore 4 km;set IsothermDepthSampling.ShallowBoundChl 2 m;set IsothermDepthSampling.DepDiffFromPeakChl1 0 m;set IsothermDepthSampling.WaitDepUndulation1 10.0 min;run" 1qycj 4 4.000000 2022-05-21T23:54:17.766Z,1653177257.766 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=1qycj 2022-05-21T23:54:17.767Z,1653177257.767 [CommandExec](IMPORTANT): Scheduled #36 (#4 of 4 with id='1qycj'): "set IsothermDepthSampling.MinOffshore 4 km;set IsothermDepthSampling.ShallowBoundChl 2 m;set IsothermDepthSampling.DepDiffFromPeakChl1 0 m;set IsothermDepthSampling.WaitDepUndulation1 10.0 min;run", AFTER MISSION 2022-05-21T23:58:39.932Z,1653177519.932 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T00:01:55.297Z,1653177715.297 [ElevatorServo](FAULT): Overload Error 2022-05-22T00:06:36.005Z,1653177996.005 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T00:14:16.732Z,1653178456.732 [ElevatorServo](FAULT): Overload Error 2022-05-22T00:42:13.123Z,1653180133.123 [MassServo](FAULT): Invalid EZ Servo response:"\377" 2022-05-22T00:49:27.303Z,1653180567.303 [CBIT](IMPORTANT): Beginning ground fault scan 2022-05-22T00:49:38.322Z,1653180578.322 [CBIT](IMPORTANT): Ground fault detected mA: CHAN A0 (Batt): -0.002446 CHAN A1 (24V): 0.001469 CHAN A2 (12V): -0.005475 CHAN A3 (5V): -0.003822 CHAN B0 (3.3V): -0.001281 CHAN B1 (3.15aV): -0.001525 CHAN B2 (3.15bV): -0.000820 CHAN B3 (GND): 0.030437 OPEN: 0.012534 Full Scale: +/- 1 mA 2022-05-22T01:13:06.159Z,1653181986.159 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T01:43:28.703Z,1653183808.703 [trackPatch_yoyo:D.OffshoreEnvelope](IMPORTANT): ACTIVE 2022-05-22T01:53:40.161Z,1653184420.161 [CommandExec](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml 2022-05-22T01:53:55.729Z,1653184435.729 [CommandExec](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml 2022-05-22T01:53:56.123Z,1653184436.123 [MissionManager](IMPORTANT): Started mission Default 2022-05-22T01:54:15.076Z,1653184455.076 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 4 hour 2022-05-22T01:54:15.077Z,1653184455.077 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 1 hour 2022-05-22T01:54:15.078Z,1653184455.078 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNotAchievingDepth 0.5 hour 2022-05-22T01:54:41.525Z,1653184481.525 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendSampleStatusAndData 0 bool 2022-05-22T01:54:41.526Z,1653184481.526 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1 bool 2022-05-22T01:54:41.527Z,1653184481.527 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.InitialDiveDuration 1 minute 2022-05-22T01:54:41.527Z,1653184481.527 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide1 210 degree 2022-05-22T01:55:05.865Z,1653184505.865 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide2 270 degree 2022-05-22T01:55:05.865Z,1653184505.865 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide3 30 degree 2022-05-22T01:55:05.866Z,1653184505.866 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide4 120 degree 2022-05-22T01:55:05.867Z,1653184505.867 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxDepth 30 meter 2022-05-22T01:55:28.740Z,1653184528.740 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MinOffshore 4 kilometer 2022-05-22T01:55:28.741Z,1653184528.741 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 2 meter 2022-05-22T01:55:28.741Z,1653184528.741 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChl1 0 meter 2022-05-22T01:55:28.742Z,1653184528.742 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 10.0 minute 2022-05-22T01:55:28.742Z,1653184528.742 [CommandExec](IMPORTANT): got command run 2022-05-22T01:55:28.745Z,1653184528.745 [CommandExec](IMPORTANT): Running 2022-05-22T01:55:29.040Z,1653184529.040 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling 2022-05-22T01:55:29.102Z,1653184529.102 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2022-05-22T01:59:58.223Z,1653184798.223 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T02:00:38.298Z,1653184838.298 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.461930 m 2022-05-22T02:01:47.962Z,1653184907.962 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -10.000000 count , nan K , 0.000000 m , nan degC , nan m , 10.000000 min . 2022-05-22T02:03:53.166Z,1653185033.166 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 20.094048 m 2022-05-22T02:06:10.126Z,1653185170.126 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 14.830724 m , 1.716000 ug/l , 4.414584 m , 13.305397 degC , 12.288120 ug/l 2022-05-22T02:06:10.535Z,1653185170.535 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Target depth = 4.414584 m . 2022-05-22T02:06:10.950Z,1653185170.950 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -10.000000 count 2022-05-22T02:09:15.214Z,1653185355.214 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 4.393441 m , settling for 10.000000 min 2022-05-22T02:19:28.119Z,1653185968.119 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-05-22T02:19:28.461Z,1653185968.461 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-05-22T02:20:24.466Z,1653186024.466 [ESPComponent](IMPORTANT): ESP has connected as client 2022-05-22T02:20:25.278Z,1653186025.278 [ESPComponent](IMPORTANT): [sample #3] Setting cartridge argument to -10 2022-05-22T02:20:25.278Z,1653186025.278 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-05-22T02:20:25.743Z,1653186025.743 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.711226 m in 13.256769 degC water with 10.665200 ug/l chlorophyll fluorescence. 2022-05-22T02:20:26.077Z,1653186026.077 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_LOG 2022-05-22T02:20:26.481Z,1653186026.481 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-05-22T02:20:28.501Z,1653186028.501 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_LOADING_CARTRIDGE 2022-05-22T02:21:08.953Z,1653186068.953 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PRIMING (@46,daSPR_phyto) 2022-05-22T02:21:10.973Z,1653186070.973 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PRIMING (@100.0) 2022-05-22T02:21:11.379Z,1653186071.379 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-05-22T02:21:11.781Z,1653186071.781 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-05-22T02:23:28.333Z,1653186208.333 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_FILTERING (@0.0) 2022-05-22T02:23:28.741Z,1653186208.741 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-05-22T03:08:52.355Z,1653188932.355 [ThrusterHE](FAULT): Could not get status:serial timeout 2022-05-22T03:09:12.333Z,1653188952.333 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PAUSED (@672.7) 2022-05-22T03:09:12.737Z,1653188952.737 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PROCESSING (@672.7) 2022-05-22T03:09:14.758Z,1653188954.758 [ESPComponent](IMPORTANT): Processing timeoutMinutes=120.000000 issuedCmd_=Cmd.startProcessing 2022-05-22T03:09:15.162Z,1653188955.162 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PROCESSED (@46,daSPR_phyto) 2022-05-22T04:07:29.450Z,1653192449.450 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPING (@46,daSPR_phyto) 2022-05-22T04:07:40.358Z,1653192460.358 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-05-22T04:07:40.770Z,1653192460.770 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPED 2022-05-22T04:12:40.537Z,1653192760.537 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence completed normally. 2022-05-22T04:12:41.438Z,1653192761.438 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed. 2022-05-22T04:12:44.352Z,1653192764.352 [MissionManager](IMPORTANT): Started mission Default 2022-05-22T04:18:10.048Z,1653193090.048 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T04:23:30.451Z,1653193410.451 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.762086 min 2022-05-22T04:23:32.073Z,1653193412.073 [NAL9602](FAULT): GPS failed to acquire within timeout. 2022-05-22T04:29:17.194Z,1653193757.194 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T04:34:00.490Z,1653194040.490 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.263080 min 2022-05-22T04:37:48.957Z,1653194268.957 [DataOverHttps](IMPORTANT): SBD MTMSN=20220522T043745 2022-05-22T04:37:56.574Z,1653194276.574 [CommandExec](IMPORTANT): got command schedule "load Science/trackPatch_yoyo.xml;set trackPatch_yoyo.MissionTimeout 2 h;set trackPatch_yoyo.MaxDepth 25 m;set trackPatch_yoyo.MinOffshore 4000 m;set trackPatch_yoyo.NeedCommsTimePatchTracking 60 min" 1rbip 1 2.000000 2022-05-22T04:37:56.575Z,1653194276.575 [CommandExec](IMPORTANT): Scheduling command #1 of 2 with id=1rbip 2022-05-22T04:37:56.577Z,1653194276.577 [CommandExec](IMPORTANT): Scheduled #37 (#1 of 2 with id='1rbip'): "load Science/trackPatch_yoyo.xml;set trackPatch_yoyo.MissionTimeout 2 h;set trackPatch_yoyo.MaxDepth 25 m;set trackPatch_yoyo.MinOffshore 4000 m;set trackPatch_yoyo.NeedCommsTimePatchTracking 60 min", AFTER MISSION 2022-05-22T04:37:57.542Z,1653194277.542 [DataOverHttps](IMPORTANT): SBD MTMSN=20220522T043753 2022-05-22T04:38:05.226Z,1653194285.226 [CommandExec](IMPORTANT): got command schedule "set trackPatch_yoyo.WpMaxDistance 1000 m;set trackPatch_yoyo.PatchTrackingYoYoMaxDepth 20 m;set trackPatch_yoyo.PeakShallowBnd 2 m;run" 1rbip 2 2.000000 2022-05-22T04:38:05.226Z,1653194285.226 [CommandExec](IMPORTANT): Scheduling command #2 of 2 with id=1rbip 2022-05-22T04:38:05.227Z,1653194285.227 [CommandExec](IMPORTANT): Scheduled #38 (#2 of 2 with id='1rbip'): "set trackPatch_yoyo.WpMaxDistance 1000 m;set trackPatch_yoyo.PatchTrackingYoYoMaxDepth 20 m;set trackPatch_yoyo.PeakShallowBnd 2 m;run", AFTER MISSION 2022-05-22T04:38:05.378Z,1653194285.378 [CommandExec](IMPORTANT): got command load ./Missions/Science/trackPatch_yoyo.xml 2022-05-22T04:38:08.558Z,1653194288.558 [CommandExec](IMPORTANT): Loaded ./Missions/Science/trackPatch_yoyo.xml 2022-05-22T04:38:26.155Z,1653194306.155 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.MissionTimeout 2 hour 2022-05-22T04:38:26.155Z,1653194306.155 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.MaxDepth 25 meter 2022-05-22T04:38:26.156Z,1653194306.156 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.MinOffshore 4000 meter 2022-05-22T04:38:26.157Z,1653194306.157 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.NeedCommsTimePatchTracking 60 minute 2022-05-22T04:38:40.218Z,1653194320.218 [DataOverHttps](IMPORTANT): SBD MTMSN=20220522T043836 2022-05-22T04:38:45.099Z,1653194325.099 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.WpMaxDistance 1000 meter 2022-05-22T04:38:45.105Z,1653194325.105 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.PatchTrackingYoYoMaxDepth 20 meter 2022-05-22T04:38:45.106Z,1653194325.106 [CommandExec](IMPORTANT): got command set trackPatch_yoyo.PeakShallowBnd 2 meter 2022-05-22T04:38:45.106Z,1653194325.106 [CommandExec](IMPORTANT): got command run 2022-05-22T04:38:45.110Z,1653194325.110 [CommandExec](IMPORTANT): Running 2022-05-22T04:38:45.330Z,1653194325.330 [MissionManager](IMPORTANT): Started mission trackPatch_yoyo 2022-05-22T04:38:50.452Z,1653194330.452 [CommandExec](IMPORTANT): got command schedule "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 4 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.MaxWaitNotAchievingDepth 0.5 h" 1rbjn 1 4.000000 2022-05-22T04:38:50.461Z,1653194330.461 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=1rbjn 2022-05-22T04:38:50.462Z,1653194330.462 [CommandExec](IMPORTANT): Scheduled #39 (#1 of 4 with id='1rbjn'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 4 h;set IsothermDepthSampling.MaxWaitNoFiring 1 h;set IsothermDepthSampling.MaxWaitNotAchievingDepth 0.5 h", AFTER MISSION 2022-05-22T04:38:50.784Z,1653194330.784 [DataOverHttps](IMPORTANT): SBD MOMSN=16817109, MTMSN=20220522T043847 2022-05-22T04:38:58.303Z,1653194338.303 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.SendSampleStatusAndData 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.InitialDiveDuration 1 min;set IsothermDepthSampling.HeadingLeavingSide1 210 degree" 1rbjn 2 4.000000 2022-05-22T04:38:58.316Z,1653194338.316 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=1rbjn 2022-05-22T04:38:58.317Z,1653194338.317 [CommandExec](IMPORTANT): Scheduled #40 (#2 of 4 with id='1rbjn'): "set IsothermDepthSampling.SendSampleStatusAndData 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.InitialDiveDuration 1 min;set IsothermDepthSampling.HeadingLeavingSide1 210 degree", AFTER MISSION 2022-05-22T04:38:59.540Z,1653194339.540 [DataOverHttps](IMPORTANT): SBD MTMSN=20220522T043855 2022-05-22T04:39:07.359Z,1653194347.359 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.HeadingLeavingSide2 270 degree;set IsothermDepthSampling.HeadingLeavingSide3 30 degree;set IsothermDepthSampling.HeadingLeavingSide4 120 degree;set IsothermDepthSampling.MaxDepth 30 m" 1rbjn 3 4.000000 2022-05-22T04:39:07.359Z,1653194347.359 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=1rbjn 2022-05-22T04:39:07.360Z,1653194347.360 [CommandExec](IMPORTANT): Scheduled #41 (#3 of 4 with id='1rbjn'): "set IsothermDepthSampling.HeadingLeavingSide2 270 degree;set IsothermDepthSampling.HeadingLeavingSide3 30 degree;set IsothermDepthSampling.HeadingLeavingSide4 120 degree;set IsothermDepthSampling.MaxDepth 30 m", AFTER MISSION 2022-05-22T04:39:08.541Z,1653194348.541 [DataOverHttps](IMPORTANT): SBD MOMSN=16817114, MTMSN=20220522T043904 2022-05-22T04:39:16.116Z,1653194356.116 [CommandExec](IMPORTANT): got command schedule "set IsothermDepthSampling.MinOffshore 4 km;set IsothermDepthSampling.ShallowBoundChl 2 m;set IsothermDepthSampling.DepDiffFromPeakChl1 0 m;set IsothermDepthSampling.WaitDepUndulation1 10.0 min;run" 1rbjn 4 4.000000 2022-05-22T04:39:16.116Z,1653194356.116 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=1rbjn 2022-05-22T04:39:16.117Z,1653194356.117 [CommandExec](IMPORTANT): Scheduled #42 (#4 of 4 with id='1rbjn'): "set IsothermDepthSampling.MinOffshore 4 km;set IsothermDepthSampling.ShallowBoundChl 2 m;set IsothermDepthSampling.DepDiffFromPeakChl1 0 m;set IsothermDepthSampling.WaitDepUndulation1 10.0 min;run", AFTER MISSION 2022-05-22T04:41:48.112Z,1653194508.112 [trackPatch_yoyo:PatchTracking:CrossLegToPatchCenter:B](IMPORTANT): Bearing for CrossLeg: 270.000000 arcdeg 2022-05-22T04:41:50.891Z,1653194510.891 [trackPatch_yoyo:PatchTracking:CrossLegSearchForPeak:B](IMPORTANT): Bearing for continued CrossLeg: 270.000000 arcdeg 2022-05-22T04:58:43.166Z,1653195523.166 [trackPatch_yoyo:SyslogWpMaxDistanceReached:SyslogWpMaxDistanceReachedChl:A](IMPORTANT): WpMaxDistance reached. Patch value is 6.322289 ug/l at 34.400750 arcdeg , -119.985544 arcdeg , distance = 915.160029 m 2022-05-22T04:58:45.992Z,1653195525.992 [trackPatch_yoyo:PatchTracking:InLeg:B](IMPORTANT): Bearing for InLeg: 90.000000 arcdeg 2022-05-22T05:14:11.570Z,1653196451.570 [trackPatch_yoyo:PatchTracking:InLeg:F](IMPORTANT): Bearing for OutLeg: 0.000000 arcdeg 2022-05-22T05:31:04.906Z,1653197464.906 [trackPatch_yoyo:SyslogWpMaxDistanceReached:SyslogWpMaxDistanceReachedChl:A](IMPORTANT): WpMaxDistance reached. Patch value is 6.322289 ug/l at 34.400750 arcdeg , -119.985544 arcdeg , distance = 1452.848386 m 2022-05-22T05:31:08.570Z,1653197468.570 [trackPatch_yoyo:PatchTracking:CrossLegToPatchCenterThenSurface:B](IMPORTANT): Bearing for CrossLeg: 180.000000 arcdeg 2022-05-22T06:03:11.113Z,1653199391.113 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T06:03:50.447Z,1653199430.447 [trackPatch_yoyo:PatchTracking:CrossLegSearchForPeak:B](IMPORTANT): Bearing for continued CrossLeg: 180.000000 arcdeg 2022-05-22T06:08:50.410Z,1653199730.410 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T06:20:45.969Z,1653200445.969 [trackPatch_yoyo:SyslogWpMaxDistanceReached:SyslogWpMaxDistanceReachedChl:A](IMPORTANT): WpMaxDistance reached. Patch value is 10.322123 ug/l at 34.400388 arcdeg , -120.009043 arcdeg , distance = 492.513068 m 2022-05-22T06:20:48.813Z,1653200448.813 [trackPatch_yoyo:PatchTracking:InLeg:B](IMPORTANT): Bearing for InLeg: 0.000000 arcdeg 2022-05-22T06:29:07.791Z,1653200947.791 [trackPatch_yoyo:PatchTracking:InLeg:F](IMPORTANT): Bearing for OutLeg: -270.000000 arcdeg 2022-05-22T06:35:08.585Z,1653201308.585 [trackPatch_yoyo:PatchTracking:ChlPatchDetection:B](IMPORTANT): Peak found before WpMaxDistance is reached. Patch value is 11.120977 ug/l at 34.397900 arcdeg , -120.012451 arcdeg , distance = 309.549269 m 2022-05-22T06:35:11.016Z,1653201311.016 [trackPatch_yoyo:PatchTracking:CrossLegToPatchCenter:B](IMPORTANT): Bearing for CrossLeg: -90.000000 arcdeg 2022-05-22T06:38:45.709Z,1653201525.709 [CommandExec](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml 2022-05-22T06:39:01.546Z,1653201541.546 [CommandExec](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml 2022-05-22T06:39:01.745Z,1653201541.745 [MissionManager](IMPORTANT): Started mission Default 2022-05-22T06:39:20.569Z,1653201560.569 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 4 hour 2022-05-22T06:39:20.570Z,1653201560.570 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 1 hour 2022-05-22T06:39:20.571Z,1653201560.571 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNotAchievingDepth 0.5 hour 2022-05-22T06:39:46.191Z,1653201586.191 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendSampleStatusAndData 0 bool 2022-05-22T06:39:46.193Z,1653201586.193 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1 bool 2022-05-22T06:39:46.193Z,1653201586.193 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.InitialDiveDuration 1 minute 2022-05-22T06:39:46.194Z,1653201586.194 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide1 210 degree 2022-05-22T06:40:09.318Z,1653201609.318 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide2 270 degree 2022-05-22T06:40:09.319Z,1653201609.319 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide3 30 degree 2022-05-22T06:40:09.321Z,1653201609.321 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.HeadingLeavingSide4 120 degree 2022-05-22T06:40:09.322Z,1653201609.322 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxDepth 30 meter 2022-05-22T06:40:31.110Z,1653201631.110 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MinOffshore 4 kilometer 2022-05-22T06:40:31.111Z,1653201631.111 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 2 meter 2022-05-22T06:40:31.111Z,1653201631.111 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChl1 0 meter 2022-05-22T06:40:31.112Z,1653201631.112 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 10.0 minute 2022-05-22T06:40:31.113Z,1653201631.113 [CommandExec](IMPORTANT): got command run 2022-05-22T06:40:31.115Z,1653201631.115 [CommandExec](IMPORTANT): Running 2022-05-22T06:40:31.384Z,1653201631.384 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling 2022-05-22T06:40:31.446Z,1653201631.446 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2022-05-22T06:44:35.003Z,1653201875.003 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-05-22T06:45:13.049Z,1653201913.049 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.466418 m 2022-05-22T06:46:22.524Z,1653201982.524 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -10.000000 count , nan K , 0.000000 m , nan degC , nan m , 10.000000 min . 2022-05-22T06:48:28.178Z,1653202108.178 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 20.074287 m 2022-05-22T06:49:38.580Z,1653202178.580 [CBIT](IMPORTANT): Beginning ground fault scan 2022-05-22T06:49:49.424Z,1653202189.424 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): 0.007150 CHAN A1 (24V): 0.001683 CHAN A2 (12V): -0.006105 CHAN A3 (5V): -0.003549 CHAN B0 (3.3V): -0.001220 CHAN B1 (3.15aV): -0.000887 CHAN B2 (3.15bV): -0.000960 CHAN B3 (GND): -0.000644 OPEN: 0.012487 Full Scale: +/- 1 mA 2022-05-22T06:50:29.828Z,1653202229.828 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 14.825778 m , 0.551200 ug/l , 6.023341 m , 12.659731 degC , 10.455641 ug/l 2022-05-22T06:50:30.180Z,1653202230.180 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Target depth = 6.023341 m . 2022-05-22T06:50:30.599Z,1653202230.599 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -10.000000 count 2022-05-22T06:53:15.876Z,1653202395.876 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 6.008841 m , settling for 10.000000 min 2022-05-22T07:03:28.785Z,1653203008.785 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-05-22T07:03:29.125Z,1653203009.125 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-05-22T07:04:24.365Z,1653203064.365 [ESPComponent](IMPORTANT): ESP has connected as client 2022-05-22T07:04:25.176Z,1653203065.176 [ESPComponent](IMPORTANT): [sample #4] Setting cartridge argument to -10 2022-05-22T07:04:25.176Z,1653203065.176 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-05-22T07:04:25.720Z,1653203065.720 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.351759 m in 13.479609 degC water with 7.155200 ug/l chlorophyll fluorescence. 2022-05-22T07:04:25.981Z,1653203065.981 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_LOG 2022-05-22T07:04:26.388Z,1653203066.388 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-05-22T07:04:28.404Z,1653203068.404 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_LOADING_CARTRIDGE 2022-05-22T07:05:08.809Z,1653203108.809 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PRIMING (@45,daSPR_phyto) 2022-05-22T07:05:11.239Z,1653203111.239 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PRIMING (@100.0) 2022-05-22T07:05:11.241Z,1653203111.241 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-05-22T07:05:11.637Z,1653203111.637 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-05-22T07:07:26.977Z,1653203246.977 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_FILTERING (@0.0) 2022-05-22T07:07:27.381Z,1653203247.381 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-05-22T07:39:18.784Z,1653205158.784 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PAUSED (@484.3) 2022-05-22T07:39:19.193Z,1653205159.193 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PROCESSING (@484.3) 2022-05-22T07:39:21.211Z,1653205161.211 [ESPComponent](IMPORTANT): Processing timeoutMinutes=120.000000 issuedCmd_=Cmd.startProcessing 2022-05-22T07:39:21.613Z,1653205161.613 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PROCESSED (@45,daSPR_phyto) 2022-05-22T08:14:35.016Z,1653207275.016 [Power24vConverter](FAULT): LCB fault: Invalid Command. Hardware Overcurrent Shutdown. 2022-05-22T08:28:37.108Z,1653208117.108 [ThrusterHE](IMPORTANT): Status lock Detected. 2022-05-22T08:28:37.487Z,1653208117.487 [ThrusterHE](IMPORTANT): Status lock Detected. 2022-05-22T08:28:37.884Z,1653208117.884 [ThrusterHE](IMPORTANT): Status lock Detected. 2022-05-22T08:28:38.296Z,1653208118.296 [ThrusterHE](IMPORTANT): Status lock Detected. 2022-05-22T08:28:38.680Z,1653208118.680 [ThrusterHE](IMPORTANT): Status lock Detected. 2022-05-22T08:28:39.133Z,1653208119.133 [ThrusterHE](IMPORTANT): Status lock Detected. 2022-05-22T08:40:15.505Z,1653208815.505 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPING (@45,daSPR_phyto) 2022-05-22T08:40:26.418Z,1653208826.418 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-05-22T08:40:26.889Z,1653208826.889 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPED 2022-05-22T08:45:26.641Z,1653209126.641 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence completed normally. 2022-05-22T08:45:27.580Z,1653209127.580 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed. 2022-05-22T08:45:30.544Z,1653209130.544 [MissionManager](IMPORTANT): Started mission Default