2023-04-19T18:06:48.990Z,1681927608.990 [CommandExec](IMPORTANT): got command restart logs 2023-04-19T18:15:26.597Z,1681928126.597 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 9.621788 min 2023-04-19T18:21:16.481Z,1681928476.481 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 15.453000 min 2023-04-19T18:21:57.408Z,1681928517.408 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=15963, MT Status=1, MTMSN=1187 2023-04-19T18:21:58.166Z,1681928518.166 [CommandExec](IMPORTANT): got command schedule asap "load Science/front_sampling.xml;set FrontSampling.NumSamplers 5 count;set FrontSampling.MissionTimeout 24 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.RepeatTransects 2 count;set FrontSampling.FrontAutonomousDetection 0 bool" 3fmrv 1 7.000000 2023-04-19T18:21:58.167Z,1681928518.167 [CommandExec](IMPORTANT): Scheduling command #1 of 7 with id=3fmrv 2023-04-19T18:21:58.167Z,1681928518.167 [CommandExec](IMPORTANT): Scheduled #38 (#1 of 7 with id='3fmrv'): "load Science/front_sampling.xml;set FrontSampling.NumSamplers 5 count;set FrontSampling.MissionTimeout 24 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.RepeatTransects 2 count;set FrontSampling.FrontAutonomousDetection 0 bool", ASAP 2023-04-19T18:21:59.683Z,1681928519.683 [DataOverHttps](IMPORTANT): SBD MTMSN=20230419T182158 2023-04-19T18:27:02.457Z,1681928822.457 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.219165 min 2023-04-19T18:27:17.265Z,1681928837.265 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonFrontMinDesignated -121.978 degree;set FrontSampling.LonFrontMaxDesignated -121.977 degree;set FrontSampling.Lat1 36.863 degree;set FrontSampling.Lon1 -121.975 degree;set FrontSampling.Lat2 36.863 degree" 3fmrv 2 7.000000 2023-04-19T18:27:17.266Z,1681928837.266 [CommandExec](IMPORTANT): Scheduling command #2 of 7 with id=3fmrv 2023-04-19T18:27:17.266Z,1681928837.266 [CommandExec](IMPORTANT): Scheduled #39 (#2 of 7 with id='3fmrv'): "set FrontSampling.LonFrontMinDesignated -121.978 degree;set FrontSampling.LonFrontMaxDesignated -121.977 degree;set FrontSampling.Lat1 36.863 degree;set FrontSampling.Lon1 -121.975 degree;set FrontSampling.Lat2 36.863 degree", ASAP 2023-04-19T18:27:18.190Z,1681928838.190 [DataOverHttps](IMPORTANT): SBD MTMSN=20230419T182717 2023-04-19T18:27:25.802Z,1681928845.802 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.Lon2 -122.06 degree;set FrontSampling.LatUpwellingEnd 36.863 degree;set FrontSampling.LonUpwellingEnd -122.06 degree;set FrontSampling.DeltaLatUpwellingSide1 0 degree" 3fmrv 3 7.000000 2023-04-19T18:27:25.802Z,1681928845.802 [CommandExec](IMPORTANT): Scheduling command #3 of 7 with id=3fmrv 2023-04-19T18:27:25.803Z,1681928845.803 [CommandExec](IMPORTANT): Scheduled #40 (#3 of 7 with id='3fmrv'): "set FrontSampling.Lon2 -122.06 degree;set FrontSampling.LatUpwellingEnd 36.863 degree;set FrontSampling.LonUpwellingEnd -122.06 degree;set FrontSampling.DeltaLatUpwellingSide1 0 degree", ASAP 2023-04-19T18:27:26.985Z,1681928846.985 [DataOverHttps](IMPORTANT): SBD MOMSN=18239473, MTMSN=20230419T182725 2023-04-19T18:27:34.133Z,1681928854.133 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonUpwellingSide1 -0.017 degree;set FrontSampling.DeltaLatUpwellingSide2 0 degree;set FrontSampling.DeltaLonUpwellingSide2 -0.034 degree;set FrontSampling.DeltaLatUpwellingSide3 0 degree" 3fmrv 4 7.000000 2023-04-19T18:27:34.133Z,1681928854.133 [CommandExec](IMPORTANT): Scheduling command #4 of 7 with id=3fmrv 2023-04-19T18:27:34.134Z,1681928854.134 [CommandExec](IMPORTANT): Scheduled #41 (#4 of 7 with id='3fmrv'): "set FrontSampling.DeltaLonUpwellingSide1 -0.017 degree;set FrontSampling.DeltaLatUpwellingSide2 0 degree;set FrontSampling.DeltaLonUpwellingSide2 -0.034 degree;set FrontSampling.DeltaLatUpwellingSide3 0 degree", ASAP 2023-04-19T18:27:35.283Z,1681928855.283 [DataOverHttps](IMPORTANT): SBD MTMSN=20230419T182734 2023-04-19T18:27:43.109Z,1681928863.109 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonUpwellingSide3 -0.051 degree;set FrontSampling.DeltaLatUpwellingSide4 0 degree;set FrontSampling.DeltaLonUpwellingSide4 -0.068 degree;set FrontSampling.LatStratifiedEnd 36.863 degree" 3fmrv 5 7.000000 2023-04-19T18:27:43.110Z,1681928863.110 [CommandExec](IMPORTANT): Scheduling command #5 of 7 with id=3fmrv 2023-04-19T18:27:43.111Z,1681928863.111 [CommandExec](IMPORTANT): Scheduled #42 (#5 of 7 with id='3fmrv'): "set FrontSampling.DeltaLonUpwellingSide3 -0.051 degree;set FrontSampling.DeltaLatUpwellingSide4 0 degree;set FrontSampling.DeltaLonUpwellingSide4 -0.068 degree;set FrontSampling.LatStratifiedEnd 36.863 degree", ASAP 2023-04-19T18:27:44.340Z,1681928864.340 [DataOverHttps](IMPORTANT): SBD MOMSN=18239478, MTMSN=20230419T182743 2023-04-19T18:27:51.673Z,1681928871.673 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonStratifiedEnd -121.97 degree;set FrontSampling.SpeedTransit 0.8 m/s;set FrontSampling.SpeedFindChlPeakAndSample 0.8 m/s;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m" 3fmrv 6 7.000000 2023-04-19T18:27:51.674Z,1681928871.674 [CommandExec](IMPORTANT): Scheduling command #6 of 7 with id=3fmrv 2023-04-19T18:27:51.675Z,1681928871.675 [CommandExec](IMPORTANT): Scheduled #43 (#6 of 7 with id='3fmrv'): "set FrontSampling.LonStratifiedEnd -121.97 degree;set FrontSampling.SpeedTransit 0.8 m/s;set FrontSampling.SpeedFindChlPeakAndSample 0.8 m/s;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m", ASAP 2023-04-19T18:27:52.819Z,1681928872.819 [DataOverHttps](IMPORTANT): SBD MTMSN=20230419T182751 2023-04-19T18:28:00.008Z,1681928880.008 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m;set FrontSampling.WaitDepthUndulation 5 min;run" 3fmrv 7 7.000000 2023-04-19T18:28:00.009Z,1681928880.009 [CommandExec](IMPORTANT): Scheduling command #7 of 7 with id=3fmrv 2023-04-19T18:28:00.010Z,1681928880.010 [CommandExec](IMPORTANT): Scheduled #44 (#7 of 7 with id='3fmrv'): "set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m;set FrontSampling.WaitDepthUndulation 5 min;run", ASAP 2023-04-19T18:28:00.133Z,1681928880.133 [CommandExec](IMPORTANT): got command load ./Missions/Science/front_sampling.xml 2023-04-19T18:28:08.030Z,1681928888.030 [CommandExec](IMPORTANT): Loaded ./Missions/Science/front_sampling.xml 2023-04-19T18:28:42.675Z,1681928922.675 [CommandExec](IMPORTANT): got command set FrontSampling.NumSamplers 5 count 2023-04-19T18:28:42.676Z,1681928922.676 [CommandExec](IMPORTANT): got command set FrontSampling.MissionTimeout 24 hour 2023-04-19T18:28:42.676Z,1681928922.676 [CommandExec](IMPORTANT): got command set FrontSampling.MaxDepth 60 meter 2023-04-19T18:28:42.700Z,1681928922.700 [CommandExec](IMPORTANT): got command set FrontSampling.RepeatTransects 2 count 2023-04-19T18:28:42.700Z,1681928922.700 [CommandExec](IMPORTANT): got command set FrontSampling.FrontAutonomousDetection 0 bool 2023-04-19T18:29:12.456Z,1681928952.456 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMinDesignated -121.978 degree 2023-04-19T18:29:12.457Z,1681928952.457 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMaxDesignated -121.977 degree 2023-04-19T18:29:12.458Z,1681928952.458 [CommandExec](IMPORTANT): got command set FrontSampling.Lat1 36.863 degree 2023-04-19T18:29:12.459Z,1681928952.459 [CommandExec](IMPORTANT): got command set FrontSampling.Lon1 -121.975 degree 2023-04-19T18:29:12.459Z,1681928952.459 [CommandExec](IMPORTANT): got command set FrontSampling.Lat2 36.863 degree 2023-04-19T18:29:40.386Z,1681928980.386 [CommandExec](IMPORTANT): got command set FrontSampling.Lon2 -122.06 degree 2023-04-19T18:29:40.386Z,1681928980.386 [CommandExec](IMPORTANT): got command set FrontSampling.LatUpwellingEnd 36.863 degree 2023-04-19T18:29:40.387Z,1681928980.387 [CommandExec](IMPORTANT): got command set FrontSampling.LonUpwellingEnd -122.06 degree 2023-04-19T18:29:40.388Z,1681928980.388 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide1 0 degree 2023-04-19T18:30:18.200Z,1681929018.200 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide1 -0.017 degree 2023-04-19T18:30:18.200Z,1681929018.200 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide2 0 degree 2023-04-19T18:30:18.209Z,1681929018.209 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide2 -0.034 degree 2023-04-19T18:30:18.210Z,1681929018.210 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide3 0 degree 2023-04-19T18:30:54.302Z,1681929054.302 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide3 -0.051 degree 2023-04-19T18:30:54.303Z,1681929054.303 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide4 0 degree 2023-04-19T18:30:54.303Z,1681929054.303 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide4 -0.068 degree 2023-04-19T18:30:54.304Z,1681929054.304 [CommandExec](IMPORTANT): got command set FrontSampling.LatStratifiedEnd 36.863 degree 2023-04-19T18:31:34.166Z,1681929094.166 [CommandExec](IMPORTANT): got command set FrontSampling.LonStratifiedEnd -121.97 degree 2023-04-19T18:31:34.166Z,1681929094.166 [CommandExec](IMPORTANT): got command set FrontSampling.SpeedTransit 0.8 meter_per_second 2023-04-19T18:31:34.167Z,1681929094.167 [CommandExec](IMPORTANT): got command set FrontSampling.SpeedFindChlPeakAndSample 0.8 meter_per_second 2023-04-19T18:31:34.168Z,1681929094.168 [CommandExec](IMPORTANT): got command set FrontSampling.ShallowBoundChl 3 meter 2023-04-19T18:31:34.168Z,1681929094.168 [CommandExec](IMPORTANT): got command set FrontSampling.DeepBoundChl 20 meter 2023-04-19T18:31:55.483Z,1681929115.483 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMinDepth 2 meter 2023-04-19T18:31:55.483Z,1681929115.483 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMaxDepth 50 meter 2023-04-19T18:31:55.484Z,1681929115.484 [CommandExec](IMPORTANT): got command set FrontSampling.WaitDepthUndulation 5 minute 2023-04-19T18:31:55.489Z,1681929115.489 [CommandExec](IMPORTANT): got command run 2023-04-19T18:31:55.492Z,1681929115.492 [CommandExec](IMPORTANT): Running 2023-04-19T18:31:55.534Z,1681929115.534 [MissionManager](IMPORTANT): Started mission FrontSampling 2023-04-19T18:33:33.375Z,1681929213.375 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862999,-121.974998 2023-04-19T19:06:57.556Z,1681931217.556 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Reached waypoint: 36.862999,-121.974998 2023-04-19T19:06:57.558Z,1681931217.558 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt2:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862999,-122.059998 2023-04-19T19:22:54.536Z,1681932174.536 [FrontSampling:CrossFrontFromStratifiedToUpwelling:P](IMPORTANT): Crossed the front from stratified to upwelling. VTHI = 0.253656 degC . VSHI = 0.030282 psu . Front latitude, longitude = 36.862824 arcdeg , -121.977391 arcdeg . Lat1, Lon1 = 36.862824 arcdeg , -121.994391 arcdeg . Lat2, Lon2 = 36.862824 arcdeg , -122.011391 arcdeg . Lat3, Lon3 = 36.862824 arcdeg , -122.028391 arcdeg . Lat4, Lon4 = 36.862824 arcdeg , -122.045391 arcdeg . Lat5, Lon5 = 36.862999 arcdeg , -122.059998 arcdeg . Lat6, Lon6 = 36.862999 arcdeg , -121.970001 arcdeg . Sample count = 1.000000 count . 2023-04-19T19:22:57.762Z,1681932177.762 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862999,-122.059998 2023-04-19T19:22:58.964Z,1681932178.964 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862824,-121.994391 2023-04-19T19:22:59.386Z,1681932179.386 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862999,-122.059998 2023-04-19T19:22:59.799Z,1681932179.799 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862824,-121.994391 2023-04-19T19:23:01.421Z,1681932181.421 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.296591 m 2023-04-19T19:25:51.923Z,1681932351.923 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.057220 m 2023-04-19T19:27:22.022Z,1681932442.022 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 20.342085 m , 1.937200 ug/l , 3.585732 m , 10.946928 ug/l 2023-04-19T19:27:25.658Z,1681932445.658 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Latitude, longitude = 36.862598 arcdeg , -121.977362 arcdeg . Target depth = 3.585732 m . 2023-04-19T19:27:26.046Z,1681932446.046 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-19T19:31:38.978Z,1681932698.978 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 3.586790 m , settling for 5.000000 min 2023-04-19T19:36:51.741Z,1681933011.741 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-19T19:36:51.922Z,1681933011.922 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-19T19:37:42.210Z,1681933062.210 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-19T19:37:43.027Z,1681933063.027 [ESPComponent](IMPORTANT): [sample #9] Setting cartridge argument to -6 2023-04-19T19:37:43.028Z,1681933063.028 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-19T19:37:43.470Z,1681933063.470 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.703156 m in 11.045771 degC water with 10.608200 ug/l chlorophyll fluorescence. 2023-04-19T19:37:43.825Z,1681933063.825 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-19T19:37:44.229Z,1681933064.229 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-19T19:37:46.251Z,1681933066.251 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-19T19:38:18.171Z,1681933098.171 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PRIMING (@34,archive_bac) 2023-04-19T19:38:21.011Z,1681933101.011 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PRIMING (@100.0) 2023-04-19T19:38:21.012Z,1681933101.012 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-19T19:38:21.401Z,1681933101.401 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-19T19:41:04.221Z,1681933264.221 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_FILTERING (@0.0) 2023-04-19T19:41:04.613Z,1681933264.613 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-19T20:32:49.553Z,1681936369.553 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2023-04-19T20:32:50.161Z,1681936370.161 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-19T20:44:32.775Z,1681937072.775 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PAUSED (@1000.0) 2023-04-19T20:44:33.183Z,1681937073.183 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-19T20:44:35.199Z,1681937075.199 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-19T20:44:35.611Z,1681937075.611 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PROCESSED (@34,archive_bac) 2023-04-19T20:56:39.198Z,1681937799.198 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPING (@34,archive_bac) 2023-04-19T20:56:49.283Z,1681937809.283 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-19T20:56:49.719Z,1681937809.719 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPED 2023-04-19T21:01:49.492Z,1681938109.492 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence completed normally. 2023-04-19T21:01:50.283Z,1681938110.283 [ESPComponent](FAULT): LCB 2 fault: LCB Watchdog Reset. Invalid Command. 2023-04-19T21:01:50.300Z,1681938110.300 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed. 2023-04-19T22:02:39.705Z,1681941759.705 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-19T22:02:51.065Z,1681941771.065 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003063 CHAN A1 (24V): -0.000893 CHAN A2 (12V): -0.000854 CHAN A3 (5V): -0.000634 CHAN B0 (3.3V): 0.000024 CHAN B1 (3.15aV): 0.000668 CHAN B2 (3.15bV): 0.000659 CHAN B3 (GND): -0.000192 OPEN: 0.007552 Full Scale: +/- 1 mA 2023-04-19T22:19:31.759Z,1681942771.759 [FrontSampling:NeedComms:C:A_Timeout:A](IMPORTANT): first GPS update timeout 2023-04-19T22:22:34.412Z,1681942954.412 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-04-19T22:29:18.062Z,1681943358.062 [FrontSampling:NeedComms:E:A_Timeout:A](IMPORTANT): second GPS update timeout 2023-04-19T22:55:29.597Z,1681944929.597 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt1:A.Waypoint](IMPORTANT): Reached waypoint: 36.862824,-121.994391 2023-04-19T23:04:42.644Z,1681945482.644 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.242568 m 2023-04-19T23:08:56.777Z,1681945736.777 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.099571 m 2023-04-19T23:10:51.097Z,1681945851.097 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 20.346840 m , 1.438400 ug/l , 11.280664 m , 5.163654 ug/l 2023-04-19T23:10:53.526Z,1681945853.526 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 2.000000 count . Latitude, longitude = 36.862307 arcdeg , -121.995863 arcdeg . Target depth = 11.280664 m . 2023-04-19T23:10:53.944Z,1681945853.944 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-19T23:14:08.683Z,1681946048.683 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 11.247005 m , settling for 5.000000 min 2023-04-19T23:19:21.521Z,1681946361.521 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-19T23:19:21.662Z,1681946361.662 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-19T23:20:14.790Z,1681946414.790 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-19T23:20:15.602Z,1681946415.602 [ESPComponent](IMPORTANT): [sample #10] Setting cartridge argument to -6 2023-04-19T23:20:15.602Z,1681946415.602 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-19T23:20:16.036Z,1681946416.036 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 10.975609 m in 9.977991 degC water with 5.260600 ug/l chlorophyll fluorescence. 2023-04-19T23:20:16.407Z,1681946416.407 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-19T23:20:16.811Z,1681946416.811 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-19T23:20:18.830Z,1681946418.830 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-19T23:20:50.756Z,1681946450.756 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PRIMING (@33,archive_bac) 2023-04-19T23:20:53.574Z,1681946453.574 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PRIMING (@100.0) 2023-04-19T23:20:53.575Z,1681946453.575 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-19T23:20:53.978Z,1681946453.978 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-19T23:23:35.991Z,1681946615.991 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_FILTERING (@0.0) 2023-04-19T23:23:36.403Z,1681946616.403 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-20T00:14:42.384Z,1681949682.384 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PAUSED (@1000.0) 2023-04-20T00:14:42.785Z,1681949682.785 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-20T00:14:44.807Z,1681949684.807 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-20T00:14:45.210Z,1681949685.210 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PROCESSED (@33,archive_bac) 2023-04-20T00:26:48.786Z,1681950408.786 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPING (@33,archive_bac) 2023-04-20T00:26:58.968Z,1681950418.968 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-20T00:26:59.487Z,1681950419.487 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPED 2023-04-20T00:31:59.187Z,1681950719.187 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence completed normally. 2023-04-20T00:32:00.103Z,1681950720.103 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 2.000000 count completed. 2023-04-20T00:32:04.546Z,1681950724.546 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt2:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862824,-122.011391 2023-04-20T00:41:50.905Z,1681951310.905 [BuoyancyServo](FAULT): Invalid EZ Servo response:"" 2023-04-20T00:41:50.906Z,1681951310.906 [BuoyancyServo](FAULT): Buoyancy engine reporting null position 2023-04-20T01:25:46.439Z,1681953946.439 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt2:A.Waypoint](IMPORTANT): Reached waypoint: 36.862824,-122.011391 2023-04-20T01:36:31.012Z,1681954591.012 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.325547 m 2023-04-20T01:40:27.781Z,1681954827.781 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.122044 m 2023-04-20T01:42:30.593Z,1681954950.593 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 20.336472 m , 3.456800 ug/l , 14.256640 m , 3.406782 ug/l 2023-04-20T01:42:33.017Z,1681954953.017 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 3.000000 count . Latitude, longitude = 36.861414 arcdeg , -122.013119 arcdeg . Target depth = 14.256640 m . 2023-04-20T01:42:33.409Z,1681954953.409 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-20T01:45:19.486Z,1681955119.486 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 14.229769 m , settling for 5.000000 min 2023-04-20T01:50:32.257Z,1681955432.257 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-20T01:50:32.433Z,1681955432.433 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-20T01:51:25.078Z,1681955485.078 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-20T01:51:25.898Z,1681955485.898 [ESPComponent](IMPORTANT): [sample #11] Setting cartridge argument to -6 2023-04-20T01:51:25.898Z,1681955485.898 [ESPComponent](IMPORTANT): [sample #11] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-20T01:51:26.341Z,1681955486.341 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 13.409967 m in 9.680567 degC water with 3.521760 ug/l chlorophyll fluorescence. 2023-04-20T01:51:26.722Z,1681955486.722 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-20T01:51:27.108Z,1681955487.108 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-20T01:51:29.119Z,1681955489.119 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-20T01:52:01.035Z,1681955521.035 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PRIMING (@32,archive_bac) 2023-04-20T01:52:03.863Z,1681955523.863 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PRIMING (@100.0) 2023-04-20T01:52:03.865Z,1681955523.865 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-20T01:52:04.275Z,1681955524.275 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-20T01:54:46.282Z,1681955686.282 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_FILTERING (@0.0) 2023-04-20T01:54:46.692Z,1681955686.692 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-20T02:02:32.905Z,1681956152.905 [DVL_micro](FAULT): Failed to initialize within timeout. 2023-04-20T02:47:39.317Z,1681958859.317 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PAUSED (@1000.0) 2023-04-20T02:47:39.744Z,1681958859.744 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-20T02:47:41.757Z,1681958861.757 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-20T02:47:42.152Z,1681958862.152 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PROCESSED (@32,archive_bac) 2023-04-20T02:59:45.738Z,1681959585.738 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_STOPPING (@32,archive_bac) 2023-04-20T02:59:55.819Z,1681959595.819 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-20T02:59:56.228Z,1681959596.228 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_STOPPED 2023-04-20T03:04:55.995Z,1681959895.995 [ESPComponent](IMPORTANT): [sample #11] ESP sampling sequence completed normally. 2023-04-20T03:04:56.843Z,1681959896.843 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 3.000000 count completed. 2023-04-20T03:05:01.284Z,1681959901.284 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt3:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862824,-122.028391 2023-04-20T03:39:40.584Z,1681961980.584 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt3:A.Waypoint](IMPORTANT): Reached waypoint: 36.862824,-122.028391 2023-04-20T03:46:28.361Z,1681962388.361 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.378706 m 2023-04-20T03:50:13.018Z,1681962613.018 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.180813 m 2023-04-20T03:52:19.486Z,1681962739.486 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 20.292389 m , 2.111200 ug/l , 3.561691 m , 2.526332 ug/l 2023-04-20T03:52:21.918Z,1681962741.918 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 4.000000 count . Latitude, longitude = 36.862869 arcdeg , -122.030849 arcdeg . Target depth = 3.561691 m . 2023-04-20T03:52:22.297Z,1681962742.297 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-20T03:56:26.742Z,1681962986.742 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 3.563885 m , settling for 5.000000 min 2023-04-20T04:01:39.525Z,1681963299.525 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-20T04:01:39.686Z,1681963299.686 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-20T04:02:32.060Z,1681963352.060 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-20T04:02:32.861Z,1681963352.861 [ESPComponent](IMPORTANT): [sample #12] Setting cartridge argument to -6 2023-04-20T04:02:32.862Z,1681963352.862 [ESPComponent](IMPORTANT): [sample #12] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-20T04:02:33.302Z,1681963353.302 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.212523 m in 9.558374 degC water with 2.605360 ug/l chlorophyll fluorescence. 2023-04-20T04:02:33.671Z,1681963353.671 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-20T04:02:34.078Z,1681963354.078 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-20T04:02:36.103Z,1681963356.103 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-20T04:02:51.176Z,1681963371.176 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-20T04:03:02.080Z,1681963382.080 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.002220 CHAN A1 (24V): -0.000997 CHAN A2 (12V): -0.000299 CHAN A3 (5V): -0.000528 CHAN B0 (3.3V): -0.000462 CHAN B1 (3.15aV): 0.003411 CHAN B2 (3.15bV): 0.000450 CHAN B3 (GND): 0.000275 OPEN: 0.007610 Full Scale: +/- 1 mA 2023-04-20T04:03:08.058Z,1681963388.058 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_WAITING_FOR_PRIMING (@31,archive_bac) 2023-04-20T04:03:10.884Z,1681963390.884 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_PRIMING (@100.0) 2023-04-20T04:03:10.885Z,1681963390.885 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-20T04:03:11.288Z,1681963391.288 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-20T04:05:55.343Z,1681963555.343 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_FILTERING (@0.0) 2023-04-20T04:05:55.718Z,1681963555.718 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-20T04:47:31.228Z,1681966051.228 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-20T05:00:21.707Z,1681966821.707 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_PAUSED (@1000.0) 2023-04-20T05:00:22.131Z,1681966822.131 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-20T05:00:24.147Z,1681966824.147 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-20T05:00:24.539Z,1681966824.539 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_WAITING_FOR_PROCESSED (@31,archive_bac) 2023-04-20T05:12:28.106Z,1681967548.106 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_STOPPING (@31,archive_bac) 2023-04-20T05:12:38.204Z,1681967558.204 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-20T05:12:38.611Z,1681967558.611 [ESPComponent](IMPORTANT): [sample #12] ESP sampling state: S_STOPPED 2023-04-20T05:17:38.401Z,1681967858.401 [ESPComponent](IMPORTANT): [sample #12] ESP sampling sequence completed normally. 2023-04-20T05:17:39.250Z,1681967859.250 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 4.000000 count completed. 2023-04-20T05:17:43.674Z,1681967863.674 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt4:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.862824,-122.045391 2023-04-20T05:41:30.561Z,1681969290.561 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt4:A.Waypoint](IMPORTANT): Reached waypoint: 36.862824,-122.045391 2023-04-20T05:45:52.181Z,1681969552.181 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.270660 m 2023-04-20T05:49:57.343Z,1681969797.343 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.179085 m 2023-04-20T05:51:50.044Z,1681969910.044 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 20.298006 m , 2.134400 ug/l , 4.130892 m , 2.899276 ug/l 2023-04-20T05:51:52.483Z,1681969912.483 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 5.000000 count . Latitude, longitude = 36.862880 arcdeg , -122.047376 arcdeg . Target depth = 4.130892 m . 2023-04-20T05:51:52.870Z,1681969912.870 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-20T05:53:04.793Z,1681969984.793 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 4.014646 m , settling for 5.000000 min 2023-04-20T05:58:17.565Z,1681970297.565 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-20T05:58:17.726Z,1681970297.726 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-20T05:59:09.986Z,1681970349.986 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-20T05:59:10.790Z,1681970350.790 [ESPComponent](IMPORTANT): [sample #13] Setting cartridge argument to -6 2023-04-20T05:59:10.790Z,1681970350.790 [ESPComponent](IMPORTANT): [sample #13] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-20T05:59:11.243Z,1681970351.243 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.891908 m in 9.560572 degC water with 2.690040 ug/l chlorophyll fluorescence. 2023-04-20T05:59:11.603Z,1681970351.603 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-20T05:59:12.017Z,1681970352.017 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-20T05:59:14.032Z,1681970354.032 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-20T05:59:45.951Z,1681970385.951 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PRIMING (@30,archive_bac) 2023-04-20T05:59:48.763Z,1681970388.763 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PRIMING (@100.0) 2023-04-20T05:59:48.764Z,1681970388.764 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-20T05:59:49.167Z,1681970389.167 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-20T06:02:32.786Z,1681970552.786 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_FILTERING (@0.0) 2023-04-20T06:02:33.190Z,1681970553.190 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-20T06:58:03.794Z,1681973883.794 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PAUSED (@1000.0) 2023-04-20T06:58:04.201Z,1681973884.201 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-20T06:58:06.221Z,1681973886.221 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-20T06:58:06.623Z,1681973886.623 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PROCESSED (@30,archive_bac) 2023-04-20T07:08:05.546Z,1681974485.546 [ThrusterHE](FAULT): Could not get status:serial timeout 2023-04-20T07:10:10.204Z,1681974610.204 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_STOPPING (@30,archive_bac) 2023-04-20T07:10:20.302Z,1681974620.302 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-20T07:10:20.715Z,1681974620.715 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_STOPPED 2023-04-20T07:15:20.474Z,1681974920.474 [ESPComponent](IMPORTANT): [sample #13] ESP sampling sequence completed normally. 2023-04-20T07:15:21.305Z,1681974921.305 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 5.000000 count completed. 2023-04-20T07:19:13.178Z,1681975153.178 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16040, MT Status=1, MTMSN=1188 2023-04-20T07:19:13.744Z,1681975153.744 [CommandExec](IMPORTANT): got command stop 2023-04-20T07:19:13.744Z,1681975153.744 [CommandExec](IMPORTANT): Scheduling is paused 2023-04-20T07:19:13.745Z,1681975153.745 [CommandExec](IMPORTANT): got command schedule clear 2023-04-20T07:19:13.745Z,1681975153.745 [CommandExec](IMPORTANT): Cleared 0 scheduled commands. 2023-04-20T07:19:13.745Z,1681975153.745 [CommandExec](IMPORTANT): got command schedule resume 2023-04-20T07:19:13.745Z,1681975153.745 [CommandExec](IMPORTANT): Scheduling is resumed 2023-04-20T07:19:14.213Z,1681975154.213 [MissionManager](IMPORTANT): Started mission Default 2023-04-20T07:26:01.161Z,1681975561.161 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 6.775807 min 2023-04-20T07:26:11.445Z,1681975571.445 [DataOverHttps](IMPORTANT): SBD MOMSN=18241123, MTMSN=20230420T072611 2023-04-20T07:26:19.019Z,1681975579.019 [CommandExec](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2023-04-20T07:26:20.855Z,1681975580.855 [CommandExec](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2023-04-20T07:26:43.117Z,1681975603.117 [CommandExec](IMPORTANT): got command set profile_station.MissionTimeout 12 hour 2023-04-20T07:26:43.118Z,1681975603.118 [CommandExec](IMPORTANT): got command set profile_station.YoYoMaxDepth 30 meter 2023-04-20T07:26:43.119Z,1681975603.119 [CommandExec](IMPORTANT): got command set profile_station.MaxDepth 60 meter 2023-04-20T07:26:43.119Z,1681975603.119 [CommandExec](IMPORTANT): got command run 2023-04-20T07:26:43.122Z,1681975603.122 [CommandExec](IMPORTANT): Running 2023-04-20T07:26:43.239Z,1681975603.239 [MissionManager](IMPORTANT): Started mission profile_station 2023-04-20T07:31:05.784Z,1681975865.784 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-20T07:35:36.194Z,1681976136.194 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2023-04-20T07:37:05.787Z,1681976225.787 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-20T08:35:15.164Z,1681979715.164 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16041, MT Status=1, MTMSN=1189 2023-04-20T08:35:15.942Z,1681979715.942 [CommandExec](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2023-04-20T08:35:17.935Z,1681979717.935 [CommandExec](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2023-04-20T08:35:18.014Z,1681979718.014 [MissionManager](IMPORTANT): Started mission Default 2023-04-20T08:35:32.353Z,1681979732.353 [CommandExec](IMPORTANT): got command set profile_station.MissionTimeout 12 hour 2023-04-20T08:35:32.354Z,1681979732.354 [CommandExec](IMPORTANT): got command set profile_station.YoYoMaxDepth 30 meter 2023-04-20T08:35:32.354Z,1681979732.354 [CommandExec](IMPORTANT): got command set profile_station.MaxDepth 60 meter 2023-04-20T08:35:32.355Z,1681979732.355 [CommandExec](IMPORTANT): got command run 2023-04-20T08:35:32.357Z,1681979732.357 [CommandExec](IMPORTANT): Running 2023-04-20T08:35:32.549Z,1681979732.549 [MissionManager](IMPORTANT): Started mission profile_station 2023-04-20T10:03:02.316Z,1681984982.316 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-20T10:03:13.208Z,1681984993.208 [CBIT](IMPORTANT): Ground fault detected mA: CHAN A0 (Batt): -0.002853 CHAN A1 (24V): -0.000956 CHAN A2 (12V): -0.000201 CHAN A3 (5V): -0.000013 CHAN B0 (3.3V): -0.000023 CHAN B1 (3.15aV): -0.000146 CHAN B2 (3.15bV): 0.000032 CHAN B3 (GND): 0.075440 OPEN: 0.007652 Full Scale: +/- 1 mA 2023-04-20T10:45:55.346Z,1681987555.346 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-20T11:35:39.089Z,1681990539.089 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2023-04-20T11:52:10.620Z,1681991530.620 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-20T12:09:27.925Z,1681992567.925 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-20T12:44:39.085Z,1681994679.085 [ThrusterHE](FAULT): Could not get status:serial timeout 2023-04-20T14:34:23.937Z,1682001263.937 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-20T16:03:13.352Z,1682006593.352 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-20T16:03:24.376Z,1682006604.376 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.002715 CHAN A1 (24V): -0.000952 CHAN A2 (12V): -0.000975 CHAN A3 (5V): -0.000766 CHAN B0 (3.3V): 0.000764 CHAN B1 (3.15aV): -0.000259 CHAN B2 (3.15bV): -0.001159 CHAN B3 (GND): 0.000534 OPEN: 0.007316 Full Scale: +/- 1 mA 2023-04-20T16:12:34.896Z,1682007154.896 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16055, MT Status=1, MTMSN=1190 2023-04-20T16:12:35.636Z,1682007155.636 [CommandExec](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2023-04-20T16:12:37.591Z,1682007157.591 [CommandExec](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2023-04-20T16:12:37.755Z,1682007157.755 [MissionManager](IMPORTANT): Started mission Default 2023-04-20T16:12:58.244Z,1682007178.244 [CommandExec](IMPORTANT): got command set profile_station.MissionTimeout 12 hour 2023-04-20T16:12:58.244Z,1682007178.244 [CommandExec](IMPORTANT): got command set profile_station.NeedCommsTime 20 minute 2023-04-20T16:12:58.245Z,1682007178.245 [CommandExec](IMPORTANT): got command set profile_station.YoYoMaxDepth 20 meter 2023-04-20T16:12:58.246Z,1682007178.246 [CommandExec](IMPORTANT): got command set profile_station.MaxDepth 60 meter 2023-04-20T16:12:58.247Z,1682007178.247 [CommandExec](IMPORTANT): got command run 2023-04-20T16:12:58.248Z,1682007178.248 [CommandExec](IMPORTANT): Running 2023-04-20T16:12:58.336Z,1682007178.336 [MissionManager](IMPORTANT): Started mission profile_station 2023-04-20T16:37:46.077Z,1682008666.077 [BuoyancyServo](FAULT): Buoyancy getPosition uart error.serial timeout 2023-04-20T17:13:48.306Z,1682010828.306 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-20T18:29:12.076Z,1682015352.076 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16066, MT Status=1, MTMSN=1191 2023-04-20T18:29:12.626Z,1682015352.626 [CommandExec](IMPORTANT): got command stop 2023-04-20T18:29:12.626Z,1682015352.626 [CommandExec](IMPORTANT): Scheduling is paused 2023-04-20T18:29:13.315Z,1682015353.315 [MissionManager](IMPORTANT): Started mission Default 2023-04-20T18:35:39.336Z,1682015739.336 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 6.426744 min 2023-04-20T18:41:24.743Z,1682016084.743 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 12.183827 min 2023-04-20T18:47:03.725Z,1682016423.725 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 17.833435 min 2023-04-20T18:52:35.013Z,1682016755.013 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 23.355048 min 2023-04-20T18:58:17.626Z,1682017097.626 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 29.065175 min 2023-04-20T19:03:57.006Z,1682017437.006 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 34.721228 min 2023-04-20T19:06:43.050Z,1682017603.050 [DataOverHttps](IMPORTANT): SBD MTMSN=20230420T190643