2023-04-22T16:03:39.337Z,1682179419.337 [CommandExec](IMPORTANT): got command restart logs 2023-04-22T17:42:33.270Z,1682185353.270 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-22T18:18:10.348Z,1682187490.348 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-22T19:51:48.237Z,1682193108.237 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-22T19:51:59.090Z,1682193119.090 [CBIT](IMPORTANT): Ground fault detected mA: CHAN A0 (Batt): -0.002788 CHAN A1 (24V): -0.001050 CHAN A2 (12V): 0.014977 CHAN A3 (5V): -0.000763 CHAN B0 (3.3V): 0.000168 CHAN B1 (3.15aV): 0.085438 CHAN B2 (3.15bV): -0.002321 CHAN B3 (GND): 0.000231 OPEN: 0.007887 Full Scale: +/- 1 mA 2023-04-22T23:48:42.547Z,1682207322.547 [ThrusterHE](FAULT): Could not get status:serial timeout 2023-04-23T00:54:01.454Z,1682211241.454 [MissionManager](IMPORTANT): Started mission Default 2023-04-23T01:02:17.283Z,1682211737.283 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 8.256778 min 2023-04-23T01:02:45.216Z,1682211765.216 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16415, MT Status=1, MTMSN=1202 2023-04-23T01:02:45.992Z,1682211765.992 [CommandExec](IMPORTANT): got command schedule asap "load Science/front_sampling.xml;set FrontSampling.NumSamplers 5 count;set FrontSampling.MissionTimeout 36 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.RepeatTransects 2 count;set FrontSampling.FrontAutonomousDetection 0 bool" 3lpgk 1 7.000000 2023-04-23T01:02:45.993Z,1682211765.993 [CommandExec](IMPORTANT): Scheduling command #1 of 7 with id=3lpgk 2023-04-23T01:02:45.994Z,1682211765.994 [CommandExec](IMPORTANT): Scheduled #8 (#1 of 7 with id='3lpgk'): "load Science/front_sampling.xml;set FrontSampling.NumSamplers 5 count;set FrontSampling.MissionTimeout 36 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.RepeatTransects 2 count;set FrontSampling.FrontAutonomousDetection 0 bool", ASAP 2023-04-23T01:05:22.545Z,1682211922.545 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16416, MT Status=1, MTMSN=1203 2023-04-23T01:05:23.341Z,1682211923.341 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonFrontMinDesignated -122.201 degree;set FrontSampling.LonFrontMaxDesignated -122.2 degree;set FrontSampling.Lat1 36.7 degree;set FrontSampling.Lon1 -122.198 degree;set FrontSampling.Lat2 36.7 degree" 3lpgk 2 7.000000 2023-04-23T01:05:23.341Z,1682211923.341 [CommandExec](IMPORTANT): Scheduling command #2 of 7 with id=3lpgk 2023-04-23T01:05:23.342Z,1682211923.342 [CommandExec](IMPORTANT): Scheduled #9 (#2 of 7 with id='3lpgk'): "set FrontSampling.LonFrontMinDesignated -122.201 degree;set FrontSampling.LonFrontMaxDesignated -122.2 degree;set FrontSampling.Lat1 36.7 degree;set FrontSampling.Lon1 -122.198 degree;set FrontSampling.Lat2 36.7 degree", ASAP 2023-04-23T01:06:31.929Z,1682211991.929 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=16417, MT Status=1, MTMSN=1204 2023-04-23T01:06:32.629Z,1682211992.629 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.Lon2 -122.65 degree;set FrontSampling.LatUpwellingEnd 36.7 degree;set FrontSampling.LonUpwellingEnd -122.65 degree;set FrontSampling.DeltaLatUpwellingSide1 0 degree" 3lpgk 3 7.000000 2023-04-23T01:06:32.630Z,1682211992.630 [CommandExec](IMPORTANT): Scheduling command #3 of 7 with id=3lpgk 2023-04-23T01:06:32.631Z,1682211992.631 [CommandExec](IMPORTANT): Scheduled #10 (#3 of 7 with id='3lpgk'): "set FrontSampling.Lon2 -122.65 degree;set FrontSampling.LatUpwellingEnd 36.7 degree;set FrontSampling.LonUpwellingEnd -122.65 degree;set FrontSampling.DeltaLatUpwellingSide1 0 degree", ASAP 2023-04-23T01:06:50.569Z,1682212010.569 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=16418, MT Status=1, MTMSN=1205 2023-04-23T01:06:51.385Z,1682212011.385 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonUpwellingSide1 -0.11 degree;set FrontSampling.DeltaLatUpwellingSide2 0 degree;set FrontSampling.DeltaLonUpwellingSide2 -0.22 degree;set FrontSampling.DeltaLatUpwellingSide3 0 degree" 3lpgk 4 7.000000 2023-04-23T01:06:51.385Z,1682212011.385 [CommandExec](IMPORTANT): Scheduling command #4 of 7 with id=3lpgk 2023-04-23T01:06:51.386Z,1682212011.386 [CommandExec](IMPORTANT): Scheduled #11 (#4 of 7 with id='3lpgk'): "set FrontSampling.DeltaLonUpwellingSide1 -0.11 degree;set FrontSampling.DeltaLatUpwellingSide2 0 degree;set FrontSampling.DeltaLonUpwellingSide2 -0.22 degree;set FrontSampling.DeltaLatUpwellingSide3 0 degree", ASAP 2023-04-23T01:07:23.893Z,1682212043.893 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=16419, MT Status=1, MTMSN=1206 2023-04-23T01:07:24.635Z,1682212044.635 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonUpwellingSide3 -0.33 degree;set FrontSampling.DeltaLatUpwellingSide4 0 degree;set FrontSampling.DeltaLonUpwellingSide4 -0.44 degree;set FrontSampling.LatStratifiedEnd 36.7 degree" 3lpgk 5 7.000000 2023-04-23T01:07:24.637Z,1682212044.637 [CommandExec](IMPORTANT): Scheduling command #5 of 7 with id=3lpgk 2023-04-23T01:07:24.639Z,1682212044.639 [CommandExec](IMPORTANT): Scheduled #12 (#5 of 7 with id='3lpgk'): "set FrontSampling.DeltaLonUpwellingSide3 -0.33 degree;set FrontSampling.DeltaLatUpwellingSide4 0 degree;set FrontSampling.DeltaLonUpwellingSide4 -0.44 degree;set FrontSampling.LatStratifiedEnd 36.7 degree", ASAP 2023-04-23T01:07:49.517Z,1682212069.517 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=16420, MT Status=1, MTMSN=1207 2023-04-23T01:07:50.253Z,1682212070.253 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonStratifiedEnd -122.198 degree;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 25 m;set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m" 3lpgk 6 7.000000 2023-04-23T01:07:50.254Z,1682212070.254 [CommandExec](IMPORTANT): Scheduling command #6 of 7 with id=3lpgk 2023-04-23T01:07:50.255Z,1682212070.255 [CommandExec](IMPORTANT): Scheduled #13 (#6 of 7 with id='3lpgk'): "set FrontSampling.LonStratifiedEnd -122.198 degree;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 25 m;set FrontSampling.YoYoMinDepth 2 m;set FrontSampling.YoYoMaxDepth 50 m", ASAP 2023-04-23T01:08:17.051Z,1682212097.051 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=16421, MT Status=1, MTMSN=1208 2023-04-23T01:08:17.796Z,1682212097.796 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.WaitDepthUndulation 5 min;set FrontSampling.RiseFromTurningDepth 4 m;set FrontSampling.DepChangeThreshAttitudeFlip 3 m;run" 3lpgk 7 7.000000 2023-04-23T01:08:17.797Z,1682212097.797 [CommandExec](IMPORTANT): Scheduling command #7 of 7 with id=3lpgk 2023-04-23T01:08:17.798Z,1682212097.798 [CommandExec](IMPORTANT): Scheduled #14 (#7 of 7 with id='3lpgk'): "set FrontSampling.WaitDepthUndulation 5 min;set FrontSampling.RiseFromTurningDepth 4 m;set FrontSampling.DepChangeThreshAttitudeFlip 3 m;run", ASAP 2023-04-23T01:08:17.946Z,1682212097.946 [CommandExec](IMPORTANT): got command load ./Missions/Science/front_sampling.xml 2023-04-23T01:08:23.912Z,1682212103.912 [CommandExec](IMPORTANT): Loaded ./Missions/Science/front_sampling.xml 2023-04-23T01:08:41.882Z,1682212121.882 [CommandExec](IMPORTANT): got command set FrontSampling.NumSamplers 5 count 2023-04-23T01:08:41.883Z,1682212121.883 [CommandExec](IMPORTANT): got command set FrontSampling.MissionTimeout 36 hour 2023-04-23T01:08:41.883Z,1682212121.883 [CommandExec](IMPORTANT): got command set FrontSampling.MaxDepth 60 meter 2023-04-23T01:08:41.897Z,1682212121.897 [CommandExec](IMPORTANT): got command set FrontSampling.RepeatTransects 2 count 2023-04-23T01:08:41.897Z,1682212121.897 [CommandExec](IMPORTANT): got command set FrontSampling.FrontAutonomousDetection 0 bool 2023-04-23T01:08:57.903Z,1682212137.903 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMinDesignated -122.201 degree 2023-04-23T01:08:57.904Z,1682212137.904 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMaxDesignated -122.2 degree 2023-04-23T01:08:57.904Z,1682212137.904 [CommandExec](IMPORTANT): got command set FrontSampling.Lat1 36.7 degree 2023-04-23T01:08:57.905Z,1682212137.905 [CommandExec](IMPORTANT): got command set FrontSampling.Lon1 -122.198 degree 2023-04-23T01:08:57.906Z,1682212137.906 [CommandExec](IMPORTANT): got command set FrontSampling.Lat2 36.7 degree 2023-04-23T01:09:11.413Z,1682212151.413 [CommandExec](IMPORTANT): got command set FrontSampling.Lon2 -122.65 degree 2023-04-23T01:09:11.414Z,1682212151.414 [CommandExec](IMPORTANT): got command set FrontSampling.LatUpwellingEnd 36.7 degree 2023-04-23T01:09:11.415Z,1682212151.415 [CommandExec](IMPORTANT): got command set FrontSampling.LonUpwellingEnd -122.65 degree 2023-04-23T01:09:11.415Z,1682212151.415 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide1 0 degree 2023-04-23T01:09:28.810Z,1682212168.810 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide1 -0.11 degree 2023-04-23T01:09:28.811Z,1682212168.811 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide2 0 degree 2023-04-23T01:09:28.811Z,1682212168.811 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide2 -0.22 degree 2023-04-23T01:09:28.812Z,1682212168.812 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide3 0 degree 2023-04-23T01:09:46.313Z,1682212186.313 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide3 -0.33 degree 2023-04-23T01:09:46.314Z,1682212186.314 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide4 0 degree 2023-04-23T01:09:46.314Z,1682212186.314 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide4 -0.44 degree 2023-04-23T01:09:46.315Z,1682212186.315 [CommandExec](IMPORTANT): got command set FrontSampling.LatStratifiedEnd 36.7 degree 2023-04-23T01:10:02.610Z,1682212202.610 [CommandExec](IMPORTANT): got command set FrontSampling.LonStratifiedEnd -122.198 degree 2023-04-23T01:10:02.611Z,1682212202.611 [CommandExec](IMPORTANT): got command set FrontSampling.ShallowBoundChl 3 meter 2023-04-23T01:10:02.612Z,1682212202.612 [CommandExec](IMPORTANT): got command set FrontSampling.DeepBoundChl 25 meter 2023-04-23T01:10:02.612Z,1682212202.612 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMinDepth 2 meter 2023-04-23T01:10:02.613Z,1682212202.613 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMaxDepth 50 meter 2023-04-23T01:10:15.992Z,1682212215.992 [CommandExec](IMPORTANT): got command set FrontSampling.WaitDepthUndulation 5 minute 2023-04-23T01:10:15.993Z,1682212215.993 [CommandExec](IMPORTANT): got command set FrontSampling.RiseFromTurningDepth 4 meter 2023-04-23T01:10:15.993Z,1682212215.993 [CommandExec](IMPORTANT): got command set FrontSampling.DepChangeThreshAttitudeFlip 3 meter 2023-04-23T01:10:15.994Z,1682212215.994 [CommandExec](IMPORTANT): got command run 2023-04-23T01:10:15.996Z,1682212215.996 [CommandExec](IMPORTANT): Running 2023-04-23T01:10:16.278Z,1682212216.278 [MissionManager](IMPORTANT): Started mission FrontSampling 2023-04-23T01:14:24.459Z,1682212464.459 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.700001,-122.197998 2023-04-23T01:51:59.396Z,1682214719.396 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-23T01:52:10.287Z,1682214730.287 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003129 CHAN A1 (24V): -0.000232 CHAN A2 (12V): -0.000760 CHAN A3 (5V): -0.000384 CHAN B0 (3.3V): 0.004021 CHAN B1 (3.15aV): 0.000085 CHAN B2 (3.15bV): -0.000165 CHAN B3 (GND): 0.000402 OPEN: 0.007839 Full Scale: +/- 1 mA 2023-04-23T02:08:26.396Z,1682215706.396 [RudderServo](FAULT): Invalid EZ Servo response:"" 2023-04-23T02:19:59.963Z,1682216399.963 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Reached waypoint: 36.700001,-122.197998 2023-04-23T02:19:59.966Z,1682216399.966 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt2:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.700001,-122.650002 2023-04-23T02:54:42.326Z,1682218482.326 [FrontSampling:CrossFrontFromStratifiedToUpwelling:P](IMPORTANT): Crossed the front from stratified to upwelling. VTHI = 0.203271 degC . VSHI = 0.004819 psu . Front latitude, longitude = 36.699099 arcdeg , -122.197622 arcdeg . Lat1, Lon1 = 36.699099 arcdeg , -122.307622 arcdeg . Lat2, Lon2 = 36.699099 arcdeg , -122.417622 arcdeg . Lat3, Lon3 = 36.699099 arcdeg , -122.527622 arcdeg . Lat4, Lon4 = 36.699099 arcdeg , -122.637622 arcdeg . Lat5, Lon5 = 36.700001 arcdeg , -122.650002 arcdeg . Lat6, Lon6 = 36.700001 arcdeg , -122.197998 arcdeg . Sample count = 1.000000 count . 2023-04-23T02:54:45.532Z,1682218485.532 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.700001,-122.650002 2023-04-23T02:54:46.734Z,1682218486.734 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.699099,-122.307622 2023-04-23T02:54:47.148Z,1682218487.148 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.700001,-122.650002 2023-04-23T02:54:47.563Z,1682218487.563 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.699099,-122.307622 2023-04-23T02:54:49.169Z,1682218489.169 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.186819 m 2023-04-23T02:56:51.190Z,1682218611.190 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 28.239754 m 2023-04-23T02:57:46.191Z,1682218666.191 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 23.714844 m , 0.440800 ug/l , 10.058505 m , 0.508661 ug/l 2023-04-23T02:57:49.779Z,1682218669.779 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Latitude, longitude = 36.698895 arcdeg , -122.197980 arcdeg . Target depth = 10.058505 m . 2023-04-23T02:57:50.206Z,1682218670.206 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-23T03:01:49.368Z,1682218909.368 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 10.019374 m , settling for 5.000000 min 2023-04-23T03:07:02.144Z,1682219222.144 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-23T03:07:02.353Z,1682219222.353 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-23T03:07:55.083Z,1682219275.083 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-23T03:07:55.903Z,1682219275.903 [ESPComponent](IMPORTANT): [sample #6] Setting cartridge argument to -6 2023-04-23T03:07:55.903Z,1682219275.903 [ESPComponent](IMPORTANT): [sample #6] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-23T03:07:56.343Z,1682219276.343 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 9.459280 m in 9.818415 degC water with 0.497640 ug/l chlorophyll fluorescence. 2023-04-23T03:07:56.694Z,1682219276.694 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-23T03:07:57.117Z,1682219277.117 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-23T03:07:59.132Z,1682219279.132 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-23T03:08:31.045Z,1682219311.045 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PRIMING (@24,archive_bac) 2023-04-23T03:08:33.859Z,1682219313.859 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PRIMING (@100.0) 2023-04-23T03:08:33.860Z,1682219313.860 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-23T03:08:34.262Z,1682219314.262 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-23T03:11:16.285Z,1682219476.285 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_FILTERING (@0.0) 2023-04-23T03:11:16.679Z,1682219476.679 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-23T04:02:00.872Z,1682222520.872 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PAUSED (@1000.0) 2023-04-23T04:02:01.283Z,1682222521.283 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-23T04:02:03.300Z,1682222523.300 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-23T04:02:03.702Z,1682222523.702 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PROCESSED (@24,archive_bac) 2023-04-23T04:14:07.299Z,1682223247.299 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_STOPPING (@24,archive_bac) 2023-04-23T04:14:17.397Z,1682223257.397 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-23T04:14:17.828Z,1682223257.828 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_STOPPED 2023-04-23T04:19:17.602Z,1682223557.602 [ESPComponent](IMPORTANT): [sample #6] ESP sampling sequence completed normally. 2023-04-23T04:19:18.834Z,1682223558.834 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:D](IMPORTANT): Sample No. 1.000000 count completed. 2023-04-23T04:38:30.241Z,1682224710.241 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2023-04-23T04:38:30.884Z,1682224710.884 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-23T05:05:06.724Z,1682226306.724 [RudderServo](FAULT): Reading outside of valid range:0.000000 2023-04-23T05:05:07.352Z,1682226307.352 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2023-04-23T07:52:10.410Z,1682236330.410 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-23T07:52:21.387Z,1682236341.387 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.004049 CHAN A1 (24V): -0.001489 CHAN A2 (12V): -0.000351 CHAN A3 (5V): 0.000446 CHAN B0 (3.3V): 0.000056 CHAN B1 (3.15aV): 0.000044 CHAN B2 (3.15bV): -0.000217 CHAN B3 (GND): 0.000290 OPEN: 0.007805 Full Scale: +/- 1 mA 2023-04-23T08:11:04.646Z,1682237464.646 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-23T08:48:39.172Z,1682239719.172 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-23T08:49:14.912Z,1682239754.912 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt1:A.Waypoint](IMPORTANT): Reached waypoint: 36.699099,-122.307622 2023-04-23T09:07:38.143Z,1682240858.143 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.262020 m 2023-04-23T09:09:48.224Z,1682240988.224 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 28.134741 m 2023-04-23T09:10:37.903Z,1682241037.903 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 23.738178 m , 0.348000 ug/l , 4.598779 m , 0.505192 ug/l 2023-04-23T09:10:40.345Z,1682241040.345 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 2.000000 count . Latitude, longitude = 36.699807 arcdeg , -122.301468 arcdeg . Target depth = 4.598779 m . 2023-04-23T09:10:40.738Z,1682241040.738 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-23T09:15:21.939Z,1682241321.939 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 4.564008 m , settling for 5.000000 min 2023-04-23T09:20:34.713Z,1682241634.713 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-23T09:20:34.866Z,1682241634.866 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-23T09:21:23.606Z,1682241683.606 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-23T09:21:24.410Z,1682241684.410 [ESPComponent](IMPORTANT): [sample #7] Setting cartridge argument to -6 2023-04-23T09:21:24.410Z,1682241684.410 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-23T09:21:24.847Z,1682241684.847 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.544128 m in 9.827387 degC water with 0.458200 ug/l chlorophyll fluorescence. 2023-04-23T09:21:25.233Z,1682241685.233 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-23T09:21:25.635Z,1682241685.635 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-23T09:21:27.647Z,1682241687.647 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-23T09:21:59.556Z,1682241719.556 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PRIMING (@23,archive_bac) 2023-04-23T09:22:02.383Z,1682241722.383 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PRIMING (@100.0) 2023-04-23T09:22:02.384Z,1682241722.384 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-23T09:22:02.785Z,1682241722.785 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-23T09:24:46.814Z,1682241886.814 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_FILTERING (@0.0) 2023-04-23T09:24:47.219Z,1682241887.219 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-23T10:11:29.032Z,1682244689.032 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PAUSED (@1000.0) 2023-04-23T10:11:29.426Z,1682244689.426 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-23T10:11:31.447Z,1682244691.447 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-23T10:11:31.849Z,1682244691.849 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PROCESSED (@23,archive_bac) 2023-04-23T10:23:35.452Z,1682245415.452 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPING (@23,archive_bac) 2023-04-23T10:23:45.521Z,1682245425.521 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-23T10:23:45.928Z,1682245425.928 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPED 2023-04-23T10:28:45.698Z,1682245725.698 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence completed normally. 2023-04-23T10:28:46.934Z,1682245726.934 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:D](IMPORTANT): Sample No. 2.000000 count completed. 2023-04-23T10:28:51.361Z,1682245731.361 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt2:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.699099,-122.417622 2023-04-23T12:58:37.850Z,1682254717.850 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-23T13:52:21.699Z,1682257941.699 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-23T13:52:32.711Z,1682257952.711 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003191 CHAN A1 (24V): -0.000997 CHAN A2 (12V): -0.000257 CHAN A3 (5V): -0.000519 CHAN B0 (3.3V): -0.000281 CHAN B1 (3.15aV): -0.000132 CHAN B2 (3.15bV): 0.000263 CHAN B3 (GND): 0.000302 OPEN: 0.007727 Full Scale: +/- 1 mA 2023-04-23T14:41:39.574Z,1682260899.574 [FrontSampling:NeedComms:D:A_Timeout:A](IMPORTANT): Comms timed out after 30.000000 min minutes 2023-04-23T17:12:01.821Z,1682269921.821 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-23T17:12:03.072Z,1682269923.072 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt2:A.Waypoint](IMPORTANT): Reached waypoint: 36.699099,-122.417622 2023-04-23T17:23:55.038Z,1682270635.038 [NAL9602](FAULT): Verify xmit timeout failure. 2023-04-23T17:40:49.698Z,1682271649.698 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.288384 m 2023-04-23T17:43:32.524Z,1682271812.524 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 28.199585 m 2023-04-23T17:44:25.046Z,1682271865.046 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 23.667326 m , 1.194800 ug/l , 7.968825 m , 1.503966 ug/l 2023-04-23T17:44:27.487Z,1682271867.487 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 3.000000 count . Latitude, longitude = 36.690382 arcdeg , -122.419121 arcdeg . Target depth = 7.968825 m . 2023-04-23T17:44:27.872Z,1682271867.872 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-23T17:48:27.461Z,1682272107.461 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 7.966127 m , settling for 5.000000 min 2023-04-23T17:53:40.260Z,1682272420.260 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-23T17:53:40.410Z,1682272420.410 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-23T17:54:32.687Z,1682272472.687 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-23T17:54:33.496Z,1682272473.496 [ESPComponent](IMPORTANT): [sample #8] Setting cartridge argument to -6 2023-04-23T17:54:33.499Z,1682272473.499 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-23T17:54:33.971Z,1682272473.971 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.807950 m in 9.988794 degC water with 1.684320 ug/l chlorophyll fluorescence. 2023-04-23T17:54:34.298Z,1682272474.298 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-23T17:54:34.710Z,1682272474.710 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-23T17:54:36.722Z,1682272476.722 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-23T17:55:08.643Z,1682272508.643 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PRIMING (@22,archive_bac) 2023-04-23T17:55:11.467Z,1682272511.467 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PRIMING (@100.0) 2023-04-23T17:55:11.468Z,1682272511.468 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-23T17:55:11.886Z,1682272511.886 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-23T17:57:54.278Z,1682272674.278 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_FILTERING (@0.0) 2023-04-23T17:57:54.683Z,1682272674.683 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-23T18:49:58.054Z,1682275798.054 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PAUSED (@1000.0) 2023-04-23T18:49:58.487Z,1682275798.487 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-23T18:50:00.496Z,1682275800.496 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-23T18:50:00.882Z,1682275800.882 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PROCESSED (@22,archive_bac) 2023-04-23T19:02:04.470Z,1682276524.470 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPING (@22,archive_bac) 2023-04-23T19:02:14.562Z,1682276534.562 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-23T19:02:14.954Z,1682276534.954 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPED 2023-04-23T19:07:14.764Z,1682276834.764 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence completed normally. 2023-04-23T19:07:15.955Z,1682276835.955 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:D](IMPORTANT): Sample No. 3.000000 count completed. 2023-04-23T19:07:20.415Z,1682276840.415 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt3:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.699099,-122.527622 2023-04-23T19:52:32.976Z,1682279552.976 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-23T19:52:44.402Z,1682279564.402 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003357 CHAN A1 (24V): -0.001282 CHAN A2 (12V): -0.000648 CHAN A3 (5V): -0.000481 CHAN B0 (3.3V): 0.000189 CHAN B1 (3.15aV): 0.000468 CHAN B2 (3.15bV): 0.000478 CHAN B3 (GND): 0.000367 OPEN: 0.004958 Full Scale: +/- 1 mA 2023-04-23T20:56:49.564Z,1682283409.564 [FrontSampling:NeedComms:D:A_Timeout:A](IMPORTANT): Comms timed out after 30.000000 min minutes 2023-04-23T21:01:11.800Z,1682283671.800 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2023-04-23T21:01:12.476Z,1682283672.476 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2023-04-23T22:03:49.762Z,1682287429.762 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-23T23:13:27.339Z,1682291607.339 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt3:A.Waypoint](IMPORTANT): Reached waypoint: 36.699099,-122.527622 2023-04-23T23:20:39.206Z,1682292039.206 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-23T23:31:40.430Z,1682292700.430 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.249919 m 2023-04-23T23:34:22.842Z,1682292862.842 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 28.113583 m 2023-04-23T23:35:14.958Z,1682292914.958 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 23.638796 m , 1.647200 ug/l , 7.751959 m , 1.528332 ug/l 2023-04-23T23:35:17.388Z,1682292917.388 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 4.000000 count . Latitude, longitude = 36.694375 arcdeg , -122.532574 arcdeg . Target depth = 7.751959 m . 2023-04-23T23:35:17.810Z,1682292917.810 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-23T23:36:08.298Z,1682292968.298 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 7.606122 m , settling for 5.000000 min 2023-04-23T23:41:21.081Z,1682293281.081 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-23T23:41:21.242Z,1682293281.242 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-23T23:42:14.348Z,1682293334.348 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-23T23:42:15.187Z,1682293335.187 [ESPComponent](IMPORTANT): [sample #9] Setting cartridge argument to -6 2023-04-23T23:42:15.187Z,1682293335.187 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-23T23:42:15.613Z,1682293335.613 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.200307 m in 9.933252 degC water with 1.554400 ug/l chlorophyll fluorescence. 2023-04-23T23:42:15.966Z,1682293335.966 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-23T23:42:16.378Z,1682293336.378 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-23T23:42:18.417Z,1682293338.417 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-23T23:42:50.307Z,1682293370.307 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PRIMING (@21,archive_bac) 2023-04-23T23:42:52.739Z,1682293372.739 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PRIMING (@100.0) 2023-04-23T23:42:52.740Z,1682293372.740 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-23T23:42:53.133Z,1682293373.133 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-23T23:45:36.758Z,1682293536.758 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_FILTERING (@0.0) 2023-04-23T23:45:37.174Z,1682293537.174 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-24T00:36:37.921Z,1682296597.921 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PAUSED (@1000.0) 2023-04-24T00:36:38.309Z,1682296598.309 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-24T00:36:40.322Z,1682296600.322 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-24T00:36:40.721Z,1682296600.721 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PROCESSED (@21,archive_bac) 2023-04-24T00:48:44.316Z,1682297324.316 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPING (@21,archive_bac) 2023-04-24T00:48:54.412Z,1682297334.412 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-24T00:48:54.824Z,1682297334.824 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPED 2023-04-24T00:53:54.573Z,1682297634.573 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence completed normally. 2023-04-24T00:53:55.844Z,1682297635.844 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:D](IMPORTANT): Sample No. 4.000000 count completed. 2023-04-24T00:54:00.259Z,1682297640.259 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt4:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.699099,-122.637622 2023-04-24T00:59:54.973Z,1682297994.973 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-24T01:52:44.689Z,1682301164.689 [CBIT](IMPORTANT): Beginning ground fault scan 2023-04-24T01:52:55.559Z,1682301175.559 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.002866 CHAN A1 (24V): -0.000947 CHAN A2 (12V): -0.000784 CHAN A3 (5V): -0.000935 CHAN B0 (3.3V): -0.000110 CHAN B1 (3.15aV): 0.029526 CHAN B2 (3.15bV): 0.000228 CHAN B3 (GND): 0.000129 OPEN: 0.007695 Full Scale: +/- 1 mA 2023-04-24T02:26:39.112Z,1682303199.112 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-24T03:47:05.273Z,1682308025.273 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-24T04:10:48.668Z,1682309448.668 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2023-04-24T04:22:18.420Z,1682310138.420 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt4:A.Waypoint](IMPORTANT): Reached waypoint: 36.699099,-122.637622 2023-04-24T04:36:51.349Z,1682311011.349 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.209293 m 2023-04-24T04:39:43.877Z,1682311183.877 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 28.102373 m 2023-04-24T04:40:35.570Z,1682311235.570 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 23.690245 m , 3.236400 ug/l , 3.695737 m , 4.831431 ug/l 2023-04-24T04:40:37.990Z,1682311237.990 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 5.000000 count . Latitude, longitude = 36.683530 arcdeg , -122.647112 arcdeg . Target depth = 3.695737 m . 2023-04-24T04:40:38.414Z,1682311238.414 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2023-04-24T04:45:32.930Z,1682311532.930 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 3.690572 m , settling for 5.000000 min 2023-04-24T04:50:45.715Z,1682311845.715 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-04-24T04:50:45.885Z,1682311845.885 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-04-24T04:51:38.611Z,1682311898.611 [ESPComponent](IMPORTANT): ESP has connected as client 2023-04-24T04:51:39.410Z,1682311899.410 [ESPComponent](IMPORTANT): [sample #10] Setting cartridge argument to -6 2023-04-24T04:51:39.410Z,1682311899.410 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-04-24T04:51:39.846Z,1682311899.846 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.076436 m in 10.318232 degC water with 5.003080 ug/l chlorophyll fluorescence. 2023-04-24T04:51:40.214Z,1682311900.214 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_LOG 2023-04-24T04:51:40.665Z,1682311900.665 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-04-24T04:51:42.647Z,1682311902.647 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_LOADING_CARTRIDGE 2023-04-24T04:52:14.568Z,1682311934.568 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PRIMING (@20,archive_bac) 2023-04-24T04:52:16.579Z,1682311936.579 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PRIMING (@100.0) 2023-04-24T04:52:16.979Z,1682311936.979 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-04-24T04:52:17.392Z,1682311937.392 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-04-24T04:55:01.811Z,1682312101.811 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_FILTERING (@0.0) 2023-04-24T04:55:02.238Z,1682312102.238 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-04-24T05:12:22.713Z,1682313142.713 [Radio_Surface](FAULT): LCB fault: LCB Watchdog Reset. 2023-04-24T05:20:12.022Z,1682313612.022 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Reached waypoint: 36.700001,-122.650002 2023-04-24T05:20:12.418Z,1682313612.418 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt6:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.700001,-122.197998 2023-04-24T05:44:54.307Z,1682315094.307 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PAUSED (@1000.0) 2023-04-24T05:44:54.710Z,1682315094.710 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PROCESSING (@1000.0) 2023-04-24T05:44:56.731Z,1682315096.731 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-04-24T05:44:57.137Z,1682315097.137 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PROCESSED (@20,archive_bac) 2023-04-24T05:57:00.706Z,1682315820.706 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPING (@20,archive_bac) 2023-04-24T05:57:10.807Z,1682315830.807 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-04-24T05:57:11.287Z,1682315831.287 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPED 2023-04-24T06:02:11.031Z,1682316131.031 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence completed normally. 2023-04-24T06:02:12.298Z,1682316132.298 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:D](IMPORTANT): Sample No. 5.000000 count completed. 2023-04-24T06:07:44.738Z,1682316464.738 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-04-24T06:22:24.666Z,1682317344.666 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=16631, MT Status=1, MTMSN=1209 2023-04-24T06:22:25.198Z,1682317345.198 [CommandExec](IMPORTANT): got command stop 2023-04-24T06:22:25.202Z,1682317345.202 [CommandExec](IMPORTANT): Scheduling is paused 2023-04-24T06:22:25.887Z,1682317345.887 [MissionManager](IMPORTANT): Started mission Default 2023-04-24T06:25:16.232Z,1682317516.232 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16632, MT Status=1, MTMSN=1210 2023-04-24T06:25:16.760Z,1682317516.760 [CommandExec](IMPORTANT): got command schedule clear 2023-04-24T06:25:16.760Z,1682317516.760 [CommandExec](IMPORTANT): Cleared 0 scheduled commands. 2023-04-24T06:25:16.760Z,1682317516.760 [CommandExec](IMPORTANT): got command schedule resume 2023-04-24T06:25:16.761Z,1682317516.761 [CommandExec](IMPORTANT): Scheduling is resumed 2023-04-24T06:27:53.665Z,1682317673.665 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16633, MT Status=1, MTMSN=1211