2020-10-12T20:29:19.158Z,1602534559.158 [Supervisor](IMPORTANT): Reading configuration overrides from Data/persisted.cfg 2020-10-12T20:29:23.309Z,1602534563.309 [SBIT](IMPORTANT): git: 2020-10-07_A 2020-10-12T20:29:23.310Z,1602534563.310 [SBIT](IMPORTANT): Kernel Release: 2.6.27.8 2020-10-12T20:29:23.905Z,1602534563.905 [CommandLine](IMPORTANT): got command ! echo 1 > /dev/loadB2 2020-10-12T20:29:24.865Z,1602534564.865 [RudderServo](FAULT): Rudder failed to initialize 2020-10-12T20:29:24.999Z,1602534564.999 [AMEcho](IMPORTANT): Setting 'enabled' to 0 2020-10-12T20:29:55.295Z,1602534595.295 [DAT](FAULT): failed to enter command mode 2020-10-12T20:30:39.748Z,1602534639.748 [SBIT](IMPORTANT): Beginning Startup BIT 2020-10-12T20:30:39.752Z,1602534639.752 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-12T20:30:50.712Z,1602534650.712 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005360 CHAN A1 (24V): 0.002953 CHAN A2 (12V): -0.003874 CHAN A3 (5V): -0.002305 CHAN B0 (3.3V): -0.000802 CHAN B1 (3.15aV): -0.000920 CHAN B2 (3.15bV): -0.000852 CHAN B3 (GND): -0.000338 OPEN: 0.007421 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-12T20:31:33.535Z,1602534693.535 [SBIT](IMPORTANT): SBIT PASSED 2020-10-12T20:31:33.536Z,1602534693.536 [SBIT](IMPORTANT): Listing configuration overrides from Data/persisted.cfg 2020-10-12T20:31:33.536Z,1602534693.536 [SBIT](IMPORTANT): AMEcho.enabled=0 bool; 2020-10-12T20:31:33.536Z,1602534693.536 [SBIT](IMPORTANT): Aanderaa_O2.loadAtStartup=0 bool; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): BPC1.batteryMissingStickThreshold=2 count; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): CANONSampler.loadAtStartup=1 bool; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): CANONSampler.rotateOnly=0 bool; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): CANONSampler.simulateHardware=0 bool; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): CBIT.gf24Offset=148.5 microampere; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): DAT.loadAtStartup=1 bool; 2020-10-12T20:31:33.537Z,1602534693.537 [SBIT](IMPORTANT): DVL_micro.loadAtStartup=0 bool; 2020-10-12T20:31:33.538Z,1602534693.538 [SBIT](IMPORTANT): ESPComponent.loadAtStartup=1 bool; 2020-10-12T20:31:33.538Z,1602534693.538 [SBIT](IMPORTANT): ESPComponent.sampleTimeout=25 minute; 2020-10-12T20:31:33.538Z,1602534693.538 [SBIT](IMPORTANT): ESPComponent.simulateHardware=0 bool; 2020-10-12T20:31:33.538Z,1602534693.538 [SBIT](IMPORTANT): Express none WetLabsBB2FL.bin_mean_mass_concentration_of_chlorophyll_in_sea_water; 2020-10-12T20:31:33.538Z,1602534693.538 [SBIT](IMPORTANT): VerticalControl.buoyancyNeutral=280 cubic_centimeter; 2020-10-12T20:31:33.538Z,1602534693.538 [SBIT](IMPORTANT): VerticalControl.massDefault=14 millimeter; 2020-10-12T20:31:33.895Z,1602534693.895 [MissionManager](IMPORTANT): Started mission Startup 2020-10-12T20:33:34.636Z,1602534814.636 [MissionManager](IMPORTANT): Started mission Default 2020-10-12T20:38:57.178Z,1602535137.178 [DataOverHttps](IMPORTANT): SBD MTMSN=20201012T203855 2020-10-12T20:39:04.530Z,1602535144.530 [CommandLine](IMPORTANT): got command schedule clear 2020-10-12T20:39:04.530Z,1602535144.530 [CommandLine](IMPORTANT): #Cleared scheduled commands:0 2020-10-12T20:39:04.531Z,1602535144.531 [CommandLine](IMPORTANT): got command schedule resume 2020-10-12T20:39:35.542Z,1602535175.542 [CommandLine](IMPORTANT): got command show variable DAT.surface 2020-10-12T20:39:35.565Z,1602535175.565 [CommandLine](IMPORTANT): DAT.surfaceThreshold (meter) 2020-10-12T20:39:43.269Z,1602535183.269 [CommandLine](IMPORTANT): got command get DAT.surfaceThreshold 2020-10-12T20:39:43.270Z,1602535183.270 [CommandLine](IMPORTANT): DAT.surfaceThreshold 0.500000 m 2020-10-12T20:40:10.649Z,1602535210.649 [CommandLine](IMPORTANT): got command show stack 2020-10-12T20:40:10.649Z,1602535210.649 [CommandLine](IMPORTANT): Behavior Stack: 2020-10-12T20:40:10.649Z,1602535210.649 [Default](IMPORTANT): Priority 0: Default:B.GoToSurface 2020-10-12T20:40:10.650Z,1602535210.650 [Default:CheckIn](IMPORTANT): Priority 1: Default:CheckIn:C.Wait 2020-10-12T20:40:17.334Z,1602535217.334 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 6.704830 min 2020-10-12T20:49:39.499Z,1602535779.499 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 16.074382 min 2020-10-12T20:50:14.299Z,1602535814.299 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40132, MT Status=1, MTMSN=2787 2020-10-12T20:50:15.042Z,1602535815.042 [CommandLine](IMPORTANT): got command schedule asap "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2gn4k 1 3.000000 2020-10-12T20:50:15.042Z,1602535815.042 [CommandLine](IMPORTANT): Scheduling command #1 of 3 with id=2gn4k 2020-10-12T20:50:15.043Z,1602535815.043 [CommandLine](IMPORTANT): Scheduled #1 (#1 of 3 with id='2gn4k'): "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" ASAP 2020-10-12T20:50:36.518Z,1602535836.518 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40133, MT Status=1, MTMSN=2788 2020-10-12T20:50:37.258Z,1602535837.258 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter;set IsothermDepthCANON.Lat1 36.8219 degree" 2gn4k 2 3.000000 2020-10-12T20:50:37.259Z,1602535837.259 [CommandLine](IMPORTANT): Scheduling command #2 of 3 with id=2gn4k 2020-10-12T20:50:37.260Z,1602535837.260 [CommandLine](IMPORTANT): Scheduled #2 (#2 of 3 with id='2gn4k'): "set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter;set IsothermDepthCANON.Lat1 36.8219 degree" ASAP 2020-10-12T20:50:49.248Z,1602535849.248 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40134, MT Status=1, MTMSN=2789 2020-10-12T20:50:49.967Z,1602535849.967 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthCANON.Lon1 -121.8856 degree;set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin;set IsothermDepthCANON.WaitDepUndulation1 20 minute;run" 2gn4k 3 3.000000 2020-10-12T20:50:49.967Z,1602535849.967 [CommandLine](IMPORTANT): Scheduling command #3 of 3 with id=2gn4k 2020-10-12T20:50:49.968Z,1602535849.968 [CommandLine](IMPORTANT): Scheduled #3 (#3 of 3 with id='2gn4k'): "set IsothermDepthCANON.Lon1 -121.8856 degree;set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin;set IsothermDepthCANON.WaitDepUndulation1 20 minute;run" ASAP 2020-10-12T20:50:50.132Z,1602535850.132 [CommandLine](IMPORTANT): got command load ./Missions/Science/isotherm_depth_canon.xml 2020-10-12T20:51:08.408Z,1602535868.408 [CommandLine](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_canon.xml 2020-10-12T20:51:21.764Z,1602535881.764 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.MissionTimeout 3.000000 hour 2020-10-12T20:51:21.765Z,1602535881.765 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.InitialDiveDuration 2.000000 minute 2020-10-12T20:51:21.766Z,1602535881.766 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SpeedSampling 0.000000 meter_per_second 2020-10-12T20:51:36.096Z,1602535896.096 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.ShallowBoundChl 3.000000 meter 2020-10-12T20:51:36.099Z,1602535896.099 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMinDepth 3.000000 meter 2020-10-12T20:51:36.100Z,1602535896.100 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMaxDepth 50.000000 meter 2020-10-12T20:51:36.101Z,1602535896.101 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lat1 36.821899 degree 2020-10-12T20:51:49.390Z,1602535909.390 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lon1 -121.885597 degree 2020-10-12T20:51:49.390Z,1602535909.390 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleType1 2.000000 count 2020-10-12T20:51:49.391Z,1602535909.391 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TempDiffFromPeakChl1 0.000000 kelvin 2020-10-12T20:51:49.392Z,1602535909.392 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.WaitDepUndulation1 20.000000 minute 2020-10-12T20:51:49.392Z,1602535909.392 [CommandLine](IMPORTANT): got command run 2020-10-12T20:51:49.403Z,1602535909.403 [CommandLine](IMPORTANT): Running 2020-10-12T20:51:49.563Z,1602535909.563 [MissionManager](IMPORTANT): Started mission IsothermDepthCANON 2020-10-12T20:51:49.616Z,1602535909.616 [IsothermDepthCANON:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-10-12T21:18:29.697Z,1602537509.697 [IsothermDepthCANON:TransitToSamplingLocationNoComms:Wpt1NoComms:C.Waypoint](IMPORTANT): Reached Waypoint: 36.821898,-121.885600 2020-10-12T21:19:40.429Z,1602537580.429 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.635151 m 2020-10-12T21:19:49.322Z,1602537589.322 [IsothermDepthCANON:SetSampleOptions:Q](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , 0.000000 K , nan m , nan degC , nan m , 20.000000 min . 2020-10-12T21:27:09.305Z,1602538029.305 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 30.037767 m 2020-10-12T21:31:38.366Z,1602538298.366 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 28.422897 m , 0.384200 ug/l , 5.959395 m , 13.642360 degC , 6.434793 ug/l 2020-10-12T21:31:41.229Z,1602538301.229 [IsothermDepthCANON:TakeSamples:FiringOnTemp:C](IMPORTANT): Taking sample No. 1.000000 count . Target temp = 13.642360 degC . 2020-10-12T21:31:42.009Z,1602538302.009 [IsothermDepthCANON:DampDown:A](IMPORTANT): Wait 20.000000 min for the depth undulation to damp down. 2020-10-12T21:51:45.563Z,1602539505.563 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-12T21:51:47.127Z,1602539507.127 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::' 2020-10-12T21:51:57.267Z,1602539517.267 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-12T21:51:57.569Z,1602539517.569 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-12T21:52:36.501Z,1602539556.501 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-12T21:52:37.315Z,1602539557.315 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -6 2020-10-12T21:52:37.316Z,1602539557.316 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-12T21:52:37.766Z,1602539557.766 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 4.839117 m in 13.711298 degC water with 6.463600 ug/l chlorophyll fluorescence. 2020-10-12T21:52:38.521Z,1602539558.521 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-12T21:52:38.925Z,1602539558.925 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-12T21:52:40.543Z,1602539560.543 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-12T21:53:13.674Z,1602539593.674 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PRIMING (@32,archive_bac) 2020-10-12T21:53:15.290Z,1602539595.290 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PRIMING (@100.0) 2020-10-12T21:53:16.906Z,1602539596.906 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-12T21:55:48.815Z,1602539748.815 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_FILTERING (@0.0) 2020-10-12T21:55:49.218Z,1602539749.218 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-12T22:02:18.387Z,1602540138.387 [IsothermDepthCANON:Sample:SampleWrapper:CANONSampler:AcquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 5.711941 m in 13.817591 degC water with 5.548300 ug/l chlorophyll fluorescence. 2020-10-12T22:05:07.180Z,1602540307.180 [CANONSampler](IMPORTANT): Sample 1, err_code=0 2020-10-12T22:13:53.230Z,1602540833.230 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PAUSED (@317.343) 2020-10-12T22:13:53.630Z,1602540833.630 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PROCESSING (@317.343) 2020-10-12T22:13:56.055Z,1602540836.055 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PROCESSED (@32,archive_bac) 2020-10-12T22:26:00.074Z,1602541560.074 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING (@32,archive_bac) 2020-10-12T22:26:25.526Z,1602541585.526 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-12T22:26:25.934Z,1602541585.934 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2020-10-12T22:31:25.706Z,1602541885.706 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence completed normally. 2020-10-12T22:31:26.110Z,1602541886.110 [ESPComponent](IMPORTANT): [sample #1] ESP log summary report (3 messages): @14:52:50.47 Selecting Cartridge 32 @15:13:50.90 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @15:13:51.09 Sampled 317.3ml 2020-10-12T22:31:28.590Z,1602541888.590 [IsothermDepthCANON:TakeSamples:FiringOnTemp:TriggerOnTemp:C](IMPORTANT): Sample No. 1.000000 count completed. 2020-10-12T22:31:31.398Z,1602541891.398 [MissionManager](IMPORTANT): Started mission Default 2020-10-12T22:31:31.461Z,1602541891.461 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-10-12T22:45:17.203Z,1602542717.203 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 13.756705 min 2020-10-12T22:45:44.726Z,1602542744.726 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40167, MT Status=1, MTMSN=2790 2020-10-12T22:45:45.499Z,1602542745.499 [CommandLine](IMPORTANT): got command schedule asap "load Science/esp_sample_station.xml;set esp_sample_station.MissionTimeout 15 hour;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.StartWaypoint 2 count;set esp_sample_station.TransitYoYoMinDepth 3 meter" 2gsj6 1 2.000000 2020-10-12T22:45:45.499Z,1602542745.499 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2gsj6 2020-10-12T22:45:45.500Z,1602542745.500 [CommandLine](IMPORTANT): Scheduled #4 (#1 of 2 with id='2gsj6'): "load Science/esp_sample_station.xml;set esp_sample_station.MissionTimeout 15 hour;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.StartWaypoint 2 count;set esp_sample_station.TransitYoYoMinDepth 3 meter" ASAP 2020-10-12T22:46:08.122Z,1602542768.122 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40168, MT Status=1, MTMSN=2791 2020-10-12T22:46:08.909Z,1602542768.909 [CommandLine](IMPORTANT): got command schedule asap "set esp_sample_station.SampleYoYoMinDepth 3 meter;set esp_sample_station.Lat1 NaN degree;set esp_sample_station.Lon1 NaN degree;set esp_sample_station.Speed 0.75 meter_per_second;run" 2gsj6 2 2.000000 2020-10-12T22:46:08.910Z,1602542768.910 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2gsj6 2020-10-12T22:46:08.911Z,1602542768.911 [CommandLine](IMPORTANT): Scheduled #5 (#2 of 2 with id='2gsj6'): "set esp_sample_station.SampleYoYoMinDepth 3 meter;set esp_sample_station.Lat1 NaN degree;set esp_sample_station.Lon1 NaN degree;set esp_sample_station.Speed 0.75 meter_per_second;run" ASAP 2020-10-12T22:46:09.043Z,1602542769.043 [CommandLine](IMPORTANT): got command load ./Missions/Science/esp_sample_station.xml 2020-10-12T22:46:11.819Z,1602542771.819 [CommandLine](IMPORTANT): Loaded ./Missions/Science/esp_sample_station.xml 2020-10-12T22:46:29.272Z,1602542789.272 [CommandLine](IMPORTANT): got command set esp_sample_station.MissionTimeout 15.000000 hour 2020-10-12T22:46:29.273Z,1602542789.273 [CommandLine](IMPORTANT): got command set esp_sample_station.NeedCommsTime 60.000000 minute 2020-10-12T22:46:29.273Z,1602542789.273 [CommandLine](IMPORTANT): got command set esp_sample_station.StartWaypoint 2.000000 count 2020-10-12T22:46:29.274Z,1602542789.274 [CommandLine](IMPORTANT): got command set esp_sample_station.TransitYoYoMinDepth 3.000000 meter 2020-10-12T22:46:41.492Z,1602542801.492 [CommandLine](IMPORTANT): got command set esp_sample_station.SampleYoYoMinDepth 3.000000 meter 2020-10-12T22:46:41.493Z,1602542801.493 [CommandLine](IMPORTANT): got command set esp_sample_station.Lat1 nan degree 2020-10-12T22:46:41.494Z,1602542801.494 [CommandLine](IMPORTANT): got command set esp_sample_station.Lon1 nan degree 2020-10-12T22:46:41.495Z,1602542801.495 [CommandLine](IMPORTANT): got command set esp_sample_station.Speed 0.750000 meter_per_second 2020-10-12T22:46:41.495Z,1602542801.495 [CommandLine](IMPORTANT): got command run 2020-10-12T22:46:41.499Z,1602542801.499 [CommandLine](IMPORTANT): Running 2020-10-12T22:46:41.833Z,1602542801.833 [MissionManager](IMPORTANT): Started mission esp_sample_station 2020-10-12T22:47:38.851Z,1602542858.851 [esp_sample_station:Lap:A](IMPORTANT): Starting Lap 2020-10-12T22:47:39.261Z,1602542859.261 [esp_sample_station:Lap:B](IMPORTANT): Lat[1] = nan arcdeg Lon[1] = nan arcdeg 2020-10-12T22:47:39.675Z,1602542859.675 [esp_sample_station:Lap:C](IMPORTANT): Lat[2] = 36.821900 arcdeg Lon[2] = -121.885600 arcdeg 2020-10-12T22:47:40.068Z,1602542860.068 [esp_sample_station:Lap:D](IMPORTANT): Lat[3] = 36.766000 arcdeg Lon[3] = -121.864900 arcdeg 2020-10-12T22:47:40.464Z,1602542860.464 [esp_sample_station:Lap:E](IMPORTANT): Lat[4] = nan arcdeg Lon[4] = nan arcdeg 2020-10-12T22:47:44.914Z,1602542864.914 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.821900 arcdeg , -121.885600 arcdeg 2020-10-12T22:54:20.036Z,1602543260.036 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.821900,-121.885600 2020-10-12T22:59:04.586Z,1602543544.586 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-12T22:59:16.370Z,1602543556.370 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-12T22:59:16.517Z,1602543556.517 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-12T22:59:55.449Z,1602543595.449 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-12T22:59:56.269Z,1602543596.269 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -6 2020-10-12T22:59:56.269Z,1602543596.269 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-12T22:59:56.698Z,1602543596.698 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 11.943211 m in 12.671808 degC water with 1.163900 ug/l chlorophyll fluorescence. 2020-10-12T22:59:57.066Z,1602543597.066 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-12T22:59:57.469Z,1602543597.469 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-12T22:59:59.489Z,1602543599.489 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-12T23:00:32.662Z,1602543632.662 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PRIMING (@31,archive_bac) 2020-10-12T23:00:34.279Z,1602543634.279 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PRIMING (@100.0) 2020-10-12T23:00:35.925Z,1602543635.925 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-12T23:03:27.606Z,1602543807.606 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_FILTERING (@0.0) 2020-10-12T23:03:28.009Z,1602543808.009 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-12T23:42:59.183Z,1602546179.183 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PAUSED (@649.276) 2020-10-12T23:42:59.586Z,1602546179.586 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PROCESSING (@649.276) 2020-10-12T23:43:02.010Z,1602546182.010 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PROCESSED (@30,archive_bac) 2020-10-12T23:55:05.586Z,1602546905.586 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPING (@30,archive_bac) 2020-10-12T23:55:31.038Z,1602546931.038 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-12T23:55:31.441Z,1602546931.441 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPED 2020-10-13T00:00:31.226Z,1602547231.226 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence completed normally. 2020-10-13T00:00:31.630Z,1602547231.630 [ESPComponent](IMPORTANT): [sample #2] ESP log summary report (6 messages): @16:00:09.40 Selecting Cartridge 31 @16:00:40.92 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 34% to 29.2psia @16:00:40.93 Retry #1 of 2 @16:00:40.98 Selecting Cartridge 30 @16:42:56.66 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @16:42:56.85 Sampled 649.3ml 2020-10-13T00:09:28.715Z,1602547768.715 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.766000 arcdeg , -121.864900 arcdeg 2020-10-13T01:09:58.853Z,1602551398.853 [BPC1](FAULT): Battery stick #38 (s/n: 0139) reported TERMINATE_DISCHARGE_ALARM. Status code: 0x4AD0. 2020-10-13T02:30:50.956Z,1602556250.956 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-13T02:31:01.815Z,1602556261.815 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005273 CHAN A1 (24V): 0.003290 CHAN A2 (12V): -0.004414 CHAN A3 (5V): -0.002481 CHAN B0 (3.3V): -0.000836 CHAN B1 (3.15aV): -0.000916 CHAN B2 (3.15bV): -0.001241 CHAN B3 (GND): -0.000204 OPEN: 0.007301 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-13T02:31:19.510Z,1602556279.510 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2020-10-13T03:28:19.213Z,1602559699.213 [NAL9602](FAULT): received: +CSQ:0 OK212, 2, 0, 0, 0 OK 2020-10-13T03:34:46.909Z,1602560086.909 [esp_sample_station:NeedComms:C:A_Timeout:A](IMPORTANT): first GPS update timeout 2020-10-13T03:48:40.177Z,1602560920.177 [BPC1](FAULT): Failed to receive data from 2 sticks prior to timeout. Missing stick IDs are: 38, 39. 2020-10-13T03:53:44.801Z,1602561224.801 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.766000,-121.864900 2020-10-13T03:57:31.121Z,1602561451.121 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T03:57:42.909Z,1602561462.909 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T03:57:43.046Z,1602561463.046 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T03:58:21.585Z,1602561501.585 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T03:58:22.396Z,1602561502.396 [ESPComponent](IMPORTANT): [sample #3] Setting cartridge argument to -6 2020-10-13T03:58:22.397Z,1602561502.397 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T03:58:22.852Z,1602561502.852 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 16.286579 m in 12.611811 degC water with 0.904000 ug/l chlorophyll fluorescence. 2020-10-13T03:58:23.202Z,1602561503.202 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T03:58:23.606Z,1602561503.606 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T03:58:25.626Z,1602561505.626 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T03:58:58.756Z,1602561538.756 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PRIMING (@29,archive_bac) 2020-10-13T03:59:00.370Z,1602561540.370 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T03:59:01.985Z,1602561541.985 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T04:01:30.674Z,1602561690.674 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T04:01:31.077Z,1602561691.077 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T04:18:59.605Z,1602562739.605 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2020-10-13T04:19:00.253Z,1602562740.253 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2020-10-13T04:33:31.762Z,1602563611.762 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PAUSED (@551.599) 2020-10-13T04:33:32.166Z,1602563612.166 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PROCESSING (@551.599) 2020-10-13T04:33:34.590Z,1602563614.590 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PROCESSED (@29,archive_bac) 2020-10-13T04:45:38.183Z,1602564338.183 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPING (@29,archive_bac) 2020-10-13T04:46:03.638Z,1602564363.638 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T04:46:04.042Z,1602564364.042 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPED 2020-10-13T04:51:03.825Z,1602564663.825 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence completed normally. 2020-10-13T04:51:04.230Z,1602564664.230 [ESPComponent](IMPORTANT): [sample #3] ESP log summary report (3 messages): @20:58:35.27 Selecting Cartridge 29 @21:33:28.92 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @21:33:29.12 Sampled 551.6ml 2020-10-13T04:53:17.974Z,1602564797.974 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40219, MT Status=1, MTMSN=2792 2020-10-13T04:53:18.550Z,1602564798.550 [CommandLine](IMPORTANT): got command schedule clear 2020-10-13T04:53:18.550Z,1602564798.550 [CommandLine](IMPORTANT): #Cleared scheduled commands:0 2020-10-13T04:53:18.550Z,1602564798.550 [CommandLine](IMPORTANT): got command schedule resume 2020-10-13T04:53:46.385Z,1602564826.385 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40220, MT Status=1, MTMSN=2793 2020-10-13T04:53:47.166Z,1602564827.166 [CommandLine](IMPORTANT): got command schedule "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2h7tf 1 3.000000 2020-10-13T04:53:47.166Z,1602564827.166 [CommandLine](IMPORTANT): Scheduling command #1 of 3 with id=2h7tf 2020-10-13T04:53:47.167Z,1602564827.167 [CommandLine](IMPORTANT): Scheduled #6 (#1 of 3 with id='2h7tf'): "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2020-10-13T04:54:00.603Z,1602564840.603 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40221, MT Status=1, MTMSN=2794 2020-10-13T04:54:01.393Z,1602564841.393 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter;set IsothermDepthCANON.Lat1 36.766 degree;set IsothermDepthCANON.Lon1 -121.8649 degree" 2h7tf 2 3.000000 2020-10-13T04:54:01.393Z,1602564841.393 [CommandLine](IMPORTANT): Scheduling command #2 of 3 with id=2h7tf 2020-10-13T04:54:01.394Z,1602564841.394 [CommandLine](IMPORTANT): Scheduled #7 (#2 of 3 with id='2h7tf'): "set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter;set IsothermDepthCANON.Lat1 36.766 degree;set IsothermDepthCANON.Lon1 -121.8649 degree" 2020-10-13T04:54:53.837Z,1602564893.837 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40222, MT Status=1, MTMSN=2795 2020-10-13T04:54:54.566Z,1602564894.566 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin;set IsothermDepthCANON.WaitDepUndulation1 20 minute;run" 2h7tf 3 3.000000 2020-10-13T04:54:54.566Z,1602564894.566 [CommandLine](IMPORTANT): Scheduling command #3 of 3 with id=2h7tf 2020-10-13T04:54:54.567Z,1602564894.567 [CommandLine](IMPORTANT): Scheduled #8 (#3 of 3 with id='2h7tf'): "set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin;set IsothermDepthCANON.WaitDepUndulation1 20 minute;run" 2020-10-13T04:55:30.986Z,1602564930.986 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40223, MT Status=1, MTMSN=2796 2020-10-13T04:55:31.711Z,1602564931.711 [CommandLine](IMPORTANT): got command schedule "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2h84l 1 3.000000 2020-10-13T04:55:31.712Z,1602564931.712 [CommandLine](IMPORTANT): Scheduling command #1 of 3 with id=2h84l 2020-10-13T04:55:31.712Z,1602564931.712 [CommandLine](IMPORTANT): Scheduled #9 (#1 of 3 with id='2h84l'): "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2020-10-13T04:56:34.393Z,1602564994.393 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40224, MT Status=1, MTMSN=2797 2020-10-13T04:56:35.222Z,1602564995.222 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter;set IsothermDepthCANON.Lat1 36.797 degree;set IsothermDepthCANON.Lon1 -121.847 degree" 2h84l 2 3.000000 2020-10-13T04:56:35.222Z,1602564995.222 [CommandLine](IMPORTANT): Scheduling command #2 of 3 with id=2h84l 2020-10-13T04:56:35.223Z,1602564995.223 [CommandLine](IMPORTANT): Scheduled #10 (#2 of 3 with id='2h84l'): "set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter;set IsothermDepthCANON.Lat1 36.797 degree;set IsothermDepthCANON.Lon1 -121.847 degree" 2020-10-13T04:57:17.931Z,1602565037.931 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40225, MT Status=1, MTMSN=2798 2020-10-13T04:57:18.612Z,1602565038.612 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin;set IsothermDepthCANON.WaitDepUndulation1 20 minute;run " 2h84l 3 3.000000 2020-10-13T04:57:18.612Z,1602565038.612 [CommandLine](IMPORTANT): Scheduling command #3 of 3 with id=2h84l 2020-10-13T04:57:18.613Z,1602565038.613 [CommandLine](IMPORTANT): Scheduled #11 (#3 of 3 with id='2h84l'): "set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin;set IsothermDepthCANON.WaitDepUndulation1 20 minute;run " 2020-10-13T04:57:48.969Z,1602565068.969 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40226, MT Status=1, MTMSN=2799 2020-10-13T04:57:49.735Z,1602565069.735 [CommandLine](IMPORTANT): got command schedule "load Science/esp_sample_station.xml;set esp_sample_station.MissionTimeout 16 hour;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.TransitYoYoMinDepth 3 meter;set esp_sample_station.SampleYoYoMinDepth 3 meter" 2h89b 1 2.000000 2020-10-13T04:57:49.735Z,1602565069.735 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2h89b 2020-10-13T04:57:49.736Z,1602565069.736 [CommandLine](IMPORTANT): Scheduled #12 (#1 of 2 with id='2h89b'): "load Science/esp_sample_station.xml;set esp_sample_station.MissionTimeout 16 hour;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.TransitYoYoMinDepth 3 meter;set esp_sample_station.SampleYoYoMinDepth 3 meter" 2020-10-13T04:58:15.502Z,1602565095.502 [CommandLine](IMPORTANT): got command load ./Missions/Science/isotherm_depth_canon.xml 2020-10-13T04:58:15.997Z,1602565095.997 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-10-13T04:58:28.134Z,1602565108.134 [CommandLine](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_canon.xml 2020-10-13T04:58:28.236Z,1602565108.236 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T04:58:45.593Z,1602565125.593 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.MissionTimeout 3.000000 hour 2020-10-13T04:58:45.594Z,1602565125.594 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.InitialDiveDuration 2.000000 minute 2020-10-13T04:58:45.595Z,1602565125.595 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SpeedSampling 0.000000 meter_per_second 2020-10-13T04:59:14.866Z,1602565154.866 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.ShallowBoundChl 3.000000 meter 2020-10-13T04:59:14.866Z,1602565154.866 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMinDepth 3.000000 meter 2020-10-13T04:59:14.867Z,1602565154.867 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMaxDepth 50.000000 meter 2020-10-13T04:59:14.868Z,1602565154.868 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lat1 36.765999 degree 2020-10-13T04:59:14.869Z,1602565154.869 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lon1 -121.864899 degree 2020-10-13T04:59:35.518Z,1602565175.518 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleType1 2.000000 count 2020-10-13T04:59:35.518Z,1602565175.518 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TempDiffFromPeakChl1 0.000000 kelvin 2020-10-13T04:59:35.519Z,1602565175.519 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.WaitDepUndulation1 20.000000 minute 2020-10-13T04:59:35.520Z,1602565175.520 [CommandLine](IMPORTANT): got command run 2020-10-13T04:59:35.522Z,1602565175.522 [CommandLine](IMPORTANT): Running 2020-10-13T04:59:35.616Z,1602565175.616 [MissionManager](IMPORTANT): Started mission IsothermDepthCANON 2020-10-13T04:59:35.674Z,1602565175.674 [IsothermDepthCANON:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-10-13T05:00:42.260Z,1602565242.260 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40227, MT Status=1, MTMSN=2800 2020-10-13T05:00:42.834Z,1602565242.834 [CommandLine](IMPORTANT): got command schedule "set esp_sample_station.Speed 0.75 meter_per_second;run " 2h89b 2 2.000000 2020-10-13T05:00:42.835Z,1602565242.835 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2h89b 2020-10-13T05:00:42.835Z,1602565242.835 [CommandLine](IMPORTANT): Scheduled #13 (#2 of 2 with id='2h89b'): "set esp_sample_station.Speed 0.75 meter_per_second;run " 2020-10-13T05:06:59.283Z,1602565619.283 [IsothermDepthCANON:TransitToSamplingLocationNoComms:Wpt1NoComms:C.Waypoint](IMPORTANT): Reached Waypoint: 36.766000,-121.864905 2020-10-13T05:07:42.542Z,1602565662.542 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.620794 m 2020-10-13T05:07:51.424Z,1602565671.424 [IsothermDepthCANON:SetSampleOptions:Q](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , 0.000000 K , nan m , nan degC , nan m , 20.000000 min . 2020-10-13T05:15:19.087Z,1602566119.087 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 30.021765 m 2020-10-13T05:20:30.599Z,1602566430.599 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 28.362465 m , 0.757100 ug/l , 9.765588 m , 13.897879 degC , 7.978359 ug/l 2020-10-13T05:20:33.407Z,1602566433.407 [IsothermDepthCANON:TakeSamples:FiringOnTemp:C](IMPORTANT): Taking sample No. 1.000000 count . Target temp = 13.897879 degC . 2020-10-13T05:20:34.223Z,1602566434.223 [IsothermDepthCANON:DampDown:A](IMPORTANT): Wait 20.000000 min for the depth undulation to damp down. 2020-10-13T05:40:37.801Z,1602567637.801 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T05:40:49.561Z,1602567649.561 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T05:40:49.670Z,1602567649.670 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T05:41:29.045Z,1602567689.045 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T05:41:29.856Z,1602567689.856 [ESPComponent](IMPORTANT): [sample #4] Setting cartridge argument to -6 2020-10-13T05:41:29.864Z,1602567689.864 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T05:41:30.321Z,1602567690.321 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 14.003874 m in 13.985193 degC water with 9.435500 ug/l chlorophyll fluorescence. 2020-10-13T05:41:30.662Z,1602567690.662 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T05:41:31.066Z,1602567691.066 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T05:41:33.086Z,1602567693.086 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T05:42:06.217Z,1602567726.217 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PRIMING (@28,archive_bac) 2020-10-13T05:42:07.834Z,1602567727.834 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T05:42:09.450Z,1602567729.450 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T05:44:37.733Z,1602567877.733 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T05:44:38.141Z,1602567878.141 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T05:50:56.718Z,1602568256.718 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PAUSED (@106.4) 2020-10-13T05:50:57.124Z,1602568257.124 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PROCESSING (@106.4) 2020-10-13T05:50:59.546Z,1602568259.546 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PROCESSED (@28,archive_bac) 2020-10-13T05:51:10.953Z,1602568270.953 [IsothermDepthCANON:Sample:SampleWrapper:CANONSampler:AcquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 7.715196 m in 13.939600 degC water with 11.062700 ug/l chlorophyll fluorescence. 2020-10-13T05:55:02.778Z,1602568502.778 [CANONSampler](IMPORTANT): Sample 2, err_code=0 2020-10-13T06:03:03.150Z,1602568983.150 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPING (@28,archive_bac) 2020-10-13T06:03:28.606Z,1602569008.606 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T06:03:29.010Z,1602569009.010 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPED 2020-10-13T06:08:28.798Z,1602569308.798 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence completed normally. 2020-10-13T06:08:29.203Z,1602569309.203 [ESPComponent](IMPORTANT): [sample #4] ESP log summary report (3 messages): @22:41:42.89 Selecting Cartridge 28 @22:50:53.92 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @22:50:54.12 Sampled 106.4ml 2020-10-13T06:08:31.675Z,1602569311.675 [IsothermDepthCANON:TakeSamples:FiringOnTemp:TriggerOnTemp:C](IMPORTANT): Sample No. 1.000000 count completed. 2020-10-13T06:08:33.838Z,1602569313.838 [CommandLine](IMPORTANT): got command load ./Missions/Science/isotherm_depth_canon.xml 2020-10-13T06:08:34.261Z,1602569314.261 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-10-13T06:08:46.515Z,1602569326.515 [CommandLine](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_canon.xml 2020-10-13T06:08:46.630Z,1602569326.630 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T06:08:59.499Z,1602569339.499 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.MissionTimeout 3.000000 hour 2020-10-13T06:08:59.500Z,1602569339.500 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.InitialDiveDuration 2.000000 minute 2020-10-13T06:08:59.500Z,1602569339.500 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SpeedSampling 0.000000 meter_per_second 2020-10-13T06:09:19.030Z,1602569359.030 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.ShallowBoundChl 3.000000 meter 2020-10-13T06:09:19.033Z,1602569359.033 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMinDepth 3.000000 meter 2020-10-13T06:09:19.034Z,1602569359.034 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMaxDepth 50.000000 meter 2020-10-13T06:09:19.035Z,1602569359.035 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lat1 36.797001 degree 2020-10-13T06:09:19.036Z,1602569359.036 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lon1 -121.847000 degree 2020-10-13T06:09:32.051Z,1602569372.051 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleType1 2.000000 count 2020-10-13T06:09:32.052Z,1602569372.052 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TempDiffFromPeakChl1 0.000000 kelvin 2020-10-13T06:09:32.053Z,1602569372.053 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.WaitDepUndulation1 20.000000 minute 2020-10-13T06:09:32.053Z,1602569372.053 [CommandLine](IMPORTANT): got command run 2020-10-13T06:09:32.056Z,1602569372.056 [CommandLine](IMPORTANT): Running 2020-10-13T06:09:32.291Z,1602569372.291 [MissionManager](IMPORTANT): Started mission IsothermDepthCANON 2020-10-13T06:09:32.351Z,1602569372.351 [IsothermDepthCANON:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-10-13T06:10:04.606Z,1602569404.606 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40228, MT Status=1, MTMSN=2801 2020-10-13T06:10:05.097Z,1602569405.097 [CommandLine](IMPORTANT): got command schedule list 2020-10-13T06:10:05.101Z,1602569405.101 [CommandLine](IMPORTANT): Scheduled #12 (#1 of 2 with id='2h89b'): "load Science/esp_sample_station.xml;set esp_sample_station.MissionTimeout 16 hour;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.TransitYoYoMinDepth 3 meter;set esp_sample_station.SampleYoYoMinDepth 3 meter" 2020-10-13T06:10:05.101Z,1602569405.101 [CommandLine](IMPORTANT): Scheduled #13 (#2 of 2 with id='2h89b'): "set esp_sample_station.Speed 0.75 meter_per_second;run " 2020-10-13T07:11:18.150Z,1602573078.150 [IsothermDepthCANON:TransitToSamplingLocationNoComms:Wpt1NoComms:C.Waypoint](IMPORTANT): Reached Waypoint: 36.797002,-121.846996 2020-10-13T07:12:10.275Z,1602573130.275 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.675092 m 2020-10-13T07:12:19.167Z,1602573139.167 [IsothermDepthCANON:SetSampleOptions:Q](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , 0.000000 K , nan m , nan degC , nan m , 20.000000 min . 2020-10-13T07:19:10.878Z,1602573550.878 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 30.032450 m 2020-10-13T07:25:03.575Z,1602573903.575 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 28.419817 m , 1.062200 ug/l , 5.495293 m , 13.926044 degC , 4.214895 ug/l 2020-10-13T07:25:06.438Z,1602573906.438 [IsothermDepthCANON:TakeSamples:FiringOnTemp:C](IMPORTANT): Taking sample No. 1.000000 count . Target temp = 13.926044 degC . 2020-10-13T07:25:07.239Z,1602573907.239 [IsothermDepthCANON:DampDown:A](IMPORTANT): Wait 20.000000 min for the depth undulation to damp down. 2020-10-13T07:45:10.803Z,1602575110.803 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T07:45:22.557Z,1602575122.557 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T07:45:22.662Z,1602575122.662 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T07:46:01.236Z,1602575161.236 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T07:46:02.044Z,1602575162.044 [ESPComponent](IMPORTANT): [sample #5] Setting cartridge argument to -6 2020-10-13T07:46:02.045Z,1602575162.045 [ESPComponent](IMPORTANT): [sample #5] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T07:46:02.518Z,1602575162.518 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 4.076693 m in 13.917200 degC water with 4.723400 ug/l chlorophyll fluorescence. 2020-10-13T07:46:02.850Z,1602575162.850 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T07:46:03.253Z,1602575163.253 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T07:46:05.273Z,1602575165.273 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T07:46:38.406Z,1602575198.406 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_PRIMING (@27,archive_bac) 2020-10-13T07:46:40.023Z,1602575200.023 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T07:46:41.638Z,1602575201.638 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T07:49:10.317Z,1602575350.317 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T07:49:10.722Z,1602575350.722 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T07:55:43.942Z,1602575743.942 [IsothermDepthCANON:Sample:SampleWrapper:CANONSampler:AcquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 3.161228 m in 14.037714 degC water with 5.650000 ug/l chlorophyll fluorescence. 2020-10-13T07:59:34.550Z,1602575974.550 [CANONSampler](IMPORTANT): Sample 3, err_code=0 2020-10-13T08:04:18.575Z,1602576258.575 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PAUSED (@263.082) 2020-10-13T08:04:18.978Z,1602576258.978 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PROCESSING (@263.082) 2020-10-13T08:04:21.401Z,1602576261.401 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_PROCESSED (@27,archive_bac) 2020-10-13T08:16:25.410Z,1602576985.410 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_STOPPING (@27,archive_bac) 2020-10-13T08:16:50.863Z,1602577010.863 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T08:16:51.266Z,1602577011.266 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_STOPPED 2020-10-13T08:21:51.045Z,1602577311.045 [ESPComponent](IMPORTANT): [sample #5] ESP sampling sequence completed normally. 2020-10-13T08:21:51.450Z,1602577311.450 [ESPComponent](IMPORTANT): [sample #5] ESP log summary report (3 messages): @00:46:15.50 Selecting Cartridge 27 @01:04:16.08 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @01:04:16.27 Sampled 263.1ml 2020-10-13T08:21:53.934Z,1602577313.934 [IsothermDepthCANON:TakeSamples:FiringOnTemp:TriggerOnTemp:C](IMPORTANT): Sample No. 1.000000 count completed. 2020-10-13T08:21:56.041Z,1602577316.041 [CommandLine](IMPORTANT): got command load ./Missions/Science/esp_sample_station.xml 2020-10-13T08:21:56.717Z,1602577316.717 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-10-13T08:21:58.638Z,1602577318.638 [CommandLine](IMPORTANT): Loaded ./Missions/Science/esp_sample_station.xml 2020-10-13T08:21:58.756Z,1602577318.756 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T08:22:16.011Z,1602577336.011 [CommandLine](IMPORTANT): got command set esp_sample_station.MissionTimeout 16.000000 hour 2020-10-13T08:22:16.011Z,1602577336.011 [CommandLine](IMPORTANT): got command set esp_sample_station.NeedCommsTime 60.000000 minute 2020-10-13T08:22:16.012Z,1602577336.012 [CommandLine](IMPORTANT): got command set esp_sample_station.TransitYoYoMinDepth 3.000000 meter 2020-10-13T08:22:16.013Z,1602577336.013 [CommandLine](IMPORTANT): got command set esp_sample_station.SampleYoYoMinDepth 3.000000 meter 2020-10-13T08:22:19.374Z,1602577339.374 [CommandLine](IMPORTANT): got command set esp_sample_station.Speed 0.750000 meter_per_second 2020-10-13T08:22:19.375Z,1602577339.375 [CommandLine](IMPORTANT): got command run 2020-10-13T08:22:19.381Z,1602577339.381 [CommandLine](IMPORTANT): Running 2020-10-13T08:22:19.712Z,1602577339.712 [MissionManager](IMPORTANT): Started mission esp_sample_station 2020-10-13T08:25:55.424Z,1602577555.424 [esp_sample_station:Lap:A](IMPORTANT): Starting Lap 2020-10-13T08:25:55.931Z,1602577555.931 [esp_sample_station:Lap:B](IMPORTANT): Lat[1] = 36.797000 arcdeg Lon[1] = -121.847000 arcdeg 2020-10-13T08:25:56.260Z,1602577556.260 [esp_sample_station:Lap:C](IMPORTANT): Lat[2] = 36.821900 arcdeg Lon[2] = -121.885600 arcdeg 2020-10-13T08:25:56.670Z,1602577556.670 [esp_sample_station:Lap:D](IMPORTANT): Lat[3] = 36.766000 arcdeg Lon[3] = -121.864900 arcdeg 2020-10-13T08:25:57.048Z,1602577557.048 [esp_sample_station:Lap:E](IMPORTANT): Lat[4] = nan arcdeg Lon[4] = nan arcdeg 2020-10-13T08:26:01.511Z,1602577561.511 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.797000 arcdeg , -121.847000 arcdeg 2020-10-13T08:31:02.125Z,1602577862.125 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-13T08:31:13.024Z,1602577873.024 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005798 CHAN A1 (24V): 0.003803 CHAN A2 (12V): -0.004160 CHAN A3 (5V): -0.002640 CHAN B0 (3.3V): -0.001320 CHAN B1 (3.15aV): -0.000707 CHAN B2 (3.15bV): -0.001155 CHAN B3 (GND): 0.000223 OPEN: 0.007465 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-13T08:55:29.469Z,1602579329.469 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.797000,-121.847000 2020-10-13T09:01:52.388Z,1602579712.388 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T09:02:04.173Z,1602579724.173 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T09:02:04.365Z,1602579724.365 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T09:02:42.553Z,1602579762.553 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T09:02:43.365Z,1602579763.365 [ESPComponent](IMPORTANT): [sample #6] Setting cartridge argument to -6 2020-10-13T09:02:43.365Z,1602579763.365 [ESPComponent](IMPORTANT): [sample #6] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T09:02:43.825Z,1602579763.825 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 11.196981 m in 14.193689 degC water with 5.785600 ug/l chlorophyll fluorescence. 2020-10-13T09:02:44.170Z,1602579764.170 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T09:02:44.573Z,1602579764.573 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T09:02:46.594Z,1602579766.594 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T09:03:19.730Z,1602579799.730 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PRIMING (@26,archive_bac) 2020-10-13T09:03:21.341Z,1602579801.341 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T09:03:22.957Z,1602579802.957 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T09:06:14.266Z,1602579974.266 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T09:06:14.670Z,1602579974.670 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T10:04:49.684Z,1602583489.684 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PAUSED (@990.616) 2020-10-13T10:04:50.086Z,1602583490.086 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PROCESSING (@990.616) 2020-10-13T10:04:52.510Z,1602583492.510 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PROCESSED (@25,archive_bac) 2020-10-13T10:07:51.131Z,1602583671.131 [BPC1](FAULT): Failed to receive data from 2 sticks prior to timeout. Missing stick IDs are: 38, 39. 2020-10-13T10:16:56.118Z,1602584216.118 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_STOPPING (@25,archive_bac) 2020-10-13T10:17:21.570Z,1602584241.570 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T10:17:21.974Z,1602584241.974 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_STOPPED 2020-10-13T10:22:21.757Z,1602584541.757 [ESPComponent](IMPORTANT): [sample #6] ESP sampling sequence completed normally. 2020-10-13T10:22:22.162Z,1602584542.162 [ESPComponent](IMPORTANT): [sample #6] ESP log summary report (6 messages): @02:02:56.79 Selecting Cartridge 26 @02:03:27.43 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 30% to 29.1psia @02:03:27.44 Retry #1 of 2 @02:03:27.49 Selecting Cartridge 25 @03:04:47.24 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @03:04:47.44 Sampled 990.6ml 2020-10-13T10:27:49.926Z,1602584869.926 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.821900 arcdeg , -121.885600 arcdeg 2020-10-13T11:33:21.178Z,1602588801.178 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T113319 2020-10-13T11:33:29.922Z,1602588809.922 [CommandLine](IMPORTANT): got command ! start onESPclient pitz -- slots archive_bac:3 2020-10-13T11:33:30.331Z,1602588810.331 [CommandLine](IMPORTANT): /mnt/mmc/started/2020-10-13UTC113330_onESPclient 2020-10-13T12:34:02.500Z,1602592442.500 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.821900,-121.885600 2020-10-13T12:36:39.642Z,1602592599.642 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40263, MT Status=1, MTMSN=2802 2020-10-13T12:36:40.178Z,1602592600.178 [CommandLine](IMPORTANT): got command ! check 2020-10-13T12:36:40.495Z,1602592600.495 [CommandLine](IMPORTANT): 40 => { Type: :archive_bac, State: :processed, filtered: 937.276}, 42 => { Type: :archive_bac, State: :processed, filtered: 952.165}, 44 => { Type: :archive_bac, State: :processed, filtered: 963.627}} Tue Oct 13 11:34:16 UTC 2020 -- Shutting down ESP in 20 seconds ... shutting ESP off 2020-10-13T12:42:11.079Z,1602592931.079 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T12:42:22.877Z,1602592942.877 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T12:42:22.989Z,1602592942.989 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T12:43:01.598Z,1602592981.598 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T12:43:02.408Z,1602592982.408 [ESPComponent](IMPORTANT): [sample #7] Setting cartridge argument to -6 2020-10-13T12:43:02.409Z,1602592982.409 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T12:43:02.859Z,1602592982.859 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 12.839787 m in 12.373713 degC water with 0.711900 ug/l chlorophyll fluorescence. 2020-10-13T12:43:03.219Z,1602592983.219 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T12:43:03.623Z,1602592983.623 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T12:43:05.637Z,1602592985.637 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T12:43:38.766Z,1602593018.766 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PRIMING (@24,archive_bac) 2020-10-13T12:43:40.382Z,1602593020.382 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T12:43:41.189Z,1602593021.189 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T12:46:37.350Z,1602593197.350 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T12:46:37.754Z,1602593197.754 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T13:34:32.767Z,1602596072.767 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PAUSED (@846.745) 2020-10-13T13:34:33.170Z,1602596073.170 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PROCESSING (@846.745) 2020-10-13T13:34:35.594Z,1602596075.594 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PROCESSED (@23,archive_bac) 2020-10-13T13:46:39.198Z,1602596799.198 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPING (@23,archive_bac) 2020-10-13T13:47:04.650Z,1602596824.650 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T13:47:05.055Z,1602596825.055 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPED 2020-10-13T13:52:04.838Z,1602597124.838 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence completed normally. 2020-10-13T13:52:05.244Z,1602597125.244 [ESPComponent](IMPORTANT): [sample #7] ESP log summary report (6 messages): @05:43:15.55 Selecting Cartridge 24 @05:43:50.12 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 31% to 32.6psia @05:43:50.14 Retry #1 of 2 @05:43:50.18 Selecting Cartridge 23 @06:34:30.21 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @06:34:30.41 Sampled 846.7ml 2020-10-13T13:57:49.361Z,1602597469.361 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.766000 arcdeg , -121.864900 arcdeg 2020-10-13T14:31:13.365Z,1602599473.365 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-13T14:31:24.307Z,1602599484.307 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005731 CHAN A1 (24V): 0.003125 CHAN A2 (12V): -0.004230 CHAN A3 (5V): -0.002374 CHAN B0 (3.3V): -0.000966 CHAN B1 (3.15aV): -0.000899 CHAN B2 (3.15bV): -0.001246 CHAN B3 (GND): -0.000271 OPEN: 0.007548 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-13T15:00:44.567Z,1602601244.567 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40276, MT Status=1, MTMSN=2803 2020-10-13T15:00:45.370Z,1602601245.370 [CommandLine](IMPORTANT): got command configSet BPC1.batteryMissingStickThreshold 2.000000 count persist 2020-10-13T17:01:07.790Z,1602608467.790 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.766000,-121.864900 2020-10-13T17:15:28.357Z,1602609328.357 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T17:15:40.141Z,1602609340.141 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T17:15:40.441Z,1602609340.441 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T17:16:18.630Z,1602609378.630 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T17:16:19.442Z,1602609379.442 [ESPComponent](IMPORTANT): [sample #8] Setting cartridge argument to -6 2020-10-13T17:16:19.443Z,1602609379.443 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T17:16:19.877Z,1602609379.877 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 13.223519 m in 12.623011 degC water with 1.254300 ug/l chlorophyll fluorescence. 2020-10-13T17:16:20.247Z,1602609380.247 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T17:16:20.649Z,1602609380.649 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T17:16:22.670Z,1602609382.670 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T17:16:55.799Z,1602609415.799 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PRIMING (@22,archive_bac) 2020-10-13T17:16:57.418Z,1602609417.418 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T17:16:58.221Z,1602609418.221 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T17:19:25.686Z,1602609565.686 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T17:19:26.089Z,1602609566.089 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T17:52:51.638Z,1602611571.638 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PAUSED (@563.7) 2020-10-13T17:52:52.034Z,1602611572.034 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PROCESSING (@563.7) 2020-10-13T17:52:54.458Z,1602611574.458 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PROCESSED (@22,archive_bac) 2020-10-13T18:04:58.061Z,1602612298.061 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPING (@22,archive_bac) 2020-10-13T18:05:23.520Z,1602612323.520 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T18:05:23.922Z,1602612323.922 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPED 2020-10-13T18:10:23.716Z,1602612623.716 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence completed normally. 2020-10-13T18:10:24.118Z,1602612624.118 [ESPComponent](IMPORTANT): [sample #8] ESP log summary report (3 messages): @10:16:32.70 Selecting Cartridge 22 @10:52:48.83 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @10:52:49.02 Sampled 563.7ml 2020-10-13T18:13:23.940Z,1602612803.940 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40302, MT Status=1, MTMSN=2804 2020-10-13T18:13:24.445Z,1602612804.445 [CommandLine](IMPORTANT): got command stop 2020-10-13T18:13:24.445Z,1602612804.445 [CommandLine](IMPORTANT): Scheduling is paused 2020-10-13T18:13:24.749Z,1602612804.749 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T18:14:02.787Z,1602612842.787 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40303, MT Status=1, MTMSN=2805 2020-10-13T18:14:03.503Z,1602612843.503 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.MissionTimeout 12 hour;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree" 2i9pr 1 2.000000 2020-10-13T18:14:03.503Z,1602612843.503 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2i9pr 2020-10-13T18:14:03.504Z,1602612843.504 [CommandLine](IMPORTANT): Scheduled #14 (#1 of 2 with id='2i9pr'): "load Science/profile_station.xml;set profile_station.MissionTimeout 12 hour;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree" ASAP 2020-10-13T18:14:52.709Z,1602612892.709 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40304, MT Status=1, MTMSN=2806 2020-10-13T18:14:53.478Z,1602612893.478 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.Radius 2000 meter;set profile_station.YoYoMinDepth 3 meter;set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.75 meter_per_second;set profile_station.MaxDepth 55 meter;run" 2i9pr 2 2.000000 2020-10-13T18:14:53.479Z,1602612893.479 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2i9pr 2020-10-13T18:14:53.479Z,1602612893.479 [CommandLine](IMPORTANT): Scheduled #15 (#2 of 2 with id='2i9pr'): "set profile_station.Radius 2000 meter;set profile_station.YoYoMinDepth 3 meter;set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.75 meter_per_second;set profile_station.MaxDepth 55 meter;run" ASAP 2020-10-13T18:14:53.642Z,1602612893.642 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-10-13T18:14:55.333Z,1602612895.333 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-10-13T18:15:07.734Z,1602612907.734 [CommandLine](IMPORTANT): got command set profile_station.MissionTimeout 12.000000 hour 2020-10-13T18:15:07.735Z,1602612907.735 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 30.000000 minute 2020-10-13T18:15:07.735Z,1602612907.735 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-10-13T18:15:07.736Z,1602612907.736 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-10-13T18:15:23.831Z,1602612923.831 [CommandLine](IMPORTANT): got command set profile_station.Radius 2000.000000 meter 2020-10-13T18:15:23.831Z,1602612923.831 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 3.000000 meter 2020-10-13T18:15:23.837Z,1602612923.837 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2020-10-13T18:15:23.838Z,1602612923.838 [CommandLine](IMPORTANT): got command set profile_station.Speed 0.750000 meter_per_second 2020-10-13T18:15:23.839Z,1602612923.839 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 55.000000 meter 2020-10-13T18:15:23.839Z,1602612923.839 [CommandLine](IMPORTANT): got command run 2020-10-13T18:15:23.844Z,1602612923.844 [MissionManager](IMPORTANT): Started mission profile_station 2020-10-13T18:15:23.846Z,1602612923.846 [CommandLine](IMPORTANT): Running 2020-10-13T18:52:03.497Z,1602615123.497 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40310, MT Status=1, MTMSN=2807 2020-10-13T18:52:04.089Z,1602615124.089 [CommandLine](IMPORTANT): got command stop 2020-10-13T18:52:04.732Z,1602615124.732 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T18:53:36.632Z,1602615216.632 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40311, MT Status=1, MTMSN=2808 2020-10-13T18:53:37.386Z,1602615217.386 [CommandLine](IMPORTANT): got command schedule asap "load Science/sci2.xml;set sci2.MissionTimeout 8 hour;set sci2.NeedCommsTime 30 minute;set sci2.Lat1 36.797 degree;set sci2.Lon1 -121.847 degree;set sci2.Lat2 nan degree;set sci2.Lon2 nan degree;set sci2.YoYoMinDepth 3 meter" 2ibha 1 2.000000 2020-10-13T18:53:37.386Z,1602615217.386 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2ibha 2020-10-13T18:53:37.387Z,1602615217.387 [CommandLine](IMPORTANT): Scheduled #16 (#1 of 2 with id='2ibha'): "load Science/sci2.xml;set sci2.MissionTimeout 8 hour;set sci2.NeedCommsTime 30 minute;set sci2.Lat1 36.797 degree;set sci2.Lon1 -121.847 degree;set sci2.Lat2 nan degree;set sci2.Lon2 nan degree;set sci2.YoYoMinDepth 3 meter" ASAP 2020-10-13T18:54:05.362Z,1602615245.362 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40312, MT Status=1, MTMSN=2809 2020-10-13T18:54:06.002Z,1602615246.002 [CommandLine](IMPORTANT): got command schedule asap "set sci2.YoYoMaxDepth 50 meter;set sci2.MaxDepth 55 meter;run" 2ibha 2 2.000000 2020-10-13T18:54:06.002Z,1602615246.002 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2ibha 2020-10-13T18:54:06.003Z,1602615246.003 [CommandLine](IMPORTANT): Scheduled #17 (#2 of 2 with id='2ibha'): "set sci2.YoYoMaxDepth 50 meter;set sci2.MaxDepth 55 meter;run" ASAP 2020-10-13T18:54:06.264Z,1602615246.264 [CommandLine](IMPORTANT): got command load ./Missions/Science/sci2.xml 2020-10-13T18:54:08.111Z,1602615248.111 [CommandLine](IMPORTANT): Loaded ./Missions/Science/sci2.xml 2020-10-13T18:54:21.389Z,1602615261.389 [CommandLine](IMPORTANT): got command set sci2.MissionTimeout 8.000000 hour 2020-10-13T18:54:21.390Z,1602615261.390 [CommandLine](IMPORTANT): got command set sci2.NeedCommsTime 30.000000 minute 2020-10-13T18:54:21.390Z,1602615261.390 [CommandLine](IMPORTANT): got command set sci2.Lat1 36.797001 degree 2020-10-13T18:54:21.391Z,1602615261.391 [CommandLine](IMPORTANT): got command set sci2.Lon1 -121.847000 degree 2020-10-13T18:54:21.392Z,1602615261.392 [CommandLine](IMPORTANT): got command set sci2.Lat2 nan degree 2020-10-13T18:54:21.393Z,1602615261.393 [CommandLine](IMPORTANT): got command set sci2.Lon2 nan degree 2020-10-13T18:54:21.394Z,1602615261.394 [CommandLine](IMPORTANT): got command set sci2.YoYoMinDepth 3.000000 meter 2020-10-13T18:54:25.445Z,1602615265.445 [CommandLine](IMPORTANT): got command set sci2.YoYoMaxDepth 50.000000 meter 2020-10-13T18:54:25.445Z,1602615265.445 [CommandLine](IMPORTANT): got command set sci2.MaxDepth 55.000000 meter 2020-10-13T18:54:25.446Z,1602615265.446 [CommandLine](IMPORTANT): got command run 2020-10-13T18:54:25.450Z,1602615265.450 [CommandLine](IMPORTANT): Running 2020-10-13T18:54:25.690Z,1602615265.690 [MissionManager](IMPORTANT): Started mission sci2 2020-10-13T19:34:04.935Z,1602617644.935 [sci2:Lap:Wpt1:A.Waypoint](IMPORTANT): Reached Waypoint: 36.797002,-121.846996 2020-10-13T19:40:13.814Z,1602618013.814 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T19:54:33.614Z,1602618873.614 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 14.323160 min 2020-10-13T19:59:15.835Z,1602619155.835 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T195913 2020-10-13T19:59:23.313Z,1602619163.313 [CommandLine](IMPORTANT): got command ! start onESPclient preston -- slots archive_bac:3 2020-10-13T19:59:23.584Z,1602619163.584 [CommandLine](IMPORTANT): /mnt/mmc/started/2020-10-13UTC195923_onESPclient 2020-10-13T20:00:06.156Z,1602619206.156 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 19.865346 min 2020-10-13T20:01:52.657Z,1602619312.657 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T200150 2020-10-13T20:02:00.085Z,1602619320.085 [CommandLine](IMPORTANT): got command ! check 2020-10-13T20:02:00.344Z,1602619320.344 [CommandLine](IMPORTANT): 40 => { Type: :archive_bac, State: :processed, filtered: 937.276}, 42 => { Type: :archive_bac, State: :processed, filtered: 952.165}, 44 => { Type: :archive_bac, State: :processed, filtered: 963.627}} Tue Oct 13 20:00:08 UTC 2020 -- Shutting down ESP in 20 seconds ... shutting ESP off 2020-10-13T20:05:45.180Z,1602619545.180 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 25.515955 min 2020-10-13T20:11:41.554Z,1602619901.554 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 31.455593 min 2020-10-13T20:17:21.722Z,1602620241.722 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 37.125175 min 2020-10-13T20:20:23.456Z,1602620423.456 [CommandLine](IMPORTANT): got command ! check 2020-10-13T20:20:23.684Z,1602620423.684 [CommandLine](IMPORTANT): 40 => { Type: :archive_bac, State: :processed, filtered: 937.276}, 42 => { Type: :archive_bac, State: :processed, filtered: 952.165}, 44 => { Type: :archive_bac, State: :processed, filtered: 963.627}} Tue Oct 13 20:00:08 UTC 2020 -- Shutting down ESP in 20 seconds ... shutting ESP off 2020-10-13T20:22:52.614Z,1602620572.614 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 42.639994 min 2020-10-13T20:28:31.600Z,1602620911.600 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 48.289311 min 2020-10-13T20:31:24.492Z,1602621084.492 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-13T20:31:35.412Z,1602621095.412 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005214 CHAN A1 (24V): 0.003312 CHAN A2 (12V): -0.004119 CHAN A3 (5V): -0.002345 CHAN B0 (3.3V): -0.000773 CHAN B1 (3.15aV): -0.001252 CHAN B2 (3.15bV): -0.000803 CHAN B3 (GND): -0.000619 OPEN: 0.007328 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-13T20:34:03.659Z,1602621243.659 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 53.824064 min 2020-10-13T20:34:40.180Z,1602621280.180 [DataOverHttps](IMPORTANT): SBD MOMSN=12891841, MTMSN=20201013T203438 2020-10-13T20:34:48.634Z,1602621288.634 [CommandLine](IMPORTANT): got command schedule asap "load Science/canon_esp_trianguloid.xml;set canon_esp_trianguloid.MissionTimeout 90 minute;set canon_esp_trianguloid.SampleInterval 60 minute;set canon_esp_trianguloid.SampleCANONSampler 1 bool" 2ih5k 1 5.000000 2020-10-13T20:34:48.634Z,1602621288.634 [CommandLine](IMPORTANT): Scheduling command #1 of 5 with id=2ih5k 2020-10-13T20:34:48.635Z,1602621288.635 [CommandLine](IMPORTANT): Scheduled #18 (#1 of 5 with id='2ih5k'): "load Science/canon_esp_trianguloid.xml;set canon_esp_trianguloid.MissionTimeout 90 minute;set canon_esp_trianguloid.SampleInterval 60 minute;set canon_esp_trianguloid.SampleCANONSampler 1 bool" ASAP 2020-10-13T20:34:49.595Z,1602621289.595 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T203447 2020-10-13T20:34:56.934Z,1602621296.934 [CommandLine](IMPORTANT): got command schedule asap "set canon_esp_trianguloid.Lat2 36.797 degree;set canon_esp_trianguloid.Lon2 -121.865 degree;set canon_esp_trianguloid.Lat3 36.797 degree;set canon_esp_trianguloid.Lon3 -121.847 degree;set canon_esp_trianguloid.Lat4 NaN degree" 2ih5k 2 5.000000 2020-10-13T20:34:56.935Z,1602621296.935 [CommandLine](IMPORTANT): Scheduling command #2 of 5 with id=2ih5k 2020-10-13T20:34:56.936Z,1602621296.936 [CommandLine](IMPORTANT): Scheduled #19 (#2 of 5 with id='2ih5k'): "set canon_esp_trianguloid.Lat2 36.797 degree;set canon_esp_trianguloid.Lon2 -121.865 degree;set canon_esp_trianguloid.Lat3 36.797 degree;set canon_esp_trianguloid.Lon3 -121.847 degree;set canon_esp_trianguloid.Lat4 NaN degree" ASAP 2020-10-13T20:34:57.906Z,1602621297.906 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T203455 2020-10-13T20:35:05.334Z,1602621305.334 [CommandLine](IMPORTANT): got command schedule asap "set canon_esp_trianguloid.Lon4 NaN degree;set canon_esp_trianguloid.Lat5 NaN degree;set canon_esp_trianguloid.Lon5 NaN degree;set canon_esp_trianguloid.Lat6 NaN degree;set canon_esp_trianguloid.Lon6 NaN degree" 2ih5k 3 5.000000 2020-10-13T20:35:05.334Z,1602621305.334 [CommandLine](IMPORTANT): Scheduling command #3 of 5 with id=2ih5k 2020-10-13T20:35:05.335Z,1602621305.335 [CommandLine](IMPORTANT): Scheduled #20 (#3 of 5 with id='2ih5k'): "set canon_esp_trianguloid.Lon4 NaN degree;set canon_esp_trianguloid.Lat5 NaN degree;set canon_esp_trianguloid.Lon5 NaN degree;set canon_esp_trianguloid.Lat6 NaN degree;set canon_esp_trianguloid.Lon6 NaN degree" ASAP 2020-10-13T20:35:06.306Z,1602621306.306 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T203504 2020-10-13T20:35:13.835Z,1602621313.835 [CommandLine](IMPORTANT): got command schedule asap "set canon_esp_trianguloid.Lat7 NaN degree;set canon_esp_trianguloid.Lon7 NaN degree;set canon_esp_trianguloid.Speed 0.75 meter_per_second;set canon_esp_trianguloid.YoYoMinDepth 3 meter" 2ih5k 4 5.000000 2020-10-13T20:35:13.835Z,1602621313.835 [CommandLine](IMPORTANT): Scheduling command #4 of 5 with id=2ih5k 2020-10-13T20:35:13.836Z,1602621313.836 [CommandLine](IMPORTANT): Scheduled #21 (#4 of 5 with id='2ih5k'): "set canon_esp_trianguloid.Lat7 NaN degree;set canon_esp_trianguloid.Lon7 NaN degree;set canon_esp_trianguloid.Speed 0.75 meter_per_second;set canon_esp_trianguloid.YoYoMinDepth 3 meter" ASAP 2020-10-13T20:35:14.810Z,1602621314.810 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T203512 2020-10-13T20:35:22.142Z,1602621322.142 [CommandLine](IMPORTANT): got command schedule asap "set canon_esp_trianguloid.YoYoMaxDepth1 50 meter;set canon_esp_trianguloid.MaxDepth 62 meter;run" 2ih5k 5 5.000000 2020-10-13T20:35:22.142Z,1602621322.142 [CommandLine](IMPORTANT): Scheduling command #5 of 5 with id=2ih5k 2020-10-13T20:35:22.143Z,1602621322.143 [CommandLine](IMPORTANT): Scheduled #22 (#5 of 5 with id='2ih5k'): "set canon_esp_trianguloid.YoYoMaxDepth1 50 meter;set canon_esp_trianguloid.MaxDepth 62 meter;run" ASAP 2020-10-13T20:35:22.460Z,1602621322.460 [CommandLine](IMPORTANT): got command load ./Missions/Science/canon_esp_trianguloid.xml 2020-10-13T20:35:25.733Z,1602621325.733 [CommandLine](IMPORTANT): Loaded ./Missions/Science/canon_esp_trianguloid.xml 2020-10-13T20:35:41.512Z,1602621341.512 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.MissionTimeout 90.000000 minute 2020-10-13T20:35:41.513Z,1602621341.513 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.SampleInterval 60.000000 minute 2020-10-13T20:35:41.514Z,1602621341.514 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.SampleCANONSampler 1.000000 bool 2020-10-13T20:35:57.246Z,1602621357.246 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T203555 2020-10-13T20:36:06.370Z,1602621366.370 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lat2 36.797001 degree 2020-10-13T20:36:06.370Z,1602621366.370 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lon2 -121.864998 degree 2020-10-13T20:36:06.371Z,1602621366.371 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lat3 36.797001 degree 2020-10-13T20:36:06.372Z,1602621366.372 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lon3 -121.847000 degree 2020-10-13T20:36:06.372Z,1602621366.372 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lat4 nan degree 2020-10-13T20:36:26.587Z,1602621386.587 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lon4 nan degree 2020-10-13T20:36:26.588Z,1602621386.588 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lat5 nan degree 2020-10-13T20:36:26.589Z,1602621386.589 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lon5 nan degree 2020-10-13T20:36:26.625Z,1602621386.625 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lat6 nan degree 2020-10-13T20:36:26.626Z,1602621386.626 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lon6 nan degree 2020-10-13T20:36:26.636Z,1602621386.636 [CommandLine](IMPORTANT): got command stop 2020-10-13T20:36:26.639Z,1602621386.639 [CommandLine](IMPORTANT): got command stop 2020-10-13T20:36:47.605Z,1602621407.605 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lat7 nan degree 2020-10-13T20:36:47.606Z,1602621407.606 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Lon7 nan degree 2020-10-13T20:36:47.607Z,1602621407.607 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Speed 0.750000 meter_per_second 2020-10-13T20:36:47.608Z,1602621407.608 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.YoYoMinDepth 3.000000 meter 2020-10-13T20:36:56.662Z,1602621416.662 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.YoYoMaxDepth1 50.000000 meter 2020-10-13T20:36:56.662Z,1602621416.662 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.MaxDepth 62.000000 meter 2020-10-13T20:36:56.663Z,1602621416.663 [CommandLine](IMPORTANT): got command run 2020-10-13T20:36:56.667Z,1602621416.667 [CommandLine](IMPORTANT): Running 2020-10-13T20:36:57.009Z,1602621417.009 [MissionManager](IMPORTANT): Started mission canon_esp_trianguloid 2020-10-13T20:36:57.420Z,1602621417.420 [MissionManager](IMPORTANT): Started mission Default 2020-10-13T20:42:34.478Z,1602621754.478 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T204232 2020-10-13T20:42:42.026Z,1602621762.026 [CommandLine](IMPORTANT): got command schedule asap "load Science/canon_esp_trianguloid.xml;set canon_esp_trianguloid.Speed 0.75 meter_per_second;set canon_esp_trianguloid.YoYoMinDepth 3 meter;set canon_esp_trianguloid.YoYoMaxDepth1 50 meter" 2ihif 1 2.000000 2020-10-13T20:42:42.026Z,1602621762.026 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2ihif 2020-10-13T20:42:42.027Z,1602621762.027 [CommandLine](IMPORTANT): Scheduled #23 (#1 of 2 with id='2ihif'): "load Science/canon_esp_trianguloid.xml;set canon_esp_trianguloid.Speed 0.75 meter_per_second;set canon_esp_trianguloid.YoYoMinDepth 3 meter;set canon_esp_trianguloid.YoYoMaxDepth1 50 meter" ASAP 2020-10-13T20:42:42.978Z,1602621762.978 [DataOverHttps](IMPORTANT): SBD MTMSN=20201013T204241 2020-10-13T20:42:52.850Z,1602621772.850 [CommandLine](IMPORTANT): got command schedule asap "set canon_esp_trianguloid.MaxDepth 62 meter;run" 2ihif 2 2.000000 2020-10-13T20:42:52.850Z,1602621772.850 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2ihif 2020-10-13T20:42:52.851Z,1602621772.851 [CommandLine](IMPORTANT): Scheduled #24 (#2 of 2 with id='2ihif'): "set canon_esp_trianguloid.MaxDepth 62 meter;run" ASAP 2020-10-13T20:42:53.001Z,1602621773.001 [CommandLine](IMPORTANT): got command load ./Missions/Science/canon_esp_trianguloid.xml 2020-10-13T20:42:55.978Z,1602621775.978 [CommandLine](IMPORTANT): Loaded ./Missions/Science/canon_esp_trianguloid.xml 2020-10-13T20:43:09.005Z,1602621789.005 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.Speed 0.750000 meter_per_second 2020-10-13T20:43:09.006Z,1602621789.006 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.YoYoMinDepth 3.000000 meter 2020-10-13T20:43:09.006Z,1602621789.006 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.YoYoMaxDepth1 50.000000 meter 2020-10-13T20:43:14.177Z,1602621794.177 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.MaxDepth 62.000000 meter 2020-10-13T20:43:14.177Z,1602621794.177 [CommandLine](IMPORTANT): got command run 2020-10-13T20:43:14.186Z,1602621794.186 [CommandLine](IMPORTANT): Running 2020-10-13T20:43:14.366Z,1602621794.366 [MissionManager](IMPORTANT): Started mission canon_esp_trianguloid 2020-10-13T20:44:40.152Z,1602621880.152 [canon_esp_trianguloid:Lap:A](IMPORTANT): Starting Lap 2020-10-13T20:44:40.556Z,1602621880.556 [canon_esp_trianguloid:Lap:B](IMPORTANT): Lat[1] = 36.797000 arcdeg Lon[1] = -121.847000 arcdeg 2020-10-13T20:44:40.958Z,1602621880.958 [canon_esp_trianguloid:Lap:C](IMPORTANT): Lat[2] = 36.754000 arcdeg Lon[2] = -122.015500 arcdeg 2020-10-13T20:44:41.374Z,1602621881.374 [canon_esp_trianguloid:Lap:D](IMPORTANT): Lat[3] = 36.782500 arcdeg Lon[3] = -122.018000 arcdeg 2020-10-13T20:44:41.784Z,1602621881.784 [canon_esp_trianguloid:Lap:E](IMPORTANT): Lat[4] = 36.711700 arcdeg Lon[4] = -122.191900 arcdeg 2020-10-13T20:44:42.172Z,1602621882.172 [canon_esp_trianguloid:Lap:F](IMPORTANT): Lat[5] = 36.697800 arcdeg Lon[5] = -122.056200 arcdeg 2020-10-13T20:44:42.578Z,1602621882.578 [canon_esp_trianguloid:Lap:G](IMPORTANT): Lat[6] = 36.754000 arcdeg Lon[6] = -122.015500 arcdeg 2020-10-13T20:44:43.014Z,1602621883.014 [canon_esp_trianguloid:Lap:H](IMPORTANT): Lat[7] = 36.797000 arcdeg Lon[7] = -121.847000 arcdeg 2020-10-13T20:44:43.473Z,1602621883.473 [canon_esp_trianguloid:Lap:I](IMPORTANT): Lat[8] = nan arcdeg Lon[8] = nan arcdeg 2020-10-13T20:44:44.205Z,1602621884.205 [canon_esp_trianguloid:Lap:Wpt1:A](IMPORTANT): Navigating to box start WP. YoYoMaxDepth: 50.000000 m 2020-10-13T20:44:44.608Z,1602621884.608 [canon_esp_trianguloid:SampleTimer:AcquireSample:B](IMPORTANT): Pre-sample comms. Elapsed time since last ESP sample: 0.030706 h 2020-10-13T20:50:58.341Z,1602622258.341 [canon_esp_trianguloid:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T20:51:10.125Z,1602622270.125 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T20:51:10.269Z,1602622270.269 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T20:51:48.414Z,1602622308.414 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T20:51:49.224Z,1602622309.224 [ESPComponent](IMPORTANT): [sample #9] Setting cartridge argument to -6 2020-10-13T20:51:49.224Z,1602622309.224 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T20:51:49.663Z,1602622309.663 [canon_esp_trianguloid:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 20.805628 m in 12.778406 degC water with 1.254300 ug/l chlorophyll fluorescence. 2020-10-13T20:51:50.030Z,1602622310.030 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T20:51:50.433Z,1602622310.433 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T20:51:52.454Z,1602622312.454 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T20:52:25.582Z,1602622345.582 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PRIMING (@21,archive_bac) 2020-10-13T20:52:27.198Z,1602622347.198 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T20:52:28.005Z,1602622348.005 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T20:54:56.681Z,1602622496.681 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T20:54:57.085Z,1602622497.085 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T21:16:57.475Z,1602623817.475 [canon_esp_trianguloid:Lap:Wpt1:C.Waypoint](IMPORTANT): Reached Waypoint: 36.797000,-121.847000 2020-10-13T21:16:57.876Z,1602623817.876 [canon_esp_trianguloid:Lap:Wpt2:A](IMPORTANT): Navigating to WP2. YoYoMaxDepth: 50.000000 m 2020-10-13T21:33:36.583Z,1602624816.583 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PAUSED (@647.41) 2020-10-13T21:33:36.985Z,1602624816.985 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PROCESSING (@647.41) 2020-10-13T21:33:39.410Z,1602624819.410 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PROCESSED (@21,archive_bac) 2020-10-13T21:45:43.005Z,1602625543.005 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPING (@21,archive_bac) 2020-10-13T21:46:08.458Z,1602625568.458 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T21:46:08.862Z,1602625568.862 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPED 2020-10-13T21:51:08.649Z,1602625868.649 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence completed normally. 2020-10-13T21:51:09.053Z,1602625869.053 [ESPComponent](IMPORTANT): [sample #9] ESP log summary report (3 messages): @13:52:02.17 Selecting Cartridge 21 @14:33:34.04 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @14:33:34.24 Sampled 647.4ml 2020-10-13T22:31:12.265Z,1602628272.265 [ThrusterServo](FAULT): getVelocity uart error serial timeout 2020-10-13T22:31:12.265Z,1602628272.265 [ThrusterServo](FAULT): Thruster uart error: serial timeout 2020-10-13T22:45:28.273Z,1602629128.273 [canon_esp_trianguloid:SampleTimer:AcquireSample:B](IMPORTANT): Pre-sample comms. Elapsed time since last ESP sample: 2.000104 h 2020-10-13T22:51:46.305Z,1602629506.305 [canon_esp_trianguloid:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-13T22:51:58.089Z,1602629518.089 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-13T22:51:58.216Z,1602629518.216 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-13T22:52:36.357Z,1602629556.357 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-13T22:52:37.173Z,1602629557.173 [ESPComponent](IMPORTANT): [sample #10] Setting cartridge argument to -6 2020-10-13T22:52:37.173Z,1602629557.173 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-13T22:52:37.610Z,1602629557.610 [canon_esp_trianguloid:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 15.891661 m in 12.696100 degC water with 1.062200 ug/l chlorophyll fluorescence. 2020-10-13T22:52:37.974Z,1602629557.974 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-13T22:52:38.378Z,1602629558.378 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-13T22:52:40.398Z,1602629560.398 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-13T22:53:13.525Z,1602629593.525 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PRIMING (@20,archive_bac) 2020-10-13T22:53:15.141Z,1602629595.141 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PRIMING (@100.0) 2020-10-13T22:53:15.950Z,1602629595.950 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-13T22:55:45.030Z,1602629745.030 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_FILTERING (@0.0) 2020-10-13T22:55:45.430Z,1602629745.430 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-13T23:26:59.294Z,1602631619.294 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PAUSED (@512.853) 2020-10-13T23:26:59.693Z,1602631619.693 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PROCESSING (@512.853) 2020-10-13T23:27:02.118Z,1602631622.118 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PROCESSED (@20,archive_bac) 2020-10-13T23:39:05.741Z,1602632345.741 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPING (@20,archive_bac) 2020-10-13T23:39:31.195Z,1602632371.195 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-13T23:39:31.598Z,1602632371.598 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPED 2020-10-13T23:44:31.381Z,1602632671.381 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence completed normally. 2020-10-13T23:44:31.785Z,1602632671.785 [ESPComponent](IMPORTANT): [sample #10] ESP log summary report (3 messages): @15:52:50.41 Selecting Cartridge 20 @16:26:56.46 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @16:26:56.66 Sampled 512.9ml 2020-10-13T23:48:34.222Z,1602632914.222 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40358, MT Status=1, MTMSN=2810 2020-10-13T23:48:38.713Z,1602632918.713 [CommandLine](IMPORTANT): got command set canon_esp_trianguloid.SampleInterval 3.000000 hour 2020-10-13T23:48:59.191Z,1602632939.191 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40359, MT Status=1, MTMSN=2811 2020-10-13T23:49:02.233Z,1602632942.233 [CommandLine](IMPORTANT): got command get canon_esp_trianguloid.SampleInterval 2020-10-13T23:49:02.234Z,1602632942.234 [CommandLine](IMPORTANT): canon_esp_trianguloid.SampleInterval 3.000000 h 2020-10-14T01:50:42.926Z,1602640242.926 [canon_esp_trianguloid:SampleTimer:AcquireSample:B](IMPORTANT): Pre-sample comms. Elapsed time since last ESP sample: 3.000016 h 2020-10-14T01:56:46.229Z,1602640606.229 [canon_esp_trianguloid:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-14T01:56:58.013Z,1602640618.013 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-14T01:56:58.158Z,1602640618.158 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-14T01:57:36.713Z,1602640656.713 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-14T01:57:37.524Z,1602640657.524 [ESPComponent](IMPORTANT): [sample #11] Setting cartridge argument to -6 2020-10-14T01:57:37.524Z,1602640657.524 [ESPComponent](IMPORTANT): [sample #11] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-14T01:57:37.955Z,1602640657.955 [canon_esp_trianguloid:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 14.964161 m in 13.176294 degC water with 1.695000 ug/l chlorophyll fluorescence. 2020-10-14T01:57:38.330Z,1602640658.330 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-14T01:57:38.733Z,1602640658.733 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-14T01:57:40.758Z,1602640660.758 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-14T01:58:13.885Z,1602640693.885 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PRIMING (@19,archive_bac) 2020-10-14T01:58:15.498Z,1602640695.498 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PRIMING (@100.0) 2020-10-14T01:58:16.305Z,1602640696.305 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-14T02:00:44.173Z,1602640844.173 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_FILTERING (@0.0) 2020-10-14T02:00:44.578Z,1602640844.578 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-14T02:31:35.457Z,1602642695.457 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-14T02:31:46.376Z,1602642706.376 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005538 CHAN A1 (24V): 0.002996 CHAN A2 (12V): -0.004107 CHAN A3 (5V): -0.002592 CHAN B0 (3.3V): -0.001317 CHAN B1 (3.15aV): -0.000881 CHAN B2 (3.15bV): -0.001276 CHAN B3 (GND): 0.000059 OPEN: 0.007240 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-14T02:50:13.681Z,1602643813.681 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PAUSED (@856.361) 2020-10-14T02:50:14.085Z,1602643814.085 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PROCESSING (@856.361) 2020-10-14T02:50:16.510Z,1602643816.510 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PROCESSED (@19,archive_bac) 2020-10-14T03:02:20.097Z,1602644540.097 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_STOPPING (@19,archive_bac) 2020-10-14T03:02:45.551Z,1602644565.551 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-14T03:02:45.953Z,1602644565.953 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_STOPPED 2020-10-14T03:07:45.741Z,1602644865.741 [ESPComponent](IMPORTANT): [sample #11] ESP sampling sequence completed normally. 2020-10-14T03:07:46.142Z,1602644866.142 [ESPComponent](IMPORTANT): [sample #11] ESP log summary report (3 messages): @18:57:50.85 Selecting Cartridge 19 @19:50:11.20 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @19:50:11.40 Sampled 856.4ml 2020-10-14T04:55:45.354Z,1602651345.354 [canon_esp_trianguloid:SampleTimer:AcquireSample:B](IMPORTANT): Pre-sample comms. Elapsed time since last ESP sample: 3.000024 h 2020-10-14T04:59:03.274Z,1602651543.274 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40401, MT Status=1, MTMSN=2812