2020-10-16T00:31:29.646Z,1602808289.646 [CommandLine](IMPORTANT): got command restart logs 2020-10-16T00:32:37.815Z,1602808357.815 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40880, MT Status=1, MTMSN=2875 2020-10-16T00:32:38.613Z,1602808358.613 [CommandLine](IMPORTANT): got command ! onESPclient bill -- slots [24,26,31]=\>:dry 2020-10-16T00:32:39.163Z,1602808359.163 [CommandLine](IMPORTANT): /mnt/mmc/started/2020-10-16UTC003239_onESPclient 2020-10-16T00:39:01.575Z,1602808741.575 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 32.754342 min 2020-10-16T00:39:51.060Z,1602808791.060 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40886, MT Status=1, MTMSN=2876 2020-10-16T00:39:51.630Z,1602808791.630 [CommandLine](IMPORTANT): got command ! check -200 2020-10-16T00:39:52.001Z,1602808792.001 [CommandLine](IMPORTANT): Waiting for ESP to boot... Connected. Fri Oct 16 00:33:17 UTC 2020 -- ssh esp@10.89.5.3 espclient -w bill Fri Oct 16 00:33:23 UTC 2020 -- Shutting down ESP in 20 seconds ... shutting ESP off 2020-10-16T00:46:48.489Z,1602809208.489 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 40.535819 min 2020-10-16T00:56:02.033Z,1602809762.033 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 49.761576 min 2020-10-16T01:01:57.287Z,1602810117.287 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 55.682430 min 2020-10-16T01:08:01.886Z,1602810481.886 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 61.759041 min 2020-10-16T01:08:20.134Z,1602810500.134 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40901, MT Status=1, MTMSN=2877 2020-10-16T01:08:20.886Z,1602810500.886 [CommandLine](IMPORTANT): got command ! check -200 2020-10-16T01:08:21.117Z,1602810501.117 [CommandLine](IMPORTANT): Waiting for ESP to boot... Connected. Fri Oct 16 00:33:17 UTC 2020 -- ssh esp@10.89.5.3 espclient -w bill Fri Oct 16 00:33:23 UTC 2020 -- Shutting down ESP in 20 seconds ... shutting ESP off 2020-10-16T01:14:44.926Z,1602810884.926 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 68.476497 min 2020-10-16T01:15:22.979Z,1602810922.979 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40907, MT Status=1, MTMSN=2878 2020-10-16T01:15:23.760Z,1602810923.760 [CommandLine](IMPORTANT): got command load ./Missions/Science/sci2.xml 2020-10-16T01:15:25.461Z,1602810925.461 [CommandLine](IMPORTANT): Loaded ./Missions/Science/sci2.xml 2020-10-16T01:15:34.425Z,1602810934.425 [CommandLine](IMPORTANT): got command set sci2.MissionTimeout 16.000000 hour 2020-10-16T01:15:34.426Z,1602810934.426 [CommandLine](IMPORTANT): got command set sci2.Lat1 36.797001 degree 2020-10-16T01:15:34.427Z,1602810934.427 [CommandLine](IMPORTANT): got command set sci2.Lon1 -121.847000 degree 2020-10-16T01:15:34.427Z,1602810934.427 [CommandLine](IMPORTANT): got command set sci2.Speed 0.750000 meter_per_second 2020-10-16T01:15:34.428Z,1602810934.428 [CommandLine](IMPORTANT): got command set sci2.YoYoMinDepth 3.000000 meter 2020-10-16T01:15:34.428Z,1602810934.428 [CommandLine](IMPORTANT): got command set sci2.YoYoMaxDepth 50.000000 meter 2020-10-16T01:15:34.430Z,1602810934.430 [CommandLine](IMPORTANT): got command run 2020-10-16T01:15:34.432Z,1602810934.432 [CommandLine](IMPORTANT): Running 2020-10-16T01:15:34.451Z,1602810934.451 [MissionManager](IMPORTANT): Started mission sci2 2020-10-16T02:20:23.947Z,1602814823.947 [NAL9602](IMPORTANT): SBD MO Status=2, MOMSN=40917, MT Status=1, MTMSN=2879 2020-10-16T02:22:57.068Z,1602814977.068 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40917, MT Status=1, MTMSN=2879 2020-10-16T02:22:57.899Z,1602814977.899 [CommandLine](IMPORTANT): got command schedule "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.NumSamplers 2 count;set IsothermDepthCANON.MissionTimeout 4 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2mjt4 1 4.000000 2020-10-16T02:22:57.900Z,1602814977.900 [CommandLine](IMPORTANT): Scheduling command #1 of 4 with id=2mjt4 2020-10-16T02:22:57.901Z,1602814977.901 [CommandLine](IMPORTANT): Scheduled #44 (#1 of 4 with id='2mjt4'): "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.NumSamplers 2 count;set IsothermDepthCANON.MissionTimeout 4 hour;set IsothermDepthCANON.InitialDiveDuration 2 minute;set IsothermDepthCANON.SpeedSampling 0 meter_per_second" 2020-10-16T02:23:11.210Z,1602814991.210 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40918, MT Status=1, MTMSN=2880 2020-10-16T02:23:12.161Z,1602814992.161 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.MaxDepth 60 meter;set IsothermDepthCANON.DeepBoundESP 58 meter;set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter" 2mjt4 2 4.000000 2020-10-16T02:23:12.162Z,1602814992.162 [CommandLine](IMPORTANT): Scheduling command #2 of 4 with id=2mjt4 2020-10-16T02:23:12.162Z,1602814992.162 [CommandLine](IMPORTANT): Scheduled #45 (#2 of 4 with id='2mjt4'): "set IsothermDepthCANON.MaxDepth 60 meter;set IsothermDepthCANON.DeepBoundESP 58 meter;set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter" 2020-10-16T02:23:37.791Z,1602815017.791 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40919, MT Status=1, MTMSN=2881 2020-10-16T02:23:38.738Z,1602815018.738 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.Lat1 36.797 degree;set IsothermDepthCANON.Lon1 -121.847 degree;set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.SampleType2 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin" 2mjt4 3 4.000000 2020-10-16T02:23:38.739Z,1602815018.739 [CommandLine](IMPORTANT): Scheduling command #3 of 4 with id=2mjt4 2020-10-16T02:23:38.739Z,1602815018.739 [CommandLine](IMPORTANT): Scheduled #46 (#3 of 4 with id='2mjt4'): "set IsothermDepthCANON.Lat1 36.797 degree;set IsothermDepthCANON.Lon1 -121.847 degree;set IsothermDepthCANON.SampleType1 2 count;set IsothermDepthCANON.SampleType2 2 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin" 2020-10-16T02:24:02.462Z,1602815042.462 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40920, MT Status=1, MTMSN=2882 2020-10-16T02:24:03.254Z,1602815043.254 [CommandLine](IMPORTANT): got command schedule "set IsothermDepthCANON.Dep2 50 meter;set IsothermDepthCANON.WaitDepUndulation1 30 minute;run" 2mjt4 4 4.000000 2020-10-16T02:24:03.254Z,1602815043.254 [CommandLine](IMPORTANT): Scheduling command #4 of 4 with id=2mjt4 2020-10-16T02:24:03.255Z,1602815043.255 [CommandLine](IMPORTANT): Scheduled #47 (#4 of 4 with id='2mjt4'): "set IsothermDepthCANON.Dep2 50 meter;set IsothermDepthCANON.WaitDepUndulation1 30 minute;run" 2020-10-16T03:25:48.493Z,1602818748.493 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-16T03:25:59.416Z,1602818759.416 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005195 CHAN A1 (24V): 0.002344 CHAN A2 (12V): -0.004498 CHAN A3 (5V): -0.002602 CHAN B0 (3.3V): -0.000920 CHAN B1 (3.15aV): -0.001090 CHAN B2 (3.15bV): -0.001123 CHAN B3 (GND): -0.000124 OPEN: 0.007631 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-16T03:49:02.763Z,1602820142.763 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2020-10-16T05:41:24.195Z,1602826884.195 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=40949, MT Status=1, MTMSN=2883 2020-10-16T05:41:24.930Z,1602826884.930 [CommandLine](IMPORTANT): got command schedule "load Science/profile_station.xml;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.7970 degree;set profile_station.Lon -121.8470 degree;set profile_station.YoYoMinDepth 3 meter" 2mvah 1 2.000000 2020-10-16T05:41:24.930Z,1602826884.930 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2mvah 2020-10-16T05:41:24.931Z,1602826884.931 [CommandLine](IMPORTANT): Scheduled #48 (#1 of 2 with id='2mvah'): "load Science/profile_station.xml;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.7970 degree;set profile_station.Lon -121.8470 degree;set profile_station.YoYoMinDepth 3 meter" 2020-10-16T05:41:43.994Z,1602826903.994 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=40950, MT Status=1, MTMSN=2884 2020-10-16T05:41:44.649Z,1602826904.649 [CommandLine](IMPORTANT): got command schedule "set profile_station.YoYoMaxDepth 50 meter;set profile_station.MaxDepth 55 meter;run" 2mvah 2 2.000000 2020-10-16T05:41:44.649Z,1602826904.649 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2mvah 2020-10-16T05:41:44.650Z,1602826904.650 [CommandLine](IMPORTANT): Scheduled #49 (#2 of 2 with id='2mvah'): "set profile_station.YoYoMaxDepth 50 meter;set profile_station.MaxDepth 55 meter;run" 2020-10-16T05:44:13.545Z,1602827053.545 [ElevatorOffsetCalculator](IMPORTANT): Completed estimation for commanded vars: speed 0.75 m/s, pitch 20.00 deg, mass-position 14.00 mm. Average elevator angle=-0.844880 +/- 0.098043 deg (conf. level 99.90%, sigma: 0.355347 deg). 2020-10-16T09:25:58.604Z,1602840358.604 [sci2:Lap:Wpt1:A.Waypoint](IMPORTANT): Reached Waypoint: 36.797002,-121.846996 2020-10-16T09:25:59.501Z,1602840359.501 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-16T09:26:01.189Z,1602840361.189 [CommandLine](IMPORTANT): got command load ./Missions/Science/isotherm_depth_canon.xml 2020-10-16T09:26:10.359Z,1602840370.359 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005098 CHAN A1 (24V): 0.003496 CHAN A2 (12V): -0.004042 CHAN A3 (5V): -0.002593 CHAN B0 (3.3V): -0.000873 CHAN B1 (3.15aV): -0.001005 CHAN B2 (3.15bV): -0.001097 CHAN B3 (GND): -0.000056 OPEN: 0.007418 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-16T09:26:13.462Z,1602840373.462 [CommandLine](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_canon.xml 2020-10-16T09:26:13.560Z,1602840373.560 [MissionManager](IMPORTANT): Started mission Default 2020-10-16T09:26:31.376Z,1602840391.376 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.NumSamplers 2.000000 count 2020-10-16T09:26:31.376Z,1602840391.376 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.MissionTimeout 4.000000 hour 2020-10-16T09:26:31.383Z,1602840391.383 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.InitialDiveDuration 2.000000 minute 2020-10-16T09:26:31.389Z,1602840391.389 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SpeedSampling 0.000000 meter_per_second 2020-10-16T09:26:53.309Z,1602840413.309 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.MaxDepth 60.000000 meter 2020-10-16T09:26:53.311Z,1602840413.311 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.DeepBoundESP 58.000000 meter 2020-10-16T09:26:53.314Z,1602840413.314 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.ShallowBoundChl 3.000000 meter 2020-10-16T09:26:53.315Z,1602840413.315 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMinDepth 3.000000 meter 2020-10-16T09:26:53.316Z,1602840413.316 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMaxDepth 50.000000 meter 2020-10-16T09:27:12.555Z,1602840432.555 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lat1 36.797001 degree 2020-10-16T09:27:12.556Z,1602840432.556 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lon1 -121.847000 degree 2020-10-16T09:27:12.557Z,1602840432.557 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleType1 2.000000 count 2020-10-16T09:27:12.557Z,1602840432.557 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleType2 2.000000 count 2020-10-16T09:27:12.558Z,1602840432.558 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TempDiffFromPeakChl1 0.000000 kelvin 2020-10-16T09:27:19.916Z,1602840439.916 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Dep2 50.000000 meter 2020-10-16T09:27:19.918Z,1602840439.918 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.WaitDepUndulation1 30.000000 minute 2020-10-16T09:27:19.918Z,1602840439.918 [CommandLine](IMPORTANT): got command run 2020-10-16T09:27:19.921Z,1602840439.921 [CommandLine](IMPORTANT): Running 2020-10-16T09:27:20.189Z,1602840440.189 [MissionManager](IMPORTANT): Started mission IsothermDepthCANON 2020-10-16T09:27:20.238Z,1602840440.238 [IsothermDepthCANON:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-10-16T09:33:24.636Z,1602840804.636 [IsothermDepthCANON:TransitToSamplingLocationNoComms:Wpt1NoComms:C.Waypoint](IMPORTANT): Reached Waypoint: 36.797002,-121.846996 2020-10-16T09:35:58.998Z,1602840958.998 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.670156 m 2020-10-16T09:36:07.866Z,1602840967.866 [IsothermDepthCANON:SetSampleOptions:Q](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , 0.000000 K , nan m , nan degC , nan m , 30.000000 min . 2020-10-16T09:42:54.728Z,1602841374.728 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 30.049505 m 2020-10-16T09:50:26.436Z,1602841826.436 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 28.390657 m , 0.881400 ug/l , 5.810309 m , 13.781929 degC , 5.301959 ug/l 2020-10-16T09:50:29.251Z,1602841829.251 [IsothermDepthCANON:TakeSamples:FiringOnTemp:C](IMPORTANT): Taking sample No. 1.000000 count . Target temp = 13.781929 degC . 2020-10-16T09:50:30.071Z,1602841830.071 [IsothermDepthCANON:DampDown:A](IMPORTANT): Wait 30.000000 min for the depth undulation to damp down. 2020-10-16T10:20:33.655Z,1602843633.655 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-16T10:20:45.370Z,1602843645.370 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-16T10:20:45.642Z,1602843645.642 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-16T10:21:23.773Z,1602843683.773 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-16T10:21:24.586Z,1602843684.586 [ESPComponent](IMPORTANT): [sample #7] Setting cartridge argument to -6 2020-10-16T10:21:24.586Z,1602843684.586 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-16T10:21:25.090Z,1602843685.090 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 6.366683 m in 13.764215 degC water with 5.311000 ug/l chlorophyll fluorescence. 2020-10-16T10:21:25.390Z,1602843685.390 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-16T10:21:25.794Z,1602843685.794 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-16T10:21:27.814Z,1602843687.814 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-16T10:22:00.942Z,1602843720.942 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PRIMING (@4,archive_bac) 2020-10-16T10:22:02.558Z,1602843722.558 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PRIMING (@100.0) 2020-10-16T10:22:03.367Z,1602843723.367 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-16T10:24:32.050Z,1602843872.050 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_FILTERING (@0.0) 2020-10-16T10:24:32.455Z,1602843872.455 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-16T10:31:06.479Z,1602844266.479 [IsothermDepthCANON:Sample:SampleWrapper:CANONSampler:AcquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 6.319116 m in 13.847712 degC water with 7.808300 ug/l chlorophyll fluorescence. 2020-10-16T10:34:32.422Z,1602844472.422 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PAUSED (@174.677) 2020-10-16T10:34:32.827Z,1602844472.827 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PROCESSING (@174.677) 2020-10-16T10:34:35.250Z,1602844475.250 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PROCESSED (@4,archive_bac) 2020-10-16T10:35:04.340Z,1602844504.340 [CANONSampler](IMPORTANT): Sample 11, err_code=0 2020-10-16T10:46:38.846Z,1602845198.846 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPING (@4,archive_bac) 2020-10-16T10:47:04.303Z,1602845224.303 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-16T10:47:04.706Z,1602845224.706 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPED 2020-10-16T10:52:04.502Z,1602845524.502 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence completed normally. 2020-10-16T10:52:04.906Z,1602845524.906 [ESPComponent](IMPORTANT): [sample #7] ESP log summary report (3 messages): @03:21:37.54 Selecting Cartridge 4 @03:34:29.85 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @03:34:30.05 Sampled 174.7ml 2020-10-16T10:52:07.399Z,1602845527.399 [IsothermDepthCANON:TakeSamples:FiringOnTemp:TriggerOnTemp:C](IMPORTANT): Sample No. 1.000000 count completed. 2020-10-16T10:52:23.951Z,1602845543.951 [IsothermDepthCANON:SetSampleOptions:Q](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , nan m , nan degC , 50.000000 m , 10.000000 min . 2020-10-16T10:52:26.767Z,1602845546.767 [IsothermDepthCANON:TakeSamples:FiringOnDepth:C](IMPORTANT): Taking sample No. 2.000000 count . Target depth = 50.000000 m . 2020-10-16T11:08:20.688Z,1602846500.688 [IsothermDepthCANON:TriggerSampleAtDepth:SampleWrapper:B](IMPORTANT): At 50.012058 m , settling for 10.000000 min 2020-10-16T11:18:23.888Z,1602847103.888 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-16T11:18:35.641Z,1602847115.641 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-16T11:18:35.776Z,1602847115.776 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-16T11:19:14.721Z,1602847154.721 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-16T11:19:15.532Z,1602847155.532 [ESPComponent](IMPORTANT): [sample #8] Setting cartridge argument to -6 2020-10-16T11:19:15.532Z,1602847155.532 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-16T11:19:16.012Z,1602847156.012 [IsothermDepthCANON:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 49.810162 m in 11.517694 degC water with 0.271200 ug/l chlorophyll fluorescence. 2020-10-16T11:19:16.338Z,1602847156.338 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-16T11:19:16.741Z,1602847156.741 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-16T11:19:18.761Z,1602847158.761 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-16T11:19:51.895Z,1602847191.895 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PRIMING (@3,archive_bac) 2020-10-16T11:19:53.510Z,1602847193.510 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PRIMING (@100.0) 2020-10-16T11:19:54.318Z,1602847194.318 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-16T11:22:22.598Z,1602847342.598 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_FILTERING (@0.0) 2020-10-16T11:22:23.003Z,1602847343.003 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-16T11:28:57.065Z,1602847737.065 [IsothermDepthCANON:Sample:SampleWrapper:CANONSampler:AcquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 50.399246 m in 11.490015 degC water with 0.271200 ug/l chlorophyll fluorescence. 2020-10-16T11:32:54.887Z,1602847974.887 [CANONSampler](IMPORTANT): Sample 12, err_code=0 2020-10-16T12:05:21.071Z,1602849921.071 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PAUSED (@1000.0) 2020-10-16T12:05:21.474Z,1602849921.474 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PROCESSING (@1000.0) 2020-10-16T12:05:23.906Z,1602849923.906 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PROCESSED (@3,archive_bac) 2020-10-16T12:17:27.138Z,1602850647.138 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPING (@3,archive_bac) 2020-10-16T12:17:52.599Z,1602850672.599 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-16T12:17:53.002Z,1602850673.002 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPED 2020-10-16T12:22:52.781Z,1602850972.781 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence completed normally. 2020-10-16T12:22:53.185Z,1602850973.185 [ESPComponent](IMPORTANT): [sample #8] ESP log summary report (2 messages): @04:19:28.96 Selecting Cartridge 3 @05:05:19.18 Sampled 1000.0ml 2020-10-16T12:22:56.081Z,1602850976.081 [IsothermDepthCANON:TakeSamples:FiringOnDepth:TriggerOnDepth:B](IMPORTANT): Sample No. 2.000000 count completed. 2020-10-16T12:22:58.319Z,1602850978.319 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-10-16T12:22:58.777Z,1602850978.777 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-10-16T12:23:00.524Z,1602850980.524 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-10-16T12:23:00.861Z,1602850980.861 [MissionManager](IMPORTANT): Started mission Default 2020-10-16T12:23:14.311Z,1602850994.311 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 30.000000 minute 2020-10-16T12:23:14.312Z,1602850994.312 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-10-16T12:23:14.313Z,1602850994.313 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-10-16T12:23:14.314Z,1602850994.314 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 3.000000 meter 2020-10-16T12:23:22.061Z,1602851002.061 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2020-10-16T12:23:22.062Z,1602851002.062 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 55.000000 meter 2020-10-16T12:23:22.062Z,1602851002.062 [CommandLine](IMPORTANT): got command run 2020-10-16T12:23:22.066Z,1602851002.066 [CommandLine](IMPORTANT): Running 2020-10-16T12:23:22.262Z,1602851002.262 [MissionManager](IMPORTANT): Started mission profile_station 2020-10-16T12:51:55.829Z,1602852715.829 [ThrusterServo](FAULT): getVelocity uart error serial timeout 2020-10-16T12:51:55.829Z,1602852715.829 [ThrusterServo](FAULT): Thruster uart error: serial timeout 2020-10-16T13:35:32.503Z,1602855332.503 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41004, MT Status=1, MTMSN=2885 2020-10-16T13:35:33.198Z,1602855333.198 [CommandLine](IMPORTANT): got command stop 2020-10-16T13:35:33.198Z,1602855333.198 [CommandLine](IMPORTANT): Scheduling is paused 2020-10-16T13:35:33.749Z,1602855333.749 [MissionManager](IMPORTANT): Started mission Default 2020-10-16T13:42:07.822Z,1602855727.822 [DataOverHttps](IMPORTANT): SBD MTMSN=20201016T134119 2020-10-16T13:42:15.354Z,1602855735.354 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.7970 degree;set profile_station.Lon -121.8470 degree;set profile_station.YoYoMinDepth 3 meter" 2ni0j 1 2.000000 2020-10-16T13:42:15.354Z,1602855735.354 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2ni0j 2020-10-16T13:42:15.355Z,1602855735.355 [CommandLine](IMPORTANT): Scheduled #50 (#1 of 2 with id='2ni0j'): "load Science/profile_station.xml;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.7970 degree;set profile_station.Lon -121.8470 degree;set profile_station.YoYoMinDepth 3 meter" ASAP 2020-10-16T13:42:16.283Z,1602855736.283 [DataOverHttps](IMPORTANT): SBD MTMSN=20201016T134128 2020-10-16T13:42:23.878Z,1602855743.878 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.YoYoMaxDepth 50 meter;set profile_station.MaxDepth 55 meter;run" 2ni0j 2 2.000000 2020-10-16T13:42:23.879Z,1602855743.879 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2ni0j 2020-10-16T13:42:23.879Z,1602855743.879 [CommandLine](IMPORTANT): Scheduled #51 (#2 of 2 with id='2ni0j'): "set profile_station.YoYoMaxDepth 50 meter;set profile_station.MaxDepth 55 meter;run" ASAP 2020-10-16T13:42:23.948Z,1602855743.948 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-10-16T13:42:25.204Z,1602855745.204 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-10-16T13:42:36.956Z,1602855756.956 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 30.000000 minute 2020-10-16T13:42:36.957Z,1602855756.957 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-10-16T13:42:36.958Z,1602855756.958 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-10-16T13:42:36.958Z,1602855756.958 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 3.000000 meter 2020-10-16T13:42:43.812Z,1602855763.812 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2020-10-16T13:42:43.813Z,1602855763.813 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 55.000000 meter 2020-10-16T13:42:43.814Z,1602855763.814 [CommandLine](IMPORTANT): got command run 2020-10-16T13:42:43.816Z,1602855763.816 [CommandLine](IMPORTANT): Running 2020-10-16T13:42:43.968Z,1602855763.968 [MissionManager](IMPORTANT): Started mission profile_station 2020-10-16T14:15:24.318Z,1602857724.318 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41016, MT Status=1, MTMSN=2886 2020-10-16T14:15:25.154Z,1602857725.154 [CommandLine](IMPORTANT): got command schedule asap "load Science/esp_sample_station.xml;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.TransitYoYoMinDepth 3 meter;set esp_sample_station.SampleYoYoMinDepth 3 meter" 2njbv 1 2.000000 2020-10-16T14:15:25.155Z,1602857725.155 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2njbv 2020-10-16T14:15:25.156Z,1602857725.156 [CommandLine](IMPORTANT): Scheduled #52 (#1 of 2 with id='2njbv'): "load Science/esp_sample_station.xml;set esp_sample_station.NeedCommsTime 60 minute;set esp_sample_station.TransitYoYoMinDepth 3 meter;set esp_sample_station.SampleYoYoMinDepth 3 meter" ASAP 2020-10-16T14:16:03.095Z,1602857763.095 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41017, MT Status=1, MTMSN=2887 2020-10-16T14:16:03.847Z,1602857763.847 [CommandLine](IMPORTANT): got command schedule asap "set esp_sample_station.Speed 0.75 meter_per_second;run" 2njbv 2 2.000000 2020-10-16T14:16:03.847Z,1602857763.847 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2njbv 2020-10-16T14:16:03.848Z,1602857763.848 [CommandLine](IMPORTANT): Scheduled #53 (#2 of 2 with id='2njbv'): "set esp_sample_station.Speed 0.75 meter_per_second;run" ASAP 2020-10-16T14:16:04.045Z,1602857764.045 [CommandLine](IMPORTANT): got command load ./Missions/Science/esp_sample_station.xml 2020-10-16T14:16:06.977Z,1602857766.977 [CommandLine](IMPORTANT): Loaded ./Missions/Science/esp_sample_station.xml 2020-10-16T14:16:07.182Z,1602857767.182 [MissionManager](IMPORTANT): Started mission Default 2020-10-16T14:16:21.559Z,1602857781.559 [CommandLine](IMPORTANT): got command set esp_sample_station.NeedCommsTime 60.000000 minute 2020-10-16T14:16:21.560Z,1602857781.560 [CommandLine](IMPORTANT): got command set esp_sample_station.TransitYoYoMinDepth 3.000000 meter 2020-10-16T14:16:21.561Z,1602857781.561 [CommandLine](IMPORTANT): got command set esp_sample_station.SampleYoYoMinDepth 3.000000 meter 2020-10-16T14:16:25.414Z,1602857785.414 [CommandLine](IMPORTANT): got command set esp_sample_station.Speed 0.750000 meter_per_second 2020-10-16T14:16:25.414Z,1602857785.414 [CommandLine](IMPORTANT): got command run 2020-10-16T14:16:25.418Z,1602857785.418 [CommandLine](IMPORTANT): Running 2020-10-16T14:16:25.477Z,1602857785.477 [MissionManager](IMPORTANT): Started mission esp_sample_station 2020-10-16T14:19:11.263Z,1602857951.263 [esp_sample_station:Lap:A](IMPORTANT): Starting Lap 2020-10-16T14:19:11.688Z,1602857951.688 [esp_sample_station:Lap:B](IMPORTANT): Lat[1] = 36.797000 arcdeg Lon[1] = -121.847000 arcdeg 2020-10-16T14:19:12.125Z,1602857952.125 [esp_sample_station:Lap:C](IMPORTANT): Lat[2] = 36.821900 arcdeg Lon[2] = -121.885600 arcdeg 2020-10-16T14:19:12.522Z,1602857952.522 [esp_sample_station:Lap:D](IMPORTANT): Lat[3] = 36.766000 arcdeg Lon[3] = -121.864900 arcdeg 2020-10-16T14:19:12.896Z,1602857952.896 [esp_sample_station:Lap:E](IMPORTANT): Lat[4] = nan arcdeg Lon[4] = nan arcdeg 2020-10-16T14:19:17.318Z,1602857957.318 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.797000 arcdeg , -121.847000 arcdeg 2020-10-16T14:24:12.677Z,1602858252.677 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2020-10-16T14:31:11.264Z,1602858671.264 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.797000,-121.847000 2020-10-16T14:39:48.941Z,1602859188.941 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-16T14:40:00.721Z,1602859200.721 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-16T14:40:00.855Z,1602859200.855 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-16T14:40:39.813Z,1602859239.813 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-16T14:40:40.616Z,1602859240.616 [ESPComponent](IMPORTANT): [sample #9] Setting cartridge argument to -6 2020-10-16T14:40:40.617Z,1602859240.617 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-16T14:40:41.054Z,1602859241.054 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 22.632818 m in 12.039087 degC water with 0.711900 ug/l chlorophyll fluorescence. 2020-10-16T14:40:41.421Z,1602859241.421 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-16T14:40:41.829Z,1602859241.829 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-16T14:40:43.846Z,1602859243.846 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-16T14:41:16.974Z,1602859276.974 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PRIMING (@31,archive_bac) 2020-10-16T14:41:18.590Z,1602859278.590 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PRIMING (@100.0) 2020-10-16T14:41:19.402Z,1602859279.402 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-16T14:43:48.074Z,1602859428.074 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_FILTERING (@0.0) 2020-10-16T14:43:48.478Z,1602859428.478 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-16T15:26:10.717Z,1602861970.717 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-16T15:26:21.592Z,1602861981.592 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005132 CHAN A1 (24V): 0.003085 CHAN A2 (12V): -0.004293 CHAN A3 (5V): -0.002355 CHAN B0 (3.3V): -0.001204 CHAN B1 (3.15aV): -0.001070 CHAN B2 (3.15bV): -0.001211 CHAN B3 (GND): 0.000052 OPEN: 0.007271 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-16T15:42:07.317Z,1602862927.317 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PAUSED (@1000.001) 2020-10-16T15:42:07.722Z,1602862927.722 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PROCESSING (@1000.001) 2020-10-16T15:42:10.145Z,1602862930.145 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PROCESSED (@31,archive_bac) 2020-10-16T15:54:13.766Z,1602863653.766 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPING (@31,archive_bac) 2020-10-16T15:54:39.219Z,1602863679.219 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-16T15:54:39.622Z,1602863679.622 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPED 2020-10-16T15:59:39.418Z,1602863979.418 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence completed normally. 2020-10-16T15:59:39.821Z,1602863979.821 [ESPComponent](IMPORTANT): [sample #9] ESP log summary report (2 messages): @07:40:54.04 Selecting Cartridge 31 @08:42:05.36 Sampled 1000.0ml 2020-10-16T16:05:34.137Z,1602864334.137 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:C](IMPORTANT): Navigating to WP 36.821900 arcdeg , -121.885600 arcdeg 2020-10-16T17:52:40.835Z,1602870760.835 [esp_sample_station:Lap:WaypointTrajectory:Station:GoToStation:D.Waypoint](IMPORTANT): Reached Waypoint: 36.821900,-121.885600 2020-10-16T17:59:28.177Z,1602871168.177 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:SelectESPCartridge:A](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-10-16T17:59:39.925Z,1602871179.925 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-16T17:59:40.215Z,1602871180.215 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-16T18:00:18.755Z,1602871218.755 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-16T18:00:19.590Z,1602871219.590 [ESPComponent](IMPORTANT): [sample #10] Setting cartridge argument to -6 2020-10-16T18:00:19.590Z,1602871219.590 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-16T18:00:20.023Z,1602871220.023 [esp_sample_station:Sample:SampleWrapper:ESP:AcquireESP:C](IMPORTANT): ESP sampling at 18.137941 m in 12.456507 degC water with 0.485900 ug/l chlorophyll fluorescence. 2020-10-16T18:00:20.394Z,1602871220.394 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-16T18:00:20.774Z,1602871220.774 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-16T18:00:22.809Z,1602871222.809 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-16T18:00:55.938Z,1602871255.938 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PRIMING (@26,archive_bac) 2020-10-16T18:00:57.554Z,1602871257.554 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PRIMING (@100.0) 2020-10-16T18:00:58.364Z,1602871258.364 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-16T18:03:41.582Z,1602871421.582 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_FILTERING (@0.0) 2020-10-16T18:03:41.987Z,1602871421.987 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-16T18:39:06.345Z,1602873546.345 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PAUSED (@593.273) 2020-10-16T18:39:06.750Z,1602873546.750 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PROCESSING (@593.273) 2020-10-16T18:39:09.174Z,1602873549.174 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PROCESSED (@26,archive_bac) 2020-10-16T18:51:12.781Z,1602874272.781 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPING (@26,archive_bac) 2020-10-16T18:51:38.239Z,1602874298.239 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-16T18:51:38.642Z,1602874298.642 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPED 2020-10-16T18:56:38.426Z,1602874598.426 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence completed normally. 2020-10-16T18:56:38.829Z,1602874598.829 [ESPComponent](IMPORTANT): [sample #10] ESP log summary report (5 messages): @11:00:32.94 Selecting Cartridge 26 @11:01:13.79 Cartridge::Sampler::PossibleLeak in FILTERING -- Retained only 43.1 of initial 55.5psia @11:01:13.81 Retry #1 of 2 @11:39:03.81 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @11:39:04.00 Sampled 593.3ml 2020-10-16T18:59:58.840Z,1602874798.840 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41056, MT Status=1, MTMSN=2888 2020-10-16T18:59:59.342Z,1602874799.342 [CommandLine](IMPORTANT): got command stop 2020-10-16T19:00:00.080Z,1602874800.080 [MissionManager](IMPORTANT): Started mission Default 2020-10-16T19:07:24.999Z,1602875244.999 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 7.408650 min 2020-10-16T19:09:41.007Z,1602875381.007 [DataOverHttps](IMPORTANT): SBD MTMSN=20201016T190852 2020-10-16T19:09:48.858Z,1602875388.858 [CommandLine](IMPORTANT): got command schedule asap "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.SampleAtPeakChl 1 bool;set IsothermDepthCANON.InitialDiveDuration 2 minute" 2nx61 1 4.000000 2020-10-16T19:09:48.859Z,1602875388.859 [CommandLine](IMPORTANT): Scheduling command #1 of 4 with id=2nx61 2020-10-16T19:09:48.860Z,1602875388.860 [CommandLine](IMPORTANT): Scheduled #54 (#1 of 4 with id='2nx61'): "load Science/isotherm_depth_canon.xml;set IsothermDepthCANON.MissionTimeout 3 hour;set IsothermDepthCANON.SampleAtPeakChl 1 bool;set IsothermDepthCANON.InitialDiveDuration 2 minute" ASAP 2020-10-16T19:09:49.799Z,1602875389.799 [DataOverHttps](IMPORTANT): SBD MTMSN=20201016T190901 2020-10-16T19:09:57.462Z,1602875397.462 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthCANON.SpeedSampling 0 meter_per_second;set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter" 2nx61 2 4.000000 2020-10-16T19:09:57.463Z,1602875397.463 [CommandLine](IMPORTANT): Scheduling command #2 of 4 with id=2nx61 2020-10-16T19:09:57.464Z,1602875397.464 [CommandLine](IMPORTANT): Scheduled #55 (#2 of 4 with id='2nx61'): "set IsothermDepthCANON.SpeedSampling 0 meter_per_second;set IsothermDepthCANON.ShallowBoundChl 3 meter;set IsothermDepthCANON.TransitYoYoMinDepth 3 meter;set IsothermDepthCANON.TransitYoYoMaxDepth 50 meter" ASAP 2020-10-16T19:09:58.379Z,1602875398.379 [DataOverHttps](IMPORTANT): SBD MTMSN=20201016T190910 2020-10-16T19:10:05.770Z,1602875405.770 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthCANON.Lat1 36.82543 degree;set IsothermDepthCANON.Lon1 -121.88658 degree;set IsothermDepthCANON.SampleType1 1 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin" 2nx61 3 4.000000 2020-10-16T19:10:05.770Z,1602875405.770 [CommandLine](IMPORTANT): Scheduling command #3 of 4 with id=2nx61 2020-10-16T19:10:05.771Z,1602875405.771 [CommandLine](IMPORTANT): Scheduled #56 (#3 of 4 with id='2nx61'): "set IsothermDepthCANON.Lat1 36.82543 degree;set IsothermDepthCANON.Lon1 -121.88658 degree;set IsothermDepthCANON.SampleType1 1 count;set IsothermDepthCANON.TempDiffFromPeakChl1 0 kelvin" ASAP 2020-10-16T19:10:06.686Z,1602875406.686 [DataOverHttps](IMPORTANT): SBD MTMSN=20201016T190918 2020-10-16T19:10:14.274Z,1602875414.274 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthCANON.WaitDepUndulation1 30 minute;run" 2nx61 4 4.000000 2020-10-16T19:10:14.274Z,1602875414.274 [CommandLine](IMPORTANT): Scheduling command #4 of 4 with id=2nx61 2020-10-16T19:10:14.275Z,1602875414.275 [CommandLine](IMPORTANT): Scheduled #57 (#4 of 4 with id='2nx61'): "set IsothermDepthCANON.WaitDepUndulation1 30 minute;run" ASAP 2020-10-16T19:10:14.442Z,1602875414.442 [CommandLine](IMPORTANT): got command load ./Missions/Science/isotherm_depth_canon.xml 2020-10-16T19:10:25.439Z,1602875425.439 [CommandLine](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_canon.xml 2020-10-16T19:10:38.941Z,1602875438.941 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.MissionTimeout 3.000000 hour 2020-10-16T19:10:38.942Z,1602875438.942 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleAtPeakChl 1.000000 bool 2020-10-16T19:10:38.943Z,1602875438.943 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.InitialDiveDuration 2.000000 minute 2020-10-16T19:11:00.520Z,1602875460.520 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SpeedSampling 0.000000 meter_per_second 2020-10-16T19:11:00.521Z,1602875460.521 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.ShallowBoundChl 3.000000 meter 2020-10-16T19:11:00.522Z,1602875460.522 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMinDepth 3.000000 meter 2020-10-16T19:11:00.523Z,1602875460.523 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TransitYoYoMaxDepth 50.000000 meter 2020-10-16T19:11:19.090Z,1602875479.090 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lat1 36.825432 degree 2020-10-16T19:11:19.091Z,1602875479.091 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.Lon1 -121.886581 degree 2020-10-16T19:11:19.091Z,1602875479.091 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.SampleType1 1.000000 count 2020-10-16T19:11:19.092Z,1602875479.092 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.TempDiffFromPeakChl1 0.000000 kelvin 2020-10-16T19:11:23.945Z,1602875483.945 [CommandLine](IMPORTANT): got command set IsothermDepthCANON.WaitDepUndulation1 30.000000 minute 2020-10-16T19:11:23.946Z,1602875483.946 [CommandLine](IMPORTANT): got command run 2020-10-16T19:11:23.949Z,1602875483.949 [CommandLine](IMPORTANT): Running 2020-10-16T19:11:24.204Z,1602875484.204 [MissionManager](IMPORTANT): Started mission IsothermDepthCANON 2020-10-16T19:11:24.232Z,1602875484.232 [IsothermDepthCANON:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-10-16T19:15:08.046Z,1602875708.046 [IsothermDepthCANON:TransitToSamplingLocationNoComms:Wpt1NoComms:C.Waypoint](IMPORTANT): Reached Waypoint: 36.825433,-121.886584 2020-10-16T19:17:23.770Z,1602875843.770 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2020-10-16T19:17:23.797Z,1602875843.797 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.666564 m 2020-10-16T19:17:32.687Z,1602875852.687 [IsothermDepthCANON:SetSampleOptions:Q](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , 0.000000 K , nan m , nan degC , nan m , 30.000000 min . 2020-10-16T19:24:02.563Z,1602876242.563 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 30.028788 m 2020-10-16T19:30:25.582Z,1602876625.582 [IsothermDepthCANON:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 28.377125 m , 0.305100 ug/l , 8.335260 m , 14.543505 degC , 8.920784 ug/l 2020-10-16T19:30:28.408Z,1602876628.408 [IsothermDepthCANON:TakeSamples:FiringOnTemp:C](IMPORTANT): Taking sample No. 1.000000 count . Target temp = 14.543505 degC . 2020-10-16T19:30:29.229Z,1602876629.229 [IsothermDepthCANON:DampDown:A](IMPORTANT): Wait 30.000000 min for the depth undulation to damp down. 2020-10-16T20:11:05.939Z,1602879065.939 [IsothermDepthCANON:Sample:SampleWrapper:CANONSampler:AcquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 7.663999 m in 14.575403 degC water with 10.079600 ug/l chlorophyll fluorescence. 2020-10-16T20:15:04.610Z,1602879304.610 [CANONSampler](IMPORTANT): Sample 13, err_code=0 2020-10-16T20:15:07.483Z,1602879307.483 [IsothermDepthCANON:TakeSamples:FiringOnTemp:TriggerOnTemp:C](IMPORTANT): Sample No. 1.000000 count completed. 2020-10-16T20:15:10.296Z,1602879310.296 [MissionManager](IMPORTANT): Started mission Default 2020-10-16T20:15:10.309Z,1602879310.309 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-10-16T20:18:19.440Z,1602879499.440 [NAL9602](FAULT): received: +CSQ:0 OK063, 2, 0, 0, 0 OK 2020-10-16T20:18:20.159Z,1602879500.159 [NAL9602](FAULT): LCB fault: Software Overcurrent. 2020-10-16T20:19:29.243Z,1602879569.243 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41063, MT Status=1, MTMSN=2889 2020-10-16T20:19:29.878Z,1602879569.878 [CommandLine](IMPORTANT): got command configSet AMEcho.enabled 1.000000 bool 2020-10-16T20:19:29.883Z,1602879569.883 [AMEcho](IMPORTANT): Setting 'enabled' to 1 2020-10-16T20:19:30.081Z,1602879570.081 [CommandLine](IMPORTANT): got command ! echo 1 > /dev/loadB2 2020-10-16T20:27:23.303Z,1602880043.303 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 12.210045 min 2020-10-16T20:27:47.571Z,1602880067.571 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41079, MT Status=1, MTMSN=2890 2020-10-16T20:27:48.338Z,1602880068.338 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.7970 degree;set profile_station.Lon -121.8470 degree;set profile_station.YoYoMinDepth 3 meter" 2o0ox 1 2.000000 2020-10-16T20:27:48.338Z,1602880068.338 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=2o0ox 2020-10-16T20:27:48.339Z,1602880068.339 [CommandLine](IMPORTANT): Scheduled #58 (#1 of 2 with id='2o0ox'): "load Science/profile_station.xml;set profile_station.NeedCommsTime 30 minute;set profile_station.Lat 36.7970 degree;set profile_station.Lon -121.8470 degree;set profile_station.YoYoMinDepth 3 meter" ASAP 2020-10-16T20:32:58.692Z,1602880378.692 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 17.799955 min 2020-10-16T20:33:15.260Z,1602880395.260 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41080, MT Status=1, MTMSN=2891 2020-10-16T20:33:15.898Z,1602880395.898 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.YoYoMaxDepth 50 meter;set profile_station.MaxDepth 55 meter;run" 2o0ox 2 2.000000 2020-10-16T20:33:15.898Z,1602880395.898 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=2o0ox 2020-10-16T20:33:15.899Z,1602880395.899 [CommandLine](IMPORTANT): Scheduled #59 (#2 of 2 with id='2o0ox'): "set profile_station.YoYoMaxDepth 50 meter;set profile_station.MaxDepth 55 meter;run" ASAP 2020-10-16T20:33:16.106Z,1602880396.106 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-10-16T20:33:17.357Z,1602880397.357 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-10-16T20:33:42.885Z,1602880422.885 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 30.000000 minute 2020-10-16T20:33:42.886Z,1602880422.886 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-10-16T20:33:42.887Z,1602880422.887 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-10-16T20:33:42.887Z,1602880422.887 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 3.000000 meter 2020-10-16T20:33:53.123Z,1602880433.123 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2020-10-16T20:33:53.124Z,1602880433.124 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 55.000000 meter 2020-10-16T20:33:53.124Z,1602880433.124 [CommandLine](IMPORTANT): got command run 2020-10-16T20:33:53.138Z,1602880433.138 [CommandLine](IMPORTANT): Running 2020-10-16T20:33:53.285Z,1602880433.285 [MissionManager](IMPORTANT): Started mission profile_station 2020-10-16T21:07:14.606Z,1602882434.606 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41082, MT Status=1, MTMSN=2892