2020-07-20T20:20:49.155Z,1595276449.155 [CommandLine](IMPORTANT): got command restart logs 2020-07-20T20:26:27.979Z,1595276787.979 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 88.429924 min 2020-07-20T20:32:22.048Z,1595277142.048 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 94.330713 min 2020-07-20T20:34:16.635Z,1595277256.635 [DataOverHttps](IMPORTANT): SBD MTMSN=20200720T203415 2020-07-20T20:34:26.802Z,1595277266.802 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.NeedCommsTime 20 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.YoYoMaxDepth 20 meter" 3a95o 1 2.000000 2020-07-20T20:34:26.802Z,1595277266.802 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=3a95o 2020-07-20T20:34:26.803Z,1595277266.803 [CommandLine](IMPORTANT): Scheduled #1 (#1 of 2 with id='3a95o'): "load Science/profile_station.xml;set profile_station.NeedCommsTime 20 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.YoYoMaxDepth 20 meter" ASAP 2020-07-20T20:34:27.754Z,1595277267.754 [DataOverHttps](IMPORTANT): SBD MTMSN=20200720T203427 2020-07-20T20:34:35.017Z,1595277275.017 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.MaxDepth 25 meter;run " 3a95o 2 2.000000 2020-07-20T20:34:35.017Z,1595277275.017 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=3a95o 2020-07-20T20:34:35.018Z,1595277275.018 [CommandLine](IMPORTANT): Scheduled #2 (#2 of 2 with id='3a95o'): "set profile_station.MaxDepth 25 meter;run " ASAP 2020-07-20T20:34:35.186Z,1595277275.186 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-07-20T20:34:37.452Z,1595277277.452 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-07-20T20:34:44.831Z,1595277284.831 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 20.000000 minute 2020-07-20T20:34:44.833Z,1595277284.833 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-07-20T20:34:44.833Z,1595277284.833 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-07-20T20:34:44.834Z,1595277284.834 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 20.000000 meter 2020-07-20T20:34:46.714Z,1595277286.714 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 25.000000 meter 2020-07-20T20:34:46.715Z,1595277286.715 [CommandLine](IMPORTANT): got command run 2020-07-20T20:34:46.749Z,1595277286.749 [CommandLine](IMPORTANT): Running 2020-07-20T20:34:46.764Z,1595277286.764 [MissionManager](IMPORTANT): Started mission profile_station 2020-07-20T21:18:12.439Z,1595279892.439 [BPC1](FAULT): Battery stick #6 (s/n: 00CA) reported TERMINATE_DISCHARGE_ALARM. Status code: 0x48C0. 2020-07-20T21:47:04.185Z,1595281624.185 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=1198, MT Status=1, MTMSN=80 2020-07-20T21:47:04.682Z,1595281624.682 [CommandLine](IMPORTANT): got command stop 2020-07-20T21:47:05.003Z,1595281625.003 [MissionManager](IMPORTANT): Started mission Default 2020-07-20T21:54:24.934Z,1595282064.934 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 7.325240 min 2020-07-20T21:55:00.010Z,1595282100.010 [DataOverHttps](IMPORTANT): SBD MTMSN=20200720T215459 2020-07-20T21:55:07.354Z,1595282107.354 [CommandLine](IMPORTANT): got command run ./Missions/Maintenance/ballast_and_trim.xml 2020-07-20T21:55:09.731Z,1595282109.731 [CommandLine](IMPORTANT): Running ./Missions/Maintenance/ballast_and_trim.xml 2020-07-20T21:55:09.791Z,1595282109.791 [MissionManager](IMPORTANT): Started mission ballast_and_trim 2020-07-20T22:30:16.330Z,1595284216.330 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): BallastAndTrim completed. Estimation time: 18.65 minutes. 2020-07-20T22:30:16.331Z,1595284216.331 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): Estimated buoyancyNeutral: 194.331976 +/- 6.644171 cc (conf. level 99.90%, sigma: 11.058033 cc). 2020-07-20T22:30:16.331Z,1595284216.331 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): Estimated massDefault: 4.752829 +/- 0.249994 mm (conf. level 99.90%, sigma: 0.416070 mm). 2020-07-20T22:30:16.332Z,1595284216.332 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): Changing persisted config values to: 194.331976 cc buoyancy neutral and 4.752829 mm mass default 2020-07-20T22:30:16.341Z,1595284216.341 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): Listing configuration overrides from Data/persisted.cfg 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): BPC1.batteryMissingStickThreshold=2 count; 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): CBIT.gf24Offset=-26 microampere; 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): DVL_micro.loadAtStartup=0 bool; 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): ESPComponent.loadAtStartup=1 bool; 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): ESPComponent.simulateHardware=0 bool; 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): VerticalControl.buoyancyNeutral=194.331976 cubic_centimeter; 2020-07-20T22:30:16.342Z,1595284216.342 [ballast_and_trim:RunBallastAndTrim:BallastAndTrim:BallastAndTrimAtTargetDepth:F.](IMPORTANT): VerticalControl.massDefault=4.752829 millimeter; 2020-07-20T22:34:04.279Z,1595284444.279 [MissionManager](IMPORTANT): Started mission Default 2020-07-20T22:40:11.593Z,1595284811.593 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 6.114989 min 2020-07-20T22:45:41.756Z,1595285141.756 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 11.617774 min 2020-07-20T22:51:19.540Z,1595285479.540 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 17.247677 min 2020-07-20T22:52:30.998Z,1595285550.998 [DataOverHttps](IMPORTANT): SBD MTMSN=20200720T225230 2020-07-20T22:52:38.288Z,1595285558.288 [CommandLine](IMPORTANT): got command load ./Missions/Maintenance/calibrate_sparton_compass.xml 2020-07-20T22:52:39.993Z,1595285559.993 [CommandLine](IMPORTANT): Loaded ./Missions/Maintenance/calibrate_sparton_compass.xml 2020-07-20T22:52:45.887Z,1595285565.887 [CommandLine](IMPORTANT): got command set SpartonCompassCal.YoYoMaxDepth 30.000000 meter 2020-07-20T22:52:45.888Z,1595285565.888 [CommandLine](IMPORTANT): got command set SpartonCompassCal.MaxDepth 35.000000 meter 2020-07-20T22:52:45.888Z,1595285565.888 [CommandLine](IMPORTANT): got command run 2020-07-20T22:52:45.894Z,1595285565.894 [CommandLine](IMPORTANT): Running 2020-07-20T22:52:46.012Z,1595285566.012 [MissionManager](IMPORTANT): Started mission SpartonCompassCal 2020-07-20T23:06:16.517Z,1595286376.517 [ElevatorServo](FAULT): Overload Error 2020-07-20T23:06:16.581Z,1595286376.581 [ThrusterServo](FAULT): Overload Error 2020-07-20T23:52:46.634Z,1595289166.634 [AHRS_M2](IMPORTANT): Acquired 10 calibration points. 2020-07-20T23:52:46.650Z,1595289166.650 [MissionManager](IMPORTANT): Started mission Default 2020-07-20T23:52:47.414Z,1595289167.414 [AHRS_M2](IMPORTANT): Magnetic calibration quality (0[best] to 10000) is 0.000194. 2020-07-21T00:02:55.045Z,1595289775.045 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.133208 min 2020-07-21T00:08:25.150Z,1595290105.150 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 15.634813 min 2020-07-21T00:13:54.810Z,1595290434.810 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.129240 min 2020-07-21T00:19:24.913Z,1595290764.913 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 26.630853 min 2020-07-21T00:24:44.131Z,1595291084.131 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T002443 2020-07-21T00:24:51.400Z,1595291091.400 [CommandLine](IMPORTANT): got command schedule asap "load Science/smear_yoyo_sampling.xml;set Smear.NumESPSamplers 1 count;set Smear.MissionTimeout 5 hour;set Smear.MaxWaitNoFiring 50000 day;set Smear.SpeedTransit 0.65 meter_per_second;set Smear.SpeedSampling 0.65 meter_per_second" 3ajsm 1 3.000000 2020-07-21T00:24:51.401Z,1595291091.401 [CommandLine](IMPORTANT): Scheduling command #1 of 3 with id=3ajsm 2020-07-21T00:24:51.402Z,1595291091.402 [CommandLine](IMPORTANT): Scheduled #3 (#1 of 3 with id='3ajsm'): "load Science/smear_yoyo_sampling.xml;set Smear.NumESPSamplers 1 count;set Smear.MissionTimeout 5 hour;set Smear.MaxWaitNoFiring 50000 day;set Smear.SpeedTransit 0.65 meter_per_second;set Smear.SpeedSampling 0.65 meter_per_second" ASAP 2020-07-21T00:24:52.347Z,1595291092.347 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T002451 2020-07-21T00:24:55.010Z,1595291095.010 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 32.132935 min 2020-07-21T00:24:59.506Z,1595291099.506 [CommandLine](IMPORTANT): got command schedule asap "set Smear.MaxDepth 60 meter;set Smear.TransitYoYoMinDepth 5 meter;set Smear.TransitLatitude 36.797 degree;set Smear.TransitLongitude -121.847 degree;set Smear.CenterLatitude 36.797 degree;set Smear.CenterLongitude -121.847 degree" 3ajsm 2 3.000000 2020-07-21T00:24:59.506Z,1595291099.506 [CommandLine](IMPORTANT): Scheduling command #2 of 3 with id=3ajsm 2020-07-21T00:24:59.507Z,1595291099.507 [CommandLine](IMPORTANT): Scheduled #4 (#2 of 3 with id='3ajsm'): "set Smear.MaxDepth 60 meter;set Smear.TransitYoYoMinDepth 5 meter;set Smear.TransitLatitude 36.797 degree;set Smear.TransitLongitude -121.847 degree;set Smear.CenterLatitude 36.797 degree;set Smear.CenterLongitude -121.847 degree" ASAP 2020-07-21T00:25:00.285Z,1595291100.285 [DataOverHttps](IMPORTANT): SBD MOMSN=12457705, MTMSN=20200721T002459 2020-07-21T00:25:07.634Z,1595291107.634 [CommandLine](IMPORTANT): got command schedule asap "set Smear.ShallowBoundSmear1 5 meter;set Smear.ThicknessSmear1 45 meter;set Smear.Wait1 5 minute;run " 3ajsm 3 3.000000 2020-07-21T00:25:07.635Z,1595291107.635 [CommandLine](IMPORTANT): Scheduling command #3 of 3 with id=3ajsm 2020-07-21T00:25:07.635Z,1595291107.635 [CommandLine](IMPORTANT): Scheduled #5 (#3 of 3 with id='3ajsm'): "set Smear.ShallowBoundSmear1 5 meter;set Smear.ThicknessSmear1 45 meter;set Smear.Wait1 5 minute;run " ASAP 2020-07-21T00:25:07.706Z,1595291107.706 [CommandLine](IMPORTANT): got command load ./Missions/Science/smear_yoyo_sampling.xml 2020-07-21T00:25:24.322Z,1595291124.322 [CommandLine](IMPORTANT): Loaded ./Missions/Science/smear_yoyo_sampling.xml 2020-07-21T00:25:43.472Z,1595291143.472 [CommandLine](IMPORTANT): got command set Smear.NumESPSamplers 1.000000 count 2020-07-21T00:25:43.473Z,1595291143.473 [CommandLine](IMPORTANT): got command set Smear.MissionTimeout 5.000000 hour 2020-07-21T00:25:43.473Z,1595291143.473 [CommandLine](IMPORTANT): got command set Smear.MaxWaitNoFiring 50000.000000 day 2020-07-21T00:25:43.474Z,1595291143.474 [CommandLine](IMPORTANT): got command set Smear.SpeedTransit 0.650000 meter_per_second 2020-07-21T00:25:43.475Z,1595291143.475 [CommandLine](IMPORTANT): got command set Smear.SpeedSampling 0.650000 meter_per_second 2020-07-21T00:26:02.809Z,1595291162.809 [CommandLine](IMPORTANT): got command set Smear.MaxDepth 60.000000 meter 2020-07-21T00:26:02.810Z,1595291162.810 [CommandLine](IMPORTANT): got command set Smear.TransitYoYoMinDepth 5.000000 meter 2020-07-21T00:26:02.811Z,1595291162.811 [CommandLine](IMPORTANT): got command set Smear.TransitLatitude 36.797001 degree 2020-07-21T00:26:02.811Z,1595291162.811 [CommandLine](IMPORTANT): got command set Smear.TransitLongitude -121.847000 degree 2020-07-21T00:26:02.812Z,1595291162.812 [CommandLine](IMPORTANT): got command set Smear.CenterLatitude 36.797001 degree 2020-07-21T00:26:02.813Z,1595291162.813 [CommandLine](IMPORTANT): got command set Smear.CenterLongitude -121.847000 degree 2020-07-21T00:26:13.221Z,1595291173.221 [CommandLine](IMPORTANT): got command set Smear.ShallowBoundSmear1 5.000000 meter 2020-07-21T00:26:13.222Z,1595291173.222 [CommandLine](IMPORTANT): got command set Smear.ThicknessSmear1 45.000000 meter 2020-07-21T00:26:13.222Z,1595291173.222 [CommandLine](IMPORTANT): got command set Smear.Wait1 5.000000 minute 2020-07-21T00:26:13.223Z,1595291173.223 [CommandLine](IMPORTANT): got command run 2020-07-21T00:26:13.253Z,1595291173.253 [CommandLine](IMPORTANT): Running 2020-07-21T00:26:13.468Z,1595291173.468 [MissionManager](IMPORTANT): Started mission Smear 2020-07-21T00:26:13.560Z,1595291173.560 [Smear:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-07-21T00:26:14.716Z,1595291174.716 [Smear:TakeSamples:SetOptionsOfSample1:SettingsSample1:C](IMPORTANT): Set YoYoMinDepth to 5.000000 m , YoYoMaxDepth to 50.000000 m . 2020-07-21T01:10:01.706Z,1595293801.706 [Smear:TransitToSamplingLocation:G.Waypoint](IMPORTANT): Reached Waypoint: 36.797002,-121.846996 2020-07-21T01:10:03.328Z,1595293803.328 [Smear:TakeSamples:Firing:B](IMPORTANT): Taking ESP sample No. 1.000000 count . 2020-07-21T01:10:03.330Z,1595293803.330 [Smear:Cylinder:CylinderYoYo:A](IMPORTANT): Yoyo profile on a cylinder around location 36.797002 arcdeg , -121.846996 arcdeg , at radius = 20.000000 m . 2020-07-21T01:10:04.532Z,1595293804.532 [Smear:InsertWait:A](IMPORTANT): Wait 5.000000 min before firing sample. 2020-07-21T01:15:07.143Z,1595294107.143 [Smear:ESPSampling:TriggerESP:D](IMPORTANT): Trigger ESP sampling. DepthLogged, TempLogged, ChlLogged = 42.663017 m , 12.366388 degC , 0.846800 ug/l . 2020-07-21T01:15:08.717Z,1595294108.717 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::' 2020-07-21T01:15:09.158Z,1595294109.158 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 134.89.10.51:134.89.10.60 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 134.89.10.32 ms-dns 134.89.10.10 2020-07-21T01:15:09.455Z,1595294109.455 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-07-21T01:15:44.795Z,1595294144.795 [ESPComponent](IMPORTANT): ESP has connected as client 2020-07-21T01:15:45.602Z,1595294145.602 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -6 2020-07-21T01:15:45.603Z,1595294145.603 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-07-21T01:15:46.025Z,1595294146.025 [Smear:ESPSampling:TriggerESP:G](IMPORTANT): Wait for ESP sampling to complete. 2020-07-21T01:15:46.809Z,1595294146.809 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2020-07-21T01:15:47.215Z,1595294147.215 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-07-21T01:15:48.835Z,1595294148.835 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2020-07-21T01:16:21.555Z,1595294181.555 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PRIMING (@58,archive_bac) 2020-07-21T01:16:23.169Z,1595294183.169 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PRIMING (@100.0) 2020-07-21T01:16:23.979Z,1595294183.979 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-07-21T01:17:41.953Z,1595294261.953 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING (@0.0) 2020-07-21T01:17:52.862Z,1595294272.862 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-07-21T01:17:53.278Z,1595294273.278 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2020-07-21T01:22:53.051Z,1595294573.051 [ESPComponent](CRITICAL): [sample #1] ESP sampling sequence terminated with error: In state S_WAITING_FOR_FILTERING got exception while awaiting completion of cmd='Cmd.startFiltering' exception=<>. Elapsed time since command issued: 80.4041 2020-07-21T01:22:53.178Z,1595294573.178 [CommandLine](FAULT): Scheduling is paused 2020-07-21T01:22:53.465Z,1595294573.465 [ESPComponent](IMPORTANT): [sample #1] ESP log summary report (8 messages): @18:15:58.53 Selecting Cartridge 58 @18:16:28.38 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 15% to 72.7psia @18:16:28.40 Retry #1 of 2 @18:16:28.45 Selecting Cartridge 57 @18:16:51.75 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 16% to 69.3psia @18:16:51.76 Retry #2 of 2 @18:16:51.82 Selecting Cartridge 56 @18:17:36.08 Cartridge::Sampler::Leak in FILTERING -- Serious Leak or Stall -- 0.00ml/s made only -4.8psi vacuum 2020-07-21T01:22:54.429Z,1595294574.429 [MissionManager](IMPORTANT): Started mission Default 2020-07-21T01:22:55.606Z,1595294575.606 [ESPComponent](IMPORTANT): [sample #1] No ESP summary messages. 2020-07-21T01:31:26.543Z,1595295086.543 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 8.528420 min 2020-07-21T01:34:24.991Z,1595295264.991 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T013424 2020-07-21T01:34:32.447Z,1595295272.447 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.NeedCommsTime 20 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.YoYoMinDepth 5 meter" 3an11 1 2.000000 2020-07-21T01:34:32.447Z,1595295272.447 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=3an11 2020-07-21T01:34:32.449Z,1595295272.449 [CommandLine](IMPORTANT): Scheduled #6 (#1 of 2 with id='3an11'): "load Science/profile_station.xml;set profile_station.NeedCommsTime 20 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.YoYoMinDepth 5 meter" ASAP 2020-07-21T01:34:33.415Z,1595295273.415 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T013432 2020-07-21T01:34:40.756Z,1595295280.756 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.65 meter_per_second;set profile_station.MaxDepth 60 meter;run " 3an11 2 2.000000 2020-07-21T01:34:40.756Z,1595295280.756 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=3an11 2020-07-21T01:34:40.757Z,1595295280.757 [CommandLine](IMPORTANT): Scheduled #7 (#2 of 2 with id='3an11'): "set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.65 meter_per_second;set profile_station.MaxDepth 60 meter;run " ASAP 2020-07-21T01:34:40.910Z,1595295280.910 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-07-21T01:34:42.255Z,1595295282.255 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-07-21T01:34:53.207Z,1595295293.207 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 20.000000 minute 2020-07-21T01:34:53.207Z,1595295293.207 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-07-21T01:34:53.208Z,1595295293.208 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-07-21T01:34:53.224Z,1595295293.224 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 5.000000 meter 2020-07-21T01:35:01.526Z,1595295301.526 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2020-07-21T01:35:01.527Z,1595295301.527 [CommandLine](IMPORTANT): got command set profile_station.Speed 0.650000 meter_per_second 2020-07-21T01:35:01.528Z,1595295301.528 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 60.000000 meter 2020-07-21T01:35:01.528Z,1595295301.528 [CommandLine](IMPORTANT): got command run 2020-07-21T01:35:01.530Z,1595295301.530 [CommandLine](IMPORTANT): Running 2020-07-21T01:35:01.894Z,1595295301.894 [MissionManager](IMPORTANT): Started mission profile_station 2020-07-21T01:44:12.665Z,1595295852.665 [CBIT](IMPORTANT): Beginning ground fault scan 2020-07-21T01:44:23.668Z,1595295863.668 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.004748 CHAN A1 (24V): -0.001255 CHAN A2 (12V): -0.000230 CHAN A3 (5V): 0.000042 CHAN B0 (3.3V): 0.000172 CHAN B1 (3.15aV): 0.000724 CHAN B2 (3.15bV): -0.000271 CHAN B3 (GND): 0.000984 OPEN: 0.003979 Full Scale Calc: 4.765 mA, -1.589 mA 2020-07-21T01:57:11.884Z,1595296631.884 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=1256, MT Status=1, MTMSN=81 2020-07-21T01:57:12.473Z,1595296632.473 [CommandLine](IMPORTANT): got command stop 2020-07-21T01:57:12.473Z,1595296632.473 [CommandLine](IMPORTANT): Scheduling is paused 2020-07-21T01:57:13.134Z,1595296633.134 [MissionManager](IMPORTANT): Started mission Default 2020-07-21T02:04:05.575Z,1595297045.575 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 6.867354 min 2020-07-21T02:09:37.343Z,1595297377.343 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 12.396682 min 2020-07-21T02:15:07.050Z,1595297707.050 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 17.891282 min 2020-07-21T02:16:18.054Z,1595297778.054 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T021617 2020-07-21T02:16:28.254Z,1595297788.254 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.NeedCommsTime 20 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.YoYoMinDepth 5 meter" 3aoyz 1 2.000000 2020-07-21T02:16:28.254Z,1595297788.254 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=3aoyz 2020-07-21T02:16:28.255Z,1595297788.255 [CommandLine](IMPORTANT): Scheduled #8 (#1 of 2 with id='3aoyz'): "load Science/profile_station.xml;set profile_station.NeedCommsTime 20 minute;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.YoYoMinDepth 5 meter" ASAP 2020-07-21T02:16:29.214Z,1595297789.214 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T021628 2020-07-21T02:16:36.562Z,1595297796.562 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.65 meter_per_second;set profile_station.MaxDepth 60 meter;run " 3aoyz 2 2.000000 2020-07-21T02:16:36.562Z,1595297796.562 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=3aoyz 2020-07-21T02:16:36.563Z,1595297796.563 [CommandLine](IMPORTANT): Scheduled #9 (#2 of 2 with id='3aoyz'): "set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.65 meter_per_second;set profile_station.MaxDepth 60 meter;run " ASAP 2020-07-21T02:16:36.683Z,1595297796.683 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2020-07-21T02:16:38.035Z,1595297798.035 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2020-07-21T02:16:49.217Z,1595297809.217 [CommandLine](IMPORTANT): got command set profile_station.NeedCommsTime 20.000000 minute 2020-07-21T02:16:49.218Z,1595297809.218 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2020-07-21T02:16:49.218Z,1595297809.218 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2020-07-21T02:16:49.219Z,1595297809.219 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 5.000000 meter 2020-07-21T02:16:57.805Z,1595297817.805 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2020-07-21T02:16:57.806Z,1595297817.806 [CommandLine](IMPORTANT): got command set profile_station.Speed 0.650000 meter_per_second 2020-07-21T02:16:57.807Z,1595297817.807 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 60.000000 meter 2020-07-21T02:16:57.807Z,1595297817.807 [CommandLine](IMPORTANT): got command run 2020-07-21T02:16:57.809Z,1595297817.809 [CommandLine](IMPORTANT): Running 2020-07-21T02:16:58.155Z,1595297818.155 [MissionManager](IMPORTANT): Started mission profile_station 2020-07-21T02:38:57.774Z,1595299137.774 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=1269, MT Status=1, MTMSN=82 2020-07-21T02:38:58.357Z,1595299138.357 [CommandLine](IMPORTANT): got command stop 2020-07-21T02:38:59.022Z,1595299139.022 [MissionManager](IMPORTANT): Started mission Default 2020-07-21T02:38:59.433Z,1595299139.433 [WetLabsBB2FL](FAULT): LCB fault: Software Overcurrent. 2020-07-21T02:43:08.078Z,1595299388.078 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T024307 2020-07-21T02:43:15.421Z,1595299395.421 [CommandLine](IMPORTANT): got command stop 2020-07-21T02:46:51.368Z,1595299611.368 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 7.865697 min 2020-07-21T02:52:38.038Z,1595299958.038 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 13.643636 min 2020-07-21T02:58:08.147Z,1595300288.147 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 19.145361 min 2020-07-21T03:03:41.904Z,1595300621.904 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 24.708175 min 2020-07-21T03:09:12.455Z,1595300952.455 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 30.217432 min 2020-07-21T03:14:42.126Z,1595301282.126 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 35.711629 min 2020-07-21T03:17:04.074Z,1595301424.074 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T031703 2020-07-21T03:17:11.558Z,1595301431.558 [CommandLine](IMPORTANT): got command schedule clear 2020-07-21T03:17:11.558Z,1595301431.558 [CommandLine](IMPORTANT): #Cleared scheduled commands:0 2020-07-21T03:17:12.538Z,1595301432.538 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T031711 2020-07-21T03:17:20.061Z,1595301440.061 [CommandLine](IMPORTANT): got command schedule resume 2020-07-21T03:17:20.061Z,1595301440.061 [CommandLine](IMPORTANT): Scheduling is resumed 2020-07-21T03:20:12.242Z,1595301612.242 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 41.213566 min 2020-07-21T03:26:13.510Z,1595301973.510 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 47.234831 min 2020-07-21T03:30:22.108Z,1595302222.108 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T033021 2020-07-21T03:30:30.543Z,1595302230.543 [CommandLine](IMPORTANT): got command schedule asap "load Science/smear_yoyo_sampling.xml;set Smear.NumESPSamplers 1 count;set Smear.MissionTimeout 5 hour;set Smear.MaxWaitNoFiring 50000 day;set Smear.SpeedTransit 0.65 meter_per_second;set Smear.SpeedSampling 0.65 meter_per_second" 3asdp 1 3.000000 2020-07-21T03:30:30.543Z,1595302230.543 [CommandLine](IMPORTANT): Scheduling command #1 of 3 with id=3asdp 2020-07-21T03:30:30.544Z,1595302230.544 [CommandLine](IMPORTANT): Scheduled #10 (#1 of 3 with id='3asdp'): "load Science/smear_yoyo_sampling.xml;set Smear.NumESPSamplers 1 count;set Smear.MissionTimeout 5 hour;set Smear.MaxWaitNoFiring 50000 day;set Smear.SpeedTransit 0.65 meter_per_second;set Smear.SpeedSampling 0.65 meter_per_second" ASAP 2020-07-21T03:30:31.526Z,1595302231.526 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T033031 2020-07-21T03:30:38.882Z,1595302238.882 [CommandLine](IMPORTANT): got command schedule asap "set Smear.MaxDepth 60 meter;set Smear.TransitYoYoMinDepth 5 meter;set Smear.TransitLatitude 36.797 degree;set Smear.TransitLongitude -121.847 degree;set Smear.CenterLatitude 36.797 degree;set Smear.CenterLongitude -121.847 degree" 3asdp 2 3.000000 2020-07-21T03:30:38.883Z,1595302238.883 [CommandLine](IMPORTANT): Scheduling command #2 of 3 with id=3asdp 2020-07-21T03:30:38.883Z,1595302238.883 [CommandLine](IMPORTANT): Scheduled #11 (#2 of 3 with id='3asdp'): "set Smear.MaxDepth 60 meter;set Smear.TransitYoYoMinDepth 5 meter;set Smear.TransitLatitude 36.797 degree;set Smear.TransitLongitude -121.847 degree;set Smear.CenterLatitude 36.797 degree;set Smear.CenterLongitude -121.847 degree" ASAP 2020-07-21T03:30:39.835Z,1595302239.835 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T033039 2020-07-21T03:30:47.184Z,1595302247.184 [CommandLine](IMPORTANT): got command schedule asap "set Smear.ShallowBoundSmear1 5 meter;set Smear.ThicknessSmear1 45 meter;set Smear.Wait1 5 minute;run " 3asdp 3 3.000000 2020-07-21T03:30:47.185Z,1595302247.185 [CommandLine](IMPORTANT): Scheduling command #3 of 3 with id=3asdp 2020-07-21T03:30:47.186Z,1595302247.186 [CommandLine](IMPORTANT): Scheduled #12 (#3 of 3 with id='3asdp'): "set Smear.ShallowBoundSmear1 5 meter;set Smear.ThicknessSmear1 45 meter;set Smear.Wait1 5 minute;run " ASAP 2020-07-21T03:30:47.452Z,1595302247.452 [CommandLine](IMPORTANT): got command load ./Missions/Science/smear_yoyo_sampling.xml 2020-07-21T03:30:56.383Z,1595302256.383 [CommandLine](IMPORTANT): Loaded ./Missions/Science/smear_yoyo_sampling.xml 2020-07-21T03:31:08.360Z,1595302268.360 [CommandLine](IMPORTANT): got command set Smear.NumESPSamplers 1.000000 count 2020-07-21T03:31:08.361Z,1595302268.361 [CommandLine](IMPORTANT): got command set Smear.MissionTimeout 5.000000 hour 2020-07-21T03:31:08.361Z,1595302268.361 [CommandLine](IMPORTANT): got command set Smear.MaxWaitNoFiring 50000.000000 day 2020-07-21T03:31:08.362Z,1595302268.362 [CommandLine](IMPORTANT): got command set Smear.SpeedTransit 0.650000 meter_per_second 2020-07-21T03:31:08.364Z,1595302268.364 [CommandLine](IMPORTANT): got command set Smear.SpeedSampling 0.650000 meter_per_second 2020-07-21T03:31:26.502Z,1595302286.502 [CommandLine](IMPORTANT): got command set Smear.MaxDepth 60.000000 meter 2020-07-21T03:31:26.503Z,1595302286.503 [CommandLine](IMPORTANT): got command set Smear.TransitYoYoMinDepth 5.000000 meter 2020-07-21T03:31:26.503Z,1595302286.503 [CommandLine](IMPORTANT): got command set Smear.TransitLatitude 36.797001 degree 2020-07-21T03:31:26.504Z,1595302286.504 [CommandLine](IMPORTANT): got command set Smear.TransitLongitude -121.847000 degree 2020-07-21T03:31:26.505Z,1595302286.505 [CommandLine](IMPORTANT): got command set Smear.CenterLatitude 36.797001 degree 2020-07-21T03:31:26.506Z,1595302286.506 [CommandLine](IMPORTANT): got command set Smear.CenterLongitude -121.847000 degree 2020-07-21T03:31:33.510Z,1595302293.510 [CommandLine](IMPORTANT): got command set Smear.ShallowBoundSmear1 5.000000 meter 2020-07-21T03:31:33.510Z,1595302293.510 [CommandLine](IMPORTANT): got command set Smear.ThicknessSmear1 45.000000 meter 2020-07-21T03:31:33.511Z,1595302293.511 [CommandLine](IMPORTANT): got command set Smear.Wait1 5.000000 minute 2020-07-21T03:31:33.511Z,1595302293.511 [CommandLine](IMPORTANT): got command run 2020-07-21T03:31:33.521Z,1595302293.521 [CommandLine](IMPORTANT): Running 2020-07-21T03:31:33.893Z,1595302293.893 [MissionManager](IMPORTANT): Started mission Smear 2020-07-21T03:31:33.973Z,1595302293.973 [Smear:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 21.00 hours. 2020-07-21T03:31:35.120Z,1595302295.120 [Smear:TakeSamples:SetOptionsOfSample1:SettingsSample1:C](IMPORTANT): Set YoYoMinDepth to 5.000000 m , YoYoMaxDepth to 50.000000 m . 2020-07-21T03:40:25.999Z,1595302825.999 [Smear:TransitToSamplingLocation:G.Waypoint](IMPORTANT): Reached Waypoint: 36.797002,-121.846996 2020-07-21T03:40:27.614Z,1595302827.614 [Smear:TakeSamples:Firing:B](IMPORTANT): Taking ESP sample No. 1.000000 count . 2020-07-21T03:40:27.615Z,1595302827.615 [Smear:Cylinder:CylinderYoYo:A](IMPORTANT): Yoyo profile on a cylinder around location 36.797002 arcdeg , -121.846996 arcdeg , at radius = 20.000000 m . 2020-07-21T03:40:28.816Z,1595302828.816 [Smear:InsertWait:A](IMPORTANT): Wait 5.000000 min before firing sample. 2020-07-21T03:45:31.428Z,1595303131.428 [Smear:ESPSampling:TriggerESP:D](IMPORTANT): Trigger ESP sampling. DepthLogged, TempLogged, ChlLogged = 15.576448 m , 13.336298 degC , 4.524000 ug/l . 2020-07-21T03:45:33.549Z,1595303133.549 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 134.89.10.51:134.89.10.60 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 134.89.10.32 ms-dns 134.89.10.10 2020-07-21T03:45:33.683Z,1595303133.683 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-07-21T03:46:08.579Z,1595303168.579 [ESPComponent](IMPORTANT): ESP has connected as client 2020-07-21T03:46:09.418Z,1595303169.418 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -6 2020-07-21T03:46:09.419Z,1595303169.419 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-07-21T03:46:09.835Z,1595303169.835 [Smear:ESPSampling:TriggerESP:G](IMPORTANT): Wait for ESP sampling to complete. 2020-07-21T03:46:10.189Z,1595303170.189 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG 2020-07-21T03:46:10.594Z,1595303170.594 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-07-21T03:46:12.609Z,1595303172.609 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE 2020-07-21T03:46:45.343Z,1595303205.343 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PRIMING (@58,archive_bac) 2020-07-21T03:46:46.953Z,1595303206.953 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PRIMING (@100.0) 2020-07-21T03:46:47.762Z,1595303207.762 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-07-21T03:51:39.471Z,1595303499.471 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_FILTERING (@0.0) 2020-07-21T03:51:39.876Z,1595303499.876 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-07-21T03:51:48.764Z,1595303508.764 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPING (@0.0) 2020-07-21T03:51:59.673Z,1595303519.673 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-07-21T03:52:00.072Z,1595303520.072 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPED 2020-07-21T03:56:59.855Z,1595303819.855 [ESPComponent](CRITICAL): [sample #2] ESP sampling sequence terminated with error: In state S_WAITING_FOR_PAUSED got exception while awaiting completion of cmd='Cmd.startFiltering' exception=<>. Elapsed time since command issued: 303.4246 2020-07-21T03:56:59.942Z,1595303819.942 [CommandLine](FAULT): Scheduling is paused 2020-07-21T03:57:00.260Z,1595303820.260 [ESPComponent](IMPORTANT): [sample #2] ESP log summary report (8 messages): @20:46:22.16 Selecting Cartridge 58 @20:46:55.29 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 16% to 44.7psia @20:46:55.30 Retry #1 of 2 @20:46:55.35 Selecting Cartridge 57 @20:47:17.65 Cartridge::Sampler::Leak in FILTERING -- Isolated toroid pressure fell by 16% to 49.6psia @20:47:17.66 Retry #2 of 2 @20:47:17.72 Selecting Cartridge 56 @20:51:42.51 Cartridge::Sampler::Leak in FILTERING -- Serious Leak or Stall -- 0.00ml/s made only -6.0psi vacuum 2020-07-21T03:57:01.166Z,1595303821.166 [MissionManager](IMPORTANT): Started mission Default 2020-07-21T03:57:02.312Z,1595303822.312 [ESPComponent](IMPORTANT): [sample #2] No ESP summary messages. 2020-07-21T04:05:22.148Z,1595304322.148 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 8.343137 min 2020-07-21T04:11:33.169Z,1595304693.169 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 14.526469 min 2020-07-21T04:16:58.851Z,1595305018.851 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 19.954956 min 2020-07-21T04:22:30.536Z,1595305350.536 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 25.482821 min 2020-07-21T04:23:42.495Z,1595305422.495 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T042341 2020-07-21T04:23:49.806Z,1595305429.806 [CommandLine](IMPORTANT): got command schedule asap "load Science/esp_sample_at_depth.xml;set esp_sample_at_depth.TargetDepth 25 meter;set esp_sample_at_depth.ESPCartridgeType_1 -6 count;set esp_sample_at_depth.ESPCartridgeType_2 NaN count" 3auuv 1 2.000000 2020-07-21T04:23:49.806Z,1595305429.806 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=3auuv 2020-07-21T04:23:49.807Z,1595305429.807 [CommandLine](IMPORTANT): Scheduled #13 (#1 of 2 with id='3auuv'): "load Science/esp_sample_at_depth.xml;set esp_sample_at_depth.TargetDepth 25 meter;set esp_sample_at_depth.ESPCartridgeType_1 -6 count;set esp_sample_at_depth.ESPCartridgeType_2 NaN count" ASAP 2020-07-21T04:23:50.730Z,1595305430.730 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T042350 2020-07-21T04:23:58.269Z,1595305438.269 [CommandLine](IMPORTANT): got command schedule asap "set esp_sample_at_depth.ApproachSpeed 0.65 meter_per_second;set esp_sample_at_depth.MaxDepth 50 meter;run " 3auuv 2 2.000000 2020-07-21T04:23:58.270Z,1595305438.270 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=3auuv 2020-07-21T04:23:58.271Z,1595305438.271 [CommandLine](IMPORTANT): Scheduled #14 (#2 of 2 with id='3auuv'): "set esp_sample_at_depth.ApproachSpeed 0.65 meter_per_second;set esp_sample_at_depth.MaxDepth 50 meter;run " ASAP 2020-07-21T04:23:58.349Z,1595305438.349 [CommandLine](IMPORTANT): got command load ./Missions/Science/esp_sample_at_depth.xml 2020-07-21T04:24:00.549Z,1595305440.549 [CommandLine](IMPORTANT): Loaded ./Missions/Science/esp_sample_at_depth.xml 2020-07-21T04:24:13.765Z,1595305453.765 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.TargetDepth 25.000000 meter 2020-07-21T04:24:13.765Z,1595305453.765 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.ESPCartridgeType_1 -6.000000 count 2020-07-21T04:24:13.766Z,1595305453.766 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.ESPCartridgeType_2 nan count 2020-07-21T04:24:21.261Z,1595305461.261 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.ApproachSpeed 0.650000 meter_per_second 2020-07-21T04:24:21.262Z,1595305461.262 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.MaxDepth 50.000000 meter 2020-07-21T04:24:21.262Z,1595305461.262 [CommandLine](IMPORTANT): got command run 2020-07-21T04:24:21.272Z,1595305461.272 [MissionManager](IMPORTANT): Started mission esp_sample_at_depth 2020-07-21T04:24:21.283Z,1595305461.283 [CommandLine](IMPORTANT): Running 2020-07-21T04:29:04.484Z,1595305744.484 [esp_sample_at_depth:SampleESP:sample:A](IMPORTANT): At 1.464792 m , settling for 3.000000 min 2020-07-21T04:32:05.474Z,1595305925.474 [esp_sample_at_depth:SampleESP:sample:C](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-07-21T04:32:07.585Z,1595305927.585 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 134.89.10.51:134.89.10.60 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 134.89.10.32 ms-dns 134.89.10.10 2020-07-21T04:32:07.723Z,1595305927.723 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-07-21T04:32:42.635Z,1595305962.635 [ESPComponent](IMPORTANT): ESP has connected as client 2020-07-21T04:32:43.446Z,1595305963.446 [ESPComponent](IMPORTANT): [sample #3] Setting cartridge argument to -6 2020-07-21T04:32:43.447Z,1595305963.447 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-07-21T04:32:43.863Z,1595305963.863 [esp_sample_at_depth:SampleESP:sample:F](IMPORTANT): ESP sampling at 0.460392 m in 14.643763 degC water with 4.077400 ug/l chlorophyll fluorescence. 2020-07-21T04:32:44.251Z,1595305964.251 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_LOG 2020-07-21T04:32:44.651Z,1595305964.651 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-07-21T04:32:46.675Z,1595305966.675 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_LOADING_CARTRIDGE 2020-07-21T04:33:19.407Z,1595305999.407 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PRIMING (@55,archive_bac) 2020-07-21T04:33:21.015Z,1595306001.015 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PRIMING (@100.0) 2020-07-21T04:33:21.818Z,1595306001.818 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-07-21T04:36:45.444Z,1595306205.444 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_FILTERING (@0.0) 2020-07-21T04:36:45.847Z,1595306205.847 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-07-21T04:37:23.817Z,1595306243.817 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PAUSED (@6.309) 2020-07-21T04:37:24.227Z,1595306244.227 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PROCESSING (@6.309) 2020-07-21T04:37:26.656Z,1595306246.656 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PROCESSED (@55,archive_bac) 2020-07-21T04:49:30.247Z,1595306970.247 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPING (@55,archive_bac) 2020-07-21T04:49:55.708Z,1595306995.708 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-07-21T04:49:56.109Z,1595306996.109 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPED 2020-07-21T04:54:55.886Z,1595307295.886 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence completed normally. 2020-07-21T04:54:56.311Z,1595307296.311 [ESPComponent](IMPORTANT): [sample #3] ESP log summary report (3 messages): @21:32:56.24 Selecting Cartridge 55 @21:37:21.22 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @21:37:21.42 Sampled 6.3ml 2020-07-21T04:54:57.923Z,1595307297.923 [MissionManager](IMPORTANT): Started mission Default 2020-07-21T05:05:20.592Z,1595307920.592 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.371082 min 2020-07-21T05:10:51.084Z,1595308251.084 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 15.879356 min 2020-07-21T05:16:20.746Z,1595308580.746 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.373661 min 2020-07-21T05:18:43.559Z,1595308723.559 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T051842 2020-07-21T05:18:51.030Z,1595308731.030 [CommandLine](IMPORTANT): got command schedule asap "load Science/esp_sample_at_depth.xml;set esp_sample_at_depth.TargetDepth 25 meter;set esp_sample_at_depth.ESPCartridgeType_1 -6 count;set esp_sample_at_depth.ESPCartridgeType_2 NaN count" 3axer 1 2.000000 2020-07-21T05:18:51.031Z,1595308731.031 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=3axer 2020-07-21T05:18:51.031Z,1595308731.031 [CommandLine](IMPORTANT): Scheduled #15 (#1 of 2 with id='3axer'): "load Science/esp_sample_at_depth.xml;set esp_sample_at_depth.TargetDepth 25 meter;set esp_sample_at_depth.ESPCartridgeType_1 -6 count;set esp_sample_at_depth.ESPCartridgeType_2 NaN count" ASAP 2020-07-21T05:18:51.958Z,1595308731.958 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T051851 2020-07-21T05:18:59.350Z,1595308739.350 [CommandLine](IMPORTANT): got command schedule asap "set esp_sample_at_depth.ApproachSpeed 0.65 meter_per_second;set esp_sample_at_depth.MaxDepth 50 meter;run " 3axer 2 2.000000 2020-07-21T05:18:59.350Z,1595308739.350 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=3axer 2020-07-21T05:18:59.351Z,1595308739.351 [CommandLine](IMPORTANT): Scheduled #16 (#2 of 2 with id='3axer'): "set esp_sample_at_depth.ApproachSpeed 0.65 meter_per_second;set esp_sample_at_depth.MaxDepth 50 meter;run " ASAP 2020-07-21T05:18:59.429Z,1595308739.429 [CommandLine](IMPORTANT): got command load ./Missions/Science/esp_sample_at_depth.xml 2020-07-21T05:19:00.861Z,1595308740.861 [CommandLine](IMPORTANT): Loaded ./Missions/Science/esp_sample_at_depth.xml 2020-07-21T05:19:13.476Z,1595308753.476 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.TargetDepth 25.000000 meter 2020-07-21T05:19:13.477Z,1595308753.477 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.ESPCartridgeType_1 -6.000000 count 2020-07-21T05:19:13.478Z,1595308753.478 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.ESPCartridgeType_2 nan count 2020-07-21T05:19:20.955Z,1595308760.955 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.ApproachSpeed 0.650000 meter_per_second 2020-07-21T05:19:20.989Z,1595308760.989 [CommandLine](IMPORTANT): got command set esp_sample_at_depth.MaxDepth 50.000000 meter 2020-07-21T05:19:20.989Z,1595308760.989 [CommandLine](IMPORTANT): got command run 2020-07-21T05:19:20.991Z,1595308760.991 [CommandLine](IMPORTANT): Running 2020-07-21T05:19:21.002Z,1595308761.002 [MissionManager](IMPORTANT): Started mission esp_sample_at_depth 2020-07-21T05:24:50.758Z,1595309090.758 [esp_sample_at_depth:SampleESP:sample:A](IMPORTANT): At 0.927153 m , settling for 3.000000 min 2020-07-21T05:27:51.751Z,1595309271.751 [esp_sample_at_depth:SampleESP:sample:C](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 2020-07-21T05:27:53.873Z,1595309273.873 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 134.89.10.51:134.89.10.60 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 134.89.10.32 ms-dns 134.89.10.10 2020-07-21T05:27:54.050Z,1595309274.050 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-07-21T05:28:28.910Z,1595309308.910 [ESPComponent](IMPORTANT): ESP has connected as client 2020-07-21T05:28:29.726Z,1595309309.726 [ESPComponent](IMPORTANT): [sample #4] Setting cartridge argument to -6 2020-07-21T05:28:29.726Z,1595309309.726 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-07-21T05:28:30.141Z,1595309310.141 [esp_sample_at_depth:SampleESP:sample:F](IMPORTANT): ESP sampling at 10.864597 m in 13.257929 degC water with 4.551840 ug/l chlorophyll fluorescence. 2020-07-21T05:28:30.526Z,1595309310.526 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_LOG 2020-07-21T05:28:30.927Z,1595309310.927 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-07-21T05:28:32.950Z,1595309312.950 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_LOADING_CARTRIDGE 2020-07-21T05:29:05.670Z,1595309345.670 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PRIMING (@58,archive_bac) 2020-07-21T05:29:07.292Z,1595309347.292 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PRIMING (@100.0) 2020-07-21T05:29:08.103Z,1595309348.103 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-07-21T05:32:25.671Z,1595309545.671 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_FILTERING (@0.0) 2020-07-21T05:32:26.076Z,1595309546.076 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-07-21T05:34:06.676Z,1595309646.676 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PAUSED (@20.494) 2020-07-21T05:34:07.074Z,1595309647.074 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PROCESSING (@20.494) 2020-07-21T05:34:09.496Z,1595309649.496 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PROCESSED (@58,archive_bac) 2020-07-21T05:46:12.679Z,1595310372.679 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPING (@58,archive_bac) 2020-07-21T05:46:38.126Z,1595310398.126 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-07-21T05:46:38.541Z,1595310398.541 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPED 2020-07-21T05:51:38.312Z,1595310698.312 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence completed normally. 2020-07-21T05:51:38.719Z,1595310698.719 [ESPComponent](IMPORTANT): [sample #4] ESP log summary report (3 messages): @22:28:42.67 Selecting Cartridge 58 @22:34:04.33 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @22:34:04.53 Sampled 20.5ml 2020-07-21T05:51:40.350Z,1595310700.350 [MissionManager](IMPORTANT): Started mission Default 2020-07-21T06:02:38.919Z,1595311358.919 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.969232 min 2020-07-21T06:08:08.614Z,1595311688.614 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 16.464225 min 2020-07-21T06:13:37.866Z,1595312017.866 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.952136 min 2020-07-21T06:19:08.116Z,1595312348.116 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 27.456069 min 2020-07-21T06:24:38.611Z,1595312678.611 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 32.964276 min 2020-07-21T06:30:07.888Z,1595313007.888 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 38.451949 min 2020-07-21T06:35:41.610Z,1595313341.610 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 44.014290 min 2020-07-21T06:39:49.626Z,1595313589.626 [DataOverHttps](IMPORTANT): SBD MTMSN=20200721T063948 2020-07-21T06:39:57.137Z,1595313597.137 [CommandLine](IMPORTANT): got command restart application 2020-07-21T06:40:00.606Z,1595313600.606 [CBIT](FAULT): Humidity exceeds 50% of running average: Humidity:2285304 %. Average:282287.125000