2023-04-27T09:21:59.850Z,1682587319.850 [Supervisor](IMPORTANT): Reading configuration overrides from Data/persisted.cfg
2023-04-27T09:22:03.964Z,1682587323.964 [StratificationFrontDetector](IMPORTANT): thresholdVTHI set to: 0.100000 degC
2023-04-27T09:22:05.056Z,1682587325.056 [SBIT](IMPORTANT): git: 2023-04-17
2023-04-27T09:22:05.057Z,1682587325.057 [SBIT](IMPORTANT): Kernel Release: 2.6.27.8
2023-04-27T09:24:19.653Z,1682587459.653 [SBIT](IMPORTANT): Beginning Startup BIT
2023-04-27T09:24:19.657Z,1682587459.657 [CBIT](IMPORTANT): Beginning ground fault scan
2023-04-27T09:24:31.104Z,1682587471.104 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): -0.003074
CHAN A1 (24V): 0.009726
CHAN A2 (12V): -0.000199
CHAN A3 (5V): -0.000040
CHAN B0 (3.3V): 0.000780
CHAN B1 (3.15aV): 0.000215
CHAN B2 (3.15bV): 0.000534
CHAN B3 (GND): 0.002697
OPEN: 0.002839
Full Scale: +/- 1 mA
2023-04-27T09:25:13.774Z,1682587513.774 [SBIT](IMPORTANT): SBIT PASSED
2023-04-27T09:25:13.774Z,1682587513.774 [SBIT](IMPORTANT): Listing configuration overrides from Data/persisted.cfg
2023-04-27T09:25:13.775Z,1682587513.775 [SBIT](IMPORTANT): BPC1.batteryMissingStickThreshold=12 count;

2023-04-27T09:25:13.775Z,1682587513.775 [SBIT](IMPORTANT): DAT.sbdAddress=-1 enum;

2023-04-27T09:25:13.775Z,1682587513.775 [SBIT](IMPORTANT): Depth_Keller.offset=3.6 decibar;

2023-04-27T09:25:13.775Z,1682587513.775 [SBIT](IMPORTANT): Express none CBIT.ampHoursUsed;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_salinity_homogeneity_index 0.050000 practical_salinity_unit;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_temperature_homogeneity_index 0.050000 celsius;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): Express none WetLabsBB2FL.bin_mean_mass_concentration_of_chlorophyll_in_sea_water;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): Express none height_above_sea_floor;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): StratificationFrontDetector.loadAtStartup=1 bool;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): StratificationFrontDetector.threshold_VTHI=0.1 celsius;

2023-04-27T09:25:13.776Z,1682587513.776 [SBIT](IMPORTANT): StratificationFrontDetector.verbosity=1 count;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalControl.buoyancyNeutral=208.617815 cubic_centimeter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalControl.massDefault=28.688721 millimeter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalControl.massPositionLimitAft=-28 millimeter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalControl.massPositionLimitFwd=32 millimeter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalControl.surfaceThreshold=1.5 meter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth1=3 meter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth2=5 meter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth3=10 meter;

2023-04-27T09:25:13.777Z,1682587513.777 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth4=15 meter;

2023-04-27T09:25:13.778Z,1682587513.778 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.loadAtStartup=1 bool;

2023-04-27T09:25:14.185Z,1682587514.185 [MissionManager](IMPORTANT): Started mission Startup
2023-04-27T09:26:15.223Z,1682587575.223 [MissionManager](IMPORTANT): Started mission Default
2023-04-27T09:34:01.919Z,1682588041.919 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 7.771632 min 
2023-04-27T09:37:04.185Z,1682588224.185 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T093703
2023-04-27T09:37:14.583Z,1682588234.583 [CommandExec](IMPORTANT): got command schedule asap "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 2.5 h;set IsothermDepthSampling.MaxWaitNoFiring 0.5 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool" 3ts1p 1 3.000000
2023-04-27T09:37:14.584Z,1682588234.584 [CommandExec](IMPORTANT): Scheduling command #1 of 3 with id=3ts1p
2023-04-27T09:37:14.585Z,1682588234.585 [CommandExec](IMPORTANT): Scheduled #1 (#1 of 3 with id='3ts1p'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.MissionTimeout 2.5 h;set IsothermDepthSampling.MaxWaitNoFiring 0.5 h;set IsothermDepthSampling.SendSampleStatusAndData 0 bool", ASAP
2023-04-27T09:37:15.493Z,1682588235.493 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T093714
2023-04-27T09:37:26.672Z,1682588246.672 [CommandExec](IMPORTANT): got command schedule asap "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m" 3ts1p 2 3.000000
2023-04-27T09:37:26.672Z,1682588246.672 [CommandExec](IMPORTANT): Scheduling command #2 of 3 with id=3ts1p
2023-04-27T09:37:26.673Z,1682588246.673 [CommandExec](IMPORTANT): Scheduled #2 (#2 of 3 with id='3ts1p'): "set IsothermDepthSampling.SendDataToMultipleModems 0 bool;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.ShallowBoundChl 3 m;set IsothermDepthSampling.DeepBoundChl 20 m", ASAP
2023-04-27T09:37:27.637Z,1682588247.637 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T093726
2023-04-27T09:37:35.310Z,1682588255.310 [CommandExec](IMPORTANT): got command schedule asap "set IsothermDepthSampling.Lat1 36.653 degree;set IsothermDepthSampling.Lon1 -121.874 degree;set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run" 3ts1p 3 3.000000
2023-04-27T09:37:35.311Z,1682588255.311 [CommandExec](IMPORTANT): Scheduling command #3 of 3 with id=3ts1p
2023-04-27T09:37:35.312Z,1682588255.312 [CommandExec](IMPORTANT): Scheduled #3 (#3 of 3 with id='3ts1p'): "set IsothermDepthSampling.Lat1 36.653 degree;set IsothermDepthSampling.Lon1 -121.874 degree;set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 m;set IsothermDepthSampling.WaitDepUndulation1 5 min;run", ASAP
2023-04-27T09:37:35.469Z,1682588255.469 [CommandExec](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml
2023-04-27T09:37:57.894Z,1682588277.894 [CommandExec](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml
2023-04-27T09:38:25.355Z,1682588305.355 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 2.5 hour
2023-04-27T09:38:25.356Z,1682588305.356 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 0.5 hour
2023-04-27T09:38:25.357Z,1682588305.357 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendSampleStatusAndData 0 bool
2023-04-27T09:38:54.831Z,1682588334.831 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SendDataToMultipleModems 0 bool
2023-04-27T09:38:54.833Z,1682588334.833 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1 bool
2023-04-27T09:38:54.833Z,1682588334.833 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 3 meter
2023-04-27T09:38:54.834Z,1682588334.834 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DeepBoundChl 20 meter
2023-04-27T09:39:13.812Z,1682588353.812 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.Lat1 36.653 degree
2023-04-27T09:39:13.813Z,1682588353.813 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.Lon1 -121.874 degree
2023-04-27T09:39:13.814Z,1682588353.814 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 meter
2023-04-27T09:39:13.827Z,1682588353.827 [CommandExec](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 5 minute
2023-04-27T09:39:13.827Z,1682588353.827 [CommandExec](IMPORTANT): got command run
2023-04-27T09:39:13.858Z,1682588353.858 [CommandExec](IMPORTANT): Running 
2023-04-27T09:39:14.157Z,1682588354.157 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling
2023-04-27T09:39:14.312Z,1682588354.312 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 240.00 hours.
2023-04-27T09:44:13.152Z,1682588653.152 [IsothermDepthSampling:TransitToSamplingLocationNoCommsOrNoTransit:Wpt1NoComms:Wpt1:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.653000,-121.874001
2023-04-27T09:45:44.869Z,1682588744.869 [StratificationFrontDetector](IMPORTANT): Initial vertical homogeneity condition (0 = stratified, 1 = homogeneous), VTHI_, thresholdVTHI_ = 0, 0.706628, 0.100000
2023-04-27T10:01:06.397Z,1682589666.397 [IsothermDepthSampling:TransitToSamplingLocationNoCommsOrNoTransit:Wpt1NoComms:Wpt1:G.Waypoint](IMPORTANT): Reached waypoint: 36.653000,-121.874001
2023-04-27T10:02:58.351Z,1682589778.351 [IsothermDepthSampling:StayAwayFromWallsAndShore:FarEnoughFromWallsAndShore:A](IMPORTANT): Far enough from walls and shore. Latitude = 36.653002 arcdeg , longitude = -121.874057 arcdeg . Distance from shore = 3316.230225 m . OffshoreDistanceTurnBack (MinOffshore + MarginMinOffshore) = 2000.000000 m + 100.000000 m . Shortest distance nan m to side. MinDistFromPolygonSides = 200.000000 m .
2023-04-27T10:03:00.338Z,1682589780.338 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.669579 m 
2023-04-27T10:04:09.857Z,1682589849.857 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min .
2023-04-27T10:05:58.543Z,1682589958.543 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 23.078825 m 
2023-04-27T10:11:30.642Z,1682590290.642 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 20.497604 m , 2.447600 ug/l , 7.398442 m , 11.271608 degC , 10.995623 ug/l 
2023-04-27T10:11:31.006Z,1682590291.006 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Target depth = 7.398442 m .
2023-04-27T10:11:31.423Z,1682590291.423 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 
2023-04-27T10:15:28.580Z,1682590528.580 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 7.390924 m , settling for 5.000000 min 
2023-04-27T10:20:31.122Z,1682590831.122 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::|summary:'
2023-04-27T10:20:41.267Z,1682590841.267 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-04-27T10:20:41.560Z,1682590841.560 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-04-27T10:21:35.090Z,1682590895.090 [ESPComponent](IMPORTANT): ESP has connected as client
2023-04-27T10:21:35.885Z,1682590895.885 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -6
2023-04-27T10:21:35.886Z,1682590895.886 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-04-27T10:21:36.356Z,1682590896.356 [IsothermDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.543483 m in 11.301111 degC water with 10.605880 ug/l chlorophyll fluorescence.
2023-04-27T10:21:37.088Z,1682590897.088 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG
2023-04-27T10:21:37.504Z,1682590897.504 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-04-27T10:21:39.117Z,1682590899.117 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE
2023-04-27T10:22:11.024Z,1682590931.024 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PRIMING (@5,archive_bac)
2023-04-27T10:22:13.859Z,1682590933.859 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PRIMING (@100.0)
2023-04-27T10:22:13.861Z,1682590933.861 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-04-27T10:22:14.266Z,1682590934.266 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-04-27T10:24:57.900Z,1682591097.900 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_FILTERING (@0.0)
2023-04-27T10:24:58.288Z,1682591098.288 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-04-27T11:17:31.176Z,1682594251.176 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PAUSED (@797.1)
2023-04-27T11:17:31.588Z,1682594251.588 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PROCESSING (@797.1)
2023-04-27T11:17:33.610Z,1682594253.610 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-04-27T11:17:34.017Z,1682594254.017 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PROCESSED (@5,archive_bac)
2023-04-27T11:29:37.590Z,1682594977.590 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING (@5,archive_bac)
2023-04-27T11:29:47.695Z,1682594987.695 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-04-27T11:29:48.103Z,1682594988.103 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED
2023-04-27T11:34:47.852Z,1682595287.852 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence completed normally.
2023-04-27T11:34:48.717Z,1682595288.717 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed.
2023-04-27T11:34:51.743Z,1682595291.743 [MissionManager](IMPORTANT): Started mission Default
2023-04-27T11:40:21.821Z,1682595621.821 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T11:45:48.794Z,1682595948.794 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.943798 min 
2023-04-27T11:45:50.384Z,1682595950.384 [NAL9602](FAULT): GPS failed to acquire within timeout.
2023-04-27T11:51:35.833Z,1682596295.833 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T11:56:19.134Z,1682596579.134 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.450020 min 
2023-04-27T12:01:23.805Z,1682596883.805 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T12:01:52.091Z,1682596912.091 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 26.999428 min 
2023-04-27T12:03:20.492Z,1682597000.492 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T120319
2023-04-27T12:03:27.910Z,1682597007.910 [CommandExec](IMPORTANT): got command schedule asap "load Science/front_sampling.xml;set FrontSampling.NumSamplers 3 count;set FrontSampling.MissionTimeout 15 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.RepeatTransects 2 count;set FrontSampling.FrontAutonomousDetection 0 bool" 3tyt2 1 6.000000
2023-04-27T12:03:27.910Z,1682597007.910 [CommandExec](IMPORTANT): Scheduling command #1 of 6 with id=3tyt2
2023-04-27T12:03:27.911Z,1682597007.911 [CommandExec](IMPORTANT): Scheduled #4 (#1 of 6 with id='3tyt2'): "load Science/front_sampling.xml;set FrontSampling.NumSamplers 3 count;set FrontSampling.MissionTimeout 15 h;set FrontSampling.MaxDepth 60 m;set FrontSampling.RepeatTransects 2 count;set FrontSampling.FrontAutonomousDetection 0 bool", ASAP
2023-04-27T12:03:28.896Z,1682597008.896 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T120327
2023-04-27T12:03:36.354Z,1682597016.354 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonFrontMinDesignated -121.871 degree;set FrontSampling.LonFrontMaxDesignated -121.87 degree;set FrontSampling.Lat1 36.66 degree;set FrontSampling.Lon1 -121.88 degree;set FrontSampling.Lat2 36.81 degree" 3tyt2 2 6.000000
2023-04-27T12:03:36.354Z,1682597016.354 [CommandExec](IMPORTANT): Scheduling command #2 of 6 with id=3tyt2
2023-04-27T12:03:36.355Z,1682597016.355 [CommandExec](IMPORTANT): Scheduled #5 (#2 of 6 with id='3tyt2'): "set FrontSampling.LonFrontMinDesignated -121.871 degree;set FrontSampling.LonFrontMaxDesignated -121.87 degree;set FrontSampling.Lat1 36.66 degree;set FrontSampling.Lon1 -121.88 degree;set FrontSampling.Lat2 36.81 degree", ASAP
2023-04-27T12:03:37.324Z,1682597017.324 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T120336
2023-04-27T12:03:44.798Z,1682597024.798 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.Lon2 -121.847 degree;set FrontSampling.LatUpwellingEnd 36.81 degree;set FrontSampling.LonUpwellingEnd -121.847 degree;set FrontSampling.DeltaLatUpwellingSide1 0.05 degree" 3tyt2 3 6.000000
2023-04-27T12:03:44.799Z,1682597024.799 [CommandExec](IMPORTANT): Scheduling command #3 of 6 with id=3tyt2
2023-04-27T12:03:44.800Z,1682597024.800 [CommandExec](IMPORTANT): Scheduled #6 (#3 of 6 with id='3tyt2'): "set FrontSampling.Lon2 -121.847 degree;set FrontSampling.LatUpwellingEnd 36.81 degree;set FrontSampling.LonUpwellingEnd -121.847 degree;set FrontSampling.DeltaLatUpwellingSide1 0.05 degree", ASAP
2023-04-27T12:03:45.800Z,1682597025.800 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T120344
2023-04-27T12:03:53.127Z,1682597033.127 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.DeltaLonUpwellingSide1 0.01 degree;set FrontSampling.DeltaLatUpwellingSide2 0.1 degree;set FrontSampling.DeltaLonUpwellingSide2 0.02 degree;set FrontSampling.LatStratifiedEnd 36.8 degree" 3tyt2 4 6.000000
2023-04-27T12:03:53.128Z,1682597033.128 [CommandExec](IMPORTANT): Scheduling command #4 of 6 with id=3tyt2
2023-04-27T12:03:53.128Z,1682597033.128 [CommandExec](IMPORTANT): Scheduled #7 (#4 of 6 with id='3tyt2'): "set FrontSampling.DeltaLonUpwellingSide1 0.01 degree;set FrontSampling.DeltaLatUpwellingSide2 0.1 degree;set FrontSampling.DeltaLonUpwellingSide2 0.02 degree;set FrontSampling.LatStratifiedEnd 36.8 degree", ASAP
2023-04-27T12:03:54.124Z,1682597034.124 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T120353
2023-04-27T12:04:03.343Z,1682597043.343 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.LonStratifiedEnd -121.84 degree;set FrontSampling.SpeedTransit 1.1 m/s;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m;set FrontSampling.YoYoMinDepth 2 m" 3tyt2 5 6.000000
2023-04-27T12:04:03.343Z,1682597043.343 [CommandExec](IMPORTANT): Scheduling command #5 of 6 with id=3tyt2
2023-04-27T12:04:03.344Z,1682597043.344 [CommandExec](IMPORTANT): Scheduled #8 (#5 of 6 with id='3tyt2'): "set FrontSampling.LonStratifiedEnd -121.84 degree;set FrontSampling.SpeedTransit 1.1 m/s;set FrontSampling.ShallowBoundChl 3 m;set FrontSampling.DeepBoundChl 20 m;set FrontSampling.YoYoMinDepth 2 m", ASAP
2023-04-27T12:04:04.324Z,1682597044.324 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T120403
2023-04-27T12:04:14.146Z,1682597054.146 [CommandExec](IMPORTANT): got command schedule asap "set FrontSampling.YoYoMaxDepth 50 m;set FrontSampling.WaitDepthUndulation 5 min;set FrontSampling.RiseFromTurningDepth 4 m;set FrontSampling.DepChangeThreshAttitudeFlip 3 m;run" 3tyt2 6 6.000000
2023-04-27T12:04:14.146Z,1682597054.146 [CommandExec](IMPORTANT): Scheduling command #6 of 6 with id=3tyt2
2023-04-27T12:04:14.147Z,1682597054.147 [CommandExec](IMPORTANT): Scheduled #9 (#6 of 6 with id='3tyt2'): "set FrontSampling.YoYoMaxDepth 50 m;set FrontSampling.WaitDepthUndulation 5 min;set FrontSampling.RiseFromTurningDepth 4 m;set FrontSampling.DepChangeThreshAttitudeFlip 3 m;run", ASAP
2023-04-27T12:04:14.269Z,1682597054.269 [CommandExec](IMPORTANT): got command load ./Missions/Science/front_sampling.xml
2023-04-27T12:04:27.616Z,1682597067.616 [CommandExec](IMPORTANT): Loaded ./Missions/Science/front_sampling.xml
2023-04-27T12:04:59.546Z,1682597099.546 [CommandExec](IMPORTANT): got command set FrontSampling.NumSamplers 3 count
2023-04-27T12:04:59.547Z,1682597099.547 [CommandExec](IMPORTANT): got command set FrontSampling.MissionTimeout 15 hour
2023-04-27T12:04:59.547Z,1682597099.547 [CommandExec](IMPORTANT): got command set FrontSampling.MaxDepth 60 meter
2023-04-27T12:04:59.548Z,1682597099.548 [CommandExec](IMPORTANT): got command set FrontSampling.RepeatTransects 2 count
2023-04-27T12:04:59.548Z,1682597099.548 [CommandExec](IMPORTANT): got command set FrontSampling.FrontAutonomousDetection 0 bool
2023-04-27T12:05:19.387Z,1682597119.387 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMinDesignated -121.871 degree
2023-04-27T12:05:19.388Z,1682597119.388 [CommandExec](IMPORTANT): got command set FrontSampling.LonFrontMaxDesignated -121.87 degree
2023-04-27T12:05:19.388Z,1682597119.388 [CommandExec](IMPORTANT): got command set FrontSampling.Lat1 36.66 degree
2023-04-27T12:05:19.389Z,1682597119.389 [CommandExec](IMPORTANT): got command set FrontSampling.Lon1 -121.88 degree
2023-04-27T12:05:19.394Z,1682597119.394 [CommandExec](IMPORTANT): got command set FrontSampling.Lat2 36.81 degree
2023-04-27T12:05:40.575Z,1682597140.575 [CommandExec](IMPORTANT): got command set FrontSampling.Lon2 -121.847 degree
2023-04-27T12:05:40.577Z,1682597140.577 [CommandExec](IMPORTANT): got command set FrontSampling.LatUpwellingEnd 36.81 degree
2023-04-27T12:05:40.578Z,1682597140.578 [CommandExec](IMPORTANT): got command set FrontSampling.LonUpwellingEnd -121.847 degree
2023-04-27T12:05:40.596Z,1682597140.596 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide1 0.05 degree
2023-04-27T12:06:12.139Z,1682597172.139 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide1 0.01 degree
2023-04-27T12:06:12.140Z,1682597172.140 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLatUpwellingSide2 0.1 degree
2023-04-27T12:06:12.140Z,1682597172.140 [CommandExec](IMPORTANT): got command set FrontSampling.DeltaLonUpwellingSide2 0.02 degree
2023-04-27T12:06:12.141Z,1682597172.141 [CommandExec](IMPORTANT): got command set FrontSampling.LatStratifiedEnd 36.8 degree
2023-04-27T12:06:47.746Z,1682597207.746 [CommandExec](IMPORTANT): got command set FrontSampling.LonStratifiedEnd -121.84 degree
2023-04-27T12:06:47.748Z,1682597207.748 [CommandExec](IMPORTANT): got command set FrontSampling.SpeedTransit 1.1 meter_per_second
2023-04-27T12:06:47.748Z,1682597207.748 [CommandExec](IMPORTANT): got command set FrontSampling.ShallowBoundChl 3 meter
2023-04-27T12:06:47.749Z,1682597207.749 [CommandExec](IMPORTANT): got command set FrontSampling.DeepBoundChl 20 meter
2023-04-27T12:06:47.750Z,1682597207.750 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMinDepth 2 meter
2023-04-27T12:07:12.446Z,1682597232.446 [CommandExec](IMPORTANT): got command set FrontSampling.YoYoMaxDepth 50 meter
2023-04-27T12:07:12.447Z,1682597232.447 [CommandExec](IMPORTANT): got command set FrontSampling.WaitDepthUndulation 5 minute
2023-04-27T12:07:12.447Z,1682597232.447 [CommandExec](IMPORTANT): got command set FrontSampling.RiseFromTurningDepth 4 meter
2023-04-27T12:07:12.449Z,1682597232.449 [CommandExec](IMPORTANT): got command set FrontSampling.DepChangeThreshAttitudeFlip 3 meter
2023-04-27T12:07:12.449Z,1682597232.449 [CommandExec](IMPORTANT): got command run
2023-04-27T12:07:12.455Z,1682597232.455 [CommandExec](IMPORTANT): Running 
2023-04-27T12:07:12.646Z,1682597232.646 [MissionManager](IMPORTANT): Started mission FrontSampling
2023-04-27T12:12:18.580Z,1682597538.580 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T12:12:26.281Z,1682597546.281 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.660000,-121.879997
2023-04-27T12:18:10.077Z,1682597890.077 [StratificationFrontDetector](IMPORTANT): Front detection is now enabled. Present vertical homogeneity condition (0 = stratified, 1 = homogeneous), VTHI, thresholdVTHI_, countSameWaterType_, countOppositeWaterType_ = 0, 0.304529, 0.100000, 5, 0
2023-04-27T12:21:20.626Z,1682598080.626 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T12:21:20.714Z,1682598080.714 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T12:21:20.715Z,1682598080.715 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T12:28:43.551Z,1682598523.551 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T12:29:27.709Z,1682598567.709 [FrontSampling:CrossFrontFromStratifiedToUpwelling:P](IMPORTANT): Crossed the front from stratified to upwelling. VTHI = 0.484552 degC . VSHI = 0.013226 psu . Front latitude, longitude = 36.660994 arcdeg , -121.871557 arcdeg . Lat1, Lon1 = 36.710994 arcdeg , -121.861557 arcdeg . Lat2, Lon2 = 36.760994 arcdeg , -121.851557 arcdeg . Lat3, Lon3 = nan arcdeg , nan arcdeg . Lat4, Lon4 = nan arcdeg , nan arcdeg . Lat5, Lon5 = 36.810001 arcdeg , -121.847000 arcdeg . Lat6, Lon6 = 36.799999 arcdeg , -121.839996 arcdeg . Sample count = 1.000000 count .
2023-04-27T12:29:30.949Z,1682598570.949 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.810001,-121.847000
2023-04-27T12:29:32.200Z,1682598572.200 [FrontSampling:BackAndForth:Waypoints:SampleFrontFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.710994,-121.861557
2023-04-27T12:29:32.545Z,1682598572.545 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt5:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.810001,-121.847000
2023-04-27T12:29:32.970Z,1682598572.970 [FrontSampling:BackAndForth:Waypoints:SampleFrontNotFirstPass:Wpt1:A.Waypoint](IMPORTANT): Navigating to waypoint: 36.710994,-121.861557
2023-04-27T12:29:34.580Z,1682598574.580 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoDown:B](IMPORTANT): Go down from 0.131066 m 
2023-04-27T12:31:20.015Z,1682598680.015 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:LogGoUp:B](IMPORTANT): Go up from 23.111217 m 
2023-04-27T12:32:10.536Z,1682598730.536 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:SetChlPeakDepth:J](IMPORTANT): Set peak-chl depth. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlSet = 18.687737 m , 1.531200 ug/l , 5.566199 m , 9.386730 ug/l 
2023-04-27T12:32:14.157Z,1682598734.157 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Latitude, longitude = 36.660726 arcdeg , -121.871445 arcdeg . Target depth = 5.566199 m .
2023-04-27T12:32:14.568Z,1682598734.568 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:F](IMPORTANT): Requesting ESP cartridge type: -6.000000 count 
2023-04-27T12:35:48.323Z,1682598948.323 [FrontSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 5.542919 m , settling for 5.000000 min 
2023-04-27T12:41:01.074Z,1682599261.074 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-04-27T12:41:01.207Z,1682599261.207 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-04-27T12:42:02.391Z,1682599322.391 [ESPComponent](IMPORTANT): ESP has connected as client
2023-04-27T12:42:03.216Z,1682599323.216 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -6
2023-04-27T12:42:03.217Z,1682599323.217 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-04-27T12:42:03.659Z,1682599323.659 [FrontSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.018750 m in 11.227289 degC water with 10.063000 ug/l chlorophyll fluorescence.
2023-04-27T12:42:04.011Z,1682599324.011 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG
2023-04-27T12:42:04.412Z,1682599324.412 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-04-27T12:42:06.433Z,1682599326.433 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE
2023-04-27T12:42:38.352Z,1682599358.352 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PRIMING (@4,archive_bac)
2023-04-27T12:42:41.184Z,1682599361.184 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PRIMING (@100.0)
2023-04-27T12:42:41.185Z,1682599361.185 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-04-27T12:42:41.596Z,1682599361.596 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-04-27T12:45:25.205Z,1682599525.205 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_FILTERING (@0.0)
2023-04-27T12:45:25.607Z,1682599525.607 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-04-27T13:50:03.643Z,1682603403.643 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PAUSED (@983.1)
2023-04-27T13:50:04.049Z,1682603404.049 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PROCESSING (@983.1)
2023-04-27T13:50:06.073Z,1682603406.073 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-04-27T13:50:06.473Z,1682603406.473 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PROCESSED (@4,archive_bac)
2023-04-27T14:02:10.042Z,1682604130.042 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPING (@4,archive_bac)
2023-04-27T14:02:20.147Z,1682604140.147 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-04-27T14:02:20.553Z,1682604140.553 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPED
2023-04-27T14:07:20.325Z,1682604440.325 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence completed normally.
2023-04-27T14:07:21.549Z,1682604441.549 [FrontSampling:FindPeakChlVerticalAndSample:FindPeakTakeSample:FiringOnDepth:TriggerOnDepth:D](IMPORTANT): Sample No. 1.000000 count completed.
2023-04-27T14:07:26.166Z,1682604446.166 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T14:07:26.254Z,1682604446.254 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T14:07:26.254Z,1682604446.254 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T14:12:53.720Z,1682604773.720 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T15:14:11.145Z,1682608451.145 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T15:14:11.238Z,1682608451.238 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T15:14:11.238Z,1682608451.238 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T15:21:24.943Z,1682608884.943 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T152123
2023-04-27T15:21:32.975Z,1682608892.975 [CommandExec](IMPORTANT): got command stop
2023-04-27T15:21:32.975Z,1682608892.975 [CommandExec](IMPORTANT): Scheduling is paused
2023-04-27T15:21:33.708Z,1682608893.708 [MissionManager](IMPORTANT): Started mission Default
2023-04-27T15:21:33.915Z,1682608893.915 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T152132
2023-04-27T15:21:41.482Z,1682608901.482 [CommandExec](IMPORTANT): got command stop
2023-04-27T15:22:08.812Z,1682608928.812 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T15:24:31.141Z,1682609071.141 [CBIT](IMPORTANT): Beginning ground fault scan
2023-04-27T15:24:42.054Z,1682609082.054 [CBIT](IMPORTANT): Ground fault detected
mA:
CHAN A0 (Batt): 0.033121
CHAN A1 (24V): -0.000632
CHAN A2 (12V): -0.000472
CHAN A3 (5V): -0.000337
CHAN B0 (3.3V): -0.000028
CHAN B1 (3.15aV): -0.000223
CHAN B2 (3.15bV): 0.000327
CHAN B3 (GND): 0.000231
OPEN: 0.007777
Full Scale: +/- 1 mA
2023-04-27T15:27:50.750Z,1682609270.750 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 6.277132 min 
2023-04-27T15:28:02.089Z,1682609282.089 [DataOverHttps](IMPORTANT): SBD MOMSN=18271073, MTMSN=20230427T152800
2023-04-27T15:28:13.018Z,1682609293.018 [CommandExec](IMPORTANT): got command schedule asap "load Science/smear_waypoint_sampling.xml;set Smear.NumSamplers 2 count;set Smear.MissionTimeout 8 h;set Smear.WaitDuration 5 min;set Smear.NeedCommsTimeInTransit 60 min;set Smear.MaxWaitNoFiring 1 h;set Smear.SpeedSampling 1.1 m/s" 3u8aa 1 2.000000
2023-04-27T15:28:13.018Z,1682609293.018 [CommandExec](IMPORTANT): Scheduling command #1 of 2 with id=3u8aa
2023-04-27T15:28:13.019Z,1682609293.019 [CommandExec](IMPORTANT): Scheduled #10 (#1 of 2 with id='3u8aa'): "load Science/smear_waypoint_sampling.xml;set Smear.NumSamplers 2 count;set Smear.MissionTimeout 8 h;set Smear.WaitDuration 5 min;set Smear.NeedCommsTimeInTransit 60 min;set Smear.MaxWaitNoFiring 1 h;set Smear.SpeedSampling 1.1 m/s", ASAP
2023-04-27T15:28:14.160Z,1682609294.160 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T152812
2023-04-27T15:28:23.927Z,1682609303.927 [CommandExec](IMPORTANT): got command schedule asap "set Smear.MaxDepth 40 m;set Smear.SamplingYoYoMinDepth 3 m;set Smear.SamplingYoYoMaxDepth 30 m;set Smear.SamplingEndLatitude 36.807 degree;set Smear.SamplingEndLongitude -121.824 degree;set Smear.NeedCommsTimeVeryLong 2 h;run" 3u8aa 2 2.000000
2023-04-27T15:28:23.927Z,1682609303.927 [CommandExec](IMPORTANT): Scheduling command #2 of 2 with id=3u8aa
2023-04-27T15:28:23.928Z,1682609303.928 [CommandExec](IMPORTANT): Scheduled #11 (#2 of 2 with id='3u8aa'): "set Smear.MaxDepth 40 m;set Smear.SamplingYoYoMinDepth 3 m;set Smear.SamplingYoYoMaxDepth 30 m;set Smear.SamplingEndLatitude 36.807 degree;set Smear.SamplingEndLongitude -121.824 degree;set Smear.NeedCommsTimeVeryLong 2 h;run", ASAP
2023-04-27T15:28:24.086Z,1682609304.086 [CommandExec](IMPORTANT): got command load ./Missions/Science/smear_waypoint_sampling.xml
2023-04-27T15:28:31.017Z,1682609311.017 [CommandExec](IMPORTANT): Loaded ./Missions/Science/smear_waypoint_sampling.xml
2023-04-27T15:29:06.669Z,1682609346.669 [CommandExec](IMPORTANT): got command set Smear.NumSamplers 2 count
2023-04-27T15:29:06.669Z,1682609346.669 [CommandExec](IMPORTANT): got command set Smear.MissionTimeout 8 hour
2023-04-27T15:29:06.670Z,1682609346.670 [CommandExec](IMPORTANT): got command set Smear.WaitDuration 5 minute
2023-04-27T15:29:06.671Z,1682609346.671 [CommandExec](IMPORTANT): got command set Smear.NeedCommsTimeInTransit 60 minute
2023-04-27T15:29:06.672Z,1682609346.672 [CommandExec](IMPORTANT): got command set Smear.MaxWaitNoFiring 1 hour
2023-04-27T15:29:06.672Z,1682609346.672 [CommandExec](IMPORTANT): got command set Smear.SpeedSampling 1.1 meter_per_second
2023-04-27T15:29:43.380Z,1682609383.380 [CommandExec](IMPORTANT): got command set Smear.MaxDepth 40 meter
2023-04-27T15:29:43.380Z,1682609383.380 [CommandExec](IMPORTANT): got command set Smear.SamplingYoYoMinDepth 3 meter
2023-04-27T15:29:43.381Z,1682609383.381 [CommandExec](IMPORTANT): got command set Smear.SamplingYoYoMaxDepth 30 meter
2023-04-27T15:29:43.382Z,1682609383.382 [CommandExec](IMPORTANT): got command set Smear.SamplingEndLatitude 36.807 degree
2023-04-27T15:29:43.382Z,1682609383.382 [CommandExec](IMPORTANT): got command set Smear.SamplingEndLongitude -121.824 degree
2023-04-27T15:29:43.383Z,1682609383.383 [CommandExec](IMPORTANT): got command set Smear.NeedCommsTimeVeryLong 2 hour
2023-04-27T15:29:43.384Z,1682609383.384 [CommandExec](IMPORTANT): got command run
2023-04-27T15:29:43.411Z,1682609383.411 [CommandExec](IMPORTANT): Running 
2023-04-27T15:29:43.640Z,1682609383.640 [MissionManager](IMPORTANT): Started mission Smear
2023-04-27T15:32:55.147Z,1682609575.147 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T15:33:48.498Z,1682609628.498 [Smear:SamplingToWaypoint:E.Waypoint](IMPORTANT): Navigating to waypoint: 36.806999,-121.823997
2023-04-27T15:34:23.621Z,1682609663.621 [DVL_micro](FAULT): Failed to initialize within timeout.
2023-04-27T15:34:26.123Z,1682609666.123 [Smear:SamplingToWaypoint:TakeSamples:Firing:B](IMPORTANT): Taking sample No. 1.000000 count .
2023-04-27T15:34:28.970Z,1682609668.970 [Smear:TriggerSampling:TriggerESP:D](IMPORTANT): Trigger ESP sampling. DepthLogged, TempLogged, ChlLogged = 4.092931 m , 11.653101 degC , 4.582000 ug/l .
2023-04-27T15:34:40.774Z,1682609680.774 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-04-27T15:34:40.890Z,1682609680.890 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-04-27T15:35:34.047Z,1682609734.047 [ESPComponent](IMPORTANT): ESP has connected as client
2023-04-27T15:35:34.867Z,1682609734.867 [ESPComponent](IMPORTANT): [sample #3] Setting cartridge argument to -6
2023-04-27T15:35:34.867Z,1682609734.867 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-04-27T15:35:35.252Z,1682609735.252 [Smear:TriggerSampling:TriggerESP:G](IMPORTANT): Wait for ESP sampling to complete.
2023-04-27T15:35:35.644Z,1682609735.644 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_LOG
2023-04-27T15:35:36.053Z,1682609736.053 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-04-27T15:35:38.089Z,1682609738.089 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_LOADING_CARTRIDGE
2023-04-27T15:36:10.011Z,1682609770.011 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PRIMING (@3,archive_bac)
2023-04-27T15:36:12.011Z,1682609772.011 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PRIMING (@100.0)
2023-04-27T15:36:12.410Z,1682609772.410 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-04-27T15:36:12.825Z,1682609772.825 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-04-27T15:38:46.359Z,1682609926.359 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T15:38:54.441Z,1682609934.441 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_FILTERING (@0.0)
2023-04-27T15:38:54.816Z,1682609934.816 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-04-27T16:06:58.269Z,1682611618.269 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout
2023-04-27T16:07:03.778Z,1682611623.778 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T16:07:03.866Z,1682611623.866 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T16:07:03.867Z,1682611623.867 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T16:07:13.038Z,1682611633.038 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T16:07:13.126Z,1682611633.126 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T16:07:13.126Z,1682611633.126 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T16:07:22.374Z,1682611642.374 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T16:07:22.470Z,1682611642.470 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T16:07:22.470Z,1682611642.470 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T16:07:30.882Z,1682611650.882 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T16:07:30.970Z,1682611650.970 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T16:07:30.970Z,1682611650.970 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T16:07:39.350Z,1682611659.350 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-04-27T16:07:39.438Z,1682611659.438 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-04-27T16:07:39.438Z,1682611659.438 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-04-27T16:07:39.444Z,1682611659.444 [CBIT](CRITICAL): Communications Fault in component: ThrusterHE
2023-04-27T16:07:39.720Z,1682611659.720 [CommandExec](FAULT): Scheduling is paused
2023-04-27T16:07:40.377Z,1682611660.377 [ESPComponent](IMPORTANT): [sample #3] Stop sampling requested.
2023-04-27T16:07:40.388Z,1682611660.388 [MissionManager](IMPORTANT): Started mission Default
2023-04-27T16:07:40.783Z,1682611660.783 [ESPComponent](IMPORTANT): Stop sampling requested. Issuing Cmd.pauseFiltering
2023-04-27T16:07:47.264Z,1682611667.264 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PAUSED (@579.3)
2023-04-27T16:07:47.648Z,1682611667.648 [ESPComponent](IMPORTANT): In S_PAUSED received result: 579.3

2023-04-27T16:07:48.052Z,1682611668.052 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PROCESSING (@579.3)
2023-04-27T16:07:50.073Z,1682611670.073 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-04-27T16:07:50.476Z,1682611670.476 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PROCESSED (@3,archive_bac)
2023-04-27T16:13:31.300Z,1682612011.300 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T16:18:55.431Z,1682612335.431 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 11.244343 min 
2023-04-27T16:18:57.041Z,1682612337.041 [NAL9602](FAULT): GPS failed to acquire within timeout.
2023-04-27T16:19:53.992Z,1682612393.992 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPING (@3,archive_bac)
2023-04-27T16:20:04.093Z,1682612404.093 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-04-27T16:20:04.518Z,1682612404.518 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPED
2023-04-27T16:24:06.241Z,1682612646.241 [DataOverHttps](IMPORTANT): SBD MOMSN=18271207, MTMSN=20230427T162404
2023-04-27T16:24:16.351Z,1682612656.351 [CommandExec](IMPORTANT): got command schedule asap "load Science/smear_waypoint_sampling.xml;set Smear.MissionTimeout 8 h;set Smear.WaitDuration 5 min;set Smear.NeedCommsTimeInTransit 60 min;set Smear.MaxWaitNoFiring 1 h;set Smear.SpeedSampling 1 m/s;set Smear.MaxDepth 40 m" 3uavp 1 2.000000
2023-04-27T16:24:16.352Z,1682612656.352 [CommandExec](IMPORTANT): Scheduling command #1 of 2 with id=3uavp
2023-04-27T16:24:16.353Z,1682612656.353 [CommandExec](IMPORTANT): Scheduled #12 (#1 of 2 with id='3uavp'): "load Science/smear_waypoint_sampling.xml;set Smear.MissionTimeout 8 h;set Smear.WaitDuration 5 min;set Smear.NeedCommsTimeInTransit 60 min;set Smear.MaxWaitNoFiring 1 h;set Smear.SpeedSampling 1 m/s;set Smear.MaxDepth 40 m", ASAP
2023-04-27T16:24:17.602Z,1682612657.602 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T162415
2023-04-27T16:24:27.723Z,1682612667.723 [CommandExec](IMPORTANT): got command schedule asap "set Smear.SamplingYoYoMinDepth 3 m;set Smear.SamplingYoYoMaxDepth 30 m;set Smear.SamplingEndLatitude 36.807 degree;set Smear.SamplingEndLongitude -121.824 degree;set Smear.NeedCommsTimeVeryLong 2 h;run" 3uavp 2 2.000000
2023-04-27T16:24:27.723Z,1682612667.723 [CommandExec](IMPORTANT): Scheduling command #2 of 2 with id=3uavp
2023-04-27T16:24:27.724Z,1682612667.724 [CommandExec](IMPORTANT): Scheduled #13 (#2 of 2 with id='3uavp'): "set Smear.SamplingYoYoMinDepth 3 m;set Smear.SamplingYoYoMaxDepth 30 m;set Smear.SamplingEndLatitude 36.807 degree;set Smear.SamplingEndLongitude -121.824 degree;set Smear.NeedCommsTimeVeryLong 2 h;run", ASAP
2023-04-27T16:24:27.862Z,1682612667.862 [CommandExec](IMPORTANT): got command load ./Missions/Science/smear_waypoint_sampling.xml
2023-04-27T16:24:32.515Z,1682612672.515 [CommandExec](IMPORTANT): Loaded ./Missions/Science/smear_waypoint_sampling.xml
2023-04-27T16:24:55.131Z,1682612695.131 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T16:25:04.424Z,1682612704.424 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence completed normally.
2023-04-27T16:25:04.986Z,1682612704.986 [CommandExec](IMPORTANT): got command set Smear.MissionTimeout 8 hour
2023-04-27T16:25:04.987Z,1682612704.987 [CommandExec](IMPORTANT): got command set Smear.WaitDuration 5 minute
2023-04-27T16:25:04.988Z,1682612704.988 [CommandExec](IMPORTANT): got command set Smear.NeedCommsTimeInTransit 60 minute
2023-04-27T16:25:04.989Z,1682612704.989 [CommandExec](IMPORTANT): got command set Smear.MaxWaitNoFiring 1 hour
2023-04-27T16:25:04.994Z,1682612704.994 [CommandExec](IMPORTANT): got command set Smear.SpeedSampling 1 meter_per_second
2023-04-27T16:25:04.994Z,1682612704.994 [CommandExec](IMPORTANT): got command set Smear.MaxDepth 40 meter
2023-04-27T16:25:36.047Z,1682612736.047 [CommandExec](IMPORTANT): got command set Smear.SamplingYoYoMinDepth 3 meter
2023-04-27T16:25:36.048Z,1682612736.048 [CommandExec](IMPORTANT): got command set Smear.SamplingYoYoMaxDepth 30 meter
2023-04-27T16:25:36.049Z,1682612736.049 [CommandExec](IMPORTANT): got command set Smear.SamplingEndLatitude 36.807 degree
2023-04-27T16:25:36.054Z,1682612736.054 [CommandExec](IMPORTANT): got command set Smear.SamplingEndLongitude -121.824 degree
2023-04-27T16:25:36.055Z,1682612736.055 [CommandExec](IMPORTANT): got command set Smear.NeedCommsTimeVeryLong 2 hour
2023-04-27T16:25:36.055Z,1682612736.055 [CommandExec](IMPORTANT): got command run
2023-04-27T16:25:36.061Z,1682612736.061 [CommandExec](IMPORTANT): Running 
2023-04-27T16:25:36.369Z,1682612736.369 [MissionManager](IMPORTANT): Started mission Smear
2023-04-27T16:26:48.766Z,1682612808.766 [Smear:SamplingToWaypoint:E.Waypoint](IMPORTANT): Navigating to waypoint: 36.806999,-121.823997
2023-04-27T16:27:26.610Z,1682612846.610 [Smear:SamplingToWaypoint:TakeSamples:Firing:B](IMPORTANT): Taking sample No. 1.000000 count .
2023-04-27T16:27:29.458Z,1682612849.458 [Smear:TriggerSampling:TriggerESP:D](IMPORTANT): Trigger ESP sampling. DepthLogged, TempLogged, ChlLogged = 3.763170 m , 11.391412 degC , 5.672400 ug/l .
2023-04-27T16:27:41.254Z,1682612861.254 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-04-27T16:27:41.383Z,1682612861.383 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-04-27T16:28:34.523Z,1682612914.523 [ESPComponent](IMPORTANT): ESP has connected as client
2023-04-27T16:28:35.344Z,1682612915.344 [ESPComponent](IMPORTANT): [sample #4] Setting cartridge argument to -6
2023-04-27T16:28:35.345Z,1682612915.345 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-04-27T16:28:35.767Z,1682612915.767 [Smear:TriggerSampling:TriggerESP:G](IMPORTANT): Wait for ESP sampling to complete.
2023-04-27T16:28:36.144Z,1682612916.144 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_LOG
2023-04-27T16:28:36.545Z,1682612916.545 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-04-27T16:28:38.568Z,1682612918.568 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_LOADING_CARTRIDGE
2023-04-27T16:29:10.488Z,1682612950.488 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PRIMING (@2,archive_bac)
2023-04-27T16:29:12.512Z,1682612952.512 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PRIMING (@100.0)
2023-04-27T16:29:12.932Z,1682612952.932 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-04-27T16:29:13.307Z,1682612953.307 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-04-27T16:31:55.325Z,1682613115.325 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_FILTERING (@0.0)
2023-04-27T16:31:55.723Z,1682613115.723 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-04-27T17:24:32.265Z,1682616272.265 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PAUSED (@1000.0)
2023-04-27T17:24:32.665Z,1682616272.665 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PROCESSING (@1000.0)
2023-04-27T17:24:34.681Z,1682616274.681 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-04-27T17:24:35.092Z,1682616275.092 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PROCESSED (@2,archive_bac)
2023-04-27T17:36:38.669Z,1682616998.669 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPING (@2,archive_bac)
2023-04-27T17:36:49.182Z,1682617009.182 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-04-27T17:36:49.568Z,1682617009.568 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPED
2023-04-27T17:41:49.344Z,1682617309.344 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence completed normally.
2023-04-27T17:41:50.565Z,1682617310.565 [Smear:SamplingToWaypoint:TakeSamples:Firing:F](IMPORTANT): Sample No. 1.000000 count completed.
2023-04-27T17:48:49.512Z,1682617729.512 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T17:54:35.479Z,1682618075.479 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T19:58:04.729Z,1682625484.729 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T20:03:49.566Z,1682625829.566 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T20:11:40.647Z,1682626300.647 [Smear:SamplingToWaypoint:E.Waypoint](IMPORTANT): Reached waypoint: 36.806999,-121.823997
2023-04-27T20:11:41.075Z,1682626301.075 [MissionManager](IMPORTANT): Started mission Default
2023-04-27T20:14:51.127Z,1682626491.127 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T201448
2023-04-27T20:15:01.862Z,1682626501.862 [CommandExec](IMPORTANT): got command stop
2023-04-27T20:15:01.862Z,1682626501.862 [CommandExec](IMPORTANT): Scheduling is paused
2023-04-27T20:16:13.800Z,1682626573.800 [DataOverHttps](IMPORTANT): SBD MTMSN=20230427T201611
2023-04-27T20:16:23.642Z,1682626583.642 [CommandExec](IMPORTANT): got command load ./Missions/Transport/keepstation.tl
2023-04-27T20:16:23.643Z,1682626583.643 [MissionManager](IMPORTANT): Loading Compiled TethysL script from file ./Missions/Transport/keepstation.tx
2023-04-27T20:16:24.850Z,1682626584.850 [CommandExec](IMPORTANT): Loaded ./Missions/Transport/keepstation.tl
2023-04-27T20:16:44.910Z,1682626604.910 [CommandExec](IMPORTANT): got command set keepstation.NeedCommsTime 20 minute
2023-04-27T20:16:44.910Z,1682626604.910 [CommandExec](IMPORTANT): got command set keepstation.Latitude 36.806966 degree
2023-04-27T20:16:44.911Z,1682626604.911 [CommandExec](IMPORTANT): got command set keepstation.Longitude -121.824326 degree
2023-04-27T20:16:44.912Z,1682626604.912 [CommandExec](IMPORTANT): got command set keepstation.Depth 20 meter
2023-04-27T20:16:44.912Z,1682626604.912 [CommandExec](IMPORTANT): got command run
2023-04-27T20:16:44.947Z,1682626604.947 [CommandExec](IMPORTANT): Running 
2023-04-27T20:16:45.091Z,1682626605.091 [MissionManager](IMPORTANT): Started mission keepstation
2023-04-27T20:17:55.768Z,1682626675.768 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T20:18:39.837Z,1682626719.837 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Navigating to waypoint: 36.806965,-121.824326
2023-04-27T20:23:41.934Z,1682627021.934 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T20:27:05.562Z,1682627225.562 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Reached waypoint: 36.806965,-121.824326
2023-04-27T20:33:06.772Z,1682627586.772 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T21:24:42.423Z,1682630682.423 [CBIT](IMPORTANT): Beginning ground fault scan
2023-04-27T21:24:53.324Z,1682630693.324 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): -0.004439
CHAN A1 (24V): -0.008615
CHAN A2 (12V): -0.000351
CHAN A3 (5V): -0.000301
CHAN B0 (3.3V): 0.012943
CHAN B1 (3.15aV): -0.000129
CHAN B2 (3.15bV): 0.000098
CHAN B3 (GND): 0.000051
OPEN: 0.007531
Full Scale: +/- 1 mA
2023-04-27T21:51:54.195Z,1682632314.195 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-04-27T22:14:51.063Z,1682633691.063 [keepstation:StandardEnvelopes:C.OffshoreEnvelope](IMPORTANT): ACTIVE
2023-04-27T22:15:37.893Z,1682633737.893 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16885, MT Status=1, MTMSN=1231
2023-04-27T22:15:38.486Z,1682633738.486 [CommandExec](IMPORTANT): got command stop
2023-04-27T22:15:39.131Z,1682633739.131 [MissionManager](IMPORTANT): Started mission Default
2023-04-27T22:21:29.891Z,1682634089.891 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 5.839395 min 
2023-04-27T22:23:06.465Z,1682634186.465 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=16886, MT Status=1, MTMSN=1232