2023-05-08T18:59:49.878Z,1683572389.878 [Supervisor](IMPORTANT): Reading configuration overrides from Data/persisted.cfg 2023-05-08T18:59:52.325Z,1683572392.325 [StratificationFrontDetector](IMPORTANT): thresholdVTHI set to: 0.100000 degC 2023-05-08T18:59:53.308Z,1683572393.308 [SBIT](IMPORTANT): git: 2023-04-26-7-g7f279a801 2023-05-08T18:59:53.308Z,1683572393.308 [SBIT](IMPORTANT): Kernel Release: 2.6.27.8 2023-05-08T18:59:54.157Z,1683572394.157 [DropWeight](CRITICAL): DROP WEIGHT MISSING. 2023-05-08T18:59:54.181Z,1683572394.181 [CommandExec](FAULT): Scheduling is paused 2023-05-08T18:59:54.194Z,1683572394.194 [CBIT](CRITICAL): Hardware Fault in component: DropWeight 2023-05-08T19:00:16.726Z,1683572416.726 [CommandExec](IMPORTANT): got command failComponent none DropWeight 2023-05-08T19:00:16.727Z,1683572416.727 [CommandExec](IMPORTANT): DropWeight failureMode is No Fault 2023-05-08T19:00:19.428Z,1683572419.428 [CommandExec](IMPORTANT): got command failComponent 2023-05-08T19:00:19.428Z,1683572419.428 [CommandExec](IMPORTANT): Failed components: 2023-05-08T19:00:19.428Z,1683572419.428 [CommandExec](IMPORTANT): No failed Components. 2023-05-08T19:00:32.990Z,1683572432.990 [CommandExec](IMPORTANT): got command run ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:00:33.066Z,1683572433.066 [CommandExec](IMPORTANT): Running ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:01:35.142Z,1683572495.142 [CommandExec](IMPORTANT): got command ! echo 1 > /dev/loadA6; echo 1 > /dev/loadA7 2023-05-08T19:01:53.307Z,1683572513.307 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:02:07.858Z,1683572527.858 [SBIT](IMPORTANT): Beginning Startup BIT 2023-05-08T19:02:07.862Z,1683572527.862 [CBIT](IMPORTANT): Beginning ground fault scan 2023-05-08T19:02:18.848Z,1683572538.848 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): 0.006674 CHAN A1 (24V): -0.001832 CHAN A2 (12V): -0.000028 CHAN A3 (5V): 0.000246 CHAN B0 (3.3V): 0.000713 CHAN B1 (3.15aV): 0.000271 CHAN B2 (3.15bV): 0.000574 CHAN B3 (GND): 0.000293 OPEN: -0.004114 Full Scale: +/- 1 mA 2023-05-08T19:02:43.240Z,1683572563.240 [CommandExec](IMPORTANT): got command help 2023-05-08T19:03:01.599Z,1683572581.599 [SBIT](IMPORTANT): SBIT PASSED 2023-05-08T19:03:01.600Z,1683572581.600 [SBIT](IMPORTANT): Listing configuration overrides from Data/persisted.cfg 2023-05-08T19:03:01.601Z,1683572581.601 [SBIT](IMPORTANT): AHRS_M2.loadAtStartup=0 bool; 2023-05-08T19:03:01.601Z,1683572581.601 [SBIT](IMPORTANT): BPC1.batteryMissingStickThreshold=12 count; 2023-05-08T19:03:01.601Z,1683572581.601 [SBIT](IMPORTANT): CTD_Seabird.loadAtStartup=0 bool; 2023-05-08T19:03:01.601Z,1683572581.601 [SBIT](IMPORTANT): DAT.sbdAddress=-1 enum; 2023-05-08T19:03:01.602Z,1683572581.602 [SBIT](IMPORTANT): DVL_micro.loadAtStartup=0 bool; 2023-05-08T19:03:01.603Z,1683572581.603 [SBIT](IMPORTANT): Depth_Keller.offset=3.6 decibar; 2023-05-08T19:03:01.603Z,1683572581.603 [SBIT](IMPORTANT): Express none CBIT.ampHoursUsed; 2023-05-08T19:03:01.605Z,1683572581.605 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_salinity_homogeneity_index 0.050000 practical_salinity_unit; 2023-05-08T19:03:01.605Z,1683572581.605 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_temperature_homogeneity_index 0.050000 celsius; 2023-05-08T19:03:01.605Z,1683572581.605 [SBIT](IMPORTANT): Express none WetLabsBB2FL.bin_mean_mass_concentration_of_chlorophyll_in_sea_water; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): Express none height_above_sea_floor; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): StratificationFrontDetector.loadAtStartup=1 bool; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): StratificationFrontDetector.threshold_VTHI=0.1 celsius; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): StratificationFrontDetector.verbosity=1 count; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): VerticalControl.buoyancyNeutral=208.617815 cubic_centimeter; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): VerticalControl.massDefault=28.688721 millimeter; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): VerticalControl.massPositionLimitAft=-28 millimeter; 2023-05-08T19:03:01.606Z,1683572581.606 [SBIT](IMPORTANT): VerticalControl.massPositionLimitFwd=32 millimeter; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): VerticalControl.surfaceThreshold=1.5 meter; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth1=3 meter; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth2=5 meter; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth3=10 meter; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth4=15 meter; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.loadAtStartup=1 bool; 2023-05-08T19:03:01.607Z,1683572581.607 [SBIT](IMPORTANT): WetLabsBB2FL.loadAtStartup=0 bool; 2023-05-08T19:03:01.995Z,1683572581.995 [MissionManager](IMPORTANT): Started mission Startup 2023-05-08T19:03:26.799Z,1683572606.799 [CommandExec](IMPORTANT): got command run ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:03:26.869Z,1683572606.869 [CommandExec](IMPORTANT): Running ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:03:54.531Z,1683572634.531 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:04:27.059Z,1683572667.059 [CommandExec](IMPORTANT): got command run 2023-05-08T19:04:27.061Z,1683572667.061 [CommandExec](IMPORTANT): Running 2023-05-08T19:05:02.897Z,1683572702.897 [MissionManager](IMPORTANT): Started mission sample_lab 2023-05-08T19:05:02.898Z,1683572702.898 [sample_lab:SampleRepeater:A](IMPORTANT): Sampling ESP cartridge type -4.000000 count 2023-05-08T19:05:04.486Z,1683572704.486 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::|summary:' 2023-05-08T19:05:14.620Z,1683572714.620 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-05-08T19:05:14.908Z,1683572714.908 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-05-08T19:05:29.152Z,1683572729.152 [ESPComponent](IMPORTANT): ESP has connected as client 2023-05-08T19:05:30.054Z,1683572730.054 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -4 2023-05-08T19:05:30.055Z,1683572730.055 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-05-08T19:05:31.147Z,1683572731.147 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2023-05-08T19:05:31.550Z,1683572731.550 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-05-08T19:05:34.436Z,1683572734.436 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2023-05-08T19:05:36.437Z,1683572736.437 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-08T19:05:36.439Z,1683572736.439 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING 2023-05-08T19:05:55.465Z,1683572755.465 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:06:05.949Z,1683572765.949 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e \ & 2023-05-08T19:06:06.357Z,1683572766.357 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2023-05-08T19:07:56.636Z,1683572876.636 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:08:31.946Z,1683572911.946 [CommandExec](IMPORTANT): got command run ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:08:32.022Z,1683572912.022 [CommandExec](IMPORTANT): Running ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:08:32.207Z,1683572912.207 [MissionManager](IMPORTANT): Started mission sample_lab 2023-05-08T19:08:32.209Z,1683572912.209 [sample_lab:SampleRepeater:A](IMPORTANT): Sampling ESP cartridge type -4.000000 count 2023-05-08T19:09:57.453Z,1683572997.453 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:10:20.977Z,1683573020.977 [CommandExec](IMPORTANT): got command show stack 2023-05-08T19:10:20.977Z,1683573020.977 [CommandExec](IMPORTANT): Behavior Stack: 2023-05-08T19:10:20.977Z,1683573020.977 [sample_lab](IMPORTANT): Priority 0: sample_lab:A.AbortSample 2023-05-08T19:10:20.977Z,1683573020.977 [sample_lab:SampleRepeater](IMPORTANT): Priority 1: sample_lab:SampleRepeater:TriggerESP 2023-05-08T19:10:33.786Z,1683573033.786 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-08T19:11:06.126Z,1683573066.126 [ESPComponent](CRITICAL): [sample #1] ESP sampling sequence terminated with error: In state S_LOADING_CARTRIDGE got exception while awaiting result for cmd='Cmd.loadCartridge -4' exception=<>. Elapsed time since command issued: 2.0126 2023-05-08T19:11:06.942Z,1683573066.942 [MissionManager](IMPORTANT): Started mission Default 2023-05-08T19:11:58.249Z,1683573118.249 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:13:59.104Z,1683573239.104 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:16:00.249Z,1683573360.249 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:16:19.268Z,1683573379.268 [CommandExec](IMPORTANT): got command run ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:16:19.412Z,1683573379.412 [CommandExec](IMPORTANT): Running ./Missions/Maintenance/sample_lab.xml 2023-05-08T19:16:19.645Z,1683573379.645 [MissionManager](IMPORTANT): Started mission sample_lab 2023-05-08T19:16:19.646Z,1683573379.646 [sample_lab:SampleRepeater:A](IMPORTANT): Sampling ESP cartridge type -4.000000 count 2023-05-08T19:16:31.468Z,1683573391.468 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-05-08T19:16:31.596Z,1683573391.596 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-05-08T19:17:24.294Z,1683573444.294 [ESPComponent](IMPORTANT): ESP has connected as client 2023-05-08T19:17:25.120Z,1683573445.120 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -4 2023-05-08T19:17:25.120Z,1683573445.120 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-05-08T19:17:25.910Z,1683573445.910 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG 2023-05-08T19:17:26.329Z,1683573446.329 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-05-08T19:17:28.331Z,1683573448.331 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE 2023-05-08T19:17:30.350Z,1683573450.350 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPING 2023-05-08T19:17:59.862Z,1683573479.862 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e \ & 2023-05-08T19:18:00.248Z,1683573480.248 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPED 2023-05-08T19:18:01.084Z,1683573481.084 [DeadReckonUsingMultipleVelocitySources](FAULT): Unable to read the rotation from vehicle frame to navigation frame for more than 120 seconds. 2023-05-08T19:19:53.153Z,1683573593.153 [CommandExec](IMPORTANT): got command restart application 2023-05-08T19:19:54.612Z,1683573594.612 [MissionManager](IMPORTANT): Started mission Default