2023-05-12T16:17:23.340Z,1683908243.340 [Supervisor](IMPORTANT): Reading configuration overrides from Data/persisted.cfg 2023-05-12T16:17:26.415Z,1683908246.415 [StratificationFrontDetector](IMPORTANT): thresholdVTHI set to: 0.050012 degC 2023-05-12T16:17:27.268Z,1683908247.268 [SBIT](IMPORTANT): git: 2023-04-25 2023-05-12T16:17:27.268Z,1683908247.268 [SBIT](IMPORTANT): Kernel Release: 2.6.27.8 2023-05-12T16:17:28.217Z,1683908248.217 [DropWeight](CRITICAL): DROP WEIGHT MISSING. 2023-05-12T16:17:28.278Z,1683908248.278 [CommandExec](FAULT): Scheduling is paused 2023-05-12T16:17:28.290Z,1683908248.290 [CBIT](CRITICAL): Hardware Fault in component: DropWeight 2023-05-12T16:18:38.894Z,1683908318.894 [SBIT](IMPORTANT): Beginning Startup BIT 2023-05-12T16:18:38.898Z,1683908318.898 [CBIT](IMPORTANT): Beginning ground fault scan 2023-05-12T16:18:49.829Z,1683908329.829 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): 0.002492 CHAN A1 (24V): 0.000865 CHAN A2 (12V): -0.005623 CHAN A3 (5V): -0.003243 CHAN B0 (3.3V): -0.001288 CHAN B1 (3.15aV): -0.001543 CHAN B2 (3.15bV): -0.001143 CHAN B3 (GND): 0.000109 OPEN: 0.002564 Full Scale: +/- 1 mA 2023-05-12T16:19:33.031Z,1683908373.031 [SBIT](IMPORTANT): SBIT PASSED 2023-05-12T16:19:33.031Z,1683908373.031 [SBIT](IMPORTANT): Listing configuration overrides from Data/persisted.cfg 2023-05-12T16:19:33.032Z,1683908373.032 [SBIT](IMPORTANT): CBIT.gfScanTimeout=2 hour; 2023-05-12T16:19:33.032Z,1683908373.032 [SBIT](IMPORTANT): DVL_micro.loadAtStartup=1 bool; 2023-05-12T16:19:33.032Z,1683908373.032 [SBIT](IMPORTANT): Depth_Keller.offset=-0.8 decibar; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): ESPComponent.sampleTimeout=4 minute; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): ElevatorServo.offsetAngle=1 degree; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_salinity_homogeneity_index 0.050000 practical_salinity_unit; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express linearApproximation VerticalHomogeneityIndexCalculator.vertical_temperature_homogeneity_index 0.050000 celsius; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express none WetLabsBB2FL.BackscatteringCoeff470nm; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express none WetLabsBB2FL.BackscatteringCoeff650nm; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express linearApproximation depth 5.000000 meter; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express linearApproximation latitude 0.005000 degree; 2023-05-12T16:19:33.033Z,1683908373.033 [SBIT](IMPORTANT): Express linearApproximation longitude 0.005000 degree; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): StratificationFrontDetector.loadAtStartup=1 bool; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): StratificationFrontDetector.threshold_VTHI=0.05 celsius; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): StratificationFrontDetector.verbosity=1 count; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): VerticalControl.buoyancyNeutral=191.501740 cubic_centimeter; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): VerticalControl.massDefault=12.944775 millimeter; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth3=20 meter; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.depth4=30 meter; 2023-05-12T16:19:33.034Z,1683908373.034 [SBIT](IMPORTANT): VerticalHomogeneityIndexCalculator.loadAtStartup=1 bool; 2023-05-12T16:19:33.426Z,1683908373.426 [MissionManager](IMPORTANT): Started mission Startup 2023-05-12T16:21:34.342Z,1683908494.342 [MissionManager](IMPORTANT): Started mission Default 2023-05-12T16:23:11.208Z,1683908591.208 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T16:28:08.555Z,1683908888.555 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T16:32:51.794Z,1683909171.794 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 11.284008 min 2023-05-12T16:38:51.025Z,1683909531.025 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T16:43:19.312Z,1683909799.312 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.742798 min 2023-05-12T16:49:33.814Z,1683910173.814 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T16:53:48.074Z,1683910428.074 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 32.222270 min 2023-05-12T17:00:16.727Z,1683910816.727 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T17:02:29.686Z,1683910949.686 [DropWeight](CRITICAL): DROP WEIGHT MISSING. 2023-05-12T17:02:29.770Z,1683910949.770 [CBIT](CRITICAL): Hardware Fault in component: DropWeight 2023-05-12T17:04:16.366Z,1683911056.366 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 42.693803 min 2023-05-12T17:08:28.201Z,1683911308.201 [CommandExec](IMPORTANT): got command burn on 2023-05-12T17:08:28.201Z,1683911308.201 [CommandExec](IMPORTANT): Activating dropweight wire 2023-05-12T17:08:50.889Z,1683911330.889 [CommandExec](IMPORTANT): got command burn off 2023-05-12T17:08:50.889Z,1683911330.889 [CommandExec](IMPORTANT): Deactivating dropweight wire 2023-05-12T17:10:59.558Z,1683911459.558 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T17:10:59.577Z,1683911459.577 [CBIT](CRITICAL): Data Fault in component: NAL9602 2023-05-12T17:14:45.031Z,1683911685.031 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 53.171204 min 2023-05-12T17:18:00.069Z,1683911880.069 [CommandExec](IMPORTANT): got command set NAL9602.latitude_fix 36.803 degree 2023-05-12T17:18:00.070Z,1683911880.070 [CommandExec](IMPORTANT): got command set NAL9602.longitude_fix -121.788 degree 2023-05-12T17:18:00.070Z,1683911880.070 [CommandExec](IMPORTANT): got command set NAL9602.time_fix 1683911849 second 2023-05-12T17:18:00.071Z,1683911880.071 [CommandExec](IMPORTANT): got command set NAL9602.goodFix 1 bool 2023-05-12T17:18:01.792Z,1683911881.792 [CommandExec](IMPORTANT): got command get NAL9602.latitude_fix 2023-05-12T17:18:01.793Z,1683911881.793 [CommandExec](IMPORTANT): NAL9602.latitude_fix 36.803001 arcdeg 2023-05-12T17:18:11.929Z,1683911891.929 [CommandExec](IMPORTANT): got command failComponent 2023-05-12T17:18:11.929Z,1683911891.929 [CommandExec](IMPORTANT): Failed components: 2023-05-12T17:18:11.929Z,1683911891.929 [CommandExec](IMPORTANT): No failed Components. 2023-05-12T17:18:19.045Z,1683911899.045 [CommandExec](IMPORTANT): got command strobe off 2023-05-12T17:18:19.045Z,1683911899.045 [CommandExec](IMPORTANT): Deactivating strobe 2023-05-12T17:19:06.045Z,1683911946.045 [DataOverHttps](IMPORTANT): SBD MTMSN=20230512T171854 2023-05-12T17:19:13.561Z,1683911953.561 [CommandExec](IMPORTANT): got command load ./Missions/Maintenance/sample_lab.xml 2023-05-12T17:19:13.682Z,1683911953.682 [CommandExec](IMPORTANT): Loaded ./Missions/Maintenance/sample_lab.xml 2023-05-12T17:19:15.445Z,1683911955.445 [CommandExec](IMPORTANT): got command set sample_lab.CartridgeType -1 count 2023-05-12T17:19:15.445Z,1683911955.445 [CommandExec](IMPORTANT): got command run 2023-05-12T17:19:15.447Z,1683911955.447 [CommandExec](IMPORTANT): Running 2023-05-12T17:19:15.574Z,1683911955.574 [MissionManager](IMPORTANT): Started mission sample_lab 2023-05-12T17:19:15.577Z,1683911955.577 [sample_lab:SampleRepeater:A](IMPORTANT): Sampling ESP cartridge type -1.000000 count 2023-05-12T17:19:17.093Z,1683911957.093 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::|summary:' 2023-05-12T17:19:27.228Z,1683911967.228 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2023-05-12T17:19:27.365Z,1683911967.365 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2023-05-12T17:20:20.129Z,1683912020.129 [ESPComponent](IMPORTANT): ESP has connected as client 2023-05-12T17:20:20.938Z,1683912020.938 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -1 2023-05-12T17:20:20.939Z,1683912020.939 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2023-05-12T17:20:22.146Z,1683912022.146 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2023-05-12T17:20:22.550Z,1683912022.550 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2023-05-12T17:20:24.166Z,1683912024.166 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2023-05-12T17:20:56.082Z,1683912056.082 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PRIMING (@60,aRNAl_bac) 2023-05-12T17:20:58.909Z,1683912058.909 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PRIMING (@100.0) 2023-05-12T17:20:58.911Z,1683912058.911 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2023-05-12T17:20:59.313Z,1683912059.313 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2023-05-12T17:23:52.226Z,1683912232.226 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_FILTERING (@0.0) 2023-05-12T17:23:52.641Z,1683912232.641 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2023-05-12T18:00:44.274Z,1683914444.274 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PAUSED (@1000.0) 2023-05-12T18:00:44.678Z,1683914444.678 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PROCESSING (@1000.0) 2023-05-12T18:00:46.702Z,1683914446.702 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2023-05-12T18:00:47.102Z,1683914447.102 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PROCESSED (@60,aRNAl_bac) 2023-05-12T18:02:22.041Z,1683914542.041 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING (@60,aRNAl_bac) 2023-05-12T18:02:29.719Z,1683914549.719 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2023-05-12T18:02:30.123Z,1683914550.123 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2023-05-12T18:07:29.889Z,1683914849.889 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence completed normally. 2023-05-12T18:07:30.485Z,1683914850.485 [MissionManager](IMPORTANT): Started mission Default 2023-05-12T18:07:45.664Z,1683914865.664 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T18:13:31.093Z,1683915211.093 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T18:18:28.861Z,1683915508.861 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.967601 min 2023-05-12T18:18:30.495Z,1683915510.495 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T18:18:49.895Z,1683915529.895 [CBIT](IMPORTANT): Beginning ground fault scan 2023-05-12T18:19:00.799Z,1683915540.799 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.012339 CHAN A1 (24V): 0.001112 CHAN A2 (12V): -0.005499 CHAN A3 (5V): -0.002910 CHAN B0 (3.3V): -0.000979 CHAN B1 (3.15aV): -0.000775 CHAN B2 (3.15bV): -0.000982 CHAN B3 (GND): -0.000342 OPEN: 0.002563 Full Scale: +/- 1 mA 2023-05-12T18:24:15.893Z,1683915855.893 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T18:28:57.891Z,1683916137.891 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.450083 min 2023-05-12T18:34:01.741Z,1683916441.741 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T18:39:27.818Z,1683916767.818 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 31.948385 min 2023-05-12T18:39:29.404Z,1683916769.404 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T18:45:14.844Z,1683917114.844 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T18:49:56.030Z,1683917396.030 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 42.419352 min 2023-05-12T18:55:06.330Z,1683917706.330 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T19:00:25.080Z,1683918025.080 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 52.903251 min 2023-05-12T19:00:26.683Z,1683918026.683 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T19:06:12.118Z,1683918372.118 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T19:10:54.206Z,1683918654.206 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 63.388452 min 2023-05-12T19:15:57.993Z,1683918957.993 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T19:21:22.027Z,1683919282.027 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 73.852124 min 2023-05-12T19:21:23.629Z,1683919283.629 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T19:21:23.702Z,1683919283.702 [CBIT](CRITICAL): Data Fault in component: NAL9602 2023-05-12T19:28:38.737Z,1683919718.737 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T19:31:50.643Z,1683919910.643 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 84.329362 min 2023-05-12T19:36:54.442Z,1683920214.442 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T19:42:19.336Z,1683920539.336 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 94.807357 min 2023-05-12T19:42:20.953Z,1683920540.953 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T19:48:06.362Z,1683920886.362 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T19:52:49.265Z,1683921169.265 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 105.305688 min 2023-05-12T19:57:53.430Z,1683921473.430 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T20:03:18.272Z,1683921798.272 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 115.789421 min 2023-05-12T20:03:19.866Z,1683921799.866 [NAL9602](FAULT): GPS failed to acquire within timeout. 2023-05-12T20:09:38.689Z,1683922178.689 [CommandExec](IMPORTANT): got command maintain control VerticalControl.buoyancyAction 150 cubic_centimeter 2023-05-12T20:10:14.481Z,1683922214.481 [CommandExec](IMPORTANT): got command report mod VerticalControl.buoyancyAction 2023-05-12T20:10:20.438Z,1683922220.438 [CommandExec](IMPORTANT): got command report clear 2023-05-12T20:10:41.106Z,1683922241.106 [CommandExec](IMPORTANT): got command show variable platform_buoyancy 2023-05-12T20:10:41.121Z,1683922241.121 [CommandExec](IMPORTANT): platform_buoyancy_position (cubic_centimeter) 2023-05-12T20:10:41.474Z,1683922241.474 [CommandExec](IMPORTANT): BuoyancyServo.platform_buoyancy_position (cubic_centimeter) 2023-05-12T20:10:57.655Z,1683922257.655 [CommandExec](IMPORTANT): got command report mod platform_buoyancy_position 2023-05-12T20:11:05.273Z,1683922265.273 [CommandExec](IMPORTANT): got command maintain 2023-05-12T20:11:23.417Z,1683922283.417 [CommandExec](IMPORTANT): got command get platform_mass_position 2023-05-12T20:11:23.418Z,1683922283.418 [CommandExec](IMPORTANT): platform_mass_position 0.012946 m 2023-05-12T20:11:45.403Z,1683922305.403 [CommandExec](IMPORTANT): got command get VerticalControl.massPositionLimitFwd 2023-05-12T20:11:45.404Z,1683922305.404 [CommandExec](IMPORTANT): VerticalControl.massPositionLimitFwd 39.000000 mm 2023-05-12T20:12:09.826Z,1683922329.826 [CommandExec](IMPORTANT): got command maintain control VerticalControl.massPositionAction 32 millimeter 2023-05-12T20:13:52.391Z,1683922432.391 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 126.358260 min 2023-05-12T20:16:04.247Z,1683922564.247 [CommandExec](IMPORTANT): got command maintain control VerticalControl.massPositionAction 22 millimeter 2023-05-12T20:16:14.780Z,1683922574.780 [CommandExec](IMPORTANT): got command get platform_buoyancy_position 2023-05-12T20:16:14.781Z,1683922574.781 [CommandExec](IMPORTANT): platform_buoyancy_position 159.269714 cc 2023-05-12T20:18:56.623Z,1683922736.623 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T20:19:01.093Z,1683922741.093 [CBIT](IMPORTANT): Beginning ground fault scan 2023-05-12T20:19:11.990Z,1683922751.990 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.005127 CHAN A1 (24V): 0.002245 CHAN A2 (12V): -0.005478 CHAN A3 (5V): -0.004085 CHAN B0 (3.3V): -0.002223 CHAN B1 (3.15aV): -0.000996 CHAN B2 (3.15bV): -0.001228 CHAN B3 (GND): -0.001092 OPEN: 0.012142 Full Scale: +/- 1 mA 2023-05-12T20:19:24.123Z,1683922764.123 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 131.886768 min 2023-05-12T20:24:56.452Z,1683923096.452 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 137.425863 min 2023-05-12T20:30:00.510Z,1683923400.510 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T20:30:32.869Z,1683923432.869 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 143.032650 min 2023-05-12T20:35:37.597Z,1683923737.597 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T20:35:58.017Z,1683923758.017 [CommandExec](IMPORTANT): got command maintain clear 2023-05-12T20:36:07.524Z,1683923767.524 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 148.610400 min 2023-05-12T20:41:39.812Z,1683924099.812 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 154.148779 min 2023-05-12T20:46:44.582Z,1683924404.582 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2023-05-12T20:47:10.903Z,1683924430.903 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 159.666878 min 2023-05-12T20:52:41.651Z,1683924761.651 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 165.179118 min