2020-10-02T00:18:50.140Z,1601597930.140 [Supervisor](IMPORTANT): Reading configuration overrides from Data/persisted.cfg 2020-10-02T00:18:54.852Z,1601597934.852 [SBIT](IMPORTANT): git: 2020-09-23a 2020-10-02T00:18:54.853Z,1601597934.853 [SBIT](IMPORTANT): Kernel Release: 2.6.27.8 2020-10-02T00:18:56.438Z,1601597936.438 [RudderServo](FAULT): Rudder failed to initialize 2020-10-02T00:19:56.396Z,1601597996.396 [SBIT](IMPORTANT): Beginning Startup BIT 2020-10-02T00:19:56.400Z,1601597996.400 [CBIT](IMPORTANT): Beginning ground fault scan 2020-10-02T00:20:07.341Z,1601598007.341 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): 0.004516 CHAN A1 (24V): -0.000299 CHAN A2 (12V): -0.004678 CHAN A3 (5V): -0.002490 CHAN B0 (3.3V): -0.000934 CHAN B1 (3.15aV): -0.001523 CHAN B2 (3.15bV): -0.000817 CHAN B3 (GND): 0.000211 OPEN: 0.005103 Full Scale Calc: 4.765 mA, -1.589 mA 2020-10-02T00:20:50.100Z,1601598050.100 [SBIT](IMPORTANT): SBIT PASSED 2020-10-02T00:20:50.100Z,1601598050.100 [SBIT](IMPORTANT): Listing configuration overrides from Data/persisted.cfg 2020-10-02T00:20:50.101Z,1601598050.101 [SBIT](IMPORTANT): Aanderaa_O2.loadAtStartup=0 bool; 2020-10-02T00:20:50.101Z,1601598050.101 [SBIT](IMPORTANT): BPC1.batteryMissingStickThreshold=2 count; 2020-10-02T00:20:50.103Z,1601598050.103 [SBIT](IMPORTANT): CANONSampler.loadAtStartup=1 bool; 2020-10-02T00:20:50.103Z,1601598050.103 [SBIT](IMPORTANT): CANONSampler.rotateOnly=1 bool; 2020-10-02T00:20:50.104Z,1601598050.104 [SBIT](IMPORTANT): CBIT.gf24Offset=-26 microampere; 2020-10-02T00:20:50.104Z,1601598050.104 [SBIT](IMPORTANT): DVL_micro.loadAtStartup=0 bool; 2020-10-02T00:20:50.104Z,1601598050.104 [SBIT](IMPORTANT): DropWeight.loadAtStartup=0 bool; 2020-10-02T00:20:50.104Z,1601598050.104 [SBIT](IMPORTANT): VerticalControl.buoyancyNeutral=230 cubic_centimeter; 2020-10-02T00:20:50.104Z,1601598050.104 [SBIT](IMPORTANT): VerticalControl.massDefault=10.3 millimeter; 2020-10-02T00:20:50.515Z,1601598050.515 [MissionManager](IMPORTANT): Started mission Startup 2020-10-02T00:22:51.311Z,1601598171.311 [MissionManager](IMPORTANT): Started mission Default 2020-10-02T00:24:38.355Z,1601598278.355 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2020-10-02T00:29:37.329Z,1601598577.329 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T00:33:26.439Z,1601598806.439 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 10.578231 min 2020-10-02T00:40:20.504Z,1601599220.504 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T00:43:53.823Z,1601599433.823 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.035274 min 2020-10-02T00:51:04.498Z,1601599864.498 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T00:54:21.296Z,1601600061.296 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 31.492800 min 2020-10-02T01:01:47.680Z,1601600507.680 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T01:04:47.871Z,1601600687.871 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 41.936117 min 2020-10-02T01:12:30.848Z,1601601150.848 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T01:12:30.864Z,1601601150.864 [CBIT](CRITICAL): Data Fault in component: NAL9602 2020-10-02T01:12:31.273Z,1601601151.273 [CommandLine](FAULT): Scheduling is paused 2020-10-02T01:15:15.687Z,1601601315.687 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 52.399666 min 2020-10-02T01:24:43.744Z,1601601883.744 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T01:25:42.364Z,1601601942.364 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 62.843799 min 2020-10-02T01:35:28.983Z,1601602528.983 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T01:36:12.232Z,1601602572.232 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 73.341927 min 2020-10-02T01:46:12.146Z,1601603172.146 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T01:46:39.627Z,1601603199.627 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 83.798551 min 2020-10-02T01:56:57.339Z,1601603817.339 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T01:57:07.041Z,1601603827.041 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 94.255412 min 2020-10-02T02:07:33.660Z,1601604453.660 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 104.698844 min 2020-10-02T02:07:40.519Z,1601604460.519 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T02:07:40.536Z,1601604460.536 [CBIT](CRITICAL): Data Fault in component: NAL9602 2020-10-02T02:18:01.508Z,1601605081.508 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 115.163265 min 2020-10-02T02:19:54.600Z,1601605194.600 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T02:28:30.109Z,1601605710.109 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 125.639966 min 2020-10-02T02:30:37.772Z,1601605837.772 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T02:38:57.939Z,1601606337.939 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 136.104134 min 2020-10-02T02:41:20.948Z,1601606480.948 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T02:49:25.358Z,1601606965.358 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 146.560986 min 2020-10-02T02:52:06.139Z,1601607126.139 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T02:59:53.572Z,1601607593.572 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 157.030843 min 2020-10-02T03:02:49.297Z,1601607769.297 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T03:02:49.330Z,1601607769.330 [CBIT](CRITICAL): Data Fault in component: NAL9602 2020-10-02T03:10:20.600Z,1601608220.600 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 167.481331 min 2020-10-02T03:15:02.985Z,1601608502.985 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T03:20:48.432Z,1601608848.432 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 177.945133 min 2020-10-02T03:25:46.175Z,1601609146.175 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T03:31:15.863Z,1601609475.863 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 188.402507 min 2020-10-02T03:36:29.355Z,1601609789.355 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T03:41:43.684Z,1601610103.684 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 198.866325 min 2020-10-02T03:47:14.551Z,1601610434.551 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T03:52:11.928Z,1601610731.928 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 209.336605 min 2020-10-02T03:57:57.716Z,1601611077.716 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T03:57:57.793Z,1601611077.793 [CBIT](CRITICAL): Data Fault in component: NAL9602 2020-10-02T04:02:39.325Z,1601611359.325 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 219.793359 min 2020-10-02T04:07:32.092Z,1601611652.092 [CommandLine](IMPORTANT): got command get CANONSampler.rotateOnly 2020-10-02T04:07:32.092Z,1601611652.092 [CommandLine](IMPORTANT): CANONSampler.rotateOnly 1 bool 2020-10-02T04:07:44.478Z,1601611664.478 [CommandLine](IMPORTANT): got command run ./Missions/sample_test.xml 2020-10-02T04:07:45.109Z,1601611665.109 [CommandLine](IMPORTANT): Running ./Missions/sample_test.xml 2020-10-02T04:07:45.184Z,1601611665.184 [MissionManager](IMPORTANT): Started mission sample_test 2020-10-02T04:07:47.562Z,1601611667.562 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::' 2020-10-02T04:07:48.002Z,1601611668.002 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-02T04:07:48.121Z,1601611668.121 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-02T04:08:23.114Z,1601611703.114 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-02T04:08:23.929Z,1601611703.929 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -4 2020-10-02T04:08:23.929Z,1601611703.929 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-02T04:08:24.381Z,1601611704.381 [sample_test:Sample:SampleWrapper:ESP:AquireESP:C](IMPORTANT): ESP sampling at 0.017653 m in nan degC water with nan ug/l chlorophyll fluorescence. 2020-10-02T04:08:25.136Z,1601611705.136 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-02T04:08:25.539Z,1601611705.539 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-02T04:08:27.155Z,1601611707.155 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-02T04:08:29.579Z,1601611709.579 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING 2020-10-02T04:08:32.003Z,1601611712.003 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2020-10-02T04:08:32.408Z,1601611712.408 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2020-10-02T04:09:29.794Z,1601611769.794 [CommandLine](IMPORTANT): got command stop 2020-10-02T04:09:29.795Z,1601611769.795 [CommandLine](IMPORTANT): Scheduling is paused 2020-10-02T04:09:30.204Z,1601611770.204 [MissionManager](IMPORTANT): Started mission Default 2020-10-02T04:10:11.400Z,1601611811.400 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T04:10:26.256Z,1601611826.256 [CommandLine](IMPORTANT): got command stop 2020-10-02T04:10:49.105Z,1601611849.105 [CommandLine](IMPORTANT): got command load ./Missions/sample_test.xml 2020-10-02T04:10:49.478Z,1601611849.478 [CommandLine](IMPORTANT): Loaded ./Missions/sample_test.xml 2020-10-02T04:11:08.972Z,1601611868.972 [CommandLine](IMPORTANT): got command set sample_test.ESPCartridgeType -6.000000 count 2020-10-02T04:13:32.178Z,1601612012.178 [ESPComponent](CRITICAL): [sample #1] ESP sampling sequence terminated with error: In state S_LOADING_CARTRIDGE got exception while awaiting result for cmd='Cmd.loadCartridge -4' exception=<>. Elapsed time since command issued: 2.4237 2020-10-02T04:13:32.215Z,1601612012.215 [CommandLine](FAULT): Scheduling is paused 2020-10-02T04:13:32.583Z,1601612012.583 [ESPComponent](IMPORTANT): [sample #1] No ESP summary messages. 2020-10-02T04:13:34.210Z,1601612014.210 [ESPComponent](IMPORTANT): [sample #1] No ESP summary messages. 2020-10-02T04:13:55.136Z,1601612035.136 [CommandLine](IMPORTANT): got command show stack 2020-10-02T04:13:55.136Z,1601612035.136 [CommandLine](IMPORTANT): Behavior Stack: 2020-10-02T04:13:55.136Z,1601612035.136 [Default](IMPORTANT): Priority 0: Default:B.GoToSurface 2020-10-02T04:13:55.136Z,1601612035.136 [Default:CheckIn](IMPORTANT): Priority 1: Default:CheckIn:Read_GPS 2020-10-02T04:14:06.523Z,1601612046.523 [CommandLine](IMPORTANT): got command set sample_test:Sample.CANONSamplerDelay 5.000000 minute 2020-10-02T04:14:11.566Z,1601612051.566 [CommandLine](IMPORTANT): got command run 2020-10-02T04:14:11.568Z,1601612051.568 [CommandLine](IMPORTANT): Running 2020-10-02T04:14:11.833Z,1601612051.833 [MissionManager](IMPORTANT): Started mission sample_test 2020-10-02T04:14:12.234Z,1601612052.234 [MissionManager](IMPORTANT): Started mission Default 2020-10-02T04:15:50.633Z,1601612150.633 [CommandLine](IMPORTANT): got command stop 2020-10-02T04:15:50.633Z,1601612150.633 [CommandLine](IMPORTANT): Scheduling is paused 2020-10-02T04:15:57.372Z,1601612157.372 [CommandLine](IMPORTANT): got command load ./Missions/sample_test.xml 2020-10-02T04:15:57.779Z,1601612157.779 [CommandLine](IMPORTANT): Loaded ./Missions/sample_test.xml 2020-10-02T04:16:12.872Z,1601612172.872 [CommandLine](IMPORTANT): got command set sample_test:Sample.CANONSamplerDelay 5.000000 minute 2020-10-02T04:16:14.110Z,1601612174.110 [CommandLine](IMPORTANT): got command run 2020-10-02T04:16:14.117Z,1601612174.117 [CommandLine](IMPORTANT): Running 2020-10-02T04:16:14.236Z,1601612174.236 [MissionManager](IMPORTANT): Started mission sample_test 2020-10-02T04:16:17.134Z,1601612177.134 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2020-10-02T04:16:17.289Z,1601612177.289 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2020-10-02T04:16:52.174Z,1601612212.174 [ESPComponent](IMPORTANT): ESP has connected as client 2020-10-02T04:16:52.986Z,1601612212.986 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -6 2020-10-02T04:16:52.986Z,1601612212.986 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2020-10-02T04:16:53.433Z,1601612213.433 [sample_test:Sample:SampleWrapper:ESP:AquireESP:C](IMPORTANT): ESP sampling at 0.030218 m in nan degC water with nan ug/l chlorophyll fluorescence. 2020-10-02T04:16:53.791Z,1601612213.791 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG 2020-10-02T04:16:54.195Z,1601612214.195 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS 2020-10-02T04:16:56.214Z,1601612216.214 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE 2020-10-02T04:17:28.535Z,1601612248.535 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PRIMING (@58,archive_bac) 2020-10-02T04:17:30.151Z,1601612250.151 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PRIMING (@100.0) 2020-10-02T04:17:30.959Z,1601612250.959 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2020-10-02T04:19:55.186Z,1601612395.186 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_FILTERING (@0.0) 2020-10-02T04:19:55.591Z,1601612395.591 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2020-10-02T04:20:55.396Z,1601612455.396 [NAL9602](FAULT): GPS failed to acquire within timeout. 2020-10-02T04:21:47.902Z,1601612507.902 [CANONSampler](FAULT): Failed to receive prompt 2020-10-02T04:22:16.208Z,1601612536.208 [sample_test:Sample:SampleWrapper:CANONSampler:AquireCANONSampler:E](IMPORTANT): CANONSampler sampling at 0.030218 m in nan degC water with nan ug/l chlorophyll fluorescence. 2020-10-02T04:22:45.310Z,1601612565.310 [CommandLine](IMPORTANT): got command stop 2020-10-02T04:22:46.079Z,1601612566.079 [ESPComponent](IMPORTANT): [sample #2] Stop sampling requested. 2020-10-02T04:22:46.139Z,1601612566.139 [MissionManager](IMPORTANT): Started mission Default 2020-10-02T04:22:46.482Z,1601612566.482 [ESPComponent](IMPORTANT): Stop sampling requested. Issuing Cmd.pauseFiltering 2020-10-02T04:22:52.138Z,1601612572.138 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PAUSED (@71.502) 2020-10-02T04:22:52.543Z,1601612572.543 [ESPComponent](IMPORTANT): In S_PAUSED received result: 71.502 2020-10-02T04:22:52.946Z,1601612572.946 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PROCESSING (@71.502) 2020-10-02T04:22:55.371Z,1601612575.371 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PROCESSED (@58,archive_bac) 2020-10-02T04:23:15.135Z,1601612595.135 [CommandLine](IMPORTANT): got command get CANONSampler.loadControl 2020-10-02T04:23:15.135Z,1601612595.135 [CommandLine](IMPORTANT): CANONSampler.loadControl /dev/loadB6 2020-10-02T04:27:52.371Z,1601612872.371 [CommandLine](IMPORTANT): got command show stack 2020-10-02T04:27:52.371Z,1601612872.371 [CommandLine](IMPORTANT): Behavior Stack: 2020-10-02T04:27:52.375Z,1601612872.375 [Default](IMPORTANT): Priority 0: Default:B.GoToSurface 2020-10-02T04:27:52.376Z,1601612872.376 [Default:CheckIn](IMPORTANT): Priority 1: Default:CheckIn:Read_GPS 2020-10-02T04:31:11.171Z,1601613071.171 [CommandLine](IMPORTANT): got command restart application