2022-10-15T04:22:44.737Z,1665807764.737 [CommandExec](IMPORTANT): got command restart logs 2022-10-15T04:24:21.013Z,1665807861.013 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T04:29:21.784Z,1665808161.784 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T04:58:42.885Z,1665809922.885 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T04:58:45.313Z,1665809925.313 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 5.000000 m 2022-10-15T04:58:45.696Z,1665809925.696 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 28.000000 count . Target depth = 5.000000 m . 2022-10-15T04:58:46.087Z,1665809926.087 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T05:02:41.230Z,1665810161.230 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 4.978305 m , settling for 5.000000 min 2022-10-15T05:07:53.988Z,1665810473.988 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T05:07:54.343Z,1665810474.343 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T05:08:55.514Z,1665810535.514 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T05:08:56.325Z,1665810536.325 [ESPComponent](IMPORTANT): [sample #29] Setting cartridge argument to -6 2022-10-15T05:08:56.326Z,1665810536.326 [ESPComponent](IMPORTANT): [sample #29] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T05:08:56.790Z,1665810536.790 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.164089 m in 14.593622 degC water with 3.318640 ug/l chlorophyll fluorescence. 2022-10-15T05:08:57.130Z,1665810537.130 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T05:08:57.534Z,1665810537.534 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T05:08:59.554Z,1665810539.554 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T05:09:33.895Z,1665810573.895 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PRIMING (@8,archive_bac) 2022-10-15T05:09:35.918Z,1665810575.918 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T05:09:36.318Z,1665810576.318 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T05:09:36.721Z,1665810576.721 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T05:11:55.697Z,1665810715.697 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T05:11:56.102Z,1665810716.102 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T05:58:03.946Z,1665813483.946 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PAUSED (@761.2) 2022-10-15T05:58:04.349Z,1665813484.349 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PROCESSING (@761.2) 2022-10-15T05:58:06.371Z,1665813486.371 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T05:58:06.781Z,1665813486.781 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PROCESSED (@8,archive_bac) 2022-10-15T06:10:10.350Z,1665814210.350 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_STOPPING (@8,archive_bac) 2022-10-15T06:10:20.043Z,1665814220.043 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T06:10:20.446Z,1665814220.446 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_STOPPED 2022-10-15T06:15:20.213Z,1665814520.213 [ESPComponent](IMPORTANT): [sample #29] ESP sampling sequence completed normally. 2022-10-15T06:15:21.079Z,1665814521.079 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 28.000000 count completed. 2022-10-15T06:15:27.535Z,1665814527.535 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 150.000000 m , 5.000000 min . 2022-10-15T06:20:58.069Z,1665814858.069 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T06:22:08.742Z,1665814928.742 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T06:27:10.078Z,1665815230.078 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T06:55:35.008Z,1665816935.008 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T06:55:37.489Z,1665816937.489 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 150.000000 m 2022-10-15T06:55:37.825Z,1665816937.825 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 29.000000 count . Target depth = 150.000000 m . 2022-10-15T06:55:38.239Z,1665816938.239 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T07:07:27.295Z,1665817647.295 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.023041 m , settling for 5.000000 min 2022-10-15T07:12:40.060Z,1665817960.060 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T07:12:40.232Z,1665817960.232 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T07:13:42.181Z,1665818022.181 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T07:13:42.994Z,1665818022.994 [ESPComponent](IMPORTANT): [sample #30] Setting cartridge argument to -6 2022-10-15T07:13:42.994Z,1665818022.994 [ESPComponent](IMPORTANT): [sample #30] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T07:13:43.436Z,1665818023.436 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.292557 m in 9.696314 degC water with 0.078000 ug/l chlorophyll fluorescence. 2022-10-15T07:13:43.797Z,1665818023.797 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T07:13:44.202Z,1665818024.202 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T07:13:46.221Z,1665818026.221 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T07:14:20.562Z,1665818060.562 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PRIMING (@7,archive_bac) 2022-10-15T07:14:23.389Z,1665818063.389 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T07:14:23.390Z,1665818063.390 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T07:14:23.795Z,1665818063.795 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T07:16:42.769Z,1665818202.769 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T07:16:43.173Z,1665818203.173 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T08:00:44.569Z,1665820844.569 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PAUSED (@1000.0) 2022-10-15T08:00:44.973Z,1665820844.973 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PROCESSING (@1000.0) 2022-10-15T08:00:46.994Z,1665820846.994 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T08:00:47.397Z,1665820847.397 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PROCESSED (@7,archive_bac) 2022-10-15T08:12:51.373Z,1665821571.373 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_STOPPING (@7,archive_bac) 2022-10-15T08:13:01.076Z,1665821581.076 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T08:13:01.475Z,1665821581.475 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_STOPPED 2022-10-15T08:18:01.249Z,1665821881.249 [ESPComponent](IMPORTANT): [sample #30] ESP sampling sequence completed normally. 2022-10-15T08:18:02.110Z,1665821882.110 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 29.000000 count completed. 2022-10-15T08:18:08.578Z,1665821888.578 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 5.000000 m , 5.000000 min . 2022-10-15T08:35:50.660Z,1665822950.660 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T08:42:17.429Z,1665823337.429 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T08:47:18.695Z,1665823638.695 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T09:12:19.220Z,1665825139.220 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T09:12:21.638Z,1665825141.638 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 5.000000 m 2022-10-15T09:12:22.039Z,1665825142.039 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 30.000000 count . Target depth = 5.000000 m . 2022-10-15T09:12:22.452Z,1665825142.452 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T09:16:19.185Z,1665825379.185 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 5.004333 m , settling for 5.000000 min 2022-10-15T09:20:09.964Z,1665825609.964 [CBIT](IMPORTANT): Beginning ground fault scan 2022-10-15T09:20:20.862Z,1665825620.862 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.003139 CHAN A1 (24V): 0.007354 CHAN A2 (12V): -0.004098 CHAN A3 (5V): -0.003364 CHAN B0 (3.3V): -0.001708 CHAN B1 (3.15aV): -0.001776 CHAN B2 (3.15bV): -0.000832 CHAN B3 (GND): -0.000784 OPEN: 0.012215 Full Scale: +/- 1 mA 2022-10-15T09:21:31.956Z,1665825691.956 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T09:21:32.092Z,1665825692.092 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T09:22:33.729Z,1665825753.729 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T09:22:34.542Z,1665825754.542 [ESPComponent](IMPORTANT): [sample #31] Setting cartridge argument to -6 2022-10-15T09:22:34.542Z,1665825754.542 [ESPComponent](IMPORTANT): [sample #31] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T09:22:34.988Z,1665825754.988 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.486918 m in 14.634821 degC water with 4.006080 ug/l chlorophyll fluorescence. 2022-10-15T09:22:35.348Z,1665825755.348 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T09:22:35.750Z,1665825755.750 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T09:22:37.769Z,1665825757.769 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T09:23:12.111Z,1665825792.111 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PRIMING (@6,archive_bac) 2022-10-15T09:23:14.133Z,1665825794.133 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T09:23:14.537Z,1665825794.537 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T09:23:14.938Z,1665825794.938 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T09:25:33.509Z,1665825933.509 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T09:25:33.914Z,1665825933.914 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T10:14:18.915Z,1665828858.915 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PAUSED (@775.6) 2022-10-15T10:14:19.317Z,1665828859.317 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PROCESSING (@775.6) 2022-10-15T10:14:21.339Z,1665828861.339 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T10:14:21.741Z,1665828861.741 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PROCESSED (@6,archive_bac) 2022-10-15T10:26:25.313Z,1665829585.313 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_STOPPING (@6,archive_bac) 2022-10-15T10:26:35.011Z,1665829595.011 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T10:26:35.414Z,1665829595.414 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_STOPPED 2022-10-15T10:31:35.181Z,1665829895.181 [ESPComponent](IMPORTANT): [sample #31] ESP sampling sequence completed normally. 2022-10-15T10:31:36.030Z,1665829896.030 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 30.000000 count completed. 2022-10-15T10:31:42.511Z,1665829902.511 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 150.000000 m , 5.000000 min . 2022-10-15T10:37:14.982Z,1665830234.982 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T10:44:02.086Z,1665830642.086 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T10:49:03.811Z,1665830943.811 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T11:19:15.793Z,1665832755.793 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T11:19:18.261Z,1665832758.261 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 150.000000 m 2022-10-15T11:19:18.657Z,1665832758.657 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 31.000000 count . Target depth = 150.000000 m . 2022-10-15T11:19:19.025Z,1665832759.025 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T11:33:01.185Z,1665833581.185 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.997025 m , settling for 5.000000 min 2022-10-15T11:38:13.944Z,1665833893.944 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T11:38:14.278Z,1665833894.278 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T11:39:15.861Z,1665833955.861 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T11:39:16.677Z,1665833956.677 [ESPComponent](IMPORTANT): [sample #32] Setting cartridge argument to -6 2022-10-15T11:39:16.678Z,1665833956.678 [ESPComponent](IMPORTANT): [sample #32] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T11:39:17.141Z,1665833957.141 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 151.113632 m in 9.584528 degC water with 0.072800 ug/l chlorophyll fluorescence. 2022-10-15T11:39:17.477Z,1665833957.477 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T11:39:17.882Z,1665833957.882 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T11:39:19.903Z,1665833959.903 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T11:39:54.242Z,1665833994.242 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PRIMING (@5,archive_bac) 2022-10-15T11:39:56.666Z,1665833996.666 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T11:39:56.667Z,1665833996.667 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T11:39:57.075Z,1665833997.075 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T11:42:16.854Z,1665834136.854 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T11:42:17.257Z,1665834137.257 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T12:29:18.441Z,1665836958.441 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PAUSED (@1000.0) 2022-10-15T12:29:18.846Z,1665836958.846 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PROCESSING (@1000.0) 2022-10-15T12:29:20.866Z,1665836960.866 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T12:29:21.269Z,1665836961.269 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PROCESSED (@5,archive_bac) 2022-10-15T12:41:24.835Z,1665837684.835 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_STOPPING (@5,archive_bac) 2022-10-15T12:41:34.531Z,1665837694.531 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T12:41:34.935Z,1665837694.935 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_STOPPED 2022-10-15T12:46:34.710Z,1665837994.710 [ESPComponent](IMPORTANT): [sample #32] ESP sampling sequence completed normally. 2022-10-15T12:46:35.582Z,1665837995.582 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 31.000000 count completed. 2022-10-15T12:46:42.032Z,1665838002.032 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 5.000000 m , 5.000000 min . 2022-10-15T13:03:44.165Z,1665839024.165 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T13:04:44.680Z,1665839084.680 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T13:09:46.122Z,1665839386.122 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T13:29:59.776Z,1665840599.776 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T13:30:02.191Z,1665840602.191 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 5.000000 m 2022-10-15T13:30:02.596Z,1665840602.596 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 32.000000 count . Target depth = 5.000000 m . 2022-10-15T13:30:03.002Z,1665840603.002 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T13:34:02.190Z,1665840842.190 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 4.981446 m , settling for 5.000000 min 2022-10-15T13:39:14.944Z,1665841154.944 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T13:39:15.124Z,1665841155.124 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T13:40:17.113Z,1665841217.113 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T13:40:17.927Z,1665841217.927 [ESPComponent](IMPORTANT): [sample #33] Setting cartridge argument to -6 2022-10-15T13:40:17.927Z,1665841217.927 [ESPComponent](IMPORTANT): [sample #33] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T13:40:18.393Z,1665841218.393 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.564377 m in 14.634577 degC water with 3.725280 ug/l chlorophyll fluorescence. 2022-10-15T13:40:18.729Z,1665841218.729 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T13:40:19.133Z,1665841219.133 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T13:40:21.154Z,1665841221.154 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T13:40:55.489Z,1665841255.489 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_PRIMING (@4,archive_bac) 2022-10-15T13:40:57.915Z,1665841257.915 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T13:40:57.918Z,1665841257.918 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T13:40:58.321Z,1665841258.321 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T13:43:18.106Z,1665841398.106 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T13:43:18.509Z,1665841398.509 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T14:19:04.590Z,1665843544.590 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PAUSED (@586.3) 2022-10-15T14:19:04.993Z,1665843544.993 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PROCESSING (@586.3) 2022-10-15T14:19:07.014Z,1665843547.014 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T14:19:07.420Z,1665843547.420 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_PROCESSED (@4,archive_bac) 2022-10-15T14:31:10.993Z,1665844270.993 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_STOPPING (@4,archive_bac) 2022-10-15T14:31:20.690Z,1665844280.690 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T14:31:21.097Z,1665844281.097 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_STOPPED 2022-10-15T14:36:20.869Z,1665844580.869 [ESPComponent](IMPORTANT): [sample #33] ESP sampling sequence completed normally. 2022-10-15T14:36:21.716Z,1665844581.716 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 32.000000 count completed. 2022-10-15T14:36:28.186Z,1665844588.186 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 150.000000 m , 5.000000 min . 2022-10-15T14:42:01.119Z,1665844921.119 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T14:43:09.810Z,1665844989.810 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T14:48:11.333Z,1665845291.333 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T15:05:45.440Z,1665846345.440 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T15:05:47.839Z,1665846347.839 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 150.000000 m 2022-10-15T15:05:48.256Z,1665846348.256 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 33.000000 count . Target depth = 150.000000 m . 2022-10-15T15:05:48.643Z,1665846348.643 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T15:17:43.020Z,1665847063.020 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.997482 m , settling for 5.000000 min 2022-10-15T15:20:20.984Z,1665847220.984 [CBIT](IMPORTANT): Beginning ground fault scan 2022-10-15T15:20:31.847Z,1665847231.847 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.000915 CHAN A1 (24V): 0.012758 CHAN A2 (12V): -0.002232 CHAN A3 (5V): -0.003284 CHAN B0 (3.3V): -0.001690 CHAN B1 (3.15aV): -0.001293 CHAN B2 (3.15bV): -0.000717 CHAN B3 (GND): -0.001345 OPEN: 0.012248 Full Scale: +/- 1 mA 2022-10-15T15:22:55.696Z,1665847375.696 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T15:22:56.120Z,1665847376.120 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T15:23:59.337Z,1665847439.337 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T15:24:00.150Z,1665847440.150 [ESPComponent](IMPORTANT): [sample #34] Setting cartridge argument to -6 2022-10-15T15:24:00.150Z,1665847440.150 [ESPComponent](IMPORTANT): [sample #34] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T15:24:00.628Z,1665847440.628 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.217651 m in 10.082605 degC water with 0.082160 ug/l chlorophyll fluorescence. 2022-10-15T15:24:00.953Z,1665847440.953 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T15:24:01.357Z,1665847441.357 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T15:24:03.380Z,1665847443.380 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T15:24:37.717Z,1665847477.717 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_PRIMING (@3,archive_bac) 2022-10-15T15:24:40.546Z,1665847480.546 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T15:24:40.547Z,1665847480.547 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T15:24:40.950Z,1665847480.950 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T15:27:00.330Z,1665847620.330 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T15:27:00.733Z,1665847620.733 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T16:11:40.489Z,1665850300.489 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PAUSED (@1000.0) 2022-10-15T16:11:40.894Z,1665850300.894 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PROCESSING (@1000.0) 2022-10-15T16:11:42.916Z,1665850302.916 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T16:11:43.317Z,1665850303.317 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_PROCESSED (@3,archive_bac) 2022-10-15T16:23:46.890Z,1665851026.890 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_STOPPING (@3,archive_bac) 2022-10-15T16:23:56.587Z,1665851036.587 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T16:23:57.026Z,1665851037.026 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_STOPPED 2022-10-15T16:28:56.793Z,1665851336.793 [ESPComponent](IMPORTANT): [sample #34] ESP sampling sequence completed normally. 2022-10-15T16:28:57.668Z,1665851337.668 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 33.000000 count completed. 2022-10-15T16:29:04.145Z,1665851344.145 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 5.000000 m , 5.000000 min . 2022-10-15T16:47:00.792Z,1665852420.792 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T16:48:08.065Z,1665852488.065 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T16:53:08.568Z,1665852788.568 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T17:11:49.464Z,1665853909.464 [ThrusterHE](FAULT): Could not get status:serial timeout 2022-10-15T17:20:47.046Z,1665854447.046 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T17:20:49.483Z,1665854449.483 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 5.000000 m 2022-10-15T17:20:49.868Z,1665854449.868 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 34.000000 count . Target depth = 5.000000 m . 2022-10-15T17:20:50.281Z,1665854450.281 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T17:24:40.178Z,1665854680.178 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 4.974266 m , settling for 5.000000 min 2022-10-15T17:29:52.933Z,1665854992.933 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T17:29:53.079Z,1665854993.079 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T17:30:55.092Z,1665855055.092 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T17:30:55.904Z,1665855055.904 [ESPComponent](IMPORTANT): [sample #35] Setting cartridge argument to -6 2022-10-15T17:30:55.905Z,1665855055.905 [ESPComponent](IMPORTANT): [sample #35] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T17:30:56.369Z,1665855056.369 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.177103 m in 14.396814 degC water with 4.525040 ug/l chlorophyll fluorescence. 2022-10-15T17:30:56.709Z,1665855056.709 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T17:30:57.122Z,1665855057.122 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T17:30:59.133Z,1665855059.133 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T17:31:33.473Z,1665855093.473 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_PRIMING (@2,archive_bac) 2022-10-15T17:31:35.898Z,1665855095.898 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PRIMING (@100.0) 2022-10-15T17:31:35.899Z,1665855095.899 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering 2022-10-15T17:31:36.301Z,1665855096.301 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2022-10-15T17:33:55.685Z,1665855235.685 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_FILTERING (@0.0) 2022-10-15T17:33:56.091Z,1665855236.091 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2022-10-15T18:21:00.901Z,1665858060.901 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PAUSED (@745.3) 2022-10-15T18:21:01.305Z,1665858061.305 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PROCESSING (@745.3) 2022-10-15T18:21:03.326Z,1665858063.326 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2022-10-15T18:21:03.729Z,1665858063.729 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_PROCESSED (@2,archive_bac) 2022-10-15T18:33:07.305Z,1665858787.305 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_STOPPING (@2,archive_bac) 2022-10-15T18:33:17.002Z,1665858797.002 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T18:33:17.411Z,1665858797.411 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_STOPPED 2022-10-15T18:38:17.181Z,1665859097.181 [ESPComponent](IMPORTANT): [sample #35] ESP sampling sequence completed normally. 2022-10-15T18:38:18.032Z,1665859098.032 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 34.000000 count completed. 2022-10-15T18:38:24.517Z,1665859104.517 [LocationDepthSampling:SetSampleOptions:G](IMPORTANT): Set CartridgeType (if ESP), DepthDesignated, WaitDepthUndulation to -6.000000 count , 150.000000 m , 5.000000 min . 2022-10-15T18:43:57.043Z,1665859437.043 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T18:45:03.660Z,1665859503.660 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Navigating to waypoint: 36.800001,-122.096994 2022-10-15T18:50:04.191Z,1665859804.191 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T19:25:39.813Z,1665861939.813 [LocationDepthSampling:GoToLocationToSample:GoToSamplingLocation:LevelRun:G.Waypoint](IMPORTANT): Reached waypoint: 36.800001,-122.096994 2022-10-15T19:25:42.209Z,1665861942.209 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:D](IMPORTANT): DepthSet = 150.000000 m 2022-10-15T19:25:42.618Z,1665861942.618 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:E](IMPORTANT): Taking sample No. 35.000000 count . Target depth = 150.000000 m . 2022-10-15T19:25:43.045Z,1665861943.045 [LocationDepthSampling:GoToLocationToSample:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2022-10-15T19:37:26.798Z,1665862646.798 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.025284 m , settling for 5.000000 min 2022-10-15T19:42:39.564Z,1665862959.564 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2022-10-15T19:42:39.784Z,1665862959.784 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2022-10-15T19:43:40.996Z,1665863020.996 [ESPComponent](IMPORTANT): ESP has connected as client 2022-10-15T19:43:41.809Z,1665863021.809 [ESPComponent](IMPORTANT): [sample #36] Setting cartridge argument to -6 2022-10-15T19:43:41.809Z,1665863021.809 [ESPComponent](IMPORTANT): [sample #36] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2022-10-15T19:43:42.259Z,1665863022.259 [LocationDepthSampling:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.394333 m in 9.926264 degC water with 0.073840 ug/l chlorophyll fluorescence. 2022-10-15T19:43:42.613Z,1665863022.613 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PREPARING_SHOW_LOG 2022-10-15T19:43:43.017Z,1665863023.017 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PREPARING_SHOW_STATUS 2022-10-15T19:43:45.039Z,1665863025.039 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_LOADING_CARTRIDGE 2022-10-15T19:43:47.063Z,1665863027.063 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_STOPPING 2022-10-15T19:43:49.483Z,1665863029.483 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2022-10-15T19:43:49.886Z,1665863029.886 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_STOPPED 2022-10-15T19:48:49.662Z,1665863329.662 [ESPComponent](CRITICAL): [sample #36] ESP sampling sequence terminated with error: In state S_LOADING_CARTRIDGE got exception while awaiting result for cmd='Cmd.loadCartridge -6' exception=<>. Elapsed time since command issued: 2.0205 2022-10-15T19:48:49.829Z,1665863329.829 [CommandExec](FAULT): Scheduling is paused for 1 commands 2022-10-15T19:48:50.834Z,1665863330.834 [MissionManager](IMPORTANT): Started mission Default 2022-10-15T19:59:49.214Z,1665863989.214 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2022-10-15T20:04:48.765Z,1665864288.765 [DataOverHttps](IMPORTANT): SBD MTMSN=20221015T200445 2022-10-15T20:05:00.778Z,1665864300.778 [CommandExec](IMPORTANT): got command schedule resume 2022-10-15T20:05:00.778Z,1665864300.778 [CommandExec](IMPORTANT): Scheduling is resumed 2022-10-15T20:05:49.495Z,1665864349.495 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 16.970923 min 2022-10-15T20:05:51.105Z,1665864351.105 [NAL9602](FAULT): GPS failed to acquire within timeout. 2022-10-15T20:07:51.899Z,1665864471.899 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=47326, MT Status=1, MTMSN=3524 2022-10-15T20:07:52.943Z,1665864472.943 [CommandExec](IMPORTANT): got command schedule "load Transport/keepstation.tl;set keepstation.MissionTimeout 48 h;set keepstation.NeedCommsTime 120 min;set keepstation.Latitude 36.797 degree;set keepstation.Longitude -121.847 degree;set keepstation.Speed 0.8 m/s;run" 2022-10-15T20:07:52.943Z,1665864472.943 [CommandExec](IMPORTANT): Scheduled #5: "load Transport/keepstation.tl;set keepstation.MissionTimeout 48 h;set keepstation.NeedCommsTime 120 min;set keepstation.Latitude 36.797 degree;set keepstation.Longitude -121.847 degree;set keepstation.Speed 0.8 m/s;run", AFTER MISSION 2022-10-15T20:07:53.075Z,1665864473.075 [CommandExec](IMPORTANT): got command load ./Missions/Transport/keepstation.tl 2022-10-15T20:07:53.082Z,1665864473.082 [MissionManager](IMPORTANT): Loading Compiled TethysL script from file ./Missions/Transport/keepstation.tx 2022-10-15T20:07:54.278Z,1665864474.278 [CommandExec](IMPORTANT): Loaded ./Missions/Transport/keepstation.tl 2022-10-15T20:08:09.757Z,1665864489.757 [CommandExec](IMPORTANT): got command set keepstation.MissionTimeout 48 hour 2022-10-15T20:08:09.758Z,1665864489.758 [CommandExec](IMPORTANT): got command set keepstation.NeedCommsTime 120 minute 2022-10-15T20:08:09.758Z,1665864489.758 [CommandExec](IMPORTANT): got command set keepstation.Latitude 36.797 degree 2022-10-15T20:08:09.759Z,1665864489.759 [CommandExec](IMPORTANT): got command set keepstation.Longitude -121.847 degree 2022-10-15T20:08:09.760Z,1665864489.760 [CommandExec](IMPORTANT): got command set keepstation.Speed 0.8 meter_per_second 2022-10-15T20:08:09.760Z,1665864489.760 [CommandExec](IMPORTANT): got command run 2022-10-15T20:08:09.763Z,1665864489.763 [CommandExec](IMPORTANT): Running 2022-10-15T20:08:09.933Z,1665864489.933 [MissionManager](IMPORTANT): Started mission keepstation 2022-10-15T20:09:21.910Z,1665864561.910 [DataOverHttps](IMPORTANT): SBD MTMSN=20221015T200917 2022-10-15T20:09:33.417Z,1665864573.417 [CommandExec](IMPORTANT): got command load ./Missions/Transport/keepstation.tl 2022-10-15T20:09:33.431Z,1665864573.431 [MissionManager](IMPORTANT): Loading Compiled TethysL script from file ./Missions/Transport/keepstation.tx 2022-10-15T20:09:33.937Z,1665864573.937 [CommandExec](IMPORTANT): Loaded ./Missions/Transport/keepstation.tl 2022-10-15T20:09:34.015Z,1665864574.015 [MissionManager](IMPORTANT): Started mission Default 2022-10-15T20:10:01.945Z,1665864601.945 [CommandExec](IMPORTANT): got command set keepstation.MissionTimeout 48 hour 2022-10-15T20:10:01.946Z,1665864601.946 [CommandExec](IMPORTANT): got command set keepstation.NeedCommsTime 120 minute 2022-10-15T20:10:01.946Z,1665864601.946 [CommandExec](IMPORTANT): got command set keepstation.Latitude 36.797 degree 2022-10-15T20:10:01.947Z,1665864601.947 [CommandExec](IMPORTANT): got command set keepstation.Longitude -121.847 degree 2022-10-15T20:10:01.952Z,1665864601.952 [CommandExec](IMPORTANT): got command set keepstation.Speed 0.8 meter_per_second 2022-10-15T20:10:01.952Z,1665864601.952 [CommandExec](IMPORTANT): got command run 2022-10-15T20:10:01.954Z,1665864601.954 [CommandExec](IMPORTANT): Running 2022-10-15T20:10:02.124Z,1665864602.124 [MissionManager](IMPORTANT): Started mission keepstation 2022-10-15T20:11:13.273Z,1665864673.273 [DataOverHttps](IMPORTANT): SBD MTMSN=20221015T201109