2024-09-22T12:01:03.883Z,1727006463.883 [CommandExec](IMPORTANT): got command restart logs
2024-09-22T12:11:23.569Z,1727007083.569 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.031217 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 7.000000 count 
2024-09-22T12:11:27.269Z,1727007087.269 [ElevatorServo](FAULT): Overload Error
2024-09-22T12:13:03.772Z,1727007183.772 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At  149.686020 m , settling for  1.000000 min 
2024-09-22T12:14:16.586Z,1727007256.586 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2024-09-22T12:14:16.723Z,1727007256.723 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2024-09-22T12:15:09.398Z,1727007309.398 [ESPComponent](IMPORTANT): ESP has connected as client
2024-09-22T12:15:10.210Z,1727007310.210 [ESPComponent](IMPORTANT): [sample #27] Setting cartridge argument to -1
2024-09-22T12:15:10.210Z,1727007310.210 [ESPComponent](IMPORTANT): [sample #27] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2024-09-22T12:15:10.652Z,1727007310.652 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at  150.668976 m  in  9.458063 degC  water with  0.072800 ug/l chlorophyll fluorescence.
2024-09-22T12:15:11.018Z,1727007311.018 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PREPARING_SHOW_LOG
2024-09-22T12:15:11.419Z,1727007311.419 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PREPARING_SHOW_STATUS
2024-09-22T12:15:13.442Z,1727007313.442 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_LOADING_CARTRIDGE
2024-09-22T12:15:44.954Z,1727007344.954 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_PRIMING (@22,aRNAl_bac)
2024-09-22T12:15:47.379Z,1727007347.379 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PRIMING (@100.0)
2024-09-22T12:15:47.380Z,1727007347.380 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering
2024-09-22T12:15:47.783Z,1727007347.783 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2024-09-22T12:18:44.347Z,1727007524.347 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_FILTERING (@0.0)
2024-09-22T12:18:44.752Z,1727007524.752 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2024-09-22T12:32:27.137Z,1727008347.137 [ThrusterHE](FAULT): Could not get status:serial timeout
2024-09-22T13:05:11.249Z,1727010311.249 [ThrusterHE](FAULT): getSpeed error serial timeout
2024-09-22T13:05:11.249Z,1727010311.249 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2024-09-22T13:07:04.940Z,1727010424.940 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PAUSED (@1000.0)
2024-09-22T13:07:05.343Z,1727010425.343 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PROCESSING (@1000.0)
2024-09-22T13:07:07.364Z,1727010427.364 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2024-09-22T13:07:07.767Z,1727010427.767 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_PROCESSED (@22,aRNAl_bac)
2024-09-22T13:08:43.125Z,1727010523.125 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_STOPPING (@22,aRNAl_bac)
2024-09-22T13:08:50.792Z,1727010530.792 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2024-09-22T13:08:51.195Z,1727010531.195 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_STOPPED
2024-09-22T13:13:50.967Z,1727010830.967 [ESPComponent](IMPORTANT): [sample #27] ESP sampling sequence completed normally.
2024-09-22T13:13:53.438Z,1727010833.438 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 7.000000 count 
2024-09-22T13:13:53.864Z,1727010833.864 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2024-09-22T13:13:54.254Z,1727010834.254 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2024-09-22T13:13:54.640Z,1727010834.640 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2024-09-22T13:13:55.047Z,1727010835.047 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.460739 m 
2024-09-22T13:13:55.455Z,1727010835.455 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.442713 degC 
2024-09-22T13:13:55.867Z,1727010835.867 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.072800 ug/l 
2024-09-22T13:13:56.657Z,1727010836.657 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 7.000000 count completed.
2024-09-22T13:14:01.135Z,1727010841.135 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min .
2024-09-22T13:25:30.173Z,1727011530.173 [ElevatorServo](FAULT): Overload Error
2024-09-22T14:11:26.120Z,1727014286.120 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.029045 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 8.000000 count 
2024-09-22T14:12:39.255Z,1727014359.255 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At  149.681747 m , settling for  1.000000 min 
2024-09-22T14:13:52.053Z,1727014432.053 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2024-09-22T14:13:52.270Z,1727014432.270 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2024-09-22T14:14:45.047Z,1727014485.047 [ESPComponent](IMPORTANT): ESP has connected as client
2024-09-22T14:14:45.860Z,1727014485.860 [ESPComponent](IMPORTANT): [sample #28] Setting cartridge argument to -1
2024-09-22T14:14:45.860Z,1727014485.860 [ESPComponent](IMPORTANT): [sample #28] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2024-09-22T14:14:46.302Z,1727014486.302 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at  149.701920 m  in  9.502192 degC  water with  0.087360 ug/l chlorophyll fluorescence.
2024-09-22T14:14:46.667Z,1727014486.667 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PREPARING_SHOW_LOG
2024-09-22T14:14:47.067Z,1727014487.067 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PREPARING_SHOW_STATUS
2024-09-22T14:14:49.087Z,1727014489.087 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_LOADING_CARTRIDGE
2024-09-22T14:15:20.734Z,1727014520.734 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_PRIMING (@21,aRNAl_bac)
2024-09-22T14:15:22.756Z,1727014522.756 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PRIMING (@100.0)
2024-09-22T14:15:23.162Z,1727014523.162 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering
2024-09-22T14:15:23.568Z,1727014523.568 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2024-09-22T14:18:20.515Z,1727014700.515 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_FILTERING (@0.0)
2024-09-22T14:18:20.919Z,1727014700.919 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2024-09-22T14:30:41.265Z,1727015441.265 [RudderServo](FAULT): Rudder uart error: serial timeout
2024-09-22T15:04:22.597Z,1727017462.597 [CBIT](IMPORTANT): Beginning ground fault scan
2024-09-22T15:04:33.963Z,1727017473.963 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): 0.009061
CHAN A1 (24V): 0.002008
CHAN A2 (12V): 0.003859
CHAN A3 (5V): 0.000965
CHAN B0 (3.3V): 0.000172
CHAN B1 (3.15aV): 0.000254
CHAN B2 (3.15bV): 0.000148
CHAN B3 (GND): -0.000306
OPEN: 0.007605
Full Scale: +/- 1 mA
2024-09-22T15:07:47.751Z,1727017667.751 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PAUSED (@1000.0)
2024-09-22T15:07:48.157Z,1727017668.157 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PROCESSING (@1000.0)
2024-09-22T15:07:50.183Z,1727017670.183 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2024-09-22T15:07:50.579Z,1727017670.579 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_PROCESSED (@21,aRNAl_bac)
2024-09-22T15:09:25.923Z,1727017765.923 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_STOPPING (@21,aRNAl_bac)
2024-09-22T15:09:33.599Z,1727017773.599 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2024-09-22T15:09:34.003Z,1727017774.003 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_STOPPED
2024-09-22T15:14:33.786Z,1727018073.786 [ESPComponent](IMPORTANT): [sample #28] ESP sampling sequence completed normally.
2024-09-22T15:14:36.245Z,1727018076.245 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 8.000000 count 
2024-09-22T15:14:36.708Z,1727018076.708 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2024-09-22T15:14:37.070Z,1727018077.070 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2024-09-22T15:14:37.478Z,1727018077.478 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2024-09-22T15:14:37.865Z,1727018077.865 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.686539 m 
2024-09-22T15:14:38.266Z,1727018078.266 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.418909 degC 
2024-09-22T15:14:38.706Z,1727018078.706 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.041600 ug/l 
2024-09-22T15:14:39.481Z,1727018079.481 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 8.000000 count completed.
2024-09-22T15:14:43.941Z,1727018083.941 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min .
2024-09-22T15:15:17.166Z,1727018117.166 [DATMMP](FAULT): Buffer send receipt timeout failure.
2024-09-22T15:15:28.002Z,1727018128.002 [DATMMP](FAULT): Buffer send receipt timeout failure.
2024-09-22T15:33:00.601Z,1727019180.601 [ThrusterHE](FAULT): Could not get status:serial timeout
2024-09-22T16:00:13.090Z,1727020813.090 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout
2024-09-22T16:11:28.715Z,1727021488.715 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.029777 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 9.000000 count 
2024-09-22T16:12:32.612Z,1727021552.612 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At  150.284714 m , settling for  1.000000 min 
2024-09-22T16:13:45.389Z,1727021625.389 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2024-09-22T16:13:45.508Z,1727021625.508 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2024-09-22T16:14:38.614Z,1727021678.614 [ESPComponent](IMPORTANT): ESP has connected as client
2024-09-22T16:14:39.426Z,1727021679.426 [ESPComponent](IMPORTANT): [sample #29] Setting cartridge argument to -1
2024-09-22T16:14:39.426Z,1727021679.426 [ESPComponent](IMPORTANT): [sample #29] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2024-09-22T16:14:39.885Z,1727021679.885 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at  149.998154 m  in  9.448114 degC  water with  0.064480 ug/l chlorophyll fluorescence.
2024-09-22T16:14:40.230Z,1727021680.230 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PREPARING_SHOW_LOG
2024-09-22T16:14:40.635Z,1727021680.635 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PREPARING_SHOW_STATUS
2024-09-22T16:14:42.659Z,1727021682.659 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_LOADING_CARTRIDGE
2024-09-22T16:15:14.255Z,1727021714.255 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PRIMING (@20,aRNAl_bac)
2024-09-22T16:15:16.687Z,1727021716.687 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PRIMING (@100.0)
2024-09-22T16:15:16.688Z,1727021716.688 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering
2024-09-22T16:15:17.087Z,1727021717.087 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2024-09-22T16:18:13.657Z,1727021893.657 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_FILTERING (@0.0)
2024-09-22T16:18:14.076Z,1727021894.076 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2024-09-22T17:04:06.253Z,1727024646.253 [ThrusterHE](FAULT): Could not get status:serial timeout
2024-09-22T17:05:09.499Z,1727024709.499 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PAUSED (@1000.0)
2024-09-22T17:05:09.903Z,1727024709.903 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PROCESSING (@1000.0)
2024-09-22T17:05:11.929Z,1727024711.929 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2024-09-22T17:05:12.327Z,1727024712.327 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PROCESSED (@20,aRNAl_bac)
2024-09-22T17:06:47.671Z,1727024807.671 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_STOPPING (@20,aRNAl_bac)
2024-09-22T17:06:55.348Z,1727024815.348 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2024-09-22T17:06:55.754Z,1727024815.754 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_STOPPED
2024-09-22T17:11:55.571Z,1727025115.571 [ESPComponent](IMPORTANT): [sample #29] ESP sampling sequence completed normally.
2024-09-22T17:11:58.066Z,1727025118.066 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 9.000000 count 
2024-09-22T17:11:58.453Z,1727025118.453 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2024-09-22T17:11:58.844Z,1727025118.844 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2024-09-22T17:11:59.257Z,1727025119.257 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2024-09-22T17:11:59.659Z,1727025119.659 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.916885 m 
2024-09-22T17:12:00.084Z,1727025120.084 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.464899 degC 
2024-09-22T17:12:00.456Z,1727025120.456 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.052000 ug/l 
2024-09-22T17:12:01.267Z,1727025121.267 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 9.000000 count completed.
2024-09-22T17:12:05.754Z,1727025125.754 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min .
2024-09-22T18:11:31.346Z,1727028691.346 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.029785 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 10.000000 count 
2024-09-22T18:15:13.136Z,1727028913.136 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At  149.769714 m , settling for  1.000000 min 
2024-09-22T18:16:25.922Z,1727028985.922 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2024-09-22T18:16:26.082Z,1727028986.082 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2024-09-22T18:17:18.835Z,1727029038.835 [ESPComponent](IMPORTANT): ESP has connected as client
2024-09-22T18:17:19.646Z,1727029039.646 [ESPComponent](IMPORTANT): [sample #30] Setting cartridge argument to -1
2024-09-22T18:17:19.646Z,1727029039.646 [ESPComponent](IMPORTANT): [sample #30] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2024-09-22T18:17:20.096Z,1727029040.096 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at  149.001984 m  in  9.673243 degC  water with  0.081120 ug/l chlorophyll fluorescence.
2024-09-22T18:17:20.451Z,1727029040.451 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PREPARING_SHOW_LOG
2024-09-22T18:17:20.858Z,1727029040.858 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PREPARING_SHOW_STATUS
2024-09-22T18:17:22.876Z,1727029042.876 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_LOADING_CARTRIDGE
2024-09-22T18:17:54.387Z,1727029074.387 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PRIMING (@19,aRNAl_bac)
2024-09-22T18:17:57.215Z,1727029077.215 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PRIMING (@100.0)
2024-09-22T18:17:57.216Z,1727029077.216 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering
2024-09-22T18:17:57.618Z,1727029077.618 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2024-09-22T18:19:21.253Z,1727029161.253 [ThrusterHE](FAULT): getSpeed error serial timeout
2024-09-22T18:19:21.253Z,1727029161.253 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2024-09-22T18:20:54.358Z,1727029254.358 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_FILTERING (@0.0)
2024-09-22T18:20:54.763Z,1727029254.763 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2024-09-22T18:22:02.652Z,1727029322.652 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PAUSED (@0.0)
2024-09-22T18:22:03.065Z,1727029323.065 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PROCESSING (@0.0)
2024-09-22T18:22:05.071Z,1727029325.071 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2024-09-22T18:22:05.474Z,1727029325.474 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PROCESSED (@19,aRNAl_bac)
2024-09-22T18:23:40.413Z,1727029420.413 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_STOPPING (@19,aRNAl_bac)
2024-09-22T18:23:48.091Z,1727029428.091 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2024-09-22T18:23:48.497Z,1727029428.497 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_STOPPED
2024-09-22T18:28:48.306Z,1727029728.306 [ESPComponent](IMPORTANT): [sample #30] ESP sampling sequence completed normally.
2024-09-22T18:28:50.779Z,1727029730.779 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 10.000000 count 
2024-09-22T18:28:51.232Z,1727029731.232 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2024-09-22T18:28:51.586Z,1727029731.586 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2024-09-22T18:28:52.006Z,1727029732.006 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2024-09-22T18:28:52.437Z,1727029732.437 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.636948 m 
2024-09-22T18:28:52.790Z,1727029732.790 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.529902 degC 
2024-09-22T18:28:53.202Z,1727029733.202 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.083200 ug/l 
2024-09-22T18:28:54.024Z,1727029734.024 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 10.000000 count completed.
2024-09-22T18:28:58.471Z,1727029738.471 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min .
2024-09-22T18:55:16.249Z,1727031316.249 [ThrusterHE](FAULT): getSpeed error serial timeout
2024-09-22T18:55:16.249Z,1727031316.249 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2024-09-22T20:11:33.766Z,1727035893.766 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.027848 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 11.000000 count 
2024-09-22T20:16:11.740Z,1727036171.740 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At  150.188828 m , settling for  1.000000 min 
2024-09-22T20:17:24.510Z,1727036244.510 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2024-09-22T20:17:24.615Z,1727036244.615 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2024-09-22T20:18:15.725Z,1727036295.725 [ESPComponent](IMPORTANT): ESP has connected as client
2024-09-22T20:18:16.534Z,1727036296.534 [ESPComponent](IMPORTANT): [sample #31] Setting cartridge argument to -1
2024-09-22T20:18:16.535Z,1727036296.535 [ESPComponent](IMPORTANT): [sample #31] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2024-09-22T20:18:16.990Z,1727036296.990 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at  150.299133 m  in  9.467859 degC  water with  0.070720 ug/l chlorophyll fluorescence.
2024-09-22T20:18:17.339Z,1727036297.339 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PREPARING_SHOW_LOG
2024-09-22T20:18:17.742Z,1727036297.742 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PREPARING_SHOW_STATUS
2024-09-22T20:18:19.763Z,1727036299.763 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_LOADING_CARTRIDGE
2024-09-22T20:18:51.275Z,1727036331.275 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PRIMING (@18,aRNAl_bac)
2024-09-22T20:18:53.699Z,1727036333.699 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PRIMING (@100.0)
2024-09-22T20:18:53.700Z,1727036333.700 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering
2024-09-22T20:18:54.102Z,1727036334.102 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2024-09-22T20:21:51.062Z,1727036511.062 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_FILTERING (@0.0)
2024-09-22T20:21:51.463Z,1727036511.463 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2024-09-22T20:22:59.746Z,1727036579.746 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PAUSED (@0.0)
2024-09-22T20:23:00.152Z,1727036580.152 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PROCESSING (@0.0)
2024-09-22T20:23:02.171Z,1727036582.171 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2024-09-22T20:23:02.575Z,1727036582.575 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PROCESSED (@18,aRNAl_bac)
2024-09-22T20:24:37.522Z,1727036677.522 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_STOPPING (@18,aRNAl_bac)
2024-09-22T20:24:45.204Z,1727036685.204 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2024-09-22T20:24:45.603Z,1727036685.603 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_STOPPED
2024-09-22T20:29:45.407Z,1727036985.407 [ESPComponent](IMPORTANT): [sample #31] ESP sampling sequence completed normally.
2024-09-22T20:29:47.888Z,1727036987.888 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 11.000000 count 
2024-09-22T20:29:48.270Z,1727036988.270 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2024-09-22T20:29:48.768Z,1727036988.768 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2024-09-22T20:29:49.117Z,1727036989.117 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2024-09-22T20:29:49.509Z,1727036989.509 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.223343 m 
2024-09-22T20:29:49.914Z,1727036989.914 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.551111 degC 
2024-09-22T20:29:50.290Z,1727036990.290 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.083200 ug/l 
2024-09-22T20:29:51.110Z,1727036991.110 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 11.000000 count completed.
2024-09-22T20:29:55.566Z,1727036995.566 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min .
2024-09-22T21:04:34.269Z,1727039074.269 [CBIT](IMPORTANT): Beginning ground fault scan
2024-09-22T21:04:45.633Z,1727039085.633 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): 0.004108
CHAN A1 (24V): -0.011991
CHAN A2 (12V): 0.002135
CHAN A3 (5V): -0.000824
CHAN B0 (3.3V): 0.000338
CHAN B1 (3.15aV): 0.000263
CHAN B2 (3.15bV): 0.000177
CHAN B3 (GND): -0.001796
OPEN: 0.007295
Full Scale: +/- 1 mA
2024-09-22T22:11:36.380Z,1727043096.380 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.028491 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 12.000000 count 
2024-09-22T22:13:56.249Z,1727043236.249 [ThrusterHE](FAULT): Could not get status:serial timeout
2024-09-22T22:15:03.159Z,1727043303.159 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At  150.191849 m , settling for  1.000000 min 
2024-09-22T22:16:15.930Z,1727043375.930 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2024-09-22T22:16:16.075Z,1727043376.075 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2024-09-22T22:17:03.906Z,1727043423.906 [ESPComponent](IMPORTANT): ESP has connected as client
2024-09-22T22:17:04.718Z,1727043424.718 [ESPComponent](IMPORTANT): [sample #32] Setting cartridge argument to -1
2024-09-22T22:17:04.718Z,1727043424.718 [ESPComponent](IMPORTANT): [sample #32] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2024-09-22T22:17:05.160Z,1727043425.160 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at  150.343414 m  in  9.530756 degC  water with  0.075920 ug/l chlorophyll fluorescence.
2024-09-22T22:17:05.522Z,1727043425.522 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PREPARING_SHOW_LOG
2024-09-22T22:17:05.927Z,1727043425.927 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PREPARING_SHOW_STATUS
2024-09-22T22:17:07.947Z,1727043427.947 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_LOADING_CARTRIDGE
2024-09-22T22:17:39.475Z,1727043459.475 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PRIMING (@17,aRNAl_bac)
2024-09-22T22:17:42.303Z,1727043462.303 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PRIMING (@100.0)
2024-09-22T22:17:42.303Z,1727043462.303 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering
2024-09-22T22:17:42.707Z,1727043462.707 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2024-09-22T22:20:40.080Z,1727043640.080 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_FILTERING (@0.0)
2024-09-22T22:20:40.483Z,1727043640.483 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2024-09-22T22:21:31.388Z,1727043691.388 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PAUSED (@0.1)
2024-09-22T22:21:31.791Z,1727043691.791 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PROCESSING (@0.1)
2024-09-22T22:21:33.811Z,1727043693.811 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2024-09-22T22:21:34.218Z,1727043694.218 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PROCESSED (@17,aRNAl_bac)
2024-09-22T22:23:09.171Z,1727043789.171 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_STOPPING (@17,aRNAl_bac)
2024-09-22T22:23:16.836Z,1727043796.836 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2024-09-22T22:23:17.239Z,1727043797.239 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_STOPPED
2024-09-22T22:28:17.042Z,1727044097.042 [ESPComponent](IMPORTANT): [sample #32] ESP sampling sequence completed normally.
2024-09-22T22:28:19.523Z,1727044099.523 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 12.000000 count 
2024-09-22T22:28:19.951Z,1727044099.951 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2024-09-22T22:28:20.309Z,1727044100.309 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2024-09-22T22:28:20.723Z,1727044100.723 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2024-09-22T22:28:21.141Z,1727044101.141 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.734879 m 
2024-09-22T22:28:21.523Z,1727044101.523 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.520899 degC 
2024-09-22T22:28:21.936Z,1727044101.936 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.114400 ug/l 
2024-09-22T22:28:22.803Z,1727044102.803 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 12.000000 count completed.
2024-09-22T22:28:27.211Z,1727044107.211 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min .
2024-09-22T22:29:00.401Z,1727044140.401 [DATMMP](FAULT): Buffer send receipt timeout failure.
2024-09-22T23:10:31.245Z,1727046631.245 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout
2024-09-23T00:01:04.739Z,1727049664.739 [CircleSample:CycleLogs:A:B](IMPORTANT): Restarting logs.