2023-05-23T20:31:13.928Z,1684873873.928 [CommandExec](IMPORTANT): got command restart logs
2023-05-23T20:31:52.670Z,1684873912.670 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.027498 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 13.000000 count 
2023-05-23T20:32:15.703Z,1684873935.703 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.903534 m , settling for 5.000000 min 
2023-05-23T20:37:28.489Z,1684874248.489 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-23T20:37:28.624Z,1684874248.624 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-23T20:38:17.671Z,1684874297.671 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-23T20:38:18.485Z,1684874298.485 [ESPComponent](IMPORTANT): [sample #13] Setting cartridge argument to -1
2023-05-23T20:38:18.486Z,1684874298.486 [ESPComponent](IMPORTANT): [sample #13] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-23T20:38:18.926Z,1684874298.926 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 99.626190 m in 9.442286 degC water with 0.042920 ug/l chlorophyll fluorescence.
2023-05-23T20:38:19.286Z,1684874299.286 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-23T20:38:19.694Z,1684874299.694 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-23T20:38:21.711Z,1684874301.711 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-23T20:38:53.635Z,1684874333.635 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PRIMING (@12,aRNAl_bac)
2023-05-23T20:38:56.454Z,1684874336.454 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PRIMING (@100.0)
2023-05-23T20:38:56.455Z,1684874336.455 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-23T20:38:56.863Z,1684874336.863 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-23T20:41:39.266Z,1684874499.266 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_FILTERING (@0.0)
2023-05-23T20:41:39.671Z,1684874499.671 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-23T21:32:51.375Z,1684877571.375 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PAUSED (@1000.0)
2023-05-23T21:32:51.774Z,1684877571.774 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-23T21:32:53.796Z,1684877573.796 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-23T21:32:54.203Z,1684877574.203 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PROCESSED (@12,aRNAl_bac)
2023-05-23T21:34:29.142Z,1684877669.142 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_STOPPING (@12,aRNAl_bac)
2023-05-23T21:34:36.814Z,1684877676.814 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-23T21:34:37.219Z,1684877677.219 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_STOPPED
2023-05-23T21:39:36.997Z,1684877976.997 [ESPComponent](IMPORTANT): [sample #13] ESP sampling sequence completed normally.
2023-05-23T21:39:39.457Z,1684877979.457 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 13.000000 count 
2023-05-23T21:39:39.859Z,1684877979.859 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-23T21:39:40.273Z,1684877980.273 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-23T21:39:40.659Z,1684877980.659 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-23T21:39:41.063Z,1684877981.063 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 99.822273 m 
2023-05-23T21:39:41.479Z,1684877981.479 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.450006 degC 
2023-05-23T21:39:41.882Z,1684877981.882 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.034800 ug/l 
2023-05-23T21:39:42.679Z,1684877982.679 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 13.000000 count completed.
2023-05-23T21:39:47.146Z,1684877987.146 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-23T21:39:48.777Z,1684877988.777 [DAT](FAULT): Sending transmit timeout failure.
2023-05-23T21:39:59.110Z,1684877999.110 [DAT](FAULT): Sending transmit timeout failure.
2023-05-23T22:31:55.387Z,1684881115.387 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.031510 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 14.000000 count 
2023-05-23T22:33:58.592Z,1684881238.592 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.844284 m , settling for 5.000000 min 
2023-05-23T22:39:11.381Z,1684881551.381 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-23T22:39:11.518Z,1684881551.518 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-23T22:40:05.014Z,1684881605.014 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-23T22:40:05.821Z,1684881605.821 [ESPComponent](IMPORTANT): [sample #14] Setting cartridge argument to -1
2023-05-23T22:40:05.821Z,1684881605.821 [ESPComponent](IMPORTANT): [sample #14] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-23T22:40:06.252Z,1684881606.252 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 99.017029 m in 9.521204 degC water with 0.045240 ug/l chlorophyll fluorescence.
2023-05-23T22:40:06.640Z,1684881606.640 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-23T22:40:07.035Z,1684881607.035 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-23T22:40:09.058Z,1684881609.058 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-23T22:40:40.971Z,1684881640.971 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_PRIMING (@11,aRNAl_bac)
2023-05-23T22:40:43.794Z,1684881643.794 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PRIMING (@100.0)
2023-05-23T22:40:43.795Z,1684881643.795 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-23T22:40:44.203Z,1684881644.203 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-23T22:43:26.616Z,1684881806.616 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_FILTERING (@0.0)
2023-05-23T22:43:27.010Z,1684881807.010 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-23T23:29:54.271Z,1684884594.271 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PAUSED (@1000.0)
2023-05-23T23:29:54.679Z,1684884594.679 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-23T23:29:56.695Z,1684884596.695 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-23T23:29:57.101Z,1684884597.101 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_PROCESSED (@11,aRNAl_bac)
2023-05-23T23:31:32.050Z,1684884692.050 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_STOPPING (@11,aRNAl_bac)
2023-05-23T23:31:39.732Z,1684884699.732 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-23T23:31:40.135Z,1684884700.135 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_STOPPED
2023-05-23T23:36:39.901Z,1684884999.901 [ESPComponent](IMPORTANT): [sample #14] ESP sampling sequence completed normally.
2023-05-23T23:36:42.351Z,1684885002.351 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 14.000000 count 
2023-05-23T23:36:42.751Z,1684885002.751 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-23T23:36:43.167Z,1684885003.167 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-23T23:36:43.579Z,1684885003.579 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-23T23:36:43.966Z,1684885003.966 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 99.305122 m 
2023-05-23T23:36:44.367Z,1684885004.367 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.463312 degC 
2023-05-23T23:36:44.778Z,1684885004.778 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.046400 ug/l 
2023-05-23T23:36:45.579Z,1684885005.579 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 14.000000 count completed.
2023-05-23T23:36:50.048Z,1684885010.048 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-23T23:36:51.717Z,1684885011.717 [DAT](FAULT): Sending transmit timeout failure.
2023-05-23T23:37:02.053Z,1684885022.053 [DAT](FAULT): Sending transmit timeout failure.
2023-05-23T23:54:53.736Z,1684886093.736 [CBIT](IMPORTANT): Beginning ground fault scan
2023-05-23T23:55:04.748Z,1684886104.748 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): -0.001660
CHAN A1 (24V): 0.000227
CHAN A2 (12V): -0.000130
CHAN A3 (5V): 0.000178
CHAN B0 (3.3V): 0.000413
CHAN B1 (3.15aV): 0.000386
CHAN B2 (3.15bV): 0.000104
CHAN B3 (GND): -0.000345
OPEN: 0.007663
Full Scale: +/- 1 mA
2023-05-24T00:31:57.813Z,1684888317.813 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.027287 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 15.000000 count 
2023-05-24T00:34:13.959Z,1684888453.959 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.819565 m , settling for 5.000000 min 
2023-05-24T00:39:26.754Z,1684888766.754 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T00:39:26.883Z,1684888766.883 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T00:40:19.197Z,1684888819.197 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T00:40:20.015Z,1684888820.015 [ESPComponent](IMPORTANT): [sample #15] Setting cartridge argument to -1
2023-05-24T00:40:20.015Z,1684888820.015 [ESPComponent](IMPORTANT): [sample #15] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T00:40:20.456Z,1684888820.456 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 100.330177 m in 9.417627 degC water with 0.040600 ug/l chlorophyll fluorescence.
2023-05-24T00:40:20.823Z,1684888820.823 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T00:40:21.218Z,1684888821.218 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T00:40:23.251Z,1684888823.251 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T00:40:55.183Z,1684888855.183 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_PRIMING (@10,aRNAl_bac)
2023-05-24T00:40:58.018Z,1684888858.018 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T00:40:58.019Z,1684888858.019 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T00:40:58.414Z,1684888858.414 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T00:43:40.822Z,1684889020.822 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T00:43:41.226Z,1684889021.226 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T01:00:23.325Z,1684890023.325 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T01:15:13.881Z,1684890913.881 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-05-24T01:15:13.881Z,1684890913.881 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-05-24T01:31:00.778Z,1684891860.778 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T01:31:01.255Z,1684891861.255 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T01:31:03.195Z,1684891863.195 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T01:31:03.599Z,1684891863.599 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_PROCESSED (@10,aRNAl_bac)
2023-05-24T01:32:38.557Z,1684891958.557 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_STOPPING (@10,aRNAl_bac)
2023-05-24T01:32:46.224Z,1684891966.224 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T01:32:46.623Z,1684891966.623 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_STOPPED
2023-05-24T01:37:46.393Z,1684892266.393 [ESPComponent](IMPORTANT): [sample #15] ESP sampling sequence completed normally.
2023-05-24T01:37:48.857Z,1684892268.857 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 15.000000 count 
2023-05-24T01:37:49.255Z,1684892269.255 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T01:37:49.660Z,1684892269.660 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T01:37:50.050Z,1684892270.050 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T01:37:50.462Z,1684892270.462 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 100.293404 m 
2023-05-24T01:37:50.885Z,1684892270.885 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.305811 degC 
2023-05-24T01:37:51.262Z,1684892271.262 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.023200 ug/l 
2023-05-24T01:37:52.079Z,1684892272.079 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 15.000000 count completed.
2023-05-24T01:37:56.541Z,1684892276.541 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T01:37:58.181Z,1684892278.181 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T01:38:08.513Z,1684892288.513 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T02:32:00.343Z,1684895520.343 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.028784 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 16.000000 count 
2023-05-24T02:33:33.290Z,1684895613.290 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.820351 m , settling for 5.000000 min 
2023-05-24T02:38:46.066Z,1684895926.066 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T02:38:46.199Z,1684895926.199 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T02:39:46.970Z,1684895986.970 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T02:39:47.792Z,1684895987.792 [ESPComponent](IMPORTANT): [sample #16] Setting cartridge argument to -1
2023-05-24T02:39:47.793Z,1684895987.793 [ESPComponent](IMPORTANT): [sample #16] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T02:39:48.244Z,1684895988.244 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 100.219482 m in 9.302423 degC water with 0.030160 ug/l chlorophyll fluorescence.
2023-05-24T02:39:48.607Z,1684895988.607 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T02:39:49.119Z,1684895989.119 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T02:39:51.023Z,1684895991.023 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T02:40:22.942Z,1684896022.942 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_PRIMING (@9,aRNAl_bac)
2023-05-24T02:40:24.964Z,1684896024.964 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T02:40:25.374Z,1684896025.374 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T02:40:25.779Z,1684896025.779 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T02:43:08.591Z,1684896188.591 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T02:43:09.000Z,1684896189.000 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T03:29:01.530Z,1684898941.530 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T03:29:01.935Z,1684898941.935 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T03:29:03.959Z,1684898943.959 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T03:29:04.360Z,1684898944.360 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_PROCESSED (@9,aRNAl_bac)
2023-05-24T03:30:39.300Z,1684899039.300 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_STOPPING (@9,aRNAl_bac)
2023-05-24T03:30:46.976Z,1684899046.976 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T03:30:47.383Z,1684899047.383 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_STOPPED
2023-05-24T03:35:47.158Z,1684899347.158 [ESPComponent](IMPORTANT): [sample #16] ESP sampling sequence completed normally.
2023-05-24T03:35:49.623Z,1684899349.623 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 16.000000 count 
2023-05-24T03:35:50.009Z,1684899350.009 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T03:35:50.411Z,1684899350.411 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T03:35:50.830Z,1684899350.830 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T03:35:51.242Z,1684899351.242 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 100.024620 m 
2023-05-24T03:35:51.626Z,1684899351.626 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.310999 degC 
2023-05-24T03:35:52.034Z,1684899352.034 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.034800 ug/l 
2023-05-24T03:35:52.847Z,1684899352.847 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 16.000000 count completed.
2023-05-24T03:35:57.304Z,1684899357.304 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T03:35:58.986Z,1684899358.986 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T03:36:09.317Z,1684899369.317 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T04:32:03.124Z,1684902723.124 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.032593 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 17.000000 count 
2023-05-24T04:35:43.718Z,1684902943.718 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.881187 m , settling for 5.000000 min 
2023-05-24T04:40:56.477Z,1684903256.477 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T04:40:56.612Z,1684903256.612 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T04:41:56.994Z,1684903316.994 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T04:41:57.810Z,1684903317.810 [ESPComponent](IMPORTANT): [sample #17] Setting cartridge argument to -1
2023-05-24T04:41:57.810Z,1684903317.810 [ESPComponent](IMPORTANT): [sample #17] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T04:41:58.230Z,1684903318.230 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 100.417717 m in 9.252710 degC water with 0.026680 ug/l chlorophyll fluorescence.
2023-05-24T04:41:58.611Z,1684903318.611 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T04:41:59.019Z,1684903319.019 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T04:42:01.039Z,1684903321.039 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T04:42:32.955Z,1684903352.955 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_PRIMING (@8,aRNAl_bac)
2023-05-24T04:42:35.374Z,1684903355.374 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T04:42:35.375Z,1684903355.375 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T04:42:35.778Z,1684903355.778 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T04:45:18.995Z,1684903518.995 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T04:45:19.399Z,1684903519.399 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T05:00:21.553Z,1684904421.553 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T05:24:18.517Z,1684905858.517 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T05:33:52.867Z,1684906432.867 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T05:33:53.258Z,1684906433.258 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T05:33:55.284Z,1684906435.284 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T05:33:55.683Z,1684906435.683 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_PROCESSED (@8,aRNAl_bac)
2023-05-24T05:35:30.623Z,1684906530.623 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_STOPPING (@8,aRNAl_bac)
2023-05-24T05:35:38.302Z,1684906538.302 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T05:35:38.703Z,1684906538.703 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_STOPPED
2023-05-24T05:40:38.501Z,1684906838.501 [ESPComponent](IMPORTANT): [sample #17] ESP sampling sequence completed normally.
2023-05-24T05:40:40.963Z,1684906840.963 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 17.000000 count 
2023-05-24T05:40:41.351Z,1684906841.351 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T05:40:41.756Z,1684906841.756 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T05:40:42.167Z,1684906842.167 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T05:40:42.586Z,1684906842.586 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 99.580513 m 
2023-05-24T05:40:42.967Z,1684906842.967 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.254786 degC 
2023-05-24T05:40:43.379Z,1684906843.379 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.023200 ug/l 
2023-05-24T05:40:44.170Z,1684906844.170 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 17.000000 count completed.
2023-05-24T05:40:48.670Z,1684906848.670 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T05:40:50.393Z,1684906850.393 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T05:41:00.733Z,1684906860.733 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T05:41:15.857Z,1684906875.857 [DAT](FAULT): failed to enter command mode
2023-05-24T05:55:05.048Z,1684907705.048 [CBIT](IMPORTANT): Beginning ground fault scan
2023-05-24T05:55:15.896Z,1684907715.896 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): -0.002530
CHAN A1 (24V): 0.000127
CHAN A2 (12V): -0.000137
CHAN A3 (5V): -0.000435
CHAN B0 (3.3V): -0.000570
CHAN B1 (3.15aV): 0.000521
CHAN B2 (3.15bV): 0.000162
CHAN B3 (GND): 0.000078
OPEN: 0.007624
Full Scale: +/- 1 mA
2023-05-24T06:32:05.891Z,1684909925.891 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.032308 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 18.000000 count 
2023-05-24T06:32:56.380Z,1684909976.380 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.791290 m , settling for 5.000000 min 
2023-05-24T06:38:09.221Z,1684910289.221 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T06:38:09.370Z,1684910289.370 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T06:39:02.071Z,1684910342.071 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T06:39:02.886Z,1684910342.886 [ESPComponent](IMPORTANT): [sample #18] Setting cartridge argument to -1
2023-05-24T06:39:02.886Z,1684910342.886 [ESPComponent](IMPORTANT): [sample #18] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T06:39:03.307Z,1684910343.307 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 100.521790 m in 9.232386 degC water with 0.026680 ug/l chlorophyll fluorescence.
2023-05-24T06:39:03.687Z,1684910343.687 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T06:39:04.100Z,1684910344.100 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T06:39:06.110Z,1684910346.110 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T06:39:38.036Z,1684910378.036 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_PRIMING (@7,aRNAl_bac)
2023-05-24T06:39:40.047Z,1684910380.047 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T06:39:40.451Z,1684910380.451 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T06:39:40.855Z,1684910380.855 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T06:42:23.674Z,1684910543.674 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T06:42:24.079Z,1684910544.079 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T07:03:44.541Z,1684911824.541 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout
2023-05-24T07:29:25.446Z,1684913365.446 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T07:32:12.981Z,1684913532.981 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T07:32:13.387Z,1684913533.387 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T07:32:15.408Z,1684913535.408 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T07:32:15.811Z,1684913535.811 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_PROCESSED (@7,aRNAl_bac)
2023-05-24T07:33:50.759Z,1684913630.759 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_STOPPING (@7,aRNAl_bac)
2023-05-24T07:33:58.435Z,1684913638.435 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T07:33:58.852Z,1684913638.852 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_STOPPED
2023-05-24T07:38:58.618Z,1684913938.618 [ESPComponent](IMPORTANT): [sample #18] ESP sampling sequence completed normally.
2023-05-24T07:39:01.082Z,1684913941.082 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 18.000000 count 
2023-05-24T07:39:01.460Z,1684913941.460 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T07:39:01.865Z,1684913941.865 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T07:39:02.271Z,1684913942.271 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T07:39:02.684Z,1684913942.684 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 99.968842 m 
2023-05-24T07:39:03.102Z,1684913943.102 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.394190 degC 
2023-05-24T07:39:03.521Z,1684913943.521 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.034800 ug/l 
2023-05-24T07:39:04.299Z,1684913944.299 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 18.000000 count completed.
2023-05-24T07:39:08.754Z,1684913948.754 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T07:39:10.573Z,1684913950.573 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T07:39:20.910Z,1684913960.910 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T07:39:36.034Z,1684913976.034 [DAT](FAULT): failed to enter command mode
2023-05-24T08:32:08.688Z,1684917128.688 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.033496 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 19.000000 count 
2023-05-24T08:33:36.490Z,1684917216.490 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.858650 m , settling for 5.000000 min 
2023-05-24T08:38:49.257Z,1684917529.257 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T08:38:49.378Z,1684917529.378 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T08:39:42.146Z,1684917582.146 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T08:39:42.951Z,1684917582.951 [ESPComponent](IMPORTANT): [sample #19] Setting cartridge argument to -1
2023-05-24T08:39:42.951Z,1684917582.951 [ESPComponent](IMPORTANT): [sample #19] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T08:39:43.386Z,1684917583.386 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 100.020645 m in 9.575006 degC water with 0.048720 ug/l chlorophyll fluorescence.
2023-05-24T08:39:43.764Z,1684917583.764 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T08:39:44.158Z,1684917584.158 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T08:39:46.186Z,1684917586.186 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T08:40:18.120Z,1684917618.120 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_PRIMING (@6,aRNAl_bac)
2023-05-24T08:40:20.138Z,1684917620.138 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T08:40:20.562Z,1684917620.562 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T08:40:20.946Z,1684917620.946 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T08:43:03.792Z,1684917783.792 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T08:43:04.197Z,1684917784.197 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T08:48:33.060Z,1684918113.060 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout
2023-05-24T08:56:26.788Z,1684918586.788 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T09:02:17.521Z,1684918937.521 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T09:32:22.031Z,1684920742.031 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T09:32:22.435Z,1684920742.435 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T09:32:24.476Z,1684920744.476 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T09:32:24.862Z,1684920744.862 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_PROCESSED (@6,aRNAl_bac)
2023-05-24T09:33:59.800Z,1684920839.800 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_STOPPING (@6,aRNAl_bac)
2023-05-24T09:34:07.478Z,1684920847.478 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T09:34:07.880Z,1684920847.880 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_STOPPED
2023-05-24T09:39:07.646Z,1684921147.646 [ESPComponent](IMPORTANT): [sample #19] ESP sampling sequence completed normally.
2023-05-24T09:39:10.099Z,1684921150.099 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 19.000000 count 
2023-05-24T09:39:10.514Z,1684921150.514 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T09:39:10.904Z,1684921150.904 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T09:39:11.311Z,1684921151.311 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T09:39:11.734Z,1684921151.734 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 99.038704 m 
2023-05-24T09:39:12.131Z,1684921152.131 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.574488 degC 
2023-05-24T09:39:12.513Z,1684921152.513 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.046400 ug/l 
2023-05-24T09:39:13.331Z,1684921153.331 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 19.000000 count completed.
2023-05-24T09:39:17.779Z,1684921157.779 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T09:39:19.481Z,1684921159.481 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T09:39:29.813Z,1684921169.813 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T09:40:01.330Z,1684921201.330 [DAT](FAULT): failed to enter command mode
2023-05-24T10:32:11.396Z,1684924331.396 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.031494 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 20.000000 count 
2023-05-24T10:36:02.190Z,1684924562.190 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 100.135475 m , settling for 5.000000 min 
2023-05-24T10:41:14.949Z,1684924874.949 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T10:41:15.066Z,1684924875.066 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T10:42:08.236Z,1684924928.236 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T10:42:09.053Z,1684924929.053 [ESPComponent](IMPORTANT): [sample #20] Setting cartridge argument to -1
2023-05-24T10:42:09.054Z,1684924929.054 [ESPComponent](IMPORTANT): [sample #20] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T10:42:09.489Z,1684924929.489 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 101.902252 m in 9.600733 degC water with 0.052200 ug/l chlorophyll fluorescence.
2023-05-24T10:42:09.855Z,1684924929.855 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T10:42:10.254Z,1684924930.254 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T10:42:12.274Z,1684924932.274 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T10:42:44.337Z,1684924964.337 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_PRIMING (@5,aRNAl_bac)
2023-05-24T10:42:47.163Z,1684924967.163 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T10:42:47.164Z,1684924967.164 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T10:42:47.567Z,1684924967.567 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T10:45:30.396Z,1684925130.396 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T10:45:30.782Z,1684925130.782 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T10:50:33.905Z,1684925433.905 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T10:54:23.193Z,1684925663.193 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout
2023-05-24T11:10:26.282Z,1684926626.282 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout
2023-05-24T11:37:07.698Z,1684928227.698 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T11:37:08.087Z,1684928228.087 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T11:37:10.113Z,1684928230.113 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T11:37:10.506Z,1684928230.506 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_PROCESSED (@5,aRNAl_bac)
2023-05-24T11:38:45.520Z,1684928325.520 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_STOPPING (@5,aRNAl_bac)
2023-05-24T11:38:53.124Z,1684928333.124 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T11:38:53.543Z,1684928333.543 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_STOPPED
2023-05-24T11:42:56.541Z,1684928576.541 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T11:43:53.307Z,1684928633.307 [ESPComponent](IMPORTANT): [sample #20] ESP sampling sequence completed normally.
2023-05-24T11:43:55.776Z,1684928635.776 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 20.000000 count 
2023-05-24T11:43:56.148Z,1684928636.148 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T11:43:56.566Z,1684928636.566 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T11:43:56.970Z,1684928636.970 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T11:43:57.390Z,1684928637.390 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 100.401924 m 
2023-05-24T11:43:57.775Z,1684928637.775 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.431391 degC 
2023-05-24T11:43:58.170Z,1684928638.170 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.034800 ug/l 
2023-05-24T11:43:58.975Z,1684928638.975 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 20.000000 count completed.
2023-05-24T11:44:03.447Z,1684928643.447 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T11:44:05.053Z,1684928645.053 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T11:44:15.393Z,1684928655.393 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T11:44:30.515Z,1684928670.515 [DAT](FAULT): failed to enter command mode
2023-05-24T11:55:16.316Z,1684929316.316 [CBIT](IMPORTANT): Beginning ground fault scan
2023-05-24T11:55:27.259Z,1684929327.259 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): -0.000856
CHAN A1 (24V): 0.000257
CHAN A2 (12V): 0.000458
CHAN A3 (5V): -0.000540
CHAN B0 (3.3V): 0.000159
CHAN B1 (3.15aV): 0.000297
CHAN B2 (3.15bV): 0.000389
CHAN B3 (GND): 0.000117
OPEN: 0.007342
Full Scale: +/- 1 mA
2023-05-24T12:32:13.852Z,1684931533.852 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.027962 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 21.000000 count 
2023-05-24T12:39:46.375Z,1684931986.375 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 100.192795 m , settling for 5.000000 min 
2023-05-24T12:44:59.246Z,1684932299.246 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T12:44:59.404Z,1684932299.404 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T12:45:51.663Z,1684932351.663 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T12:45:52.489Z,1684932352.489 [ESPComponent](IMPORTANT): [sample #21] Setting cartridge argument to -1
2023-05-24T12:45:52.490Z,1684932352.490 [ESPComponent](IMPORTANT): [sample #21] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T12:45:52.955Z,1684932352.955 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 98.873955 m in 9.426874 degC water with 0.035960 ug/l chlorophyll fluorescence.
2023-05-24T12:45:53.278Z,1684932353.278 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T12:45:53.699Z,1684932353.699 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T12:45:55.696Z,1684932355.696 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T12:46:27.664Z,1684932387.664 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_PRIMING (@4,aRNAl_bac)
2023-05-24T12:46:29.678Z,1684932389.678 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T12:46:30.085Z,1684932390.085 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T12:46:30.500Z,1684932390.500 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T12:49:13.327Z,1684932553.327 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T12:49:13.727Z,1684932553.727 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T13:01:38.909Z,1684933298.909 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T13:32:53.925Z,1684935173.925 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout
2023-05-24T13:38:54.150Z,1684935534.150 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T13:38:54.556Z,1684935534.556 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T13:38:56.580Z,1684935536.580 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T13:38:56.983Z,1684935536.983 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_PROCESSED (@4,aRNAl_bac)
2023-05-24T13:39:41.170Z,1684935581.170 [ThrusterHE](FAULT): getSpeed error serial timeout
2023-05-24T13:39:41.171Z,1684935581.171 [ThrusterHE](FAULT): Thruster uart error: serial timeout
2023-05-24T13:40:31.935Z,1684935631.935 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_STOPPING (@4,aRNAl_bac)
2023-05-24T13:40:39.610Z,1684935639.610 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T13:40:40.015Z,1684935640.015 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_STOPPED
2023-05-24T13:45:39.786Z,1684935939.786 [ESPComponent](IMPORTANT): [sample #21] ESP sampling sequence completed normally.
2023-05-24T13:45:42.256Z,1684935942.256 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 21.000000 count 
2023-05-24T13:45:42.649Z,1684935942.649 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T13:45:43.034Z,1684935943.034 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T13:45:43.447Z,1684935943.447 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T13:45:43.855Z,1684935943.855 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 100.039780 m 
2023-05-24T13:45:44.273Z,1684935944.273 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.368006 degC 
2023-05-24T13:45:44.650Z,1684935944.650 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.034800 ug/l 
2023-05-24T13:45:45.464Z,1684935945.464 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 21.000000 count completed.
2023-05-24T13:45:49.927Z,1684935949.927 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T13:45:51.785Z,1684935951.785 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T13:46:02.126Z,1684935962.126 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T14:32:16.505Z,1684938736.505 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.030070 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 22.000000 count 
2023-05-24T14:38:11.240Z,1684939091.240 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 100.160248 m , settling for 5.000000 min 
2023-05-24T14:43:24.005Z,1684939404.005 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T14:43:24.225Z,1684939404.225 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T14:44:16.547Z,1684939456.547 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T14:44:17.362Z,1684939457.362 [ESPComponent](IMPORTANT): [sample #22] Setting cartridge argument to -1
2023-05-24T14:44:17.364Z,1684939457.364 [ESPComponent](IMPORTANT): [sample #22] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T14:44:17.779Z,1684939457.779 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 100.135620 m in 9.318110 degC water with 0.032480 ug/l chlorophyll fluorescence.
2023-05-24T14:44:18.163Z,1684939458.163 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T14:44:18.565Z,1684939458.565 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T14:44:20.581Z,1684939460.581 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T14:44:52.498Z,1684939492.498 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_PRIMING (@3,aRNAl_bac)
2023-05-24T14:44:54.923Z,1684939494.923 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T14:44:54.924Z,1684939494.924 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T14:44:55.328Z,1684939495.328 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T14:47:38.141Z,1684939658.141 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T14:47:38.551Z,1684939658.551 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T14:57:45.513Z,1684940265.513 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T14:59:43.965Z,1684940383.965 [ThrusterHE](FAULT): Could not get status:serial timeout
2023-05-24T15:35:25.959Z,1684942525.959 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T15:35:26.368Z,1684942526.368 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T15:35:28.384Z,1684942528.384 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T15:35:28.785Z,1684942528.785 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_PROCESSED (@3,aRNAl_bac)
2023-05-24T15:37:03.783Z,1684942623.783 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_STOPPING (@3,aRNAl_bac)
2023-05-24T15:37:11.459Z,1684942631.459 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T15:37:11.863Z,1684942631.863 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_STOPPED
2023-05-24T15:42:11.641Z,1684942931.641 [ESPComponent](IMPORTANT): [sample #22] ESP sampling sequence completed normally.
2023-05-24T15:42:12.445Z,1684942932.445 [ESPComponent](FAULT): LCB 2 fault:  Current Limiter Activated.
2023-05-24T15:42:14.086Z,1684942934.086 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 22.000000 count 
2023-05-24T15:42:14.490Z,1684942934.490 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T15:42:14.907Z,1684942934.907 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T15:42:15.290Z,1684942935.290 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T15:42:15.702Z,1684942935.702 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 100.070312 m 
2023-05-24T15:42:16.103Z,1684942936.103 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.296412 degC 
2023-05-24T15:42:16.525Z,1684942936.525 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.034800 ug/l 
2023-05-24T15:42:17.308Z,1684942937.308 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 22.000000 count completed.
2023-05-24T15:42:21.795Z,1684942941.795 [circle_sample:SetSampleOptions:D](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min .
2023-05-24T15:42:23.561Z,1684942943.561 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T15:42:33.893Z,1684942953.893 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T15:42:49.013Z,1684942969.013 [DAT](FAULT): failed to enter command mode
2023-05-24T16:32:19.147Z,1684945939.147 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.030615 min ) is longer than MaxWaitFirstSample (= 0.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 23.000000 count 
2023-05-24T16:32:57.954Z,1684945977.954 [circle_sample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 99.754059 m , settling for 5.000000 min 
2023-05-24T16:38:10.706Z,1684946290.706 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.11.2:10.89.11.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8
2023-05-24T16:38:10.853Z,1684946290.853 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000)
2023-05-24T16:39:03.534Z,1684946343.534 [ESPComponent](IMPORTANT): ESP has connected as client
2023-05-24T16:39:04.349Z,1684946344.349 [ESPComponent](IMPORTANT): [sample #23] Setting cartridge argument to -1
2023-05-24T16:39:04.350Z,1684946344.350 [ESPComponent](IMPORTANT): [sample #23] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT
2023-05-24T16:39:04.781Z,1684946344.781 [circle_sample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 99.728149 m in 9.235529 degC water with 0.027840 ug/l chlorophyll fluorescence.
2023-05-24T16:39:05.152Z,1684946345.152 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PREPARING_SHOW_LOG
2023-05-24T16:39:05.553Z,1684946345.553 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PREPARING_SHOW_STATUS
2023-05-24T16:39:07.574Z,1684946347.574 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_LOADING_CARTRIDGE
2023-05-24T16:39:39.489Z,1684946379.489 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_PRIMING (@2,aRNAl_bac)
2023-05-24T16:39:41.515Z,1684946381.515 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PRIMING (@100.0)
2023-05-24T16:39:41.914Z,1684946381.914 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=92.070000 issuedCmd_=Cmd.startFiltering
2023-05-24T16:39:42.330Z,1684946382.330 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0)
2023-05-24T16:41:04.665Z,1684946464.665 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout
2023-05-24T16:42:25.159Z,1684946545.159 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_FILTERING (@0.0)
2023-05-24T16:42:25.563Z,1684946545.563 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0)
2023-05-24T17:29:45.792Z,1684949385.792 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PAUSED (@1000.0)
2023-05-24T17:29:46.189Z,1684949386.189 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PROCESSING (@1000.0)
2023-05-24T17:29:48.212Z,1684949388.212 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing
2023-05-24T17:29:48.614Z,1684949388.614 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_PROCESSED (@2,aRNAl_bac)
2023-05-24T17:31:23.559Z,1684949483.559 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_STOPPING (@2,aRNAl_bac)
2023-05-24T17:31:31.230Z,1684949491.230 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs &
2023-05-24T17:31:31.639Z,1684949491.639 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_STOPPED
2023-05-24T17:36:31.539Z,1684949791.539 [ESPComponent](IMPORTANT): [sample #23] ESP sampling sequence completed normally.
2023-05-24T17:36:33.991Z,1684949793.991 [circle_sample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 23.000000 count 
2023-05-24T17:36:34.392Z,1684949794.392 [circle_sample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 
2023-05-24T17:36:34.791Z,1684949794.791 [circle_sample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 
2023-05-24T17:36:35.199Z,1684949795.199 [circle_sample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 
2023-05-24T17:36:35.623Z,1684949795.623 [circle_sample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 101.295731 m 
2023-05-24T17:36:35.000Z,1684949796.000 [circle_sample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.098413 degC 
2023-05-24T17:36:36.410Z,1684949796.410 [circle_sample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.023200 ug/l 
2023-05-24T17:36:37.209Z,1684949797.209 [circle_sample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 23.000000 count completed.
2023-05-24T17:36:39.020Z,1684949799.020 [MissionManager](IMPORTANT): Started mission Default
2023-05-24T17:36:43.394Z,1684949803.394 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T17:36:53.733Z,1684949813.733 [DAT](FAULT): Sending transmit timeout failure.
2023-05-24T17:45:07.728Z,1684950307.728 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session.
2023-05-24T17:55:27.552Z,1684950927.552 [CBIT](IMPORTANT): Beginning ground fault scan
2023-05-24T17:55:38.449Z,1684950938.449 [CBIT](IMPORTANT): No ground fault detected
mA:
CHAN A0 (Batt): -0.002154
CHAN A1 (24V): 0.000707
CHAN A2 (12V): 0.000157
CHAN A3 (5V): 0.000090
CHAN B0 (3.3V): 0.000511
CHAN B1 (3.15aV): -0.000164
CHAN B2 (3.15bV): -0.000020
CHAN B3 (GND): -0.000219
OPEN: 0.007434
Full Scale: +/- 1 mA
2023-05-24T18:58:12.859Z,1684954692.859 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=19420, MT Status=1, MTMSN=1344
2023-05-24T18:58:13.774Z,1684954693.774 [CommandExec](IMPORTANT): got command configSet DVL_micro.loadAtStartup 1 bool persist
2023-05-24T18:58:13.778Z,1684954693.778 [CommandExec](IMPORTANT): configSet DVL_micro.loadAtStartup requires a restart to take effect.
2023-05-24T18:58:13.786Z,1684954693.786 [CommandExec](IMPORTANT): got command restart application