2024-09-20T04:27:24.580Z,1726806444.580 [CommandExec](IMPORTANT): got command schedule resume 2024-09-20T04:27:24.580Z,1726806444.580 [CommandExec](IMPORTANT): Scheduling is resumed 2024-09-20T04:27:24.581Z,1726806444.581 [CommandExec](IMPORTANT): got command restart logs 2024-09-20T04:33:49.731Z,1726806829.731 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 13.636565 min 2024-09-20T04:34:16.027Z,1726806856.027 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17393, MT Status=1, MTMSN=4127 2024-09-20T04:34:16.826Z,1726806856.826 [CommandExec](IMPORTANT): got command schedule 20240920T050000 "load Science/circle_sample.xml;set CircleSample.MissionTimeout 4 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool;set CircleSample.AcousticTargetDepth 0 m" 4flrv 1 4.000000 2024-09-20T04:34:16.826Z,1726806856.826 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=4flrv 2024-09-20T04:34:16.827Z,1726806856.827 [CommandExec](IMPORTANT): Scheduled #1 (#1 of 4 with id='4flrv'): "load Science/circle_sample.xml;set CircleSample.MissionTimeout 4 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool;set CircleSample.AcousticTargetDepth 0 m", AT TIME: 2024-09-20T05:00:00Z 2024-09-20T04:35:30.864Z,1726806930.864 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17394, MT Status=1, MTMSN=4128 2024-09-20T04:35:31.699Z,1726806931.699 [CommandExec](IMPORTANT): got command schedule 20240920T050000 "set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m;set CircleSample.CircleYoYoMinDepth 10 m" 4flrv 2 4.000000 2024-09-20T04:35:31.699Z,1726806931.699 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=4flrv 2024-09-20T04:35:31.700Z,1726806931.700 [CommandExec](IMPORTANT): Scheduled #2 (#2 of 4 with id='4flrv'): "set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m;set CircleSample.CircleYoYoMinDepth 10 m", AT TIME: 2024-09-20T05:00:00Z 2024-09-20T04:35:38.958Z,1726806938.958 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17395, MT Status=1, MTMSN=4129 2024-09-20T04:35:39.617Z,1726806939.617 [CommandExec](IMPORTANT): got command schedule 20240920T050000 "set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run" 4flrv 4 4.000000 2024-09-20T04:35:39.617Z,1726806939.617 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=4flrv 2024-09-20T04:35:39.620Z,1726806939.620 [CommandExec](IMPORTANT): Scheduled #3 (#4 of 4 with id='4flrv'): "set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run", AT TIME: 2024-09-20T05:00:00Z 2024-09-20T04:35:55.527Z,1726806955.527 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17396, MT Status=1, MTMSN=4130 2024-09-20T04:35:56.290Z,1726806956.290 [CommandExec](IMPORTANT): got command schedule 20240920T050000 "set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree;set CircleSample.TrackingUpdatePeriod 65 s" 4flrv 3 4.000000 2024-09-20T04:35:56.291Z,1726806956.291 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=4flrv 2024-09-20T04:35:56.292Z,1726806956.292 [CommandExec](IMPORTANT): Scheduled #3 (#4 of 4 with id='4flrv'): "set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run", AT TIME: 2024-09-20T05:00:00Z 2024-09-20T04:42:14.693Z,1726807334.693 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 22.052547 min 2024-09-20T04:42:46.236Z,1726807366.236 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17403, MT Status=1, MTMSN=4131 2024-09-20T04:42:46.928Z,1726807366.928 [CommandExec](IMPORTANT): got command load ./Missions/Transport/keepstation.tl 2024-09-20T04:42:48.637Z,1726807368.637 [CommandExec](IMPORTANT): Loaded ./Missions/Transport/keepstation.tl id=keepstation 2024-09-20T04:42:53.621Z,1726807373.621 [CommandExec](IMPORTANT): got command set keepstation.NeedCommsTime 10 minute 2024-09-20T04:42:53.622Z,1726807373.622 [CommandExec](IMPORTANT): got command set keepstation.Latitude 36.712468 degree 2024-09-20T04:42:53.622Z,1726807373.622 [CommandExec](IMPORTANT): got command set keepstation.Longitude -122.1869 degree 2024-09-20T04:42:53.623Z,1726807373.623 [CommandExec](IMPORTANT): got command run 2024-09-20T04:42:53.624Z,1726807373.624 [CommandExec](IMPORTANT): Running 2024-09-20T04:42:53.813Z,1726807373.813 [MissionManager](IMPORTANT): Started mission keepstation 2024-09-20T04:44:09.628Z,1726807449.628 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Navigating to waypoint: 36.712467,-122.186897 2024-09-20T04:47:22.724Z,1726807642.724 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Reached waypoint: 36.712467,-122.186897 2024-09-20T05:00:00.105Z,1726808400.105 [CommandExec](IMPORTANT): got command load ./Missions/Science/circle_sample.xml 2024-09-20T05:00:10.902Z,1726808410.902 [CommandExec](IMPORTANT): Loaded ./Missions/Science/circle_sample.xml id=CircleSample 2024-09-20T05:00:11.244Z,1726808411.244 [MissionManager](IMPORTANT): Started mission Default 2024-09-20T05:00:24.868Z,1726808424.868 [CommandExec](IMPORTANT): got command set CircleSample.MissionTimeout 4 hour 2024-09-20T05:00:24.869Z,1726808424.869 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTrackingTimeout 0.5 hour 2024-09-20T05:00:24.870Z,1726808424.870 [CommandExec](IMPORTANT): got command set CircleSample.NeedCommsMissionStart 0 bool 2024-09-20T05:00:24.870Z,1726808424.870 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTargetDepth 0 meter 2024-09-20T05:00:39.540Z,1726808439.540 [CommandExec](IMPORTANT): got command set CircleSample.SendSampleResultToWG 1 bool 2024-09-20T05:00:39.542Z,1726808439.542 [CommandExec](IMPORTANT): got command set CircleSample.WaitBeforeFiring 0 minute 2024-09-20T05:00:39.543Z,1726808439.543 [CommandExec](IMPORTANT): got command set CircleSample.MaxWaitMustFire 120 minute 2024-09-20T05:00:39.544Z,1726808439.544 [CommandExec](IMPORTANT): got command set CircleSample.DepthCircle 150 meter 2024-09-20T05:00:39.544Z,1726808439.544 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMinDepth 10 meter 2024-09-20T05:00:53.845Z,1726808453.845 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMaxDepth 280 meter 2024-09-20T05:00:53.846Z,1726808453.846 [CommandExec](IMPORTANT): got command set CircleSample.MaxDepth 290 meter 2024-09-20T05:00:53.846Z,1726808453.846 [CommandExec](IMPORTANT): got command set CircleSample.CenterLatitude 36.712468 degree 2024-09-20T05:00:53.847Z,1726808453.847 [CommandExec](IMPORTANT): got command set CircleSample.CenterLongitude -122.1869 degree 2024-09-20T05:00:53.848Z,1726808453.848 [CommandExec](IMPORTANT): got command set CircleSample.TrackingUpdatePeriod 65 second 2024-09-20T05:01:01.486Z,1726808461.486 [CommandExec](IMPORTANT): got command set CircleSample.NumberOfStartingFixesToIgnore 1 count 2024-09-20T05:01:01.487Z,1726808461.487 [CommandExec](IMPORTANT): got command set CircleSample:NeedComms.CommsTimeout 15 minute 2024-09-20T05:01:01.487Z,1726808461.487 [CommandExec](IMPORTANT): got command run 2024-09-20T05:01:01.493Z,1726808461.493 [CommandExec](IMPORTANT): Running 2024-09-20T05:01:01.773Z,1726808461.773 [MissionManager](IMPORTANT): Started mission CircleSample 2024-09-20T05:01:01.844Z,1726808461.844 [CircleSample:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 0.50 hours. 2024-09-20T05:01:03.755Z,1726808463.755 [CircleSample:MissionStartNoComms:A](IMPORTANT): Mission start no comms. 2024-09-20T05:01:04.000Z,1726808465.000 [CircleSample:CircleAndSample:B](IMPORTANT): Circle around location 36.712467 arcdeg , -122.186897 arcdeg , at radius = 20.000000 m and depth = 150.000000 m . 2024-09-20T05:01:07.025Z,1726808467.025 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min . 2024-09-20T05:11:06.552Z,1726809066.552 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 10.033087 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 1.000000 count 2024-09-20T05:17:07.362Z,1726809427.362 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.070419 m , settling for 5.000000 min 2024-09-20T05:22:09.943Z,1726809729.943 [ESPComponent](IMPORTANT): Setting regex for ESP Log summary: 'Selecting Cartridge|Sampled|Error|Underpressure|Overpressure|Fail|Retry|Cmd::|Sampler::|summary:' 2024-09-20T05:22:20.082Z,1726809740.082 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T05:22:20.212Z,1726809740.212 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T05:23:11.386Z,1726809791.386 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T05:23:12.311Z,1726809792.311 [ESPComponent](IMPORTANT): [sample #1] Setting cartridge argument to -1 2024-09-20T05:23:12.312Z,1726809792.312 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T05:23:12.655Z,1726809792.655 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 149.520172 m in 9.270014 degC water with 0.070720 ug/l chlorophyll fluorescence. 2024-09-20T05:23:13.407Z,1726809793.407 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T05:23:13.811Z,1726809793.811 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T05:23:15.427Z,1726809795.427 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T05:23:46.939Z,1726809826.939 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PRIMING (@58,aRNAl_bac) 2024-09-20T05:23:49.767Z,1726809829.767 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T05:23:49.768Z,1726809829.768 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T05:23:50.171Z,1726809830.171 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T05:26:45.919Z,1726810005.919 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T05:26:46.328Z,1726810006.328 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T06:14:51.999Z,1726812891.999 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T06:14:52.403Z,1726812892.403 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T06:14:54.423Z,1726812894.423 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T06:14:54.826Z,1726812894.826 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_WAITING_FOR_PROCESSED (@58,aRNAl_bac) 2024-09-20T06:16:29.771Z,1726812989.771 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPING (@58,aRNAl_bac) 2024-09-20T06:16:37.447Z,1726812997.447 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T06:16:37.852Z,1726812997.852 [ESPComponent](IMPORTANT): [sample #1] ESP sampling state: S_STOPPED 2024-09-20T06:21:37.627Z,1726813297.627 [ESPComponent](IMPORTANT): [sample #1] ESP sampling sequence completed normally. 2024-09-20T06:21:40.202Z,1726813300.202 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 1.000000 count 2024-09-20T06:21:40.602Z,1726813300.602 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T06:21:41.016Z,1726813301.016 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T06:21:41.392Z,1726813301.392 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T06:21:41.809Z,1726813301.809 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.764160 m 2024-09-20T06:21:42.217Z,1726813302.217 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.264490 degC 2024-09-20T06:21:42.604Z,1726813302.604 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.083200 ug/l 2024-09-20T06:21:43.424Z,1726813303.424 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 1.000000 count completed. 2024-09-20T06:21:47.868Z,1726813307.868 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-20T06:21:49.137Z,1726813309.137 [ElevatorServo](FAULT): Overload Error 2024-09-20T06:22:33.626Z,1726813353.626 [DATMMP](FAULT): Buffer send receipt timeout failure. 2024-09-20T07:08:25.685Z,1726816105.685 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-20T07:11:09.252Z,1726816269.252 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.030998 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 2.000000 count 2024-09-20T07:16:49.012Z,1726816609.012 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.273239 m , settling for 1.000000 min 2024-09-20T07:18:01.805Z,1726816681.805 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T07:18:01.911Z,1726816681.911 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T07:18:49.431Z,1726816729.431 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T07:18:50.273Z,1726816730.273 [ESPComponent](IMPORTANT): [sample #2] Setting cartridge argument to -1 2024-09-20T07:18:50.273Z,1726816730.273 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T07:18:50.708Z,1726816730.708 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.618973 m in 9.363367 degC water with 0.071760 ug/l chlorophyll fluorescence. 2024-09-20T07:18:51.046Z,1726816731.046 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T07:18:51.454Z,1726816731.454 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T07:18:53.471Z,1726816733.471 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T07:19:24.984Z,1726816764.984 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PRIMING (@57,aRNAl_bac) 2024-09-20T07:19:27.003Z,1726816767.003 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T07:19:27.412Z,1726816767.412 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T07:19:27.811Z,1726816767.811 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T07:22:24.263Z,1726816944.263 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T07:22:24.670Z,1726816944.670 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T07:35:06.041Z,1726817706.041 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2024-09-20T07:46:45.211Z,1726818405.211 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-20T07:46:45.211Z,1726818405.211 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-20T08:10:09.955Z,1726819809.955 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T08:10:10.358Z,1726819810.358 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T08:10:12.379Z,1726819812.379 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T08:10:12.785Z,1726819812.785 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_WAITING_FOR_PROCESSED (@57,aRNAl_bac) 2024-09-20T08:11:47.727Z,1726819907.727 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPING (@57,aRNAl_bac) 2024-09-20T08:11:55.407Z,1726819915.407 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T08:11:55.807Z,1726819915.807 [ESPComponent](IMPORTANT): [sample #2] ESP sampling state: S_STOPPED 2024-09-20T08:16:55.574Z,1726820215.574 [ESPComponent](IMPORTANT): [sample #2] ESP sampling sequence completed normally. 2024-09-20T08:16:58.131Z,1726820218.131 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 2.000000 count 2024-09-20T08:16:58.554Z,1726820218.554 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T08:16:58.933Z,1726820218.933 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T08:16:59.337Z,1726820219.337 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T08:16:59.769Z,1726820219.769 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.771957 m 2024-09-20T08:17:00.152Z,1726820220.152 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.352411 degC 2024-09-20T08:17:00.570Z,1726820220.570 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.072800 ug/l 2024-09-20T08:17:01.375Z,1726820221.375 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 2.000000 count completed. 2024-09-20T08:17:03.092Z,1726820223.092 [MissionManager](IMPORTANT): Started mission Default 2024-09-20T08:22:08.895Z,1726820528.895 [NAL9602](FAULT): GPS failed to acquire within timeout. 2024-09-20T08:22:09.701Z,1726820529.701 [NAL9602](FAULT): LCB fault: LCB Watchdog Reset. Current Limiter Activated. 2024-09-20T08:22:10.177Z,1726820530.177 [ElevatorServo](FAULT): Overload Error 2024-09-20T08:23:09.088Z,1726820589.088 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17415, MT Status=1, MTMSN=4132 2024-09-20T08:23:09.782Z,1726820589.782 [CommandExec](IMPORTANT): got command schedule "load Transport/keepstation.tl;set keepstation.NeedCommsTime 10 min;set keepstation.Latitude 36.712468 degree;set keepstation.Longitude -122.1869 degree;run" 2024-09-20T08:23:09.783Z,1726820589.783 [CommandExec](IMPORTANT): Scheduled #5: "load Transport/keepstation.tl;set keepstation.NeedCommsTime 10 min;set keepstation.Latitude 36.712468 degree;set keepstation.Longitude -122.1869 degree;run", AFTER MISSION 2024-09-20T08:23:09.912Z,1726820589.912 [CommandExec](IMPORTANT): got command load ./Missions/Transport/keepstation.tl 2024-09-20T08:23:11.567Z,1726820591.567 [CommandExec](IMPORTANT): Loaded ./Missions/Transport/keepstation.tl id=keepstation 2024-09-20T08:23:18.071Z,1726820598.071 [CommandExec](IMPORTANT): got command set keepstation.NeedCommsTime 10 minute 2024-09-20T08:23:18.072Z,1726820598.072 [CommandExec](IMPORTANT): got command set keepstation.Latitude 36.712468 degree 2024-09-20T08:23:18.073Z,1726820598.073 [CommandExec](IMPORTANT): got command set keepstation.Longitude -122.1869 degree 2024-09-20T08:23:18.073Z,1726820598.073 [CommandExec](IMPORTANT): got command run 2024-09-20T08:23:18.078Z,1726820598.078 [CommandExec](IMPORTANT): Running 2024-09-20T08:23:18.411Z,1726820598.411 [MissionManager](IMPORTANT): Started mission keepstation 2024-09-20T08:38:28.059Z,1726821508.059 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Navigating to waypoint: 36.712467,-122.186897 2024-09-20T08:57:32.916Z,1726822652.916 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Reached waypoint: 36.712467,-122.186897 2024-09-20T09:02:42.550Z,1726822962.550 [CBIT](IMPORTANT): Beginning ground fault scan 2024-09-20T09:02:53.446Z,1726822973.446 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.000412 CHAN A1 (24V): -0.021599 CHAN A2 (12V): -0.000884 CHAN A3 (5V): -0.001386 CHAN B0 (3.3V): -0.000431 CHAN B1 (3.15aV): -0.000323 CHAN B2 (3.15bV): -0.000034 CHAN B3 (GND): -0.000312 OPEN: 0.007277 Full Scale: +/- 1 mA 2024-09-20T11:07:59.741Z,1726830479.741 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17483, MT Status=1, MTMSN=4133 2024-09-20T11:08:00.234Z,1726830480.234 [CommandExec](IMPORTANT): got command stop 2024-09-20T11:08:00.234Z,1726830480.234 [CommandExec](IMPORTANT): Scheduling is paused 2024-09-20T11:08:00.567Z,1726830480.567 [MissionManager](IMPORTANT): Started mission Default 2024-09-20T11:11:12.082Z,1726830672.082 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17489, MT Status=1, MTMSN=4134 2024-09-20T11:11:12.663Z,1726830672.663 [CommandExec](IMPORTANT): got command schedule resume 2024-09-20T11:11:12.663Z,1726830672.663 [CommandExec](IMPORTANT): Scheduling is resumed 2024-09-20T11:11:30.671Z,1726830690.671 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17490, MT Status=1, MTMSN=4135 2024-09-20T11:11:32.957Z,1726830692.957 [CommandExec](IMPORTANT): got command configSet NAL9602.fastGPSFix 0 bool persist 2024-09-20T11:11:32.963Z,1726830692.963 [CommandExec](IMPORTANT): got command configSet Express none DATMMP.acoustic_contact_range persist 2024-09-20T11:17:16.603Z,1726831036.603 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 9.260454 min 2024-09-20T11:17:48.502Z,1726831068.502 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17496, MT Status=1, MTMSN=4136 2024-09-20T11:17:49.254Z,1726831069.254 [CommandExec](IMPORTANT): got command schedule 20240920T120000 "load Science/circle_sample.xml;set CircleSample.NumSamplers 55 count;set CircleSample.MissionTimeout 120 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool" 4g4me 1 4.000000 2024-09-20T11:17:49.255Z,1726831069.255 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=4g4me 2024-09-20T11:17:49.256Z,1726831069.256 [CommandExec](IMPORTANT): Scheduled #6 (#1 of 4 with id='4g4me'): "load Science/circle_sample.xml;set CircleSample.NumSamplers 55 count;set CircleSample.MissionTimeout 120 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool", AT TIME: 2024-09-20T12:00:00Z 2024-09-20T11:18:06.353Z,1726831086.353 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17497, MT Status=1, MTMSN=4137 2024-09-20T11:18:07.055Z,1726831087.055 [CommandExec](IMPORTANT): got command schedule 20240920T120000 "set CircleSample.AcousticTargetDepth 0 m;set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m" 4g4me 2 4.000000 2024-09-20T11:18:07.055Z,1726831087.055 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=4g4me 2024-09-20T11:18:07.057Z,1726831087.057 [CommandExec](IMPORTANT): Scheduled #7 (#2 of 4 with id='4g4me'): "set CircleSample.AcousticTargetDepth 0 m;set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m", AT TIME: 2024-09-20T12:00:00Z 2024-09-20T11:18:20.574Z,1726831100.574 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17498, MT Status=1, MTMSN=4138 2024-09-20T11:18:21.319Z,1726831101.319 [CommandExec](IMPORTANT): got command schedule 20240920T120000 "set CircleSample.CircleYoYoMinDepth 10 m;set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree" 4g4me 3 4.000000 2024-09-20T11:18:21.321Z,1726831101.321 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=4g4me 2024-09-20T11:18:21.322Z,1726831101.322 [CommandExec](IMPORTANT): Scheduled #8 (#3 of 4 with id='4g4me'): "set CircleSample.CircleYoYoMinDepth 10 m;set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree", AT TIME: 2024-09-20T12:00:00Z 2024-09-20T11:18:34.362Z,1726831114.362 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17499, MT Status=1, MTMSN=4139 2024-09-20T11:18:35.162Z,1726831115.162 [CommandExec](IMPORTANT): got command schedule 20240920T120000 "set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run" 4g4me 4 4.000000 2024-09-20T11:18:35.163Z,1726831115.163 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=4g4me 2024-09-20T11:18:35.164Z,1726831115.164 [CommandExec](IMPORTANT): Scheduled #9 (#4 of 4 with id='4g4me'): "set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run", AT TIME: 2024-09-20T12:00:00Z 2024-09-20T11:24:32.856Z,1726831472.856 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 16.531446 min 2024-09-20T11:31:13.723Z,1726831873.723 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 23.212596 min 2024-09-20T11:38:14.848Z,1726832294.848 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 30.231067 min 2024-09-20T11:45:02.204Z,1726832702.204 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 37.020504 min 2024-09-20T11:51:30.165Z,1726833090.165 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 43.486727 min 2024-09-20T11:58:00.925Z,1726833480.925 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 49.999255 min 2024-09-20T12:00:00.164Z,1726833600.164 [CommandExec](IMPORTANT): got command load ./Missions/Science/circle_sample.xml 2024-09-20T12:00:07.043Z,1726833607.043 [CommandExec](IMPORTANT): Loaded ./Missions/Science/circle_sample.xml id=CircleSample 2024-09-20T12:00:18.715Z,1726833618.715 [CommandExec](IMPORTANT): got command set CircleSample.NumSamplers 55 count 2024-09-20T12:00:18.716Z,1726833618.716 [CommandExec](IMPORTANT): got command set CircleSample.MissionTimeout 120 hour 2024-09-20T12:00:18.716Z,1726833618.716 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTrackingTimeout 0.5 hour 2024-09-20T12:00:18.717Z,1726833618.717 [CommandExec](IMPORTANT): got command set CircleSample.NeedCommsMissionStart 0 bool 2024-09-20T12:00:33.077Z,1726833633.077 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTargetDepth 0 meter 2024-09-20T12:00:33.078Z,1726833633.078 [CommandExec](IMPORTANT): got command set CircleSample.SendSampleResultToWG 1 bool 2024-09-20T12:00:33.078Z,1726833633.078 [CommandExec](IMPORTANT): got command set CircleSample.WaitBeforeFiring 0 minute 2024-09-20T12:00:33.079Z,1726833633.079 [CommandExec](IMPORTANT): got command set CircleSample.MaxWaitMustFire 120 minute 2024-09-20T12:00:33.080Z,1726833633.080 [CommandExec](IMPORTANT): got command set CircleSample.DepthCircle 150 meter 2024-09-20T12:00:46.416Z,1726833646.416 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMinDepth 10 meter 2024-09-20T12:00:46.416Z,1726833646.416 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMaxDepth 280 meter 2024-09-20T12:00:46.418Z,1726833646.418 [CommandExec](IMPORTANT): got command set CircleSample.MaxDepth 290 meter 2024-09-20T12:00:46.418Z,1726833646.418 [CommandExec](IMPORTANT): got command set CircleSample.CenterLatitude 36.712468 degree 2024-09-20T12:00:46.419Z,1726833646.419 [CommandExec](IMPORTANT): got command set CircleSample.CenterLongitude -122.1869 degree 2024-09-20T12:01:00.019Z,1726833660.019 [CommandExec](IMPORTANT): got command set CircleSample.IntervalRestartLogs 12 hour 2024-09-20T12:01:00.020Z,1726833660.020 [CommandExec](IMPORTANT): got command set CircleSample.TrackingUpdatePeriod 65 second 2024-09-20T12:01:00.020Z,1726833660.020 [CommandExec](IMPORTANT): got command set CircleSample.NumberOfStartingFixesToIgnore 1 count 2024-09-20T12:01:00.022Z,1726833660.022 [CommandExec](IMPORTANT): got command set CircleSample:NeedComms.CommsTimeout 15 minute 2024-09-20T12:01:00.022Z,1726833660.022 [CommandExec](IMPORTANT): got command run 2024-09-20T12:01:00.024Z,1726833660.024 [CommandExec](IMPORTANT): Running 2024-09-20T12:01:00.033Z,1726833660.033 [MissionManager](IMPORTANT): Started mission CircleSample 2024-09-20T12:01:00.130Z,1726833660.130 [CircleSample:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 0.50 hours. 2024-09-20T12:01:02.027Z,1726833662.027 [CircleSample:MissionStartNoComms:A](IMPORTANT): Mission start no comms. 2024-09-20T12:01:03.287Z,1726833663.287 [CircleSample:CircleAndSample:B](IMPORTANT): Circle around location 36.712467 arcdeg , -122.186897 arcdeg , at radius = 20.000000 m and depth = 150.000000 m . 2024-09-20T12:01:05.314Z,1726833665.314 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min . 2024-09-20T12:11:04.511Z,1726834264.511 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 10.027973 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 1.000000 count 2024-09-20T12:11:27.190Z,1726834287.190 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.807144 m , settling for 5.000000 min 2024-09-20T12:16:39.949Z,1726834599.949 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T12:16:40.110Z,1726834600.110 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T12:17:31.658Z,1726834651.658 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T12:17:32.470Z,1726834652.470 [ESPComponent](IMPORTANT): [sample #3] Setting cartridge argument to -1 2024-09-20T12:17:32.470Z,1726834652.470 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T12:17:32.919Z,1726834652.919 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 149.650208 m in 9.299982 degC water with 0.073840 ug/l chlorophyll fluorescence. 2024-09-20T12:17:33.275Z,1726834653.275 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T12:17:33.679Z,1726834653.679 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T12:17:35.705Z,1726834655.705 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T12:18:07.303Z,1726834687.303 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PRIMING (@56,aRNAl_bac) 2024-09-20T12:18:10.152Z,1726834690.152 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T12:18:10.155Z,1726834690.155 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T12:18:10.536Z,1726834690.536 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T12:21:03.871Z,1726834863.871 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T12:21:04.266Z,1726834864.266 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T13:07:39.948Z,1726837659.948 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T13:07:40.351Z,1726837660.351 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T13:07:42.372Z,1726837662.372 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T13:07:42.775Z,1726837662.775 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_WAITING_FOR_PROCESSED (@56,aRNAl_bac) 2024-09-20T13:09:17.733Z,1726837757.733 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPING (@56,aRNAl_bac) 2024-09-20T13:09:25.397Z,1726837765.397 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T13:09:25.803Z,1726837765.803 [ESPComponent](IMPORTANT): [sample #3] ESP sampling state: S_STOPPED 2024-09-20T13:14:25.590Z,1726838065.590 [ESPComponent](IMPORTANT): [sample #3] ESP sampling sequence completed normally. 2024-09-20T13:14:28.085Z,1726838068.085 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 1.000000 count 2024-09-20T13:14:28.518Z,1726838068.518 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T13:14:28.938Z,1726838068.938 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T13:14:29.306Z,1726838069.306 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T13:14:29.706Z,1726838069.706 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.425186 m 2024-09-20T13:14:30.111Z,1726838070.111 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.258814 degC 2024-09-20T13:14:30.504Z,1726838070.504 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.041600 ug/l 2024-09-20T13:14:31.286Z,1726838071.286 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 1.000000 count completed. 2024-09-20T13:14:35.828Z,1726838075.828 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-20T14:11:07.030Z,1726841467.030 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.028141 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 2.000000 count 2024-09-20T14:14:45.191Z,1726841685.191 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.296082 m , settling for 1.000000 min 2024-09-20T14:15:57.962Z,1726841757.962 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T14:15:58.093Z,1726841758.093 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T14:16:49.975Z,1726841809.975 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T14:16:50.781Z,1726841810.781 [ESPComponent](IMPORTANT): [sample #4] Setting cartridge argument to -1 2024-09-20T14:16:50.782Z,1726841810.782 [ESPComponent](IMPORTANT): [sample #4] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T14:16:51.246Z,1726841811.246 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.363358 m in 9.221033 degC water with 0.078000 ug/l chlorophyll fluorescence. 2024-09-20T14:16:51.587Z,1726841811.587 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T14:16:51.995Z,1726841811.995 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T14:16:54.013Z,1726841814.013 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T14:17:25.535Z,1726841845.535 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_PRIMING (@55,aRNAl_bac) 2024-09-20T14:17:28.362Z,1726841848.362 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T14:17:28.364Z,1726841848.364 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T14:17:28.767Z,1726841848.767 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T14:21:28.351Z,1726842088.351 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPING (@0.0) 2024-09-20T14:21:36.028Z,1726842096.028 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T14:21:36.431Z,1726842096.431 [ESPComponent](IMPORTANT): [sample #4] ESP sampling state: S_STOPPED 2024-09-20T14:26:36.191Z,1726842396.191 [ESPComponent](CRITICAL): [sample #4] ESP sampling sequence terminated with error: In state S_WAITING_FOR_FILTERING got exception while awaiting completion of cmd='Cmd.startFiltering' exception=<>. Elapsed time since command issued: 242.8124 2024-09-20T14:26:36.426Z,1726842396.426 [CommandExec](FAULT): Scheduling is paused 2024-09-20T14:26:37.454Z,1726842397.454 [MissionManager](IMPORTANT): Started mission Default 2024-09-20T14:31:46.581Z,1726842706.581 [ElevatorServo](FAULT): Overload Error 2024-09-20T14:31:51.041Z,1726842711.041 [BuoyancyServo](FAULT): Buoyancy getPosition uart error.serial timeout 2024-09-20T14:48:06.961Z,1726843686.961 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 21.485042 min 2024-09-20T14:57:02.057Z,1726844222.057 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 30.403330 min 2024-09-20T14:57:45.332Z,1726844265.332 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17556, MT Status=1, MTMSN=4142 2024-09-20T14:57:46.073Z,1726844266.073 [CommandExec](IMPORTANT): got command load ./Missions/Transport/keepstation.tl 2024-09-20T14:57:47.754Z,1726844267.754 [CommandExec](IMPORTANT): Loaded ./Missions/Transport/keepstation.tl id=keepstation 2024-09-20T14:57:54.198Z,1726844274.198 [CommandExec](IMPORTANT): got command set keepstation.NeedCommsTime 10 minute 2024-09-20T14:57:54.199Z,1726844274.199 [CommandExec](IMPORTANT): got command set keepstation.Latitude 36.712468 degree 2024-09-20T14:57:54.200Z,1726844274.200 [CommandExec](IMPORTANT): got command set keepstation.Longitude -122.1869 degree 2024-09-20T14:57:54.200Z,1726844274.200 [CommandExec](IMPORTANT): got command run 2024-09-20T14:57:54.202Z,1726844274.202 [CommandExec](IMPORTANT): Running 2024-09-20T14:57:54.308Z,1726844274.308 [MissionManager](IMPORTANT): Started mission keepstation 2024-09-20T14:58:48.935Z,1726844328.935 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17557, MT Status=1, MTMSN=4143 2024-09-20T14:58:49.511Z,1726844329.511 [CommandExec](IMPORTANT): got command schedule resume 2024-09-20T14:58:49.511Z,1726844329.511 [CommandExec](IMPORTANT): Scheduling is resumed 2024-09-20T15:02:09.399Z,1726844529.399 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Navigating to waypoint: 36.712467,-122.186897 2024-09-20T15:02:53.593Z,1726844573.593 [CBIT](IMPORTANT): Beginning ground fault scan 2024-09-20T15:03:04.549Z,1726844584.549 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): 0.008228 CHAN A1 (24V): -0.007499 CHAN A2 (12V): 0.004718 CHAN A3 (5V): -0.000762 CHAN B0 (3.3V): -0.000055 CHAN B1 (3.15aV): -0.000858 CHAN B2 (3.15bV): -0.000560 CHAN B3 (GND): -0.001655 OPEN: 0.007269 Full Scale: +/- 1 mA 2024-09-20T15:13:35.386Z,1726845215.386 [keepstation:TransitToStation:Wpt1.Waypoint](IMPORTANT): Reached waypoint: 36.712467,-122.186897 2024-09-20T15:15:42.292Z,1726845342.292 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17565, MT Status=1, MTMSN=4144 2024-09-20T15:15:42.915Z,1726845342.915 [CommandExec](IMPORTANT): got command ! onESPclient preston -- ‘spare:5’ 2024-09-20T15:15:43.361Z,1726845343.361 [CommandExec](IMPORTANT): /mnt/mmc/started/2024-09-20UTC151543_onESPclient 2024-09-20T15:30:56.668Z,1726846256.668 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17572, MT Status=1, MTMSN=4145 2024-09-20T15:30:57.182Z,1726846257.182 [CommandExec](IMPORTANT): got command stop 2024-09-20T15:30:57.182Z,1726846257.182 [CommandExec](IMPORTANT): Scheduling is paused 2024-09-20T15:30:57.488Z,1726846257.488 [MissionManager](IMPORTANT): Started mission Default 2024-09-20T15:31:39.958Z,1726846299.958 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17573, MT Status=1, MTMSN=4146 2024-09-20T15:31:40.519Z,1726846300.519 [CommandExec](IMPORTANT): got command ! onESPclient preston -- 'spare:5' 2024-09-20T15:31:40.892Z,1726846300.892 [CommandExec](IMPORTANT): /mnt/mmc/started/2024-09-20UTC153140_onESPclient 2024-09-20T15:32:31.701Z,1726846351.701 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17578, MT Status=1, MTMSN=4147 2024-09-20T15:32:32.267Z,1726846352.267 [CommandExec](IMPORTANT): got command schedule resume 2024-09-20T15:32:32.267Z,1726846352.267 [CommandExec](IMPORTANT): Scheduling is resumed 2024-09-20T15:38:19.527Z,1726846699.527 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 7.360540 min 2024-09-20T15:38:44.566Z,1726846724.566 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17586, MT Status=1, MTMSN=4151 2024-09-20T15:38:45.338Z,1726846725.338 [CommandExec](IMPORTANT): got command schedule 20240920T160000 "set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run" 4gglz 4 4.000000 2024-09-20T15:38:45.339Z,1726846725.339 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=4gglz 2024-09-20T15:38:45.340Z,1726846725.340 [CommandExec](IMPORTANT): Scheduled #10 (#4 of 4 with id='4gglz'): "set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run", AT TIME: 2024-09-20T16:00:00Z 2024-09-20T15:45:45.584Z,1726847145.584 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 14.794676 min 2024-09-20T15:53:03.750Z,1726847583.750 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 22.097636 min 2024-09-20T16:00:24.708Z,1726848024.708 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 29.446824 min 2024-09-20T16:09:22.547Z,1726848562.547 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 38.410722 min 2024-09-20T16:18:27.643Z,1726849107.643 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 47.495772 min 2024-09-20T16:25:23.927Z,1726849523.927 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 54.433883 min 2024-09-20T16:31:33.711Z,1726849893.711 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 60.596989 min 2024-09-20T16:34:25.972Z,1726850065.972 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17619, MT Status=1, MTMSN=4152 2024-09-20T16:34:26.683Z,1726850066.683 [CommandExec](IMPORTANT): got command schedule asap "load Science/circle_sample.xml;set CircleSample.NumSamplers 1 count;set CircleSample.MissionTimeout 120 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool" 4gjc7 1 4.000000 2024-09-20T16:34:26.684Z,1726850066.684 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=4gjc7 2024-09-20T16:34:26.689Z,1726850066.689 [CommandExec](IMPORTANT): Scheduled #11 (#1 of 4 with id='4gjc7'): "load Science/circle_sample.xml;set CircleSample.NumSamplers 1 count;set CircleSample.MissionTimeout 120 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool", ASAP 2024-09-20T16:34:55.862Z,1726850095.862 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17620, MT Status=1, MTMSN=4153 2024-09-20T16:34:56.674Z,1726850096.674 [CommandExec](IMPORTANT): got command schedule asap "set CircleSample.AcousticTargetDepth 0 m;set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m" 4gjc7 2 4.000000 2024-09-20T16:34:56.674Z,1726850096.674 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=4gjc7 2024-09-20T16:34:56.675Z,1726850096.675 [CommandExec](IMPORTANT): Scheduled #12 (#2 of 4 with id='4gjc7'): "set CircleSample.AcousticTargetDepth 0 m;set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m", ASAP 2024-09-20T16:35:27.567Z,1726850127.567 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17622, MT Status=1, MTMSN=4154 2024-09-20T16:35:28.330Z,1726850128.330 [CommandExec](IMPORTANT): got command schedule asap "set CircleSample.CircleYoYoMinDepth 10 m;set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree" 4gjc7 3 4.000000 2024-09-20T16:35:28.330Z,1726850128.330 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=4gjc7 2024-09-20T16:35:28.331Z,1726850128.331 [CommandExec](IMPORTANT): Scheduled #13 (#3 of 4 with id='4gjc7'): "set CircleSample.CircleYoYoMinDepth 10 m;set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree", ASAP 2024-09-20T16:36:10.887Z,1726850170.887 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=17625, MT Status=1, MTMSN=4155 2024-09-20T16:36:11.638Z,1726850171.638 [CommandExec](IMPORTANT): got command schedule asap "set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run" 4gjc7 4 4.000000 2024-09-20T16:36:11.638Z,1726850171.638 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=4gjc7 2024-09-20T16:36:11.639Z,1726850171.639 [CommandExec](IMPORTANT): Scheduled #14 (#4 of 4 with id='4gjc7'): "set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run", ASAP 2024-09-20T16:36:11.800Z,1726850171.800 [CommandExec](IMPORTANT): got command load ./Missions/Science/circle_sample.xml 2024-09-20T16:36:19.079Z,1726850179.079 [CommandExec](IMPORTANT): Loaded ./Missions/Science/circle_sample.xml id=CircleSample 2024-09-20T16:36:31.059Z,1726850191.059 [CommandExec](IMPORTANT): got command set CircleSample.NumSamplers 1 count 2024-09-20T16:36:31.059Z,1726850191.059 [CommandExec](IMPORTANT): got command set CircleSample.MissionTimeout 120 hour 2024-09-20T16:36:31.060Z,1726850191.060 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTrackingTimeout 0.5 hour 2024-09-20T16:36:31.061Z,1726850191.061 [CommandExec](IMPORTANT): got command set CircleSample.NeedCommsMissionStart 0 bool 2024-09-20T16:36:46.145Z,1726850206.145 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTargetDepth 0 meter 2024-09-20T16:36:46.146Z,1726850206.146 [CommandExec](IMPORTANT): got command set CircleSample.SendSampleResultToWG 1 bool 2024-09-20T16:36:46.146Z,1726850206.146 [CommandExec](IMPORTANT): got command set CircleSample.WaitBeforeFiring 0 minute 2024-09-20T16:36:46.147Z,1726850206.147 [CommandExec](IMPORTANT): got command set CircleSample.MaxWaitMustFire 120 minute 2024-09-20T16:36:46.148Z,1726850206.148 [CommandExec](IMPORTANT): got command set CircleSample.DepthCircle 150 meter 2024-09-20T16:37:00.041Z,1726850220.041 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMinDepth 10 meter 2024-09-20T16:37:00.042Z,1726850220.042 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMaxDepth 280 meter 2024-09-20T16:37:00.042Z,1726850220.042 [CommandExec](IMPORTANT): got command set CircleSample.MaxDepth 290 meter 2024-09-20T16:37:00.043Z,1726850220.043 [CommandExec](IMPORTANT): got command set CircleSample.CenterLatitude 36.712468 degree 2024-09-20T16:37:00.043Z,1726850220.043 [CommandExec](IMPORTANT): got command set CircleSample.CenterLongitude -122.1869 degree 2024-09-20T16:37:14.052Z,1726850234.052 [CommandExec](IMPORTANT): got command set CircleSample.IntervalRestartLogs 12 hour 2024-09-20T16:37:14.052Z,1726850234.052 [CommandExec](IMPORTANT): got command set CircleSample.TrackingUpdatePeriod 65 second 2024-09-20T16:37:14.057Z,1726850234.057 [CommandExec](IMPORTANT): got command set CircleSample.NumberOfStartingFixesToIgnore 1 count 2024-09-20T16:37:14.058Z,1726850234.058 [CommandExec](IMPORTANT): got command set CircleSample:NeedComms.CommsTimeout 15 minute 2024-09-20T16:37:14.058Z,1726850234.058 [CommandExec](IMPORTANT): got command run 2024-09-20T16:37:14.060Z,1726850234.060 [CommandExec](IMPORTANT): Running 2024-09-20T16:37:14.204Z,1726850234.204 [MissionManager](IMPORTANT): Started mission CircleSample 2024-09-20T16:37:14.254Z,1726850234.254 [CircleSample:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 0.50 hours. 2024-09-20T16:37:16.254Z,1726850236.254 [CircleSample:MissionStartNoComms:A](IMPORTANT): Mission start no comms. 2024-09-20T16:37:17.489Z,1726850237.489 [CircleSample:CircleAndSample:B](IMPORTANT): Circle around location 36.712467 arcdeg , -122.186897 arcdeg , at radius = 20.000000 m and depth = 150.000000 m . 2024-09-20T16:37:19.488Z,1726850239.488 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 5.000000 min . 2024-09-20T16:47:19.025Z,1726850839.025 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 1.098488 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 1.000000 count 2024-09-20T16:48:58.463Z,1726850938.463 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.951126 m , settling for 5.000000 min 2024-09-20T16:54:11.250Z,1726851251.250 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T16:54:11.372Z,1726851251.372 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T16:55:02.522Z,1726851302.522 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T16:55:03.334Z,1726851303.334 [ESPComponent](IMPORTANT): [sample #5] Setting cartridge argument to -1 2024-09-20T16:55:03.335Z,1726851303.335 [ESPComponent](IMPORTANT): [sample #5] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T16:55:03.802Z,1726851303.802 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 149.623734 m in 9.335566 degC water with 0.086320 ug/l chlorophyll fluorescence. 2024-09-20T16:55:04.139Z,1726851304.139 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T16:55:04.543Z,1726851304.543 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T16:55:06.562Z,1726851306.562 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T16:55:38.075Z,1726851338.075 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_PRIMING (@52,aRNAl_bac) 2024-09-20T16:55:40.902Z,1726851340.902 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T16:55:40.903Z,1726851340.903 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T16:55:41.307Z,1726851341.307 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T16:58:36.244Z,1726851516.244 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T16:58:36.651Z,1726851516.651 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T17:23:24.952Z,1726853004.952 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-20T17:42:43.894Z,1726854163.894 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-20T17:42:43.894Z,1726854163.894 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-20T17:46:22.295Z,1726854382.295 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T17:46:22.699Z,1726854382.699 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T17:46:24.720Z,1726854384.720 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T17:46:25.124Z,1726854385.124 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_WAITING_FOR_PROCESSED (@52,aRNAl_bac) 2024-09-20T17:47:59.807Z,1726854479.807 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_STOPPING (@52,aRNAl_bac) 2024-09-20T17:48:07.618Z,1726854487.618 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T17:48:08.114Z,1726854488.114 [ESPComponent](IMPORTANT): [sample #5] ESP sampling state: S_STOPPED 2024-09-20T17:53:07.887Z,1726854787.887 [ESPComponent](IMPORTANT): [sample #5] ESP sampling sequence completed normally. 2024-09-20T17:53:10.346Z,1726854790.346 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 1.000000 count 2024-09-20T17:53:10.765Z,1726854790.765 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T17:53:11.157Z,1726854791.157 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T17:53:11.567Z,1726854791.567 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T17:53:11.977Z,1726854791.977 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.885284 m 2024-09-20T17:53:12.403Z,1726854792.403 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.341089 degC 2024-09-20T17:53:12.774Z,1726854792.774 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.062400 ug/l 2024-09-20T17:53:13.600Z,1726854793.600 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 1.000000 count completed. 2024-09-20T17:53:15.215Z,1726854795.215 [MissionManager](IMPORTANT): Started mission Default 2024-09-20T17:53:20.196Z,1726854800.196 [DATMMP](FAULT): Buffer send receipt timeout failure. 2024-09-20T18:03:41.780Z,1726855421.780 [NAL9602](FAULT): MT Queue status failed to be acquired within timeout. Will not retry this session. 2024-09-20T18:07:33.820Z,1726855653.820 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17632, MT Status=1, MTMSN=4156 2024-09-20T18:07:34.543Z,1726855654.543 [CommandExec](IMPORTANT): got command schedule asap "load Science/circle_sample.xml;set CircleSample.NumSamplers 52 count;set CircleSample.MissionTimeout 110 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool" 4gnnc 1 4.000000 2024-09-20T18:07:34.543Z,1726855654.543 [CommandExec](IMPORTANT): Scheduling command #1 of 4 with id=4gnnc 2024-09-20T18:07:34.544Z,1726855654.544 [CommandExec](IMPORTANT): Scheduled #15 (#1 of 4 with id='4gnnc'): "load Science/circle_sample.xml;set CircleSample.NumSamplers 52 count;set CircleSample.MissionTimeout 110 h;set CircleSample.AcousticTrackingTimeout 0.5 h;set CircleSample.NeedCommsMissionStart 0 bool", ASAP 2024-09-20T18:08:09.054Z,1726855689.054 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17633, MT Status=1, MTMSN=4157 2024-09-20T18:08:09.847Z,1726855689.847 [CommandExec](IMPORTANT): got command schedule asap "set CircleSample.AcousticTargetDepth 0 m;set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m" 4gnnc 2 4.000000 2024-09-20T18:08:09.848Z,1726855689.848 [CommandExec](IMPORTANT): Scheduling command #2 of 4 with id=4gnnc 2024-09-20T18:08:09.849Z,1726855689.849 [CommandExec](IMPORTANT): Scheduled #16 (#2 of 4 with id='4gnnc'): "set CircleSample.AcousticTargetDepth 0 m;set CircleSample.SendSampleResultToWG 1 bool;set CircleSample.WaitBeforeFiring 0 min;set CircleSample.MaxWaitMustFire 120 min;set CircleSample.DepthCircle 150 m", ASAP 2024-09-20T18:08:31.724Z,1726855711.724 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17634, MT Status=1, MTMSN=4158 2024-09-20T18:08:32.456Z,1726855712.456 [CommandExec](IMPORTANT): got command schedule asap "set CircleSample.CircleYoYoMinDepth 10 m;set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree" 4gnnc 3 4.000000 2024-09-20T18:08:32.457Z,1726855712.457 [CommandExec](IMPORTANT): Scheduling command #3 of 4 with id=4gnnc 2024-09-20T18:08:32.458Z,1726855712.458 [CommandExec](IMPORTANT): Scheduled #17 (#3 of 4 with id='4gnnc'): "set CircleSample.CircleYoYoMinDepth 10 m;set CircleSample.CircleYoYoMaxDepth 280 m;set CircleSample.MaxDepth 290 m;set CircleSample.CenterLatitude 36.712468 degree;set CircleSample.CenterLongitude -122.1869 degree", ASAP 2024-09-20T18:08:56.469Z,1726855736.469 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=17635, MT Status=1, MTMSN=4159 2024-09-20T18:08:57.313Z,1726855737.313 [CommandExec](IMPORTANT): got command schedule asap "set CircleSample.Settle1 1 min;set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run" 4gnnc 4 4.000000 2024-09-20T18:08:57.313Z,1726855737.313 [CommandExec](IMPORTANT): Scheduling command #4 of 4 with id=4gnnc 2024-09-20T18:08:57.314Z,1726855737.314 [CommandExec](IMPORTANT): Scheduled #18 (#4 of 4 with id='4gnnc'): "set CircleSample.Settle1 1 min;set CircleSample.IntervalRestartLogs 12 h;set CircleSample.TrackingUpdatePeriod 65 s;set CircleSample.NumberOfStartingFixesToIgnore 1 count;set CircleSample:NeedComms.CommsTimeout 15 min;run", ASAP 2024-09-20T18:08:57.464Z,1726855737.464 [CommandExec](IMPORTANT): got command load ./Missions/Science/circle_sample.xml 2024-09-20T18:09:04.234Z,1726855744.234 [CommandExec](IMPORTANT): Loaded ./Missions/Science/circle_sample.xml id=CircleSample 2024-09-20T18:09:16.447Z,1726855756.447 [CommandExec](IMPORTANT): got command set CircleSample.NumSamplers 52 count 2024-09-20T18:09:16.455Z,1726855756.455 [CommandExec](IMPORTANT): got command set CircleSample.MissionTimeout 110 hour 2024-09-20T18:09:16.456Z,1726855756.456 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTrackingTimeout 0.5 hour 2024-09-20T18:09:16.457Z,1726855756.457 [CommandExec](IMPORTANT): got command set CircleSample.NeedCommsMissionStart 0 bool 2024-09-20T18:09:30.603Z,1726855770.603 [CommandExec](IMPORTANT): got command set CircleSample.AcousticTargetDepth 0 meter 2024-09-20T18:09:30.604Z,1726855770.604 [CommandExec](IMPORTANT): got command set CircleSample.SendSampleResultToWG 1 bool 2024-09-20T18:09:30.604Z,1726855770.604 [CommandExec](IMPORTANT): got command set CircleSample.WaitBeforeFiring 0 minute 2024-09-20T18:09:30.608Z,1726855770.608 [CommandExec](IMPORTANT): got command set CircleSample.MaxWaitMustFire 120 minute 2024-09-20T18:09:30.609Z,1726855770.609 [CommandExec](IMPORTANT): got command set CircleSample.DepthCircle 150 meter 2024-09-20T18:09:43.742Z,1726855783.742 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMinDepth 10 meter 2024-09-20T18:09:43.743Z,1726855783.743 [CommandExec](IMPORTANT): got command set CircleSample.CircleYoYoMaxDepth 280 meter 2024-09-20T18:09:43.744Z,1726855783.744 [CommandExec](IMPORTANT): got command set CircleSample.MaxDepth 290 meter 2024-09-20T18:09:43.745Z,1726855783.745 [CommandExec](IMPORTANT): got command set CircleSample.CenterLatitude 36.712468 degree 2024-09-20T18:09:43.745Z,1726855783.745 [CommandExec](IMPORTANT): got command set CircleSample.CenterLongitude -122.1869 degree 2024-09-20T18:09:59.170Z,1726855799.170 [CommandExec](IMPORTANT): got command set CircleSample.Settle1 1 minute 2024-09-20T18:09:59.170Z,1726855799.170 [CommandExec](IMPORTANT): got command set CircleSample.IntervalRestartLogs 12 hour 2024-09-20T18:09:59.171Z,1726855799.171 [CommandExec](IMPORTANT): got command set CircleSample.TrackingUpdatePeriod 65 second 2024-09-20T18:09:59.172Z,1726855799.172 [CommandExec](IMPORTANT): got command set CircleSample.NumberOfStartingFixesToIgnore 1 count 2024-09-20T18:09:59.172Z,1726855799.172 [CommandExec](IMPORTANT): got command set CircleSample:NeedComms.CommsTimeout 15 minute 2024-09-20T18:09:59.173Z,1726855799.173 [CommandExec](IMPORTANT): got command run 2024-09-20T18:09:59.210Z,1726855799.210 [CommandExec](IMPORTANT): Running 2024-09-20T18:09:59.296Z,1726855799.296 [MissionManager](IMPORTANT): Started mission CircleSample 2024-09-20T18:09:59.327Z,1726855799.327 [CircleSample:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 0.50 hours. 2024-09-20T18:10:01.319Z,1726855801.319 [CircleSample:MissionStartNoComms:A](IMPORTANT): Mission start no comms. 2024-09-20T18:10:02.550Z,1726855802.550 [CircleSample:CircleAndSample:B](IMPORTANT): Circle around location 36.712467 arcdeg , -122.186897 arcdeg , at radius = 20.000000 m and depth = 150.000000 m . 2024-09-20T18:10:04.573Z,1726855804.573 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-20T18:10:06.148Z,1726855806.148 [NAL9602](FAULT): GPS failed to acquire within timeout. 2024-09-20T18:20:04.108Z,1726856404.108 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 1.098383 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 1.000000 count 2024-09-20T18:32:32.317Z,1726857152.317 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.029114 m , settling for 1.000000 min 2024-09-20T18:33:45.101Z,1726857225.101 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T18:33:45.222Z,1726857225.222 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T18:34:36.746Z,1726857276.746 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T18:34:37.559Z,1726857277.559 [ESPComponent](IMPORTANT): [sample #6] Setting cartridge argument to -1 2024-09-20T18:34:37.559Z,1726857277.559 [ESPComponent](IMPORTANT): [sample #6] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T18:34:38.029Z,1726857278.029 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.140793 m in 9.362452 degC water with 0.056160 ug/l chlorophyll fluorescence. 2024-09-20T18:34:38.364Z,1726857278.364 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T18:34:38.767Z,1726857278.767 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T18:34:40.786Z,1726857280.786 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T18:35:12.313Z,1726857312.313 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PRIMING (@51,aRNAl_bac) 2024-09-20T18:35:14.323Z,1726857314.323 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T18:35:14.732Z,1726857314.732 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T18:35:15.131Z,1726857315.131 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T18:35:56.073Z,1726857356.073 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-20T18:38:09.658Z,1726857489.658 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T18:38:10.062Z,1726857490.062 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T19:26:30.125Z,1726860390.125 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T19:26:30.526Z,1726860390.526 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T19:26:32.547Z,1726860392.547 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T19:26:32.950Z,1726860392.950 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_WAITING_FOR_PROCESSED (@51,aRNAl_bac) 2024-09-20T19:28:07.895Z,1726860487.895 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_STOPPING (@51,aRNAl_bac) 2024-09-20T19:28:15.572Z,1726860495.572 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T19:28:15.975Z,1726860495.975 [ESPComponent](IMPORTANT): [sample #6] ESP sampling state: S_STOPPED 2024-09-20T19:33:15.935Z,1726860795.935 [ESPComponent](IMPORTANT): [sample #6] ESP sampling sequence completed normally. 2024-09-20T19:33:18.395Z,1726860798.395 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 1.000000 count 2024-09-20T19:33:18.810Z,1726860798.810 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T19:33:19.219Z,1726860799.219 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T19:33:19.607Z,1726860799.607 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T19:33:20.031Z,1726860800.031 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.543106 m 2024-09-20T19:33:20.421Z,1726860800.421 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.424585 degC 2024-09-20T19:33:20.837Z,1726860800.837 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.010400 ug/l 2024-09-20T19:33:21.629Z,1726860801.629 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 1.000000 count completed. 2024-09-20T19:33:26.120Z,1726860806.120 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-20T19:43:20.093Z,1726861400.093 [ElevatorServo](FAULT): Overload Error 2024-09-20T20:20:06.825Z,1726863606.825 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.032235 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 2.000000 count 2024-09-20T20:20:10.532Z,1726863610.532 [ElevatorServo](FAULT): Overload Error 2024-09-20T20:24:10.455Z,1726863850.455 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.729706 m , settling for 1.000000 min 2024-09-20T20:25:23.254Z,1726863923.254 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T20:25:23.353Z,1726863923.353 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T20:26:15.334Z,1726863975.334 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T20:26:16.145Z,1726863976.145 [ESPComponent](IMPORTANT): [sample #7] Setting cartridge argument to -1 2024-09-20T20:26:16.146Z,1726863976.146 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T20:26:16.590Z,1726863976.590 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.348129 m in 9.565790 degC water with 0.059280 ug/l chlorophyll fluorescence. 2024-09-20T20:26:16.950Z,1726863976.950 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T20:26:17.354Z,1726863977.354 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T20:26:19.375Z,1726863979.375 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T20:26:50.887Z,1726864010.887 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PRIMING (@50,aRNAl_bac) 2024-09-20T20:26:53.716Z,1726864013.716 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T20:26:53.717Z,1726864013.717 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T20:26:54.119Z,1726864014.119 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T20:29:49.866Z,1726864189.866 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T20:29:50.270Z,1726864190.270 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T20:30:29.258Z,1726864229.258 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-20T20:30:29.258Z,1726864229.258 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-20T20:49:46.066Z,1726865386.066 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-20T20:49:46.066Z,1726865386.066 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-20T21:03:04.834Z,1726866184.834 [CBIT](IMPORTANT): Beginning ground fault scan 2024-09-20T21:03:15.724Z,1726866195.724 [CBIT](IMPORTANT): Ground fault detected mA: CHAN A0 (Batt): 0.058017 CHAN A1 (24V): 0.118231 CHAN A2 (12V): 0.042980 CHAN A3 (5V): 0.010717 CHAN B0 (3.3V): 0.007030 CHAN B1 (3.15aV): 0.006478 CHAN B2 (3.15bV): 0.007023 CHAN B3 (GND): -0.007393 OPEN: 0.007663 Full Scale: +/- 1 mA 2024-09-20T21:11:04.973Z,1726866664.973 [RudderServo](FAULT): Rudder uart error - getPosition..serial timeout 2024-09-20T21:15:11.102Z,1726866911.102 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T21:15:11.507Z,1726866911.507 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T21:15:13.527Z,1726866913.527 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T21:15:13.931Z,1726866913.931 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_WAITING_FOR_PROCESSED (@50,aRNAl_bac) 2024-09-20T21:16:48.879Z,1726867008.879 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPING (@50,aRNAl_bac) 2024-09-20T21:16:56.559Z,1726867016.559 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T21:16:56.966Z,1726867016.966 [ESPComponent](IMPORTANT): [sample #7] ESP sampling state: S_STOPPED 2024-09-20T21:21:56.751Z,1726867316.751 [ESPComponent](IMPORTANT): [sample #7] ESP sampling sequence completed normally. 2024-09-20T21:21:59.233Z,1726867319.233 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 2.000000 count 2024-09-20T21:21:59.624Z,1726867319.624 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T21:22:00.036Z,1726867320.036 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T21:22:00.483Z,1726867320.483 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T21:22:00.828Z,1726867320.828 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 151.823105 m 2024-09-20T21:22:01.242Z,1726867321.242 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.540888 degC 2024-09-20T21:22:01.664Z,1726867321.664 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.093600 ug/l 2024-09-20T21:22:02.439Z,1726867322.439 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 2.000000 count completed. 2024-09-20T21:22:06.894Z,1726867326.894 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-20T21:22:09.162Z,1726867329.162 [DATMMP](FAULT): Buffer send receipt timeout failure. 2024-09-20T21:52:26.061Z,1726869146.061 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-20T22:20:09.456Z,1726870809.456 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.029443 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 3.000000 count 2024-09-20T22:20:13.110Z,1726870813.110 [ElevatorServo](FAULT): Overload Error 2024-09-20T22:21:58.919Z,1726870918.919 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.747314 m , settling for 1.000000 min 2024-09-20T22:23:11.901Z,1726870991.901 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-20T22:23:12.071Z,1726870992.071 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-20T22:24:03.954Z,1726871043.954 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-20T22:24:04.769Z,1726871044.769 [ESPComponent](IMPORTANT): [sample #8] Setting cartridge argument to -1 2024-09-20T22:24:04.770Z,1726871044.770 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-20T22:24:05.227Z,1726871045.227 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 148.505585 m in 9.561487 degC water with 0.061360 ug/l chlorophyll fluorescence. 2024-09-20T22:24:05.570Z,1726871045.570 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-20T22:24:05.975Z,1726871045.975 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-20T22:24:07.994Z,1726871047.994 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-20T22:24:39.511Z,1726871079.511 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PRIMING (@49,aRNAl_bac) 2024-09-20T22:24:42.339Z,1726871082.339 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PRIMING (@100.0) 2024-09-20T22:24:42.340Z,1726871082.340 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-20T22:24:42.742Z,1726871082.742 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-20T22:27:38.492Z,1726871258.492 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_FILTERING (@0.0) 2024-09-20T22:27:38.895Z,1726871258.895 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-20T22:44:33.257Z,1726872273.257 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-20T22:44:33.257Z,1726872273.257 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-20T22:57:04.089Z,1726873024.089 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-20T22:57:04.089Z,1726873024.089 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-20T23:00:38.181Z,1726873238.181 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-20T23:02:51.085Z,1726873371.085 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-20T23:16:50.457Z,1726874210.457 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PAUSED (@1000.0) 2024-09-20T23:16:50.859Z,1726874210.859 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-20T23:16:52.880Z,1726874212.880 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-20T23:16:53.283Z,1726874213.283 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_WAITING_FOR_PROCESSED (@49,aRNAl_bac) 2024-09-20T23:18:28.302Z,1726874308.302 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPING (@49,aRNAl_bac) 2024-09-20T23:18:35.980Z,1726874315.980 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-20T23:18:36.409Z,1726874316.409 [ESPComponent](IMPORTANT): [sample #8] ESP sampling state: S_STOPPED 2024-09-20T23:23:36.304Z,1726874616.304 [ESPComponent](IMPORTANT): [sample #8] ESP sampling sequence completed normally. 2024-09-20T23:23:38.802Z,1726874618.802 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 3.000000 count 2024-09-20T23:23:39.177Z,1726874619.177 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-20T23:23:39.582Z,1726874619.582 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-20T23:23:40.061Z,1726874620.061 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-20T23:23:40.408Z,1726874620.408 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.649124 m 2024-09-20T23:23:40.833Z,1726874620.833 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.430109 degC 2024-09-20T23:23:41.207Z,1726874621.207 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.072800 ug/l 2024-09-20T23:23:41.997Z,1726874621.997 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 3.000000 count completed. 2024-09-20T23:23:46.466Z,1726874626.466 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-21T00:11:10.049Z,1726877470.049 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-21T00:20:12.159Z,1726878012.159 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.032422 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 4.000000 count 2024-09-21T00:21:09.142Z,1726878069.142 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.150101 m , settling for 1.000000 min 2024-09-21T00:22:21.913Z,1726878141.913 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-21T00:22:22.105Z,1726878142.105 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-21T00:23:13.651Z,1726878193.651 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-21T00:23:14.462Z,1726878194.462 [ESPComponent](IMPORTANT): [sample #9] Setting cartridge argument to -1 2024-09-21T00:23:14.462Z,1726878194.462 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-21T00:23:14.943Z,1726878194.943 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 150.647415 m in 9.426752 degC water with 0.059280 ug/l chlorophyll fluorescence. 2024-09-21T00:23:15.267Z,1726878195.267 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-21T00:23:15.674Z,1726878195.674 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-21T00:23:17.692Z,1726878197.692 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-21T00:23:49.206Z,1726878229.206 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PRIMING (@48,aRNAl_bac) 2024-09-21T00:23:51.224Z,1726878231.224 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PRIMING (@100.0) 2024-09-21T00:23:51.637Z,1726878231.637 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-21T00:23:52.038Z,1726878232.038 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-21T00:26:55.055Z,1726878415.055 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_FILTERING (@0.0) 2024-09-21T00:26:55.455Z,1726878415.455 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-21T00:46:06.081Z,1726879566.081 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-21T00:46:06.081Z,1726879566.081 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-21T01:13:15.011Z,1726881195.011 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PAUSED (@1000.0) 2024-09-21T01:13:15.415Z,1726881195.415 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-21T01:13:17.435Z,1726881197.435 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-21T01:13:17.839Z,1726881197.839 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_WAITING_FOR_PROCESSED (@48,aRNAl_bac) 2024-09-21T01:14:53.191Z,1726881293.191 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPING (@48,aRNAl_bac) 2024-09-21T01:15:00.868Z,1726881300.868 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-21T01:15:01.271Z,1726881301.271 [ESPComponent](IMPORTANT): [sample #9] ESP sampling state: S_STOPPED 2024-09-21T01:20:01.051Z,1726881601.051 [ESPComponent](IMPORTANT): [sample #9] ESP sampling sequence completed normally. 2024-09-21T01:20:01.923Z,1726881601.923 [ESPComponent](FAULT): LCB 1 fault: LCB Watchdog Reset. Current Limiter Activated. 2024-09-21T01:20:03.609Z,1726881603.609 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 4.000000 count 2024-09-21T01:20:03.983Z,1726881603.983 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-21T01:20:04.411Z,1726881604.411 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-21T01:20:04.805Z,1726881604.805 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-21T01:20:05.193Z,1726881605.193 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.206207 m 2024-09-21T01:20:05.604Z,1726881605.604 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.531214 degC 2024-09-21T01:20:06.018Z,1726881606.018 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.041600 ug/l 2024-09-21T01:20:06.814Z,1726881606.814 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 4.000000 count completed. 2024-09-21T01:20:11.281Z,1726881611.281 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-21T01:20:44.565Z,1726881644.565 [DATMMP](FAULT): Buffer send receipt timeout failure. 2024-09-21T01:48:14.842Z,1726883294.842 [ElevatorServo](FAULT): Overload Error 2024-09-21T02:20:14.809Z,1726885214.809 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.030762 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 5.000000 count 2024-09-21T02:20:18.497Z,1726885218.497 [ElevatorServo](FAULT): Overload Error 2024-09-21T02:23:13.778Z,1726885393.778 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 149.696808 m , settling for 1.000000 min 2024-09-21T02:24:26.630Z,1726885466.630 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-21T02:24:26.754Z,1726885466.754 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-21T02:25:19.071Z,1726885519.071 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-21T02:25:19.878Z,1726885519.878 [ESPComponent](IMPORTANT): [sample #10] Setting cartridge argument to -1 2024-09-21T02:25:19.879Z,1726885519.879 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-21T02:25:20.324Z,1726885520.324 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 151.185638 m in 9.499964 degC water with 0.059280 ug/l chlorophyll fluorescence. 2024-09-21T02:25:20.682Z,1726885520.682 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-21T02:25:21.087Z,1726885521.087 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-21T02:25:23.106Z,1726885523.106 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-21T02:25:54.620Z,1726885554.620 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PRIMING (@47,aRNAl_bac) 2024-09-21T02:25:57.446Z,1726885557.446 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PRIMING (@100.0) 2024-09-21T02:25:57.447Z,1726885557.447 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-21T02:25:57.851Z,1726885557.851 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-21T02:28:53.601Z,1726885733.601 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_FILTERING (@0.0) 2024-09-21T02:28:54.002Z,1726885734.002 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-21T03:03:15.904Z,1726887795.904 [CBIT](IMPORTANT): Beginning ground fault scan 2024-09-21T03:03:26.768Z,1726887806.768 [CBIT](IMPORTANT): Ground fault detected mA: CHAN A0 (Batt): 0.022892 CHAN A1 (24V): 0.058776 CHAN A2 (12V): 0.013602 CHAN A3 (5V): 0.002057 CHAN B0 (3.3V): 0.004486 CHAN B1 (3.15aV): 0.003098 CHAN B2 (3.15bV): 0.001310 CHAN B3 (GND): -0.001604 OPEN: 0.008704 Full Scale: +/- 1 mA 2024-09-21T03:13:52.443Z,1726888432.443 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PAUSED (@1000.0) 2024-09-21T03:13:52.851Z,1726888432.851 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-21T03:13:54.867Z,1726888434.867 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-21T03:13:55.270Z,1726888435.270 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_WAITING_FOR_PROCESSED (@47,aRNAl_bac) 2024-09-21T03:15:30.222Z,1726888530.222 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPING (@47,aRNAl_bac) 2024-09-21T03:15:37.896Z,1726888537.896 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-21T03:15:38.299Z,1726888538.299 [ESPComponent](IMPORTANT): [sample #10] ESP sampling state: S_STOPPED 2024-09-21T03:18:47.606Z,1726888727.606 [ThrusterHE](FAULT): Could not get status:serial timeout 2024-09-21T03:20:38.159Z,1726888838.159 [ESPComponent](IMPORTANT): [sample #10] ESP sampling sequence completed normally. 2024-09-21T03:20:40.667Z,1726888840.667 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 5.000000 count 2024-09-21T03:20:41.037Z,1726888841.037 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-21T03:20:41.440Z,1726888841.440 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-21T03:20:41.858Z,1726888841.858 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-21T03:20:42.236Z,1726888842.236 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 149.203125 m 2024-09-21T03:20:42.650Z,1726888842.650 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.438990 degC 2024-09-21T03:20:43.063Z,1726888843.063 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.062400 ug/l 2024-09-21T03:20:43.853Z,1726888843.853 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 5.000000 count completed. 2024-09-21T03:20:48.343Z,1726888848.343 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-21T03:21:10.834Z,1726888870.834 [DATMMP](FAULT): Buffer send receipt timeout failure. 2024-09-21T04:05:44.329Z,1726891544.329 [ThrusterHE](FAULT): getSpeed error serial timeout 2024-09-21T04:05:44.330Z,1726891544.330 [ThrusterHE](FAULT): Thruster uart error: serial timeout 2024-09-21T04:20:17.231Z,1726892417.231 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:F](IMPORTANT): Elapsed time since start or last firing(= 120.026978 min ) is longer than MaxWaitFirstSample (= 10.000000 min ) for the 1st sample, or longer than MaxWaitMustFire (= 120.000000 min ) for non-1st samples. Firing sample No. 6.000000 count 2024-09-21T04:22:42.686Z,1726892562.686 [CircleSample:SampleAtDepth:sample:SampleWrapper:B](IMPORTANT): At 150.315277 m , settling for 1.000000 min 2024-09-21T04:23:55.462Z,1726892635.462 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2024-09-21T04:23:55.562Z,1726892635.562 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2024-09-21T04:24:47.923Z,1726892687.923 [ESPComponent](IMPORTANT): ESP has connected as client 2024-09-21T04:24:48.734Z,1726892688.734 [ESPComponent](IMPORTANT): [sample #11] Setting cartridge argument to -1 2024-09-21T04:24:48.734Z,1726892688.734 [ESPComponent](IMPORTANT): [sample #11] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2024-09-21T04:24:49.186Z,1726892689.186 [CircleSample:SampleAtDepth:sample:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 149.930527 m in 9.515772 degC water with 0.064480 ug/l chlorophyll fluorescence. 2024-09-21T04:24:49.542Z,1726892689.542 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PREPARING_SHOW_LOG 2024-09-21T04:24:49.943Z,1726892689.943 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PREPARING_SHOW_STATUS 2024-09-21T04:24:51.963Z,1726892691.963 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_LOADING_CARTRIDGE 2024-09-21T04:25:23.474Z,1726892723.474 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PRIMING (@46,aRNAl_bac) 2024-09-21T04:25:25.899Z,1726892725.899 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PRIMING (@100.0) 2024-09-21T04:25:25.899Z,1726892725.899 [ESPComponent](IMPORTANT): Filtering timeoutMinutes=94.169998 issuedCmd_=Cmd.startFiltering 2024-09-21T04:25:26.304Z,1726892726.304 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2024-09-21T04:28:43.877Z,1726892923.877 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_FILTERING (@0.0) 2024-09-21T04:28:44.279Z,1726892924.279 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2024-09-21T05:16:43.371Z,1726895803.371 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PAUSED (@1000.0) 2024-09-21T05:16:43.775Z,1726895803.775 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_PROCESSING (@1000.0) 2024-09-21T05:16:45.800Z,1726895805.800 [ESPComponent](IMPORTANT): Processing timeoutMinutes=30.000000 issuedCmd_=Cmd.startProcessing 2024-09-21T05:16:46.200Z,1726895806.200 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_WAITING_FOR_PROCESSED (@45,aRNAl_bac) 2024-09-21T05:18:21.142Z,1726895901.142 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_STOPPING (@45,aRNAl_bac) 2024-09-21T05:18:28.819Z,1726895908.819 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq -e 'ssh -o StrictHostKeyChecking=no' --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr*' /LRAUV/ESPlogs & 2024-09-21T05:18:29.223Z,1726895909.223 [ESPComponent](IMPORTANT): [sample #11] ESP sampling state: S_STOPPED 2024-09-21T05:23:29.075Z,1726896209.075 [ESPComponent](IMPORTANT): [sample #11] ESP sampling sequence completed normally. 2024-09-21T05:23:31.537Z,1726896211.537 [CircleSample:SendSampleInfo:SampleInfo:C](IMPORTANT): CntSamples = 6.000000 count 2024-09-21T05:23:31.945Z,1726896211.945 [CircleSample:SendSampleInfo:SampleInfo:D](IMPORTANT): Science:ESPComponent.sample_number = 0 count 2024-09-21T05:23:32.350Z,1726896212.350 [CircleSample:SendSampleInfo:SampleInfo:E](IMPORTANT): Science:ESPComponent.cartridge = -1 count 2024-09-21T05:23:32.772Z,1726896212.772 [CircleSample:SendSampleInfo:SampleInfo:F](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2024-09-21T05:23:33.147Z,1726896213.147 [CircleSample:SendSampleInfo:SampleInfo:G](IMPORTANT): Universal:depth = 150.470032 m 2024-09-21T05:23:33.571Z,1726896213.571 [CircleSample:SendSampleInfo:SampleInfo:H](IMPORTANT): Universal:sea_water_temperature = 9.432795 degC 2024-09-21T05:23:33.959Z,1726896213.959 [CircleSample:SendSampleInfo:SampleInfo:I](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 0.062400 ug/l 2024-09-21T05:23:34.776Z,1726896214.776 [CircleSample:CircleAndSample:Sampling:FiringOnMaxWaitOrNotReceivingOtherLRAUVSampleStartFirstSample:O](IMPORTANT): Sample No. 6.000000 count completed. 2024-09-21T05:23:39.267Z,1726896219.267 [CircleSample:SetSampleOptions:E](IMPORTANT): Set CartridgeType (if ESP), SettleDuration to -1.000000 count , 1.000000 min . 2024-09-21T05:23:41.401Z,1726896221.401 [DATMMP](FAULT): Buffer send receipt timeout failure. 2024-09-21T06:09:59.970Z,1726898999.970 [CircleSample:CycleLogs:A:B](IMPORTANT): Restarting logs.