2021-04-22T08:44:15.422Z,1619081055.422 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 19.818722 min 2021-04-22T08:48:36.716Z,1619081316.716 [DataOverHttps](IMPORTANT): SBD MTMSN=20210422T084835 2021-04-22T08:49:05.028Z,1619081345.028 [CommandLine](IMPORTANT): got command configSet Express linearApproximation 2021-04-22T08:49:05.028Z,1619081345.028 [CommandLine](FAULT): Incomplete syntax. Try: help configSet 2021-04-22T08:49:05.035Z,1619081345.035 [DataOverHttps](IMPORTANT): SBD MTMSN=20210422T084846 2021-04-22T08:49:16.720Z,1619081356.720 [CommandLine](IMPORTANT): got command configSet Express linearApproximation WetLabsBB2FL.bin_median_mass_concentration_of_chlorophyll_in_sea_water 0.400000 microgram_per_liter persist 2021-04-22T08:50:27.319Z,1619081427.319 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 26.017041 min 2021-04-22T08:56:23.373Z,1619081783.373 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 31.951093 min 2021-04-22T08:57:00.988Z,1619081820.988 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41796, MT Status=1, MTMSN=2976 2021-04-22T08:57:01.778Z,1619081821.778 [CommandLine](IMPORTANT): got command schedule asap "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.NumSamplers 40 count;set IsothermDepthSampling.MissionTimeout 140 hour;set IsothermDepthSampling.AcousticTrackingTimeout 3 hour" 20wmj 1 23.000000 2021-04-22T08:57:01.779Z,1619081821.779 [CommandLine](IMPORTANT): Scheduling command #1 of 23 with id=20wmj 2021-04-22T08:57:01.780Z,1619081821.780 [CommandLine](IMPORTANT): Scheduled #102 (#1 of 23 with id='20wmj'): "load Science/isotherm_depth_sampling.xml;set IsothermDepthSampling.NumSamplers 40 count;set IsothermDepthSampling.MissionTimeout 140 hour;set IsothermDepthSampling.AcousticTrackingTimeout 3 hour" ASAP 2021-04-22T08:58:45.859Z,1619081925.859 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41797, MT Status=1, MTMSN=2977 2021-04-22T08:58:46.646Z,1619081926.646 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.MaxWaitNoFiring 6 hour;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.ShallowBoundChl 5 meter;set IsothermDepthSampling.DeepBoundChl 10 meter" 20wmj 2 23.000000 2021-04-22T08:58:46.647Z,1619081926.647 [CommandLine](IMPORTANT): Scheduling command #2 of 23 with id=20wmj 2021-04-22T08:58:46.648Z,1619081926.648 [CommandLine](IMPORTANT): Scheduled #103 (#2 of 23 with id='20wmj'): "set IsothermDepthSampling.MaxWaitNoFiring 6 hour;set IsothermDepthSampling.SampleAtPeakChl 1 bool;set IsothermDepthSampling.ShallowBoundChl 5 meter;set IsothermDepthSampling.DeepBoundChl 10 meter" ASAP 2021-04-22T08:59:20.624Z,1619081960.624 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41798, MT Status=1, MTMSN=2978 2021-04-22T08:59:21.394Z,1619081961.394 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType1 -6 count;set IsothermDepthSampling.CartridgeType2 -9 count;set IsothermDepthSampling.CartridgeType3 -6 count;set IsothermDepthSampling.CartridgeType4 -9 count" 20wmj 3 23.000000 2021-04-22T08:59:21.395Z,1619081961.395 [CommandLine](IMPORTANT): Scheduling command #3 of 23 with id=20wmj 2021-04-22T08:59:21.396Z,1619081961.396 [CommandLine](IMPORTANT): Scheduled #104 (#3 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType1 -6 count;set IsothermDepthSampling.CartridgeType2 -9 count;set IsothermDepthSampling.CartridgeType3 -6 count;set IsothermDepthSampling.CartridgeType4 -9 count" ASAP 2021-04-22T09:00:16.515Z,1619082016.515 [DataOverHttps](IMPORTANT): SBD MTMSN=20210422T090015 2021-04-22T09:00:24.822Z,1619082024.822 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType21 -6 count;set IsothermDepthSampling.CartridgeType22 -9 count;set IsothermDepthSampling.CartridgeType23 -6 count;set IsothermDepthSampling.CartridgeType24 -9 count" 20wmj 8 23.000000 2021-04-22T09:00:24.823Z,1619082024.823 [CommandLine](IMPORTANT): Scheduling command #8 of 23 with id=20wmj 2021-04-22T09:00:24.824Z,1619082024.824 [CommandLine](IMPORTANT): Scheduled #105 (#8 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType21 -6 count;set IsothermDepthSampling.CartridgeType22 -9 count;set IsothermDepthSampling.CartridgeType23 -6 count;set IsothermDepthSampling.CartridgeType24 -9 count" ASAP 2021-04-22T09:00:25.787Z,1619082025.787 [DataOverHttps](IMPORTANT): SBD MTMSN=20210422T090025 2021-04-22T09:00:26.459Z,1619082026.459 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41799, MT Status=1, MTMSN=2979 2021-04-22T09:00:27.222Z,1619082027.222 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType5 -6 count;set IsothermDepthSampling.CartridgeType6 -9 count;set IsothermDepthSampling.CartridgeType7 -6 count;set IsothermDepthSampling.CartridgeType8 -9 count" 20wmj 4 23.000000 2021-04-22T09:00:27.222Z,1619082027.222 [CommandLine](IMPORTANT): Scheduling command #4 of 23 with id=20wmj 2021-04-22T09:00:27.223Z,1619082027.223 [CommandLine](IMPORTANT): Scheduled #106 (#4 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType5 -6 count;set IsothermDepthSampling.CartridgeType6 -9 count;set IsothermDepthSampling.CartridgeType7 -6 count;set IsothermDepthSampling.CartridgeType8 -9 count" ASAP 2021-04-22T09:05:19.367Z,1619082319.367 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 40.884652 min 2021-04-22T09:07:37.744Z,1619082457.744 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41800, MT Status=1, MTMSN=2980 2021-04-22T09:07:38.483Z,1619082458.483 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType9 -6 count;set IsothermDepthSampling.CartridgeType10 -9 count;set IsothermDepthSampling.CartridgeType11 -6 count;set IsothermDepthSampling.CartridgeType12 -9 count" 20wmj 5 23.000000 2021-04-22T09:07:38.484Z,1619082458.484 [CommandLine](IMPORTANT): Scheduling command #5 of 23 with id=20wmj 2021-04-22T09:07:38.485Z,1619082458.485 [CommandLine](IMPORTANT): Scheduled #107 (#5 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType9 -6 count;set IsothermDepthSampling.CartridgeType10 -9 count;set IsothermDepthSampling.CartridgeType11 -6 count;set IsothermDepthSampling.CartridgeType12 -9 count" ASAP 2021-04-22T09:08:07.707Z,1619082487.707 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41801, MT Status=1, MTMSN=2981 2021-04-22T09:08:08.467Z,1619082488.467 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType13 -6 count;set IsothermDepthSampling.CartridgeType14 -9 count;set IsothermDepthSampling.CartridgeType15 -6 count;set IsothermDepthSampling.CartridgeType16 -9 count" 20wmj 6 23.000000 2021-04-22T09:08:08.467Z,1619082488.467 [CommandLine](IMPORTANT): Scheduling command #6 of 23 with id=20wmj 2021-04-22T09:08:08.468Z,1619082488.468 [CommandLine](IMPORTANT): Scheduled #108 (#6 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType13 -6 count;set IsothermDepthSampling.CartridgeType14 -9 count;set IsothermDepthSampling.CartridgeType15 -6 count;set IsothermDepthSampling.CartridgeType16 -9 count" ASAP 2021-04-22T09:10:10.719Z,1619082610.719 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41802, MT Status=1, MTMSN=2982 2021-04-22T09:10:11.495Z,1619082611.495 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType17 -6 count;set IsothermDepthSampling.CartridgeType18 -9 count;set IsothermDepthSampling.CartridgeType19 -6 count;set IsothermDepthSampling.CartridgeType20 -9 count" 20wmj 7 23.000000 2021-04-22T09:10:11.495Z,1619082611.495 [CommandLine](IMPORTANT): Scheduling command #7 of 23 with id=20wmj 2021-04-22T09:10:11.496Z,1619082611.496 [CommandLine](IMPORTANT): Scheduled #109 (#7 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType17 -6 count;set IsothermDepthSampling.CartridgeType18 -9 count;set IsothermDepthSampling.CartridgeType19 -6 count;set IsothermDepthSampling.CartridgeType20 -9 count" ASAP 2021-04-22T09:11:03.347Z,1619082663.347 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41803, MT Status=1, MTMSN=2983 2021-04-22T09:11:04.086Z,1619082664.086 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType25 -6 count;set IsothermDepthSampling.CartridgeType26 -9 count;set IsothermDepthSampling.CartridgeType27 -6 count;set IsothermDepthSampling.CartridgeType28 -9 count" 20wmj 9 23.000000 2021-04-22T09:11:04.086Z,1619082664.086 [CommandLine](IMPORTANT): Scheduling command #9 of 23 with id=20wmj 2021-04-22T09:11:04.087Z,1619082664.087 [CommandLine](IMPORTANT): Scheduled #110 (#9 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType25 -6 count;set IsothermDepthSampling.CartridgeType26 -9 count;set IsothermDepthSampling.CartridgeType27 -6 count;set IsothermDepthSampling.CartridgeType28 -9 count" ASAP 2021-04-22T09:11:39.310Z,1619082699.310 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41804, MT Status=1, MTMSN=2984 2021-04-22T09:11:40.079Z,1619082700.079 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType29 -6 count;set IsothermDepthSampling.CartridgeType30 -9 count;set IsothermDepthSampling.CartridgeType31 -6 count;set IsothermDepthSampling.CartridgeType32 -9 count" 20wmj 10 23.000000 2021-04-22T09:11:40.079Z,1619082700.079 [CommandLine](IMPORTANT): Scheduling command #10 of 23 with id=20wmj 2021-04-22T09:11:40.080Z,1619082700.080 [CommandLine](IMPORTANT): Scheduled #111 (#10 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType29 -6 count;set IsothermDepthSampling.CartridgeType30 -9 count;set IsothermDepthSampling.CartridgeType31 -6 count;set IsothermDepthSampling.CartridgeType32 -9 count" ASAP 2021-04-22T09:11:47.807Z,1619082707.807 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41805, MT Status=1, MTMSN=2985 2021-04-22T09:11:48.595Z,1619082708.595 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType33 -6 count;set IsothermDepthSampling.CartridgeType34 -9 count;set IsothermDepthSampling.CartridgeType35 -6 count;set IsothermDepthSampling.CartridgeType36 -9 count" 20wmj 11 23.000000 2021-04-22T09:11:48.595Z,1619082708.595 [CommandLine](IMPORTANT): Scheduling command #11 of 23 with id=20wmj 2021-04-22T09:11:48.596Z,1619082708.596 [CommandLine](IMPORTANT): Scheduled #112 (#11 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType33 -6 count;set IsothermDepthSampling.CartridgeType34 -9 count;set IsothermDepthSampling.CartridgeType35 -6 count;set IsothermDepthSampling.CartridgeType36 -9 count" ASAP 2021-04-22T09:12:59.310Z,1619082779.310 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41806, MT Status=1, MTMSN=2986 2021-04-22T09:13:00.134Z,1619082780.134 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType37 -6 count;set IsothermDepthSampling.CartridgeType38 -9 count;set IsothermDepthSampling.CartridgeType39 -6 count;set IsothermDepthSampling.CartridgeType40 -9 count" 20wmj 12 23.000000 2021-04-22T09:13:00.135Z,1619082780.135 [CommandLine](IMPORTANT): Scheduling command #12 of 23 with id=20wmj 2021-04-22T09:13:00.136Z,1619082780.136 [CommandLine](IMPORTANT): Scheduled #113 (#12 of 23 with id='20wmj'): "set IsothermDepthSampling.CartridgeType37 -6 count;set IsothermDepthSampling.CartridgeType38 -9 count;set IsothermDepthSampling.CartridgeType39 -6 count;set IsothermDepthSampling.CartridgeType40 -9 count" ASAP 2021-04-22T09:13:15.083Z,1619082795.083 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41807, MT Status=1, MTMSN=2987 2021-04-22T09:13:15.859Z,1619082795.859 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 meter;set IsothermDepthSampling.WaitDepUndulation1 20 minute;set IsothermDepthSampling.WaitDepUndulation2 5 minute;set IsothermDepthSampling.WaitDepUndulation3 120 minute" 20wmj 13 23.000000 2021-04-22T09:13:15.859Z,1619082795.859 [CommandLine](IMPORTANT): Scheduling command #13 of 23 with id=20wmj 2021-04-22T09:13:15.860Z,1619082795.860 [CommandLine](IMPORTANT): Scheduled #114 (#13 of 23 with id='20wmj'): "set IsothermDepthSampling.DepDiffFromPeakChlCommon 0 meter;set IsothermDepthSampling.WaitDepUndulation1 20 minute;set IsothermDepthSampling.WaitDepUndulation2 5 minute;set IsothermDepthSampling.WaitDepUndulation3 120 minute" ASAP 2021-04-22T09:13:45.782Z,1619082825.782 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41808, MT Status=1, MTMSN=2988 2021-04-22T09:13:46.587Z,1619082826.587 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation4 5 minute;set IsothermDepthSampling.WaitDepUndulation5 120 minute;set IsothermDepthSampling.WaitDepUndulation6 5 minute;set IsothermDepthSampling.WaitDepUndulation7 120 minute" 20wmj 14 23.000000 2021-04-22T09:13:46.587Z,1619082826.587 [CommandLine](IMPORTANT): Scheduling command #14 of 23 with id=20wmj 2021-04-22T09:13:46.588Z,1619082826.588 [CommandLine](IMPORTANT): Scheduled #115 (#14 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation4 5 minute;set IsothermDepthSampling.WaitDepUndulation5 120 minute;set IsothermDepthSampling.WaitDepUndulation6 5 minute;set IsothermDepthSampling.WaitDepUndulation7 120 minute" ASAP 2021-04-22T09:35:50.957Z,1619084150.957 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41809, MT Status=1, MTMSN=2989 2021-04-22T09:35:51.711Z,1619084151.711 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation8 5 minute;set IsothermDepthSampling.WaitDepUndulation9 120 minute;set IsothermDepthSampling.WaitDepUndulation10 5 minute;set IsothermDepthSampling.WaitDepUndulation11 120 minute" 20wmj 15 23.000000 2021-04-22T09:35:51.711Z,1619084151.711 [CommandLine](IMPORTANT): Scheduling command #15 of 23 with id=20wmj 2021-04-22T09:35:51.712Z,1619084151.712 [CommandLine](IMPORTANT): Scheduled #116 (#15 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation8 5 minute;set IsothermDepthSampling.WaitDepUndulation9 120 minute;set IsothermDepthSampling.WaitDepUndulation10 5 minute;set IsothermDepthSampling.WaitDepUndulation11 120 minute" ASAP 2021-04-22T09:37:52.971Z,1619084272.971 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41810, MT Status=1, MTMSN=2990 2021-04-22T09:37:53.778Z,1619084273.778 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation12 5 minute;set IsothermDepthSampling.WaitDepUndulation13 120 minute;set IsothermDepthSampling.WaitDepUndulation14 5 minute;set IsothermDepthSampling.WaitDepUndulation15 120 minute" 20wmj 16 23.000000 2021-04-22T09:37:53.779Z,1619084273.779 [CommandLine](IMPORTANT): Scheduling command #16 of 23 with id=20wmj 2021-04-22T09:37:53.780Z,1619084273.780 [CommandLine](IMPORTANT): Scheduled #117 (#16 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation12 5 minute;set IsothermDepthSampling.WaitDepUndulation13 120 minute;set IsothermDepthSampling.WaitDepUndulation14 5 minute;set IsothermDepthSampling.WaitDepUndulation15 120 minute" ASAP 2021-04-22T09:41:56.239Z,1619084516.239 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41811, MT Status=1, MTMSN=2991 2021-04-22T09:41:57.075Z,1619084517.075 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation16 5 minute;set IsothermDepthSampling.WaitDepUndulation17 120 minute;set IsothermDepthSampling.WaitDepUndulation18 5 minute;set IsothermDepthSampling.WaitDepUndulation19 120 minute" 20wmj 17 23.000000 2021-04-22T09:41:57.075Z,1619084517.075 [CommandLine](IMPORTANT): Scheduling command #17 of 23 with id=20wmj 2021-04-22T09:41:57.076Z,1619084517.076 [CommandLine](IMPORTANT): Scheduled #118 (#17 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation16 5 minute;set IsothermDepthSampling.WaitDepUndulation17 120 minute;set IsothermDepthSampling.WaitDepUndulation18 5 minute;set IsothermDepthSampling.WaitDepUndulation19 120 minute" ASAP 2021-04-22T09:42:37.851Z,1619084557.851 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41812, MT Status=1, MTMSN=2992 2021-04-22T09:42:38.646Z,1619084558.646 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation20 5 minute;set IsothermDepthSampling.WaitDepUndulation21 120 minute;set IsothermDepthSampling.WaitDepUndulation22 5 minute;set IsothermDepthSampling.WaitDepUndulation23 120 minute" 20wmj 18 23.000000 2021-04-22T09:42:38.647Z,1619084558.647 [CommandLine](IMPORTANT): Scheduling command #18 of 23 with id=20wmj 2021-04-22T09:42:38.648Z,1619084558.648 [CommandLine](IMPORTANT): Scheduled #119 (#18 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation20 5 minute;set IsothermDepthSampling.WaitDepUndulation21 120 minute;set IsothermDepthSampling.WaitDepUndulation22 5 minute;set IsothermDepthSampling.WaitDepUndulation23 120 minute" ASAP 2021-04-22T09:43:04.102Z,1619084584.102 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41813, MT Status=1, MTMSN=2993 2021-04-22T09:43:04.871Z,1619084584.871 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation24 5 minute;set IsothermDepthSampling.WaitDepUndulation25 120 minute;set IsothermDepthSampling.WaitDepUndulation26 5 minute;set IsothermDepthSampling.WaitDepUndulation27 120 minute" 20wmj 19 23.000000 2021-04-22T09:43:04.871Z,1619084584.871 [CommandLine](IMPORTANT): Scheduling command #19 of 23 with id=20wmj 2021-04-22T09:43:04.872Z,1619084584.872 [CommandLine](IMPORTANT): Scheduled #120 (#19 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation24 5 minute;set IsothermDepthSampling.WaitDepUndulation25 120 minute;set IsothermDepthSampling.WaitDepUndulation26 5 minute;set IsothermDepthSampling.WaitDepUndulation27 120 minute" ASAP 2021-04-22T09:43:12.989Z,1619084592.989 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41814, MT Status=1, MTMSN=2994 2021-04-22T09:43:13.778Z,1619084593.778 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation28 5 minute;set IsothermDepthSampling.WaitDepUndulation29 120 minute;set IsothermDepthSampling.WaitDepUndulation30 5 minute;set IsothermDepthSampling.WaitDepUndulation31 120 minute" 20wmj 20 23.000000 2021-04-22T09:43:13.778Z,1619084593.778 [CommandLine](IMPORTANT): Scheduling command #20 of 23 with id=20wmj 2021-04-22T09:43:13.779Z,1619084593.779 [CommandLine](IMPORTANT): Scheduled #121 (#20 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation28 5 minute;set IsothermDepthSampling.WaitDepUndulation29 120 minute;set IsothermDepthSampling.WaitDepUndulation30 5 minute;set IsothermDepthSampling.WaitDepUndulation31 120 minute" ASAP 2021-04-22T09:43:29.954Z,1619084609.954 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41815, MT Status=1, MTMSN=2995 2021-04-22T09:43:30.799Z,1619084610.799 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation32 5 minute;set IsothermDepthSampling.WaitDepUndulation33 120 minute;set IsothermDepthSampling.WaitDepUndulation34 5 minute;set IsothermDepthSampling.WaitDepUndulation35 120 minute" 20wmj 21 23.000000 2021-04-22T09:43:30.799Z,1619084610.799 [CommandLine](IMPORTANT): Scheduling command #21 of 23 with id=20wmj 2021-04-22T09:43:30.800Z,1619084610.800 [CommandLine](IMPORTANT): Scheduled #122 (#21 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation32 5 minute;set IsothermDepthSampling.WaitDepUndulation33 120 minute;set IsothermDepthSampling.WaitDepUndulation34 5 minute;set IsothermDepthSampling.WaitDepUndulation35 120 minute" ASAP 2021-04-22T09:43:41.288Z,1619084621.288 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41816, MT Status=1, MTMSN=2996 2021-04-22T09:43:42.106Z,1619084622.106 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation36 5 minute;set IsothermDepthSampling.WaitDepUndulation37 120 minute;set IsothermDepthSampling.WaitDepUndulation38 5 minute;set IsothermDepthSampling.WaitDepUndulation39 120 minute" 20wmj 22 23.000000 2021-04-22T09:43:42.106Z,1619084622.106 [CommandLine](IMPORTANT): Scheduling command #22 of 23 with id=20wmj 2021-04-22T09:43:42.108Z,1619084622.108 [CommandLine](IMPORTANT): Scheduled #123 (#22 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation36 5 minute;set IsothermDepthSampling.WaitDepUndulation37 120 minute;set IsothermDepthSampling.WaitDepUndulation38 5 minute;set IsothermDepthSampling.WaitDepUndulation39 120 minute" ASAP 2021-04-22T09:43:48.963Z,1619084628.963 [NAL9602](IMPORTANT): SBD MO Status=0, MOMSN=41817, MT Status=1, MTMSN=2997 2021-04-22T09:43:49.614Z,1619084629.614 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.WaitDepUndulation40 5 minute;run" 20wmj 23 23.000000 2021-04-22T09:43:49.614Z,1619084629.614 [CommandLine](IMPORTANT): Scheduling command #23 of 23 with id=20wmj 2021-04-22T09:43:49.615Z,1619084629.615 [CommandLine](IMPORTANT): Scheduled #124 (#23 of 23 with id='20wmj'): "set IsothermDepthSampling.WaitDepUndulation40 5 minute;run" ASAP 2021-04-22T09:43:49.670Z,1619084629.670 [CommandLine](IMPORTANT): got command load ./Missions/Science/isotherm_depth_sampling.xml 2021-04-22T09:44:02.996Z,1619084642.996 [CommandLine](IMPORTANT): Loaded ./Missions/Science/isotherm_depth_sampling.xml 2021-04-22T09:44:17.736Z,1619084657.736 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.NumSamplers 40.000000 count 2021-04-22T09:44:17.737Z,1619084657.737 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.MissionTimeout 140.000000 hour 2021-04-22T09:44:17.737Z,1619084657.737 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.AcousticTrackingTimeout 3.000000 hour 2021-04-22T09:44:37.323Z,1619084677.323 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.MaxWaitNoFiring 6.000000 hour 2021-04-22T09:44:37.324Z,1619084677.324 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.SampleAtPeakChl 1.000000 bool 2021-04-22T09:44:37.325Z,1619084677.325 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.ShallowBoundChl 5.000000 meter 2021-04-22T09:44:37.326Z,1619084677.326 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.DeepBoundChl 10.000000 meter 2021-04-22T09:44:56.387Z,1619084696.387 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType1 -6.000000 count 2021-04-22T09:44:56.388Z,1619084696.388 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType2 -9.000000 count 2021-04-22T09:44:56.388Z,1619084696.388 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType3 -6.000000 count 2021-04-22T09:44:56.390Z,1619084696.390 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType4 -9.000000 count 2021-04-22T09:45:15.799Z,1619084715.799 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType5 -6.000000 count 2021-04-22T09:45:15.800Z,1619084715.800 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType6 -9.000000 count 2021-04-22T09:45:15.801Z,1619084715.801 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType7 -6.000000 count 2021-04-22T09:45:15.802Z,1619084715.802 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType8 -9.000000 count 2021-04-22T09:45:34.753Z,1619084734.753 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType9 -6.000000 count 2021-04-22T09:45:34.754Z,1619084734.754 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType10 -9.000000 count 2021-04-22T09:45:34.755Z,1619084734.755 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType11 -6.000000 count 2021-04-22T09:45:34.755Z,1619084734.755 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType12 -9.000000 count 2021-04-22T09:45:53.923Z,1619084753.923 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType13 -6.000000 count 2021-04-22T09:45:53.924Z,1619084753.924 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType14 -9.000000 count 2021-04-22T09:45:53.925Z,1619084753.925 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType15 -6.000000 count 2021-04-22T09:45:53.928Z,1619084753.928 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType16 -9.000000 count 2021-04-22T09:46:12.952Z,1619084772.952 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType17 -6.000000 count 2021-04-22T09:46:12.952Z,1619084772.952 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType18 -9.000000 count 2021-04-22T09:46:12.955Z,1619084772.955 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType19 -6.000000 count 2021-04-22T09:46:12.956Z,1619084772.956 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType20 -9.000000 count 2021-04-22T09:46:31.940Z,1619084791.940 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType21 -6.000000 count 2021-04-22T09:46:31.940Z,1619084791.940 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType22 -9.000000 count 2021-04-22T09:46:31.943Z,1619084791.943 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType23 -6.000000 count 2021-04-22T09:46:31.944Z,1619084791.944 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType24 -9.000000 count 2021-04-22T09:46:50.968Z,1619084810.968 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType25 -6.000000 count 2021-04-22T09:46:50.969Z,1619084810.969 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType26 -9.000000 count 2021-04-22T09:46:50.970Z,1619084810.970 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType27 -6.000000 count 2021-04-22T09:46:50.971Z,1619084810.971 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType28 -9.000000 count 2021-04-22T09:47:09.994Z,1619084829.994 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType29 -6.000000 count 2021-04-22T09:47:09.995Z,1619084829.995 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType30 -9.000000 count 2021-04-22T09:47:09.996Z,1619084829.996 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType31 -6.000000 count 2021-04-22T09:47:09.997Z,1619084829.997 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType32 -9.000000 count 2021-04-22T09:47:29.047Z,1619084849.047 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType33 -6.000000 count 2021-04-22T09:47:29.048Z,1619084849.048 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType34 -9.000000 count 2021-04-22T09:47:29.049Z,1619084849.049 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType35 -6.000000 count 2021-04-22T09:47:29.050Z,1619084849.050 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType36 -9.000000 count 2021-04-22T09:47:48.084Z,1619084868.084 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType37 -6.000000 count 2021-04-22T09:47:48.085Z,1619084868.085 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType38 -9.000000 count 2021-04-22T09:47:48.086Z,1619084868.086 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType39 -6.000000 count 2021-04-22T09:47:48.087Z,1619084868.087 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.CartridgeType40 -9.000000 count 2021-04-22T09:48:09.768Z,1619084889.768 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.DepDiffFromPeakChlCommon 0.000000 meter 2021-04-22T09:48:09.769Z,1619084889.769 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation1 20.000000 minute 2021-04-22T09:48:09.770Z,1619084889.770 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation2 5.000000 minute 2021-04-22T09:48:09.771Z,1619084889.771 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation3 120.000000 minute 2021-04-22T09:48:30.962Z,1619084910.962 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation4 5.000000 minute 2021-04-22T09:48:30.962Z,1619084910.962 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation5 120.000000 minute 2021-04-22T09:48:30.963Z,1619084910.963 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation6 5.000000 minute 2021-04-22T09:48:30.964Z,1619084910.964 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation7 120.000000 minute 2021-04-22T09:48:52.219Z,1619084932.219 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation8 5.000000 minute 2021-04-22T09:48:52.220Z,1619084932.220 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation9 120.000000 minute 2021-04-22T09:48:52.220Z,1619084932.220 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation10 5.000000 minute 2021-04-22T09:48:52.221Z,1619084932.221 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation11 120.000000 minute 2021-04-22T09:49:13.428Z,1619084953.428 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation12 5.000000 minute 2021-04-22T09:49:13.429Z,1619084953.429 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation13 120.000000 minute 2021-04-22T09:49:13.429Z,1619084953.429 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation14 5.000000 minute 2021-04-22T09:49:13.430Z,1619084953.430 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation15 120.000000 minute 2021-04-22T09:49:34.788Z,1619084974.788 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation16 5.000000 minute 2021-04-22T09:49:34.790Z,1619084974.790 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation17 120.000000 minute 2021-04-22T09:49:34.790Z,1619084974.790 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation18 5.000000 minute 2021-04-22T09:49:34.791Z,1619084974.791 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation19 120.000000 minute 2021-04-22T09:49:56.137Z,1619084996.137 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation20 5.000000 minute 2021-04-22T09:49:56.138Z,1619084996.138 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation21 120.000000 minute 2021-04-22T09:49:56.139Z,1619084996.139 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation22 5.000000 minute 2021-04-22T09:49:56.140Z,1619084996.140 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation23 120.000000 minute 2021-04-22T09:49:57.173Z,1619084997.173 [Default:CheckIn:E](IMPORTANT): Default mission has been running for 85.514160 min 2021-04-22T09:50:17.640Z,1619085017.640 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation24 5.000000 minute 2021-04-22T09:50:17.642Z,1619085017.642 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation25 120.000000 minute 2021-04-22T09:50:17.643Z,1619085017.643 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation26 5.000000 minute 2021-04-22T09:50:17.644Z,1619085017.644 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation27 120.000000 minute 2021-04-22T09:50:39.057Z,1619085039.057 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation28 5.000000 minute 2021-04-22T09:50:39.058Z,1619085039.058 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation29 120.000000 minute 2021-04-22T09:50:39.058Z,1619085039.058 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation30 5.000000 minute 2021-04-22T09:50:39.059Z,1619085039.059 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation31 120.000000 minute 2021-04-22T09:51:00.826Z,1619085060.826 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation32 5.000000 minute 2021-04-22T09:51:00.826Z,1619085060.826 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation33 120.000000 minute 2021-04-22T09:51:00.827Z,1619085060.827 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation34 5.000000 minute 2021-04-22T09:51:00.828Z,1619085060.828 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation35 120.000000 minute 2021-04-22T09:51:22.638Z,1619085082.638 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation36 5.000000 minute 2021-04-22T09:51:22.639Z,1619085082.639 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation37 120.000000 minute 2021-04-22T09:51:22.639Z,1619085082.639 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation38 5.000000 minute 2021-04-22T09:51:22.640Z,1619085082.640 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation39 120.000000 minute 2021-04-22T09:51:28.286Z,1619085088.286 [CommandLine](IMPORTANT): got command set IsothermDepthSampling.WaitDepUndulation40 5.000000 minute 2021-04-22T09:51:28.286Z,1619085088.286 [CommandLine](IMPORTANT): got command run 2021-04-22T09:51:28.290Z,1619085088.290 [CommandLine](IMPORTANT): Running 2021-04-22T09:51:28.537Z,1619085088.537 [MissionManager](IMPORTANT): Started mission IsothermDepthSampling 2021-04-22T09:51:28.590Z,1619085088.590 [IsothermDepthSampling:AbortDrift:B.AbortDrift](IMPORTANT): Acoustic timeout set to: 3.00 hours. 2021-04-22T09:52:28.160Z,1619085148.160 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoDown:B](IMPORTANT): Go down from 0.300373 m 2021-04-22T09:54:44.735Z,1619085284.735 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:LogGoUp:B](IMPORTANT): Go up from 13.131887 m 2021-04-22T09:57:37.645Z,1619085457.645 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 20.000000 min . 2021-04-22T09:59:25.588Z,1619085565.588 [IsothermDepthSampling:FindPeakChlVertical:FindPeak:SetChlPeakTempDepth:L](IMPORTANT): Set peak-chl depth and temp. DepthLogged, ChlLogged, PeakChlDepthSet, PeakChlTemperatureSet, PeakChlSet = 7.886116 m , 10.951200 ug/l , 5.606523 m , 11.153425 degC , 13.782972 ug/l 2021-04-22T09:59:25.910Z,1619085565.910 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 1.000000 count . Target depth = 5.606523 m . 2021-04-22T09:59:26.319Z,1619085566.319 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-22T10:01:15.020Z,1619085675.020 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.558457 m , settling for 20.000000 min 2021-04-22T10:21:27.873Z,1619086887.873 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-22T10:21:27.982Z,1619086887.982 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-22T10:22:06.209Z,1619086926.209 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-22T10:22:07.036Z,1619086927.036 [ESPComponent](IMPORTANT): [sample #13] Setting cartridge argument to -6 2021-04-22T10:22:07.037Z,1619086927.037 [ESPComponent](IMPORTANT): [sample #13] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-22T10:22:07.528Z,1619086927.528 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.157090 m in 11.060114 degC water with 12.769120 ug/l chlorophyll fluorescence. 2021-04-22T10:22:07.826Z,1619086927.826 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-22T10:22:08.230Z,1619086928.230 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-22T10:22:10.250Z,1619086930.250 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-22T10:22:49.846Z,1619086969.846 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PRIMING (@42,archive_bac) 2021-04-22T10:22:51.459Z,1619086971.459 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PRIMING (@100.0) 2021-04-22T10:22:52.267Z,1619086972.267 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-22T10:25:41.138Z,1619087141.138 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_FILTERING (@0.0) 2021-04-22T10:25:41.542Z,1619087141.542 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-22T10:30:10.181Z,1619087410.181 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-22T11:06:37.898Z,1619089597.898 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PAUSED (@583.4) 2021-04-22T11:06:38.304Z,1619089598.304 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_PROCESSING (@583.4) 2021-04-22T11:06:40.730Z,1619089600.730 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_WAITING_FOR_PROCESSED (@42,archive_bac) 2021-04-22T11:18:45.909Z,1619090325.909 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_STOPPING (@42,archive_bac) 2021-04-22T11:18:58.839Z,1619090338.839 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-22T11:18:59.242Z,1619090339.242 [ESPComponent](IMPORTANT): [sample #13] ESP sampling state: S_STOPPED 2021-04-22T11:23:59.022Z,1619090639.022 [ESPComponent](IMPORTANT): [sample #13] ESP sampling sequence completed normally. 2021-04-22T11:23:59.422Z,1619090639.422 [ESPComponent](IMPORTANT): [sample #13] ESP log summary report (3 messages): @03:22:23.46 Selecting Cartridge 42 @04:06:34.91 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @04:06:35.11 Sampled 583.4ml 2021-04-22T11:23:59.896Z,1619090639.896 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 1.000000 count completed. 2021-04-22T11:24:01.899Z,1619090641.899 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 1.000000 count 2021-04-22T11:24:03.539Z,1619090643.539 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-22T11:24:04.728Z,1619090644.728 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-22T11:24:06.344Z,1619090646.344 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.312541 m 2021-04-22T11:24:07.955Z,1619090647.955 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 13.852800 ug/l 2021-04-22T11:24:13.173Z,1619090653.173 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T11:24:24.489Z,1619090664.489 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T11:24:24.553Z,1619090664.553 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-22T11:24:28.158Z,1619090668.158 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 2.000000 count . Target depth = 5.606523 m . 2021-04-22T11:24:28.564Z,1619090668.564 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-22T11:24:35.805Z,1619090675.805 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T11:24:47.112Z,1619090687.112 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T11:24:55.228Z,1619090695.228 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.523006 m , settling for 5.000000 min 2021-04-22T11:24:58.428Z,1619090698.428 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T11:30:07.982Z,1619091007.982 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-22T11:30:08.081Z,1619091008.081 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-22T11:30:45.445Z,1619091045.445 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-22T11:30:46.257Z,1619091046.257 [ESPComponent](IMPORTANT): [sample #14] Setting cartridge argument to -9 2021-04-22T11:30:46.257Z,1619091046.257 [ESPComponent](IMPORTANT): [sample #14] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-22T11:30:46.738Z,1619091046.738 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.669298 m in 11.185022 degC water with 14.206400 ug/l chlorophyll fluorescence. 2021-04-22T11:30:47.061Z,1619091047.061 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-22T11:30:47.471Z,1619091047.471 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-22T11:30:49.485Z,1619091049.485 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-22T11:31:29.889Z,1619091089.889 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_PRIMING (@39,daSPR_bac) 2021-04-22T11:31:31.504Z,1619091091.504 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PRIMING (@100.0) 2021-04-22T11:31:32.315Z,1619091092.315 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-22T11:34:23.211Z,1619091263.211 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_FILTERING (@0.0) 2021-04-22T11:34:23.614Z,1619091263.614 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-22T12:03:13.142Z,1619092993.142 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PAUSED (@416.8) 2021-04-22T12:03:13.546Z,1619092993.546 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_PROCESSING (@416.8) 2021-04-22T12:03:15.970Z,1619092995.970 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_WAITING_FOR_PROCESSED (@39,daSPR_bac) 2021-04-22T13:00:20.741Z,1619096420.741 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_STOPPING (@39,daSPR_bac) 2021-04-22T13:00:35.287Z,1619096435.287 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-22T13:00:35.690Z,1619096435.690 [ESPComponent](IMPORTANT): [sample #14] ESP sampling state: S_STOPPED 2021-04-22T13:05:35.462Z,1619096735.462 [ESPComponent](IMPORTANT): [sample #14] ESP sampling sequence completed normally. 2021-04-22T13:05:35.862Z,1619096735.862 [ESPComponent](IMPORTANT): [sample #14] ESP log summary report (5 messages): @04:30:59.79 Selecting Cartridge 39 @05:03:09.89 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @05:03:10.08 Sampled 416.8ml @05:09:11.63 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position @05:29:17.83 Slide::Error in pump -- Slide::Error while holding Archive Syringe (ArS) in position 2021-04-22T13:05:36.349Z,1619096736.349 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 2.000000 count completed. 2021-04-22T13:05:38.349Z,1619096738.349 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 2.000000 count 2021-04-22T13:05:39.979Z,1619096739.979 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-22T13:05:41.204Z,1619096741.204 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-22T13:05:42.796Z,1619096742.796 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.283373 m 2021-04-22T13:05:44.428Z,1619096744.428 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 13.561600 ug/l 2021-04-22T13:05:49.614Z,1619096749.614 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T13:06:00.945Z,1619096760.945 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T13:06:01.008Z,1619096761.008 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-22T13:06:04.607Z,1619096764.607 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 3.000000 count . Target depth = 5.606523 m . 2021-04-22T13:06:04.997Z,1619096764.997 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-22T13:06:12.238Z,1619096772.238 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T13:06:23.554Z,1619096783.554 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T13:06:34.865Z,1619096794.865 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T13:07:02.785Z,1619096822.785 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.662568 m , settling for 120.000000 min 2021-04-22T14:17:36.437Z,1619101056.437 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-22T14:17:47.728Z,1619101067.728 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.007644 CHAN A1 (24V): -0.000293 CHAN A2 (12V): -0.005591 CHAN A3 (5V): -0.002816 CHAN B0 (3.3V): -0.001416 CHAN B1 (3.15aV): -0.001240 CHAN B2 (3.15bV): -0.000988 CHAN B3 (GND): 0.000470 OPEN: 0.007339 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-22T15:07:15.558Z,1619104035.558 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-22T15:07:15.694Z,1619104035.694 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-22T15:07:53.882Z,1619104073.882 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-22T15:07:54.694Z,1619104074.694 [ESPComponent](IMPORTANT): [sample #15] Setting cartridge argument to -6 2021-04-22T15:07:54.694Z,1619104074.694 [ESPComponent](IMPORTANT): [sample #15] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-22T15:07:55.148Z,1619104075.148 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.981091 m in 11.174768 degC water with 14.424800 ug/l chlorophyll fluorescence. 2021-04-22T15:07:55.498Z,1619104075.498 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-22T15:07:55.901Z,1619104075.901 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-22T15:07:57.923Z,1619104077.923 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-22T15:08:37.522Z,1619104117.522 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_PRIMING (@40,archive_bac) 2021-04-22T15:08:39.130Z,1619104119.130 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PRIMING (@100.0) 2021-04-22T15:08:39.942Z,1619104119.942 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-22T15:11:28.002Z,1619104288.002 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_FILTERING (@0.0) 2021-04-22T15:11:28.407Z,1619104288.407 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-22T15:44:09.068Z,1619106249.068 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PAUSED (@470.7) 2021-04-22T15:44:09.470Z,1619106249.470 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_PROCESSING (@470.7) 2021-04-22T15:44:11.894Z,1619106251.894 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_WAITING_FOR_PROCESSED (@40,archive_bac) 2021-04-22T15:56:17.100Z,1619106977.100 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_STOPPING (@40,archive_bac) 2021-04-22T15:56:30.028Z,1619106990.028 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-22T15:56:30.430Z,1619106990.430 [ESPComponent](IMPORTANT): [sample #15] ESP sampling state: S_STOPPED 2021-04-22T16:01:30.202Z,1619107290.202 [ESPComponent](IMPORTANT): [sample #15] ESP sampling sequence completed normally. 2021-04-22T16:01:30.602Z,1619107290.602 [ESPComponent](IMPORTANT): [sample #15] ESP log summary report (3 messages): @08:08:08.78 Selecting Cartridge 40 @08:44:06.38 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @08:44:06.58 Sampled 470.7ml 2021-04-22T16:01:31.078Z,1619107291.078 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 3.000000 count completed. 2021-04-22T16:01:33.102Z,1619107293.102 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 3.000000 count 2021-04-22T16:01:34.715Z,1619107294.715 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-22T16:01:35.934Z,1619107295.934 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-22T16:01:37.558Z,1619107297.558 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.235086 m 2021-04-22T16:01:39.160Z,1619107299.160 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 13.020800 ug/l 2021-04-22T16:01:44.373Z,1619107304.373 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T16:01:55.680Z,1619107315.680 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T16:01:55.711Z,1619107315.711 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-22T16:01:59.395Z,1619107319.395 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 4.000000 count . Target depth = 5.606523 m . 2021-04-22T16:01:59.759Z,1619107319.759 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-22T16:02:06.986Z,1619107326.986 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T16:02:18.299Z,1619107338.299 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T16:02:29.605Z,1619107349.605 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T16:03:08.445Z,1619107388.445 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.699814 m , settling for 5.000000 min 2021-04-22T16:08:21.181Z,1619107701.181 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-22T16:08:21.294Z,1619107701.294 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-22T16:08:58.645Z,1619107738.645 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-22T16:08:59.457Z,1619107739.457 [ESPComponent](IMPORTANT): [sample #16] Setting cartridge argument to -9 2021-04-22T16:08:59.458Z,1619107739.458 [ESPComponent](IMPORTANT): [sample #16] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-22T16:08:59.927Z,1619107739.927 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.744421 m in 11.172968 degC water with 15.479360 ug/l chlorophyll fluorescence. 2021-04-22T16:09:00.261Z,1619107740.261 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-22T16:09:00.666Z,1619107740.666 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-22T16:09:02.686Z,1619107742.686 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-22T16:09:43.090Z,1619107783.090 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_PRIMING (@37,daSPR_bac) 2021-04-22T16:09:44.702Z,1619107784.702 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PRIMING (@100.0) 2021-04-22T16:09:45.510Z,1619107785.510 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-22T16:12:37.617Z,1619107957.617 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_FILTERING (@0.0) 2021-04-22T16:12:38.023Z,1619107958.023 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-22T16:37:58.282Z,1619109478.282 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PAUSED (@369.9) 2021-04-22T16:37:58.686Z,1619109478.686 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_PROCESSING (@369.9) 2021-04-22T16:38:01.116Z,1619109481.116 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_WAITING_FOR_PROCESSED (@37,daSPR_bac) 2021-04-22T17:34:45.670Z,1619112885.670 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_STOPPING (@37,daSPR_bac) 2021-04-22T17:35:00.618Z,1619112900.618 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-22T17:35:01.023Z,1619112901.023 [ESPComponent](IMPORTANT): [sample #16] ESP sampling state: S_STOPPED 2021-04-22T17:40:00.798Z,1619113200.798 [ESPComponent](IMPORTANT): [sample #16] ESP sampling sequence completed normally. 2021-04-22T17:40:01.203Z,1619113201.203 [ESPComponent](IMPORTANT): [sample #16] ESP log summary report (4 messages): @09:09:13.85 Selecting Cartridge 37 @09:37:56.02 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @09:37:56.21 Sampled 369.9ml @09:42:33.62 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position 2021-04-22T17:40:01.678Z,1619113201.678 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 4.000000 count completed. 2021-04-22T17:40:03.700Z,1619113203.700 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 4.000000 count 2021-04-22T17:40:05.305Z,1619113205.305 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-22T17:40:06.516Z,1619113206.516 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-22T17:40:08.140Z,1619113208.140 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 7.414035 m 2021-04-22T17:40:09.756Z,1619113209.756 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 15.007200 ug/l 2021-04-22T17:40:14.963Z,1619113214.963 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T17:40:26.271Z,1619113226.271 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T17:40:26.303Z,1619113226.303 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-22T17:40:29.944Z,1619113229.944 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 5.000000 count . Target depth = 5.606523 m . 2021-04-22T17:40:30.349Z,1619113230.349 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-22T17:40:37.590Z,1619113237.590 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T17:40:48.900Z,1619113248.900 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T17:40:52.594Z,1619113252.594 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.517622 m , settling for 120.000000 min 2021-04-22T17:41:00.211Z,1619113260.211 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T19:41:05.326Z,1619120465.326 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-22T19:41:05.510Z,1619120465.510 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-22T19:41:44.122Z,1619120504.122 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-22T19:41:44.933Z,1619120504.933 [ESPComponent](IMPORTANT): [sample #17] Setting cartridge argument to -6 2021-04-22T19:41:44.933Z,1619120504.933 [ESPComponent](IMPORTANT): [sample #17] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-22T19:41:45.476Z,1619120505.476 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.970770 m in 11.177759 degC water with 18.013840 ug/l chlorophyll fluorescence. 2021-04-22T19:41:45.737Z,1619120505.737 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-22T19:41:46.141Z,1619120506.141 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-22T19:41:48.162Z,1619120508.162 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-22T19:42:27.755Z,1619120547.755 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_PRIMING (@38,archive_bac) 2021-04-22T19:42:29.374Z,1619120549.374 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PRIMING (@100.0) 2021-04-22T19:42:30.179Z,1619120550.179 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-22T19:45:19.057Z,1619120719.057 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_FILTERING (@0.0) 2021-04-22T19:45:19.462Z,1619120719.462 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-22T20:16:53.829Z,1619122613.829 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PAUSED (@451.9) 2021-04-22T20:16:54.238Z,1619122614.238 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_PROCESSING (@451.9) 2021-04-22T20:16:56.658Z,1619122616.658 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_WAITING_FOR_PROCESSED (@38,archive_bac) 2021-04-22T20:17:47.770Z,1619122667.770 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-22T20:17:58.597Z,1619122678.597 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.007083 CHAN A1 (24V): -0.000240 CHAN A2 (12V): -0.004108 CHAN A3 (5V): -0.002224 CHAN B0 (3.3V): -0.001673 CHAN B1 (3.15aV): -0.000302 CHAN B2 (3.15bV): -0.001288 CHAN B3 (GND): 0.001313 OPEN: 0.007746 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-22T20:29:01.858Z,1619123341.858 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_STOPPING (@38,archive_bac) 2021-04-22T20:29:14.786Z,1619123354.786 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-22T20:29:15.190Z,1619123355.190 [ESPComponent](IMPORTANT): [sample #17] ESP sampling state: S_STOPPED 2021-04-22T20:34:14.970Z,1619123654.970 [ESPComponent](IMPORTANT): [sample #17] ESP sampling sequence completed normally. 2021-04-22T20:34:15.374Z,1619123655.374 [ESPComponent](IMPORTANT): [sample #17] ESP log summary report (3 messages): @12:41:58.92 Selecting Cartridge 38 @13:16:51.31 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @13:16:51.51 Sampled 451.9ml 2021-04-22T20:34:15.856Z,1619123655.856 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 5.000000 count completed. 2021-04-22T20:34:17.906Z,1619123657.906 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 5.000000 count 2021-04-22T20:34:19.469Z,1619123659.469 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-22T20:34:20.677Z,1619123660.677 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-22T20:34:22.292Z,1619123662.292 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.828868 m 2021-04-22T20:34:23.906Z,1619123663.906 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 14.664001 ug/l 2021-04-22T20:34:29.137Z,1619123669.137 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T20:34:40.446Z,1619123680.446 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T20:34:40.478Z,1619123680.478 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-22T20:34:44.131Z,1619123684.131 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 6.000000 count . Target depth = 5.606523 m . 2021-04-22T20:34:44.514Z,1619123684.514 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-22T20:34:51.750Z,1619123691.750 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T20:35:03.070Z,1619123703.070 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T20:35:10.398Z,1619123710.398 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.568330 m , settling for 5.000000 min 2021-04-22T20:35:14.382Z,1619123714.382 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T20:40:23.141Z,1619124023.141 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-22T20:40:23.283Z,1619124023.283 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-22T20:41:00.604Z,1619124060.604 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-22T20:41:01.413Z,1619124061.413 [ESPComponent](IMPORTANT): [sample #18] Setting cartridge argument to -9 2021-04-22T20:41:01.413Z,1619124061.413 [ESPComponent](IMPORTANT): [sample #18] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-22T20:41:01.885Z,1619124061.885 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.283103 m in 11.509333 degC water with 18.258240 ug/l chlorophyll fluorescence. 2021-04-22T20:41:02.218Z,1619124062.218 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-22T20:41:02.622Z,1619124062.622 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-22T20:41:04.642Z,1619124064.642 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-22T20:41:45.042Z,1619124105.042 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_PRIMING (@35,daSPR_bac) 2021-04-22T20:41:46.657Z,1619124106.657 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PRIMING (@100.0) 2021-04-22T20:41:47.466Z,1619124107.466 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-22T20:44:39.982Z,1619124279.982 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_FILTERING (@0.0) 2021-04-22T20:44:40.386Z,1619124280.386 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-22T21:13:27.526Z,1619126007.526 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PAUSED (@414.8) 2021-04-22T21:13:27.930Z,1619126007.930 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_PROCESSING (@414.8) 2021-04-22T21:13:30.356Z,1619126010.356 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_WAITING_FOR_PROCESSED (@35,daSPR_bac) 2021-04-22T22:10:54.534Z,1619129454.534 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_STOPPING (@35,daSPR_bac) 2021-04-22T22:11:09.083Z,1619129469.083 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-22T22:11:09.486Z,1619129469.486 [ESPComponent](IMPORTANT): [sample #18] ESP sampling state: S_STOPPED 2021-04-22T22:16:09.269Z,1619129769.269 [ESPComponent](IMPORTANT): [sample #18] ESP sampling sequence completed normally. 2021-04-22T22:16:09.670Z,1619129769.670 [ESPComponent](IMPORTANT): [sample #18] ESP log summary report (3 messages): @13:41:15.92 Selecting Cartridge 35 @14:13:25.33 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @14:13:25.53 Sampled 414.8ml 2021-04-22T22:16:10.137Z,1619129770.137 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 6.000000 count completed. 2021-04-22T22:16:12.164Z,1619129772.164 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 6.000000 count 2021-04-22T22:16:13.779Z,1619129773.779 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-22T22:16:14.988Z,1619129774.988 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-22T22:16:16.635Z,1619129776.635 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.628195 m 2021-04-22T22:16:18.251Z,1619129778.251 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 17.555201 ug/l 2021-04-22T22:16:23.427Z,1619129783.427 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T22:16:34.733Z,1619129794.733 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T22:16:34.799Z,1619129794.799 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-22T22:16:38.401Z,1619129798.401 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 7.000000 count . Target depth = 5.606523 m . 2021-04-22T22:16:38.812Z,1619129798.812 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-22T22:16:46.049Z,1619129806.049 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T22:16:57.362Z,1619129817.362 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T22:17:08.674Z,1619129828.674 [DAT](FAULT): Ack receipt timeout failure. 2021-04-22T22:17:49.925Z,1619129869.925 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.652247 m , settling for 120.000000 min 2021-04-22T22:32:48.066Z,1619130768.066 [ThrusterServo](FAULT): Overload Error 2021-04-23T00:18:02.714Z,1619137082.714 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T00:18:02.818Z,1619137082.818 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T00:18:41.487Z,1619137121.487 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T00:18:42.296Z,1619137122.296 [ESPComponent](IMPORTANT): [sample #19] Setting cartridge argument to -6 2021-04-23T00:18:42.297Z,1619137122.297 [ESPComponent](IMPORTANT): [sample #19] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T00:18:42.771Z,1619137122.771 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.851045 m in 10.978968 degC water with 13.186160 ug/l chlorophyll fluorescence. 2021-04-23T00:18:43.101Z,1619137123.101 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T00:18:43.506Z,1619137123.506 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T00:18:45.526Z,1619137125.526 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T00:19:25.118Z,1619137165.118 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_PRIMING (@36,archive_bac) 2021-04-23T00:19:26.734Z,1619137166.734 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T00:19:27.545Z,1619137167.545 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T00:22:16.010Z,1619137336.010 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T00:22:16.414Z,1619137336.414 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T00:53:45.151Z,1619139225.151 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PAUSED (@452.5) 2021-04-23T00:53:45.550Z,1619139225.550 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_PROCESSING (@452.5) 2021-04-23T00:53:47.974Z,1619139227.974 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_WAITING_FOR_PROCESSED (@36,archive_bac) 2021-04-23T01:05:53.170Z,1619139953.170 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_STOPPING (@36,archive_bac) 2021-04-23T01:06:06.095Z,1619139966.095 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T01:06:06.499Z,1619139966.499 [ESPComponent](IMPORTANT): [sample #19] ESP sampling state: S_STOPPED 2021-04-23T01:11:06.269Z,1619140266.269 [ESPComponent](IMPORTANT): [sample #19] ESP sampling sequence completed normally. 2021-04-23T01:11:06.674Z,1619140266.674 [ESPComponent](IMPORTANT): [sample #19] ESP log summary report (3 messages): @17:18:55.92 Selecting Cartridge 36 @17:53:42.19 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @17:53:42.39 Sampled 452.5ml 2021-04-23T01:11:07.126Z,1619140267.126 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 7.000000 count completed. 2021-04-23T01:11:09.164Z,1619140269.164 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 7.000000 count 2021-04-23T01:11:10.780Z,1619140270.780 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-23T01:11:12.030Z,1619140272.030 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T01:11:13.610Z,1619140273.610 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 7.170816 m 2021-04-23T01:11:15.218Z,1619140275.218 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.887200 ug/l 2021-04-23T01:11:20.430Z,1619140280.430 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T01:11:31.742Z,1619140291.742 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T01:11:31.797Z,1619140291.797 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-23T01:11:35.404Z,1619140295.404 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 8.000000 count . Target depth = 5.606523 m . 2021-04-23T01:11:35.814Z,1619140295.814 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-23T01:11:43.053Z,1619140303.053 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T01:11:54.375Z,1619140314.375 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T01:12:02.122Z,1619140322.122 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.514033 m , settling for 5.000000 min 2021-04-23T01:12:05.687Z,1619140325.687 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T01:17:14.835Z,1619140634.835 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T01:17:14.954Z,1619140634.954 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T01:17:52.699Z,1619140672.699 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T01:17:53.509Z,1619140673.509 [ESPComponent](IMPORTANT): [sample #20] Setting cartridge argument to -9 2021-04-23T01:17:53.510Z,1619140673.510 [ESPComponent](IMPORTANT): [sample #20] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T01:17:53.981Z,1619140673.981 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.407409 m in 11.185419 degC water with 15.679039 ug/l chlorophyll fluorescence. 2021-04-23T01:17:54.313Z,1619140674.313 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T01:17:54.720Z,1619140674.720 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T01:17:56.737Z,1619140676.737 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T01:18:37.138Z,1619140717.138 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_PRIMING (@33,daSPR_bac) 2021-04-23T01:18:38.754Z,1619140718.754 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T01:18:39.563Z,1619140719.563 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T01:21:31.264Z,1619140891.264 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T01:21:31.667Z,1619140891.667 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T01:41:50.942Z,1619142110.942 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PAUSED (@292.7) 2021-04-23T01:41:51.346Z,1619142111.346 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_PROCESSING (@292.7) 2021-04-23T01:41:53.772Z,1619142113.772 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_WAITING_FOR_PROCESSED (@33,daSPR_bac) 2021-04-23T02:17:58.637Z,1619144278.637 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-23T02:18:09.473Z,1619144289.473 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006778 CHAN A1 (24V): 0.000921 CHAN A2 (12V): -0.005517 CHAN A3 (5V): -0.002822 CHAN B0 (3.3V): -0.001113 CHAN B1 (3.15aV): -0.001291 CHAN B2 (3.15bV): -0.001467 CHAN B3 (GND): 0.000100 OPEN: 0.007462 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-23T02:38:33.514Z,1619145513.514 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_STOPPING (@33,daSPR_bac) 2021-04-23T02:38:48.463Z,1619145528.463 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T02:38:48.866Z,1619145528.866 [ESPComponent](IMPORTANT): [sample #20] ESP sampling state: S_STOPPED 2021-04-23T02:43:48.638Z,1619145828.638 [ESPComponent](IMPORTANT): [sample #20] ESP sampling sequence completed normally. 2021-04-23T02:43:49.043Z,1619145829.043 [ESPComponent](IMPORTANT): [sample #20] ESP log summary report (4 messages): @18:18:07.33 Selecting Cartridge 33 @18:41:48.13 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @18:41:48.33 Sampled 292.7ml @19:38:36.58 Slide::Error in LRAUV -- Plunge Valve (PV) positionErr at locked (sensed locked - 79 counts) 2021-04-23T02:43:49.492Z,1619145829.492 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 8.000000 count completed. 2021-04-23T02:43:51.524Z,1619145831.524 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 8.000000 count 2021-04-23T02:43:53.147Z,1619145833.147 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-23T02:43:54.341Z,1619145834.341 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T02:43:55.965Z,1619145835.965 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.759680 m 2021-04-23T02:43:57.593Z,1619145837.593 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.065600 ug/l 2021-04-23T02:44:02.798Z,1619145842.798 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T02:44:14.114Z,1619145854.114 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T02:44:14.169Z,1619145854.169 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-23T02:44:17.777Z,1619145857.777 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 9.000000 count . Target depth = 5.606523 m . 2021-04-23T02:44:18.182Z,1619145858.182 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-23T02:44:25.426Z,1619145865.426 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T02:44:36.734Z,1619145876.734 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T02:44:42.450Z,1619145882.450 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.490249 m , settling for 120.000000 min 2021-04-23T02:44:48.058Z,1619145888.058 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T03:20:07.152Z,1619148007.152 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2021-04-23T03:20:07.541Z,1619148007.541 [ElevatorServo](FAULT): Command Overflow Error 2021-04-23T03:20:08.649Z,1619148008.649 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-23T04:44:55.282Z,1619153095.282 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T04:44:55.431Z,1619153095.431 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T04:45:34.034Z,1619153134.034 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T04:45:34.844Z,1619153134.844 [ESPComponent](IMPORTANT): [sample #21] Setting cartridge argument to -6 2021-04-23T04:45:34.845Z,1619153134.845 [ESPComponent](IMPORTANT): [sample #21] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T04:45:35.338Z,1619153135.338 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.243885 m in 11.060022 degC water with 11.738480 ug/l chlorophyll fluorescence. 2021-04-23T04:45:35.649Z,1619153135.649 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T04:45:36.053Z,1619153136.053 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T04:45:38.074Z,1619153138.074 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T04:46:17.671Z,1619153177.671 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_PRIMING (@34,archive_bac) 2021-04-23T04:46:19.288Z,1619153179.288 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T04:46:20.094Z,1619153180.094 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T04:49:09.374Z,1619153349.374 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T04:49:09.778Z,1619153349.778 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T05:10:45.023Z,1619154645.023 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PAUSED (@309.9) 2021-04-23T05:10:45.426Z,1619154645.426 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_PROCESSING (@309.9) 2021-04-23T05:10:47.850Z,1619154647.850 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_WAITING_FOR_PROCESSED (@34,archive_bac) 2021-04-23T05:22:53.030Z,1619155373.030 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_STOPPING (@34,archive_bac) 2021-04-23T05:23:05.959Z,1619155385.959 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T05:23:06.362Z,1619155386.362 [ESPComponent](IMPORTANT): [sample #21] ESP sampling state: S_STOPPED 2021-04-23T05:28:06.141Z,1619155686.141 [ESPComponent](IMPORTANT): [sample #21] ESP sampling sequence completed normally. 2021-04-23T05:28:06.546Z,1619155686.546 [ESPComponent](IMPORTANT): [sample #21] ESP log summary report (3 messages): @21:45:50.06 Selecting Cartridge 34 @22:10:42.56 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @22:10:42.76 Sampled 309.9ml 2021-04-23T05:28:07.024Z,1619155687.024 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 9.000000 count completed. 2021-04-23T05:28:09.049Z,1619155689.049 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 9.000000 count 2021-04-23T05:28:10.845Z,1619155690.845 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-23T05:28:11.865Z,1619155691.865 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T05:28:13.470Z,1619155693.470 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.180789 m 2021-04-23T05:28:15.118Z,1619155695.118 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 12.708800 ug/l 2021-04-23T05:28:20.297Z,1619155700.297 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T05:28:31.618Z,1619155711.618 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T05:28:31.649Z,1619155711.649 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-23T05:28:35.293Z,1619155715.293 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 10.000000 count . Target depth = 5.606523 m . 2021-04-23T05:28:35.681Z,1619155715.681 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-23T05:28:39.327Z,1619155719.327 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.664814 m , settling for 5.000000 min 2021-04-23T05:28:42.926Z,1619155722.926 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T05:28:54.233Z,1619155734.233 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T05:29:05.562Z,1619155745.562 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T05:33:52.078Z,1619156032.078 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T05:33:52.202Z,1619156032.202 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T05:34:29.977Z,1619156069.977 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T05:34:30.792Z,1619156070.792 [ESPComponent](IMPORTANT): [sample #22] Setting cartridge argument to -9 2021-04-23T05:34:30.792Z,1619156070.792 [ESPComponent](IMPORTANT): [sample #22] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T05:34:31.252Z,1619156071.252 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.887303 m in 11.023218 degC water with 12.602720 ug/l chlorophyll fluorescence. 2021-04-23T05:34:31.594Z,1619156071.594 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T05:34:31.996Z,1619156071.996 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T05:34:34.013Z,1619156074.013 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T05:35:14.422Z,1619156114.422 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_PRIMING (@31,daSPR_bac) 2021-04-23T05:35:16.038Z,1619156116.038 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T05:35:16.846Z,1619156116.846 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T05:39:03.911Z,1619156343.911 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T05:39:04.298Z,1619156344.298 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T06:05:05.378Z,1619157905.378 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PAUSED (@372.4) 2021-04-23T06:05:05.782Z,1619157905.782 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_PROCESSING (@372.4) 2021-04-23T06:05:08.206Z,1619157908.206 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_WAITING_FOR_PROCESSED (@29,daSPR_bac) 2021-04-23T07:04:48.518Z,1619161488.518 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_STOPPING (@29,daSPR_bac) 2021-04-23T07:05:03.467Z,1619161503.467 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T07:05:03.870Z,1619161503.870 [ESPComponent](IMPORTANT): [sample #22] ESP sampling state: S_STOPPED 2021-04-23T07:10:03.641Z,1619161803.641 [ESPComponent](IMPORTANT): [sample #22] ESP sampling sequence completed normally. 2021-04-23T07:10:04.047Z,1619161804.047 [ESPComponent](IMPORTANT): [sample #22] ESP log summary report (9 messages): @22:34:45.04 Selecting Cartridge 31 @22:38:24.03 Cartridge::Sampler::Leaky in FILTERING -- Isolated cartridge pressure fell by 25% to 33.9psia @22:38:24.04 Retry #1 of 2 @22:38:24.06 Cmd::SpareCartridge in FILTERING -- 1 spare remaining @22:38:24.11 Selecting Cartridge 29 @23:05:02.85 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @23:05:03.05 Sampled 372.4ml @23:06:30.43 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position @00:04:54.18 Slide::Error in LRAUV -- Archive Syringe (ArS) positionErr at 955uL (sensed 929uL) 2021-04-23T07:10:04.524Z,1619161804.524 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 10.000000 count completed. 2021-04-23T07:10:06.537Z,1619161806.537 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 10.000000 count 2021-04-23T07:10:08.152Z,1619161808.152 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-23T07:10:09.350Z,1619161809.350 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T07:10:10.974Z,1619161810.974 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.914052 m 2021-04-23T07:10:12.581Z,1619161812.581 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.658400 ug/l 2021-04-23T07:10:17.820Z,1619161817.820 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T07:10:29.117Z,1619161829.117 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T07:10:29.147Z,1619161829.147 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-23T07:10:32.801Z,1619161832.801 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 11.000000 count . Target depth = 5.606523 m . 2021-04-23T07:10:33.200Z,1619161833.200 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-23T07:10:40.432Z,1619161840.432 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T07:10:51.736Z,1619161851.736 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T07:10:58.259Z,1619161858.259 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.499673 m , settling for 120.000000 min 2021-04-23T07:11:03.050Z,1619161863.050 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T08:04:30.454Z,1619165070.454 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-23T08:18:09.595Z,1619165889.595 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-23T08:18:20.424Z,1619165900.424 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006394 CHAN A1 (24V): -0.000287 CHAN A2 (12V): -0.005600 CHAN A3 (5V): -0.002784 CHAN B0 (3.3V): -0.000769 CHAN B1 (3.15aV): -0.001328 CHAN B2 (3.15bV): -0.000997 CHAN B3 (GND): -0.000072 OPEN: 0.007533 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-23T09:11:11.122Z,1619169071.122 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T09:11:11.266Z,1619169071.266 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T09:11:49.825Z,1619169109.825 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T09:11:50.637Z,1619169110.637 [ESPComponent](IMPORTANT): [sample #23] Setting cartridge argument to -6 2021-04-23T09:11:50.638Z,1619169110.638 [ESPComponent](IMPORTANT): [sample #23] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T09:11:51.115Z,1619169111.115 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.578021 m in 10.945673 degC water with 11.617840 ug/l chlorophyll fluorescence. 2021-04-23T09:11:51.441Z,1619169111.441 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T09:11:51.848Z,1619169111.848 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T09:11:53.865Z,1619169113.865 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T09:12:33.458Z,1619169153.458 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_PRIMING (@32,archive_bac) 2021-04-23T09:12:35.074Z,1619169155.074 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T09:12:35.882Z,1619169155.882 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T09:15:24.350Z,1619169324.350 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T09:15:24.754Z,1619169324.754 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T09:19:01.429Z,1619169541.429 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2021-04-23T09:19:02.074Z,1619169542.074 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-23T09:43:41.189Z,1619171021.189 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PAUSED (@408.4) 2021-04-23T09:43:41.594Z,1619171021.594 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_PROCESSING (@408.4) 2021-04-23T09:43:44.019Z,1619171024.019 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_WAITING_FOR_PROCESSED (@32,archive_bac) 2021-04-23T09:55:49.202Z,1619171749.202 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_STOPPING (@32,archive_bac) 2021-04-23T09:56:02.130Z,1619171762.130 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T09:56:02.535Z,1619171762.535 [ESPComponent](IMPORTANT): [sample #23] ESP sampling state: S_STOPPED 2021-04-23T10:01:02.309Z,1619172062.309 [ESPComponent](IMPORTANT): [sample #23] ESP sampling sequence completed normally. 2021-04-23T10:01:02.714Z,1619172062.714 [ESPComponent](IMPORTANT): [sample #23] ESP log summary report (3 messages): @02:12:05.40 Selecting Cartridge 32 @02:43:38.90 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @02:43:39.09 Sampled 408.4ml 2021-04-23T10:01:03.203Z,1619172063.203 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 11.000000 count completed. 2021-04-23T10:01:05.233Z,1619172065.233 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 11.000000 count 2021-04-23T10:01:06.852Z,1619172066.852 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-23T10:01:08.059Z,1619172068.059 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T10:01:09.684Z,1619172069.684 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.582059 m 2021-04-23T10:01:11.294Z,1619172071.294 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.908000 ug/l 2021-04-23T10:01:16.490Z,1619172076.490 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T10:01:27.807Z,1619172087.807 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T10:01:27.864Z,1619172087.864 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-23T10:01:31.460Z,1619172091.460 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 12.000000 count . Target depth = 5.606523 m . 2021-04-23T10:01:31.867Z,1619172091.867 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-23T10:01:39.114Z,1619172099.114 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T10:01:46.445Z,1619172106.445 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.572821 m , settling for 5.000000 min 2021-04-23T10:01:50.425Z,1619172110.425 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T10:02:01.743Z,1619172121.743 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T10:06:59.174Z,1619172419.174 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T10:06:59.286Z,1619172419.286 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T10:09:29.350Z,1619172569.350 [ESPComponent](FAULT): Timeout while waiting for ESP to connect (elapsed=150.0641) 2021-04-23T10:09:41.174Z,1619172581.174 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T10:09:41.272Z,1619172581.272 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T10:10:18.685Z,1619172618.685 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T10:10:19.493Z,1619172619.493 [ESPComponent](IMPORTANT): [sample #24] Setting cartridge argument to -9 2021-04-23T10:10:19.494Z,1619172619.494 [ESPComponent](IMPORTANT): [sample #24] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T10:10:19.971Z,1619172619.971 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.056836 m in 10.943171 degC water with 11.261120 ug/l chlorophyll fluorescence. 2021-04-23T10:10:20.298Z,1619172620.298 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T10:10:20.701Z,1619172620.701 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T10:10:22.722Z,1619172622.722 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T10:11:03.525Z,1619172663.525 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_WAITING_FOR_PRIMING (@27,daSPR_bac) 2021-04-23T10:11:05.142Z,1619172665.142 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T10:11:05.950Z,1619172665.950 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T10:14:04.114Z,1619172844.114 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T10:14:04.517Z,1619172844.517 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T10:44:58.898Z,1619174698.898 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_PAUSED (@454.1) 2021-04-23T10:44:59.302Z,1619174699.302 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_PROCESSING (@454.1) 2021-04-23T10:45:01.726Z,1619174701.726 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_WAITING_FOR_PROCESSED (@27,daSPR_bac) 2021-04-23T10:46:23.313Z,1619174783.313 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-23T11:44:13.762Z,1619178253.762 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_STOPPING (@27,daSPR_bac) 2021-04-23T11:44:28.308Z,1619178268.308 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T11:44:28.764Z,1619178268.764 [ESPComponent](IMPORTANT): [sample #24] ESP sampling state: S_STOPPED 2021-04-23T11:49:28.542Z,1619178568.542 [ESPComponent](IMPORTANT): [sample #24] ESP sampling sequence completed normally. 2021-04-23T11:49:28.946Z,1619178568.946 [ESPComponent](IMPORTANT): [sample #24] ESP log summary report (4 messages): @03:10:33.46 Selecting Cartridge 27 @03:44:56.45 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @03:44:56.65 Sampled 454.1ml @03:47:49.16 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position 2021-04-23T11:49:29.427Z,1619178569.427 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 12.000000 count completed. 2021-04-23T11:49:31.431Z,1619178571.431 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 12.000000 count 2021-04-23T11:49:33.045Z,1619178573.045 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-23T11:49:34.246Z,1619178574.246 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T11:49:35.870Z,1619178575.870 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.289210 m 2021-04-23T11:49:37.489Z,1619178577.489 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.107200 ug/l 2021-04-23T11:49:42.697Z,1619178582.697 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T11:49:54.014Z,1619178594.014 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T11:49:54.045Z,1619178594.045 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-23T11:49:57.696Z,1619178597.696 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 13.000000 count . Target depth = 5.606523 m . 2021-04-23T11:49:58.089Z,1619178598.089 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-23T11:50:05.327Z,1619178605.327 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T11:50:16.648Z,1619178616.648 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T11:50:24.370Z,1619178624.370 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.526600 m , settling for 120.000000 min 2021-04-23T11:50:27.953Z,1619178627.953 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T13:50:37.162Z,1619185837.162 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T13:50:37.282Z,1619185837.282 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T13:51:15.840Z,1619185875.840 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T13:51:16.653Z,1619185876.653 [ESPComponent](IMPORTANT): [sample #25] Setting cartridge argument to -6 2021-04-23T13:51:16.653Z,1619185876.653 [ESPComponent](IMPORTANT): [sample #25] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T13:51:17.109Z,1619185877.109 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.840091 m in 10.629969 degC water with 9.051120 ug/l chlorophyll fluorescence. 2021-04-23T13:51:17.454Z,1619185877.454 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T13:51:17.857Z,1619185877.857 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T13:51:19.878Z,1619185879.878 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T13:51:59.470Z,1619185919.470 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_WAITING_FOR_PRIMING (@30,archive_bac) 2021-04-23T13:52:01.086Z,1619185921.086 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T13:52:01.894Z,1619185921.894 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T13:54:51.574Z,1619186091.574 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T13:54:51.978Z,1619186091.978 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T14:13:46.993Z,1619187226.993 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2021-04-23T14:13:47.638Z,1619187227.638 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-23T14:18:20.549Z,1619187500.549 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-23T14:18:31.409Z,1619187511.409 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.007302 CHAN A1 (24V): -0.000608 CHAN A2 (12V): -0.005508 CHAN A3 (5V): -0.002358 CHAN B0 (3.3V): -0.001252 CHAN B1 (3.15aV): -0.000832 CHAN B2 (3.15bV): -0.001150 CHAN B3 (GND): 0.000756 OPEN: 0.007246 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-23T14:41:26.518Z,1619188886.518 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_PAUSED (@694.0) 2021-04-23T14:41:26.922Z,1619188886.922 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_PROCESSING (@694.0) 2021-04-23T14:41:29.348Z,1619188889.348 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_WAITING_FOR_PROCESSED (@30,archive_bac) 2021-04-23T14:53:34.554Z,1619189614.554 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_STOPPING (@30,archive_bac) 2021-04-23T14:53:47.482Z,1619189627.482 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T14:53:47.950Z,1619189627.950 [ESPComponent](IMPORTANT): [sample #25] ESP sampling state: S_STOPPED 2021-04-23T14:58:47.726Z,1619189927.726 [ESPComponent](IMPORTANT): [sample #25] ESP sampling sequence completed normally. 2021-04-23T14:58:48.132Z,1619189928.132 [ESPComponent](IMPORTANT): [sample #25] ESP log summary report (3 messages): @06:51:31.08 Selecting Cartridge 30 @07:41:24.16 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @07:41:24.36 Sampled 694.0ml 2021-04-23T14:58:48.671Z,1619189928.671 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 13.000000 count completed. 2021-04-23T14:58:50.637Z,1619189930.637 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 13.000000 count 2021-04-23T14:58:52.241Z,1619189932.241 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-23T14:58:53.522Z,1619189933.522 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T14:58:55.094Z,1619189935.094 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.969965 m 2021-04-23T14:58:56.710Z,1619189936.710 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 5.917600 ug/l 2021-04-23T14:59:01.909Z,1619189941.909 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T14:59:13.228Z,1619189953.228 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T14:59:13.258Z,1619189953.258 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-23T14:59:16.897Z,1619189956.897 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 14.000000 count . Target depth = 5.606523 m . 2021-04-23T14:59:17.293Z,1619189957.293 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-23T14:59:24.536Z,1619189964.536 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T14:59:35.852Z,1619189975.852 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T14:59:44.793Z,1619189984.793 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.463326 m , settling for 5.000000 min 2021-04-23T14:59:47.163Z,1619189987.163 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T15:04:57.518Z,1619190297.518 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T15:04:57.659Z,1619190297.659 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T15:07:27.710Z,1619190447.710 [ESPComponent](FAULT): Timeout while waiting for ESP to connect (elapsed=150.0508) 2021-04-23T15:07:39.534Z,1619190459.534 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T15:07:39.659Z,1619190459.659 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T15:08:16.593Z,1619190496.593 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T15:08:17.404Z,1619190497.404 [ESPComponent](IMPORTANT): [sample #26] Setting cartridge argument to -9 2021-04-23T15:08:17.404Z,1619190497.404 [ESPComponent](IMPORTANT): [sample #26] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T15:08:17.862Z,1619190497.862 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.073889 m in 10.302912 degC water with 4.276480 ug/l chlorophyll fluorescence. 2021-04-23T15:08:18.210Z,1619190498.210 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T15:08:18.613Z,1619190498.613 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T15:08:20.636Z,1619190500.636 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T15:09:01.034Z,1619190541.034 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_WAITING_FOR_PRIMING (@25,daSPR_bac) 2021-04-23T15:09:02.650Z,1619190542.650 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T15:09:03.462Z,1619190543.462 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T15:11:55.966Z,1619190715.966 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T15:11:56.370Z,1619190716.370 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T15:46:42.659Z,1619192802.659 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_PAUSED (@503.3) 2021-04-23T15:46:43.061Z,1619192803.061 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_PROCESSING (@503.3) 2021-04-23T15:46:45.485Z,1619192805.485 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_WAITING_FOR_PROCESSED (@25,daSPR_bac) 2021-04-23T16:44:03.163Z,1619196243.163 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_STOPPING (@25,daSPR_bac) 2021-04-23T16:44:18.111Z,1619196258.111 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T16:44:18.514Z,1619196258.514 [ESPComponent](IMPORTANT): [sample #26] ESP sampling state: S_STOPPED 2021-04-23T16:49:18.290Z,1619196558.290 [ESPComponent](IMPORTANT): [sample #26] ESP sampling sequence completed normally. 2021-04-23T16:49:18.694Z,1619196558.694 [ESPComponent](IMPORTANT): [sample #26] ESP log summary report (3 messages): @08:08:31.54 Selecting Cartridge 25 @08:46:40.03 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @08:46:40.23 Sampled 503.3ml 2021-04-23T16:49:19.188Z,1619196559.188 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 14.000000 count completed. 2021-04-23T16:49:21.169Z,1619196561.169 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 14.000000 count 2021-04-23T16:49:22.784Z,1619196562.784 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-23T16:49:24.015Z,1619196564.015 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T16:49:25.644Z,1619196565.644 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.349970 m 2021-04-23T16:49:27.245Z,1619196567.245 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 10.358400 ug/l 2021-04-23T16:49:32.445Z,1619196572.445 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T16:49:43.762Z,1619196583.762 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T16:49:43.824Z,1619196583.824 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-23T16:49:47.428Z,1619196587.428 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 15.000000 count . Target depth = 5.606523 m . 2021-04-23T16:49:47.836Z,1619196587.836 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-23T16:49:55.083Z,1619196595.083 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T16:50:06.390Z,1619196606.390 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T16:50:17.710Z,1619196617.710 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T16:50:59.359Z,1619196659.359 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.655392 m , settling for 120.000000 min 2021-04-23T18:51:12.145Z,1619203872.145 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T18:51:12.289Z,1619203872.289 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T18:51:50.878Z,1619203910.878 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T18:51:51.691Z,1619203911.691 [ESPComponent](IMPORTANT): [sample #27] Setting cartridge argument to -6 2021-04-23T18:51:51.692Z,1619203911.692 [ESPComponent](IMPORTANT): [sample #27] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T18:51:52.162Z,1619203912.162 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.213550 m in 10.886530 degC water with 14.798160 ug/l chlorophyll fluorescence. 2021-04-23T18:51:52.496Z,1619203912.496 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T18:51:52.898Z,1619203912.898 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T18:51:54.918Z,1619203914.918 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T18:52:34.512Z,1619203954.512 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_PRIMING (@28,archive_bac) 2021-04-23T18:52:36.126Z,1619203956.126 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T18:52:36.936Z,1619203956.936 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T18:55:26.214Z,1619204126.214 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T18:55:26.617Z,1619204126.617 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T19:42:39.114Z,1619206959.114 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PAUSED (@679.8) 2021-04-23T19:42:39.518Z,1619206959.518 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_PROCESSING (@679.8) 2021-04-23T19:42:41.945Z,1619206961.945 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_WAITING_FOR_PROCESSED (@28,archive_bac) 2021-04-23T19:54:47.126Z,1619207687.126 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_STOPPING (@28,archive_bac) 2021-04-23T19:55:00.054Z,1619207700.054 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T19:55:00.459Z,1619207700.459 [ESPComponent](IMPORTANT): [sample #27] ESP sampling state: S_STOPPED 2021-04-23T20:00:00.225Z,1619208000.225 [ESPComponent](IMPORTANT): [sample #27] ESP sampling sequence completed normally. 2021-04-23T20:00:00.630Z,1619208000.630 [ESPComponent](IMPORTANT): [sample #27] ESP log summary report (3 messages): @11:52:05.13 Selecting Cartridge 28 @12:42:36.13 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @12:42:36.33 Sampled 679.8ml 2021-04-23T20:00:01.113Z,1619208001.113 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 15.000000 count completed. 2021-04-23T20:00:03.138Z,1619208003.138 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 15.000000 count 2021-04-23T20:00:04.723Z,1619208004.723 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-23T20:00:05.950Z,1619208005.950 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T20:00:07.584Z,1619208007.584 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.211394 m 2021-04-23T20:00:09.165Z,1619208009.165 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 12.771200 ug/l 2021-04-23T20:00:14.382Z,1619208014.382 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T20:00:25.697Z,1619208025.697 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T20:00:25.753Z,1619208025.753 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-23T20:00:29.384Z,1619208029.384 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 16.000000 count . Target depth = 5.606523 m . 2021-04-23T20:00:29.765Z,1619208029.765 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-23T20:00:37.009Z,1619208037.009 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T20:00:48.322Z,1619208048.322 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T20:00:59.633Z,1619208059.633 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T20:01:05.330Z,1619208065.330 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.548590 m , settling for 5.000000 min 2021-04-23T20:06:18.078Z,1619208378.078 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T20:06:18.202Z,1619208378.202 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T20:06:55.949Z,1619208415.949 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T20:06:56.766Z,1619208416.766 [ESPComponent](IMPORTANT): [sample #28] Setting cartridge argument to -9 2021-04-23T20:06:56.767Z,1619208416.767 [ESPComponent](IMPORTANT): [sample #28] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T20:06:57.232Z,1619208417.232 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.179893 m in 11.099390 degC water with 14.085760 ug/l chlorophyll fluorescence. 2021-04-23T20:06:57.565Z,1619208417.565 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T20:06:57.970Z,1619208417.970 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T20:06:59.989Z,1619208419.989 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T20:07:40.794Z,1619208460.794 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_PRIMING (@23,daSPR_bac) 2021-04-23T20:07:42.410Z,1619208462.410 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T20:07:43.223Z,1619208463.223 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T20:10:36.143Z,1619208636.143 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T20:10:36.543Z,1619208636.543 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-23T20:18:31.769Z,1619209111.769 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-23T20:18:42.737Z,1619209122.737 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006938 CHAN A1 (24V): -0.002934 CHAN A2 (12V): -0.005285 CHAN A3 (5V): -0.002452 CHAN B0 (3.3V): -0.000719 CHAN B1 (3.15aV): -0.001105 CHAN B2 (3.15bV): 0.000046 CHAN B3 (GND): 0.000046 OPEN: 0.007475 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-23T20:47:13.922Z,1619210833.922 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PAUSED (@527.0) 2021-04-23T20:47:14.326Z,1619210834.326 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_PROCESSING (@527.0) 2021-04-23T20:47:16.750Z,1619210836.750 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_WAITING_FOR_PROCESSED (@23,daSPR_bac) 2021-04-23T21:48:47.778Z,1619214527.778 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_STOPPING (@23,daSPR_bac) 2021-04-23T21:49:02.331Z,1619214542.331 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-23T21:49:02.730Z,1619214542.730 [ESPComponent](IMPORTANT): [sample #28] ESP sampling state: S_STOPPED 2021-04-23T21:54:02.502Z,1619214842.502 [ESPComponent](IMPORTANT): [sample #28] ESP sampling sequence completed normally. 2021-04-23T21:54:02.906Z,1619214842.906 [ESPComponent](IMPORTANT): [sample #28] ESP log summary report (5 messages): @13:07:10.90 Selecting Cartridge 23 @13:47:11.33 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @13:47:11.53 Sampled 527.0ml @13:53:01.32 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position @14:17:38.74 Slide::Error in pump -- Slide::Error while holding Archive Syringe (ArS) in position 2021-04-23T21:54:03.376Z,1619214843.376 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 16.000000 count completed. 2021-04-23T21:54:05.379Z,1619214845.379 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 16.000000 count 2021-04-23T21:54:07.002Z,1619214847.002 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-23T21:54:08.213Z,1619214848.213 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-23T21:54:09.827Z,1619214849.827 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.703141 m 2021-04-23T21:54:11.440Z,1619214851.440 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 19.188000 ug/l 2021-04-23T21:54:16.671Z,1619214856.671 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T21:54:27.981Z,1619214867.981 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T21:54:28.011Z,1619214868.011 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-23T21:54:31.663Z,1619214871.663 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 17.000000 count . Target depth = 5.606523 m . 2021-04-23T21:54:32.037Z,1619214872.037 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-23T21:54:39.281Z,1619214879.281 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T21:54:50.606Z,1619214890.606 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T21:54:59.148Z,1619214899.148 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.536474 m , settling for 120.000000 min 2021-04-23T21:55:01.913Z,1619214901.913 [DAT](FAULT): Ack receipt timeout failure. 2021-04-23T23:55:11.970Z,1619222111.970 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-23T23:55:12.081Z,1619222112.081 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-23T23:55:51.045Z,1619222151.045 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-23T23:55:51.856Z,1619222151.856 [ESPComponent](IMPORTANT): [sample #29] Setting cartridge argument to -6 2021-04-23T23:55:51.857Z,1619222151.857 [ESPComponent](IMPORTANT): [sample #29] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-23T23:55:52.340Z,1619222152.340 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.244416 m in 10.740015 degC water with 15.747681 ug/l chlorophyll fluorescence. 2021-04-23T23:55:52.662Z,1619222152.662 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-23T23:55:53.066Z,1619222153.066 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-23T23:55:55.086Z,1619222155.086 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-23T23:56:34.734Z,1619222194.734 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PRIMING (@26,archive_bac) 2021-04-23T23:56:36.350Z,1619222196.350 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PRIMING (@100.0) 2021-04-23T23:56:37.159Z,1619222197.159 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-23T23:59:26.845Z,1619222366.845 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_FILTERING (@0.0) 2021-04-23T23:59:27.246Z,1619222367.246 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T00:44:59.130Z,1619225099.130 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PAUSED (@656.7) 2021-04-24T00:44:59.536Z,1619225099.536 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_PROCESSING (@656.7) 2021-04-24T00:45:01.958Z,1619225101.958 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_WAITING_FOR_PROCESSED (@26,archive_bac) 2021-04-24T00:57:07.142Z,1619225827.142 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_STOPPING (@26,archive_bac) 2021-04-24T00:57:20.071Z,1619225840.071 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T00:57:20.474Z,1619225840.474 [ESPComponent](IMPORTANT): [sample #29] ESP sampling state: S_STOPPED 2021-04-24T01:02:20.243Z,1619226140.243 [ESPComponent](IMPORTANT): [sample #29] ESP sampling sequence completed normally. 2021-04-24T01:02:20.646Z,1619226140.646 [ESPComponent](IMPORTANT): [sample #29] ESP log summary report (3 messages): @16:56:05.42 Selecting Cartridge 26 @17:44:56.26 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @17:44:56.46 Sampled 656.7ml 2021-04-24T01:02:21.158Z,1619226141.158 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 17.000000 count completed. 2021-04-24T01:02:23.190Z,1619226143.190 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 17.000000 count 2021-04-24T01:02:24.787Z,1619226144.787 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-24T01:02:26.018Z,1619226146.018 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T01:02:27.637Z,1619226147.637 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 7.168129 m 2021-04-24T01:02:29.259Z,1619226149.259 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 10.712000 ug/l 2021-04-24T01:02:34.454Z,1619226154.454 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T01:02:45.782Z,1619226165.782 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T01:02:45.841Z,1619226165.841 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-24T01:02:49.467Z,1619226169.467 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 18.000000 count . Target depth = 5.606523 m . 2021-04-24T01:02:49.843Z,1619226169.843 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-24T01:02:52.280Z,1619226172.280 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.583594 m , settling for 5.000000 min 2021-04-24T01:02:57.140Z,1619226177.140 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T01:03:08.391Z,1619226188.391 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T01:03:19.706Z,1619226199.706 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T01:08:05.013Z,1619226485.013 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T01:08:05.138Z,1619226485.138 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T01:08:42.940Z,1619226522.940 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T01:08:43.749Z,1619226523.749 [ESPComponent](IMPORTANT): [sample #30] Setting cartridge argument to -9 2021-04-24T01:08:43.755Z,1619226523.755 [ESPComponent](IMPORTANT): [sample #30] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T01:08:44.211Z,1619226524.211 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 3.438102 m in 11.401392 degC water with 18.771999 ug/l chlorophyll fluorescence. 2021-04-24T01:08:44.554Z,1619226524.554 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T01:08:44.958Z,1619226524.958 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T01:08:46.980Z,1619226526.980 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T01:09:27.378Z,1619226567.378 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PRIMING (@21,daSPR_bac) 2021-04-24T01:09:28.994Z,1619226568.994 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T01:09:29.802Z,1619226569.802 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T01:12:22.722Z,1619226742.722 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T01:12:23.126Z,1619226743.126 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T01:44:10.435Z,1619228650.435 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PAUSED (@455.0) 2021-04-24T01:44:10.838Z,1619228650.838 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_PROCESSING (@455.0) 2021-04-24T01:44:13.264Z,1619228653.264 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_WAITING_FOR_PROCESSED (@21,daSPR_bac) 2021-04-24T02:18:42.765Z,1619230722.765 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-24T02:18:53.597Z,1619230733.597 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006967 CHAN A1 (24V): 0.000218 CHAN A2 (12V): -0.005287 CHAN A3 (5V): -0.002422 CHAN B0 (3.3V): -0.000728 CHAN B1 (3.15aV): 0.001538 CHAN B2 (3.15bV): -0.001112 CHAN B3 (GND): 0.000111 OPEN: 0.007383 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-24T02:41:20.542Z,1619232080.542 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_STOPPING (@21,daSPR_bac) 2021-04-24T02:41:35.492Z,1619232095.492 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T02:41:35.894Z,1619232095.894 [ESPComponent](IMPORTANT): [sample #30] ESP sampling state: S_STOPPED 2021-04-24T02:46:35.670Z,1619232395.670 [ESPComponent](IMPORTANT): [sample #30] ESP sampling sequence completed normally. 2021-04-24T02:46:36.075Z,1619232396.075 [ESPComponent](IMPORTANT): [sample #30] ESP log summary report (5 messages): @18:08:57.44 Selecting Cartridge 21 @18:44:07.27 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @18:44:07.47 Sampled 455.0ml @19:08:45.47 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position @19:41:25.66 Slide::Error in LRAUV -- Archive Syringe (ArS) positionErr at 1054uL (sensed 1030uL) 2021-04-24T02:46:36.527Z,1619232396.527 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 18.000000 count completed. 2021-04-24T02:46:38.554Z,1619232398.554 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 18.000000 count 2021-04-24T02:46:40.170Z,1619232400.170 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-24T02:46:41.378Z,1619232401.378 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T02:46:42.997Z,1619232402.997 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 3.640939 m 2021-04-24T02:46:44.605Z,1619232404.605 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 9.994400 ug/l 2021-04-24T02:46:49.829Z,1619232409.829 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T02:47:01.137Z,1619232421.137 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T02:47:01.184Z,1619232421.184 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-24T02:47:04.810Z,1619232424.810 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 19.000000 count . Target depth = 5.606523 m . 2021-04-24T02:47:05.222Z,1619232425.222 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-24T02:47:12.453Z,1619232432.453 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T02:47:13.303Z,1619232433.303 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.650458 m , settling for 120.000000 min 2021-04-24T02:47:23.766Z,1619232443.766 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T02:47:35.077Z,1619232455.077 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T04:47:26.082Z,1619239646.082 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T04:47:26.218Z,1619239646.218 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T04:48:04.813Z,1619239684.813 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T04:48:05.627Z,1619239685.627 [ESPComponent](IMPORTANT): [sample #31] Setting cartridge argument to -6 2021-04-24T04:48:05.628Z,1619239685.628 [ESPComponent](IMPORTANT): [sample #31] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T04:48:06.250Z,1619239686.250 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.520586 m in 10.474665 degC water with 4.318080 ug/l chlorophyll fluorescence. 2021-04-24T04:48:06.575Z,1619239686.575 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T04:48:06.978Z,1619239686.978 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T04:48:08.997Z,1619239688.997 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T04:48:48.590Z,1619239728.590 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PRIMING (@24,archive_bac) 2021-04-24T04:48:50.206Z,1619239730.206 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T04:48:51.014Z,1619239731.014 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T04:51:40.699Z,1619239900.699 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T04:51:41.101Z,1619239901.101 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T05:33:34.438Z,1619242414.438 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PAUSED (@634.5) 2021-04-24T05:33:34.842Z,1619242414.842 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_PROCESSING (@634.5) 2021-04-24T05:33:37.267Z,1619242417.267 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_WAITING_FOR_PROCESSED (@24,archive_bac) 2021-04-24T05:45:42.454Z,1619243142.454 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_STOPPING (@24,archive_bac) 2021-04-24T05:45:55.384Z,1619243155.384 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T05:45:55.785Z,1619243155.785 [ESPComponent](IMPORTANT): [sample #31] ESP sampling state: S_STOPPED 2021-04-24T05:50:55.562Z,1619243455.562 [ESPComponent](IMPORTANT): [sample #31] ESP sampling sequence completed normally. 2021-04-24T05:50:55.966Z,1619243455.966 [ESPComponent](IMPORTANT): [sample #31] ESP log summary report (3 messages): @21:48:19.35 Selecting Cartridge 24 @22:33:31.10 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @22:33:31.29 Sampled 634.5ml 2021-04-24T05:50:56.437Z,1619243456.437 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 19.000000 count completed. 2021-04-24T05:50:58.511Z,1619243458.511 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 19.000000 count 2021-04-24T05:51:00.056Z,1619243460.056 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-24T05:51:01.289Z,1619243461.289 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T05:51:02.900Z,1619243462.900 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 3.557471 m 2021-04-24T05:51:04.530Z,1619243464.530 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.544000 ug/l 2021-04-24T05:51:09.732Z,1619243469.732 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T05:51:21.035Z,1619243481.035 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T05:51:21.065Z,1619243481.065 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-24T05:51:24.700Z,1619243484.700 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 20.000000 count . Target depth = 5.606523 m . 2021-04-24T05:51:25.119Z,1619243485.119 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-24T05:51:32.345Z,1619243492.345 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T05:51:43.665Z,1619243503.665 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T05:51:55.096Z,1619243515.096 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T05:52:23.441Z,1619243543.441 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.693538 m , settling for 5.000000 min 2021-04-24T05:57:36.181Z,1619243856.181 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T05:57:36.289Z,1619243856.289 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T05:58:14.085Z,1619243894.085 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T05:58:14.896Z,1619243894.896 [ESPComponent](IMPORTANT): [sample #32] Setting cartridge argument to -9 2021-04-24T05:58:14.897Z,1619243894.897 [ESPComponent](IMPORTANT): [sample #32] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T05:58:15.379Z,1619243895.379 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.813804 m in 10.914667 degC water with 7.525440 ug/l chlorophyll fluorescence. 2021-04-24T05:58:15.703Z,1619243895.703 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T05:58:16.105Z,1619243896.105 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T05:58:18.126Z,1619243898.126 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T05:58:58.526Z,1619243938.526 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PRIMING (@19,daSPR_bac) 2021-04-24T05:59:00.141Z,1619243940.141 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T05:59:00.950Z,1619243940.950 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T06:01:52.798Z,1619244112.798 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T06:01:53.202Z,1619244113.202 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T06:09:00.454Z,1619244540.454 [ElevatorServo](FAULT): Elevator uart error: serial timeout 2021-04-24T06:33:24.353Z,1619246004.353 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PAUSED (@449.6) 2021-04-24T06:33:24.758Z,1619246004.758 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_PROCESSING (@449.6) 2021-04-24T06:33:27.182Z,1619246007.182 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_WAITING_FOR_PROCESSED (@19,daSPR_bac) 2021-04-24T07:31:39.023Z,1619249499.023 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_STOPPING (@19,daSPR_bac) 2021-04-24T07:31:53.566Z,1619249513.566 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T07:31:53.970Z,1619249513.970 [ESPComponent](IMPORTANT): [sample #32] ESP sampling state: S_STOPPED 2021-04-24T07:36:53.762Z,1619249813.762 [ESPComponent](IMPORTANT): [sample #32] ESP sampling sequence completed normally. 2021-04-24T07:36:54.166Z,1619249814.166 [ESPComponent](IMPORTANT): [sample #32] ESP log summary report (3 messages): @22:58:29.10 Selecting Cartridge 19 @23:33:22.09 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @23:33:22.29 Sampled 449.6ml 2021-04-24T07:36:54.644Z,1619249814.644 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 20.000000 count completed. 2021-04-24T07:36:56.676Z,1619249816.676 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 20.000000 count 2021-04-24T07:36:58.287Z,1619249818.287 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-24T07:36:59.509Z,1619249819.509 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T07:37:01.101Z,1619249821.101 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.922216 m 2021-04-24T07:37:02.725Z,1619249822.725 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 3.712800 ug/l 2021-04-24T07:37:07.949Z,1619249827.949 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T07:37:19.246Z,1619249839.246 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T07:37:19.292Z,1619249839.292 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-24T07:37:22.945Z,1619249842.945 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 21.000000 count . Target depth = 5.606523 m . 2021-04-24T07:37:23.311Z,1619249843.311 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-24T07:37:30.561Z,1619249850.561 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T07:37:41.874Z,1619249861.874 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T07:37:49.590Z,1619249869.590 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.478586 m , settling for 120.000000 min 2021-04-24T07:37:53.191Z,1619249873.191 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T08:18:53.781Z,1619252333.781 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-24T08:19:04.629Z,1619252344.629 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006175 CHAN A1 (24V): -0.000132 CHAN A2 (12V): -0.005229 CHAN A3 (5V): -0.002881 CHAN B0 (3.3V): -0.002608 CHAN B1 (3.15aV): -0.000707 CHAN B2 (3.15bV): -0.001069 CHAN B3 (GND): 0.000297 OPEN: 0.007516 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-24T09:38:02.390Z,1619257082.390 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T09:38:02.539Z,1619257082.539 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T09:38:41.557Z,1619257121.557 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T09:38:42.372Z,1619257122.372 [ESPComponent](IMPORTANT): [sample #33] Setting cartridge argument to -6 2021-04-24T09:38:42.373Z,1619257122.373 [ESPComponent](IMPORTANT): [sample #33] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T09:38:42.860Z,1619257122.860 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.782206 m in 10.182245 degC water with 3.263520 ug/l chlorophyll fluorescence. 2021-04-24T09:38:43.173Z,1619257123.173 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T09:38:43.578Z,1619257123.578 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T09:38:45.598Z,1619257125.598 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T09:39:25.189Z,1619257165.189 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_PRIMING (@22,archive_bac) 2021-04-24T09:39:26.807Z,1619257166.807 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T09:39:27.614Z,1619257167.614 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T09:42:17.294Z,1619257337.294 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T09:42:17.697Z,1619257337.697 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T10:46:51.309Z,1619261211.309 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PAUSED (@1000.0) 2021-04-24T10:46:51.715Z,1619261211.715 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_PROCESSING (@1000.0) 2021-04-24T10:46:54.138Z,1619261214.138 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_WAITING_FOR_PROCESSED (@22,archive_bac) 2021-04-24T10:58:59.326Z,1619261939.326 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_STOPPING (@22,archive_bac) 2021-04-24T10:59:12.663Z,1619261952.663 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T10:59:13.066Z,1619261953.066 [ESPComponent](IMPORTANT): [sample #33] ESP sampling state: S_STOPPED 2021-04-24T11:04:12.838Z,1619262252.838 [ESPComponent](IMPORTANT): [sample #33] ESP sampling sequence completed normally. 2021-04-24T11:04:13.242Z,1619262253.242 [ESPComponent](IMPORTANT): [sample #33] ESP log summary report (2 messages): @02:38:56.39 Selecting Cartridge 22 @03:46:49.28 Sampled 1000.0ml 2021-04-24T11:04:13.759Z,1619262253.759 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 21.000000 count completed. 2021-04-24T11:04:15.719Z,1619262255.719 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 21.000000 count 2021-04-24T11:04:17.334Z,1619262257.334 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-24T11:04:18.578Z,1619262258.578 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T11:04:20.175Z,1619262260.175 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.910996 m 2021-04-24T11:04:21.804Z,1619262261.804 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 2.527200 ug/l 2021-04-24T11:04:26.997Z,1619262266.997 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T11:04:38.311Z,1619262278.311 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T11:04:38.355Z,1619262278.355 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-24T11:04:41.984Z,1619262281.984 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 22.000000 count . Target depth = 5.606523 m . 2021-04-24T11:04:42.392Z,1619262282.392 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-24T11:04:49.623Z,1619262289.623 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T11:05:00.935Z,1619262300.935 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T11:05:10.261Z,1619262310.261 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.452109 m , settling for 5.000000 min 2021-04-24T11:05:12.246Z,1619262312.246 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T11:10:23.025Z,1619262623.025 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T11:10:23.144Z,1619262623.144 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T11:11:01.290Z,1619262661.290 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T11:11:02.100Z,1619262662.100 [ESPComponent](IMPORTANT): [sample #34] Setting cartridge argument to -9 2021-04-24T11:11:02.101Z,1619262662.101 [ESPComponent](IMPORTANT): [sample #34] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T11:11:02.564Z,1619262662.564 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.214448 m in 10.474054 degC water with 7.906080 ug/l chlorophyll fluorescence. 2021-04-24T11:11:02.905Z,1619262662.905 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T11:11:03.310Z,1619262663.310 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T11:11:05.330Z,1619262665.330 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T11:11:45.730Z,1619262705.730 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_PRIMING (@17,daSPR_bac) 2021-04-24T11:11:47.346Z,1619262707.346 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T11:11:48.154Z,1619262708.154 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T11:14:41.476Z,1619262881.476 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T11:14:41.879Z,1619262881.879 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T11:49:17.274Z,1619264957.274 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PAUSED (@480.0) 2021-04-24T11:49:17.678Z,1619264957.678 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_PROCESSING (@480.0) 2021-04-24T11:49:20.101Z,1619264960.101 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_WAITING_FOR_PROCESSED (@17,daSPR_bac) 2021-04-24T11:56:14.590Z,1619265374.590 [ElevatorServo](FAULT): Elevator uart error - getPosition.serial timeout 2021-04-24T12:48:37.862Z,1619268517.862 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_STOPPING (@17,daSPR_bac) 2021-04-24T12:48:52.815Z,1619268532.815 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T12:48:53.214Z,1619268533.214 [ESPComponent](IMPORTANT): [sample #34] ESP sampling state: S_STOPPED 2021-04-24T12:53:52.982Z,1619268832.982 [ESPComponent](IMPORTANT): [sample #34] ESP sampling sequence completed normally. 2021-04-24T12:53:53.386Z,1619268833.386 [ESPComponent](IMPORTANT): [sample #34] ESP log summary report (4 messages): @04:11:15.73 Selecting Cartridge 17 @04:49:14.33 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @04:49:14.52 Sampled 480.0ml @05:48:40.47 Slide::Error in LRAUV -- Plunge Valve (PV) positionErr at locked (sensed locked - 53 counts) 2021-04-24T12:53:53.858Z,1619268833.858 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 22.000000 count completed. 2021-04-24T12:53:55.863Z,1619268835.863 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 22.000000 count 2021-04-24T12:53:57.478Z,1619268837.478 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-24T12:53:58.690Z,1619268838.690 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T12:54:00.305Z,1619268840.305 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.777186 m 2021-04-24T12:54:01.919Z,1619268841.919 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 9.526400 ug/l 2021-04-24T12:54:07.143Z,1619268847.143 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T12:54:18.455Z,1619268858.455 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T12:54:18.484Z,1619268858.484 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-24T12:54:22.138Z,1619268862.138 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 23.000000 count . Target depth = 5.606523 m . 2021-04-24T12:54:22.534Z,1619268862.534 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-24T12:54:29.776Z,1619268869.776 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T12:54:41.085Z,1619268881.085 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T12:54:52.397Z,1619268892.397 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T12:55:20.304Z,1619268920.304 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.685461 m , settling for 120.000000 min 2021-04-24T14:19:04.986Z,1619273944.986 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-24T14:19:15.896Z,1619273955.896 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006456 CHAN A1 (24V): 0.000044 CHAN A2 (12V): -0.005367 CHAN A3 (5V): -0.002713 CHAN B0 (3.3V): -0.001064 CHAN B1 (3.15aV): -0.001184 CHAN B2 (3.15bV): -0.000955 CHAN B3 (GND): 0.000423 OPEN: 0.007633 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-24T14:55:33.099Z,1619276133.099 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T14:55:33.270Z,1619276133.270 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T14:56:12.262Z,1619276172.262 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T14:56:13.073Z,1619276173.073 [ESPComponent](IMPORTANT): [sample #35] Setting cartridge argument to -6 2021-04-24T14:56:13.073Z,1619276173.073 [ESPComponent](IMPORTANT): [sample #35] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T14:56:13.530Z,1619276173.530 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.662390 m in 10.835260 degC water with 9.207120 ug/l chlorophyll fluorescence. 2021-04-24T14:56:13.878Z,1619276173.878 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T14:56:14.282Z,1619276174.282 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T14:56:16.302Z,1619276176.302 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T14:56:55.898Z,1619276215.898 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_PRIMING (@20,archive_bac) 2021-04-24T14:56:57.514Z,1619276217.514 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T14:56:58.322Z,1619276218.322 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T14:59:48.411Z,1619276388.411 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T14:59:48.814Z,1619276388.814 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T15:37:47.390Z,1619278667.390 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PAUSED (@539.9) 2021-04-24T15:37:47.794Z,1619278667.794 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_PROCESSING (@539.9) 2021-04-24T15:37:50.218Z,1619278670.218 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_WAITING_FOR_PROCESSED (@20,archive_bac) 2021-04-24T15:49:55.414Z,1619279395.414 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_STOPPING (@20,archive_bac) 2021-04-24T15:50:08.343Z,1619279408.343 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T15:50:08.748Z,1619279408.748 [ESPComponent](IMPORTANT): [sample #35] ESP sampling state: S_STOPPED 2021-04-24T15:55:08.517Z,1619279708.517 [ESPComponent](IMPORTANT): [sample #35] ESP sampling sequence completed normally. 2021-04-24T15:55:08.922Z,1619279708.922 [ESPComponent](IMPORTANT): [sample #35] ESP log summary report (3 messages): @07:56:28.59 Selecting Cartridge 20 @08:37:45.32 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @08:37:45.52 Sampled 539.9ml 2021-04-24T15:55:09.404Z,1619279709.404 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 23.000000 count completed. 2021-04-24T15:55:11.420Z,1619279711.420 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 23.000000 count 2021-04-24T15:55:13.031Z,1619279713.031 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-24T15:55:14.243Z,1619279714.243 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T15:55:15.853Z,1619279715.853 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.955341 m 2021-04-24T15:55:17.467Z,1619279717.467 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 9.339200 ug/l 2021-04-24T15:55:22.688Z,1619279722.688 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T15:55:33.986Z,1619279733.986 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T15:55:34.044Z,1619279734.044 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-24T15:55:37.679Z,1619279737.679 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 24.000000 count . Target depth = 5.606523 m . 2021-04-24T15:55:38.060Z,1619279738.060 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-24T15:55:45.297Z,1619279745.297 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T15:55:56.620Z,1619279756.620 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T15:56:06.356Z,1619279766.356 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.481728 m , settling for 5.000000 min 2021-04-24T15:56:07.925Z,1619279767.925 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T16:01:19.098Z,1619280079.098 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T16:01:19.208Z,1619280079.208 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T16:01:56.966Z,1619280116.966 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T16:01:57.779Z,1619280117.779 [ESPComponent](IMPORTANT): [sample #36] Setting cartridge argument to -9 2021-04-24T16:01:57.780Z,1619280117.780 [ESPComponent](IMPORTANT): [sample #36] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T16:01:58.251Z,1619280118.251 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.447352 m in 10.869013 degC water with 10.635040 ug/l chlorophyll fluorescence. 2021-04-24T16:01:58.582Z,1619280118.582 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T16:01:58.985Z,1619280118.985 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T16:02:01.005Z,1619280121.005 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T16:02:41.406Z,1619280161.406 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_WAITING_FOR_PRIMING (@15,daSPR_bac) 2021-04-24T16:02:43.022Z,1619280163.022 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T16:02:43.830Z,1619280163.830 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T16:05:36.342Z,1619280336.342 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T16:05:36.745Z,1619280336.745 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T16:33:22.078Z,1619282002.078 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PAUSED (@397.9) 2021-04-24T16:33:22.483Z,1619282002.483 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_PROCESSING (@397.9) 2021-04-24T16:33:24.906Z,1619282004.906 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_WAITING_FOR_PROCESSED (@15,daSPR_bac) 2021-04-24T16:45:40.794Z,1619282740.794 [ThrusterServo](FAULT): getVelocity uart error serial timeout 2021-04-24T16:45:40.794Z,1619282740.794 [ThrusterServo](FAULT): Thruster uart error: serial timeout 2021-04-24T16:45:41.481Z,1619282741.481 [ThrusterServo](FAULT): LCB fault: Software Overcurrent. 2021-04-24T17:29:44.054Z,1619285384.054 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_STOPPING (@15,daSPR_bac) 2021-04-24T17:29:58.599Z,1619285398.599 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T17:29:59.002Z,1619285399.002 [ESPComponent](IMPORTANT): [sample #36] ESP sampling state: S_STOPPED 2021-04-24T17:34:58.787Z,1619285698.787 [ESPComponent](IMPORTANT): [sample #36] ESP sampling sequence completed normally. 2021-04-24T17:34:59.190Z,1619285699.190 [ESPComponent](IMPORTANT): [sample #36] ESP log summary report (3 messages): @09:02:11.39 Selecting Cartridge 15 @09:33:19.02 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @09:33:19.22 Sampled 397.9ml 2021-04-24T17:34:59.660Z,1619285699.660 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 24.000000 count completed. 2021-04-24T17:35:01.684Z,1619285701.684 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 24.000000 count 2021-04-24T17:35:03.280Z,1619285703.280 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-24T17:35:04.521Z,1619285704.521 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T17:35:06.129Z,1619285706.129 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.069501 m 2021-04-24T17:35:07.741Z,1619285707.741 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 12.469600 ug/l 2021-04-24T17:35:12.955Z,1619285712.955 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T17:35:24.259Z,1619285724.259 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T17:35:24.289Z,1619285724.289 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-24T17:35:27.940Z,1619285727.940 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 25.000000 count . Target depth = 5.606523 m . 2021-04-24T17:35:28.339Z,1619285728.339 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-24T17:35:32.010Z,1619285732.010 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.621290 m , settling for 120.000000 min 2021-04-24T17:35:35.575Z,1619285735.575 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T17:35:46.886Z,1619285746.886 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T17:35:58.199Z,1619285758.199 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T19:35:44.749Z,1619292944.749 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T19:35:44.882Z,1619292944.882 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T19:36:23.825Z,1619292983.825 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T19:36:24.637Z,1619292984.637 [ESPComponent](IMPORTANT): [sample #37] Setting cartridge argument to -6 2021-04-24T19:36:24.638Z,1619292984.638 [ESPComponent](IMPORTANT): [sample #37] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T19:36:25.090Z,1619292985.090 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.959829 m in 10.887629 degC water with 15.088320 ug/l chlorophyll fluorescence. 2021-04-24T19:36:25.441Z,1619292985.441 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T19:36:25.846Z,1619292985.846 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T19:36:27.866Z,1619292987.866 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T19:37:07.458Z,1619293027.458 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_WAITING_FOR_PRIMING (@18,archive_bac) 2021-04-24T19:37:09.074Z,1619293029.074 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T19:37:09.882Z,1619293029.882 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T19:39:59.566Z,1619293199.566 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T19:39:59.974Z,1619293199.974 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T20:14:06.666Z,1619295246.666 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_PAUSED (@504.4) 2021-04-24T20:14:07.070Z,1619295247.070 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_PROCESSING (@504.4) 2021-04-24T20:14:09.495Z,1619295249.495 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_WAITING_FOR_PROCESSED (@18,archive_bac) 2021-04-24T20:19:15.993Z,1619295555.993 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-24T20:19:26.833Z,1619295566.833 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.007015 CHAN A1 (24V): -0.000156 CHAN A2 (12V): -0.005347 CHAN A3 (5V): -0.002483 CHAN B0 (3.3V): -0.001440 CHAN B1 (3.15aV): -0.000969 CHAN B2 (3.15bV): -0.001202 CHAN B3 (GND): 0.000468 OPEN: 0.007602 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-24T20:26:14.686Z,1619295974.686 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_STOPPING (@18,archive_bac) 2021-04-24T20:26:27.614Z,1619295987.614 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T20:26:28.022Z,1619295988.022 [ESPComponent](IMPORTANT): [sample #37] ESP sampling state: S_STOPPED 2021-04-24T20:31:27.792Z,1619296287.792 [ESPComponent](IMPORTANT): [sample #37] ESP sampling sequence completed normally. 2021-04-24T20:31:28.195Z,1619296288.195 [ESPComponent](IMPORTANT): [sample #37] ESP log summary report (3 messages): @12:36:38.23 Selecting Cartridge 18 @13:14:03.79 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @13:14:03.99 Sampled 504.4ml 2021-04-24T20:31:28.666Z,1619296288.666 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 25.000000 count completed. 2021-04-24T20:31:30.697Z,1619296290.697 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 25.000000 count 2021-04-24T20:31:32.306Z,1619296292.306 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-24T20:31:33.509Z,1619296293.509 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T20:31:35.135Z,1619296295.135 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.682585 m 2021-04-24T20:31:36.748Z,1619296296.748 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 9.276800 ug/l 2021-04-24T20:31:41.949Z,1619296301.949 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T20:31:53.274Z,1619296313.274 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T20:31:53.330Z,1619296313.330 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-24T20:31:56.935Z,1619296316.935 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 26.000000 count . Target depth = 5.606523 m . 2021-04-24T20:31:57.347Z,1619296317.347 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-24T20:32:04.574Z,1619296324.574 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T20:32:15.886Z,1619296335.886 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T20:32:27.199Z,1619296347.199 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T20:32:55.567Z,1619296375.567 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.650010 m , settling for 5.000000 min 2021-04-24T20:38:08.278Z,1619296688.278 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-24T20:38:08.378Z,1619296688.378 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-24T20:38:46.170Z,1619296726.170 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-24T20:38:46.981Z,1619296726.981 [ESPComponent](IMPORTANT): [sample #38] Setting cartridge argument to -9 2021-04-24T20:38:46.982Z,1619296726.982 [ESPComponent](IMPORTANT): [sample #38] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-24T20:38:47.483Z,1619296727.483 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.063940 m in 10.769312 degC water with 14.256320 ug/l chlorophyll fluorescence. 2021-04-24T20:38:47.791Z,1619296727.791 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-24T20:38:48.190Z,1619296728.190 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-24T20:38:50.210Z,1619296730.210 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-24T20:39:30.609Z,1619296770.609 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_WAITING_FOR_PRIMING (@13,daSPR_bac) 2021-04-24T20:39:32.226Z,1619296772.226 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_PRIMING (@100.0) 2021-04-24T20:39:33.036Z,1619296773.036 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-24T20:42:24.737Z,1619296944.737 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_FILTERING (@0.0) 2021-04-24T20:42:25.141Z,1619296945.141 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-24T20:59:48.462Z,1619297988.462 [ThrusterServo](FAULT): getVelocity uart error serial timeout 2021-04-24T20:59:48.462Z,1619297988.462 [ThrusterServo](FAULT): Thruster uart error: serial timeout 2021-04-24T20:59:49.167Z,1619297989.167 [ThrusterServo](FAULT): LCB fault: Software Overcurrent. 2021-04-24T21:13:02.558Z,1619298782.558 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_PAUSED (@427.5) 2021-04-24T21:13:02.962Z,1619298782.962 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_PROCESSING (@427.5) 2021-04-24T21:13:05.386Z,1619298785.386 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_WAITING_FOR_PROCESSED (@13,daSPR_bac) 2021-04-24T22:12:29.949Z,1619302349.949 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_STOPPING (@13,daSPR_bac) 2021-04-24T22:12:44.903Z,1619302364.903 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-24T22:12:45.306Z,1619302365.306 [ESPComponent](IMPORTANT): [sample #38] ESP sampling state: S_STOPPED 2021-04-24T22:17:45.086Z,1619302665.086 [ESPComponent](IMPORTANT): [sample #38] ESP sampling sequence completed normally. 2021-04-24T22:17:45.490Z,1619302665.490 [ESPComponent](IMPORTANT): [sample #38] ESP log summary report (4 messages): @13:39:01.19 Selecting Cartridge 13 @14:12:59.97 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @14:13:00.17 Sampled 427.5ml @14:19:56.33 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position 2021-04-24T22:17:45.944Z,1619302665.944 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 26.000000 count completed. 2021-04-24T22:17:48.013Z,1619302668.013 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 26.000000 count 2021-04-24T22:17:49.590Z,1619302669.590 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-24T22:17:50.801Z,1619302670.801 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-24T22:17:52.428Z,1619302672.428 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.025348 m 2021-04-24T22:17:54.037Z,1619302674.037 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 12.771200 ug/l 2021-04-24T22:17:59.251Z,1619302679.251 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T22:18:10.560Z,1619302690.560 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T22:18:10.595Z,1619302690.595 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-24T22:18:14.231Z,1619302694.231 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 27.000000 count . Target depth = 5.606523 m . 2021-04-24T22:18:14.652Z,1619302694.652 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-24T22:18:21.870Z,1619302701.870 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T22:18:33.177Z,1619302713.177 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T22:18:44.494Z,1619302724.494 [DAT](FAULT): Ack receipt timeout failure. 2021-04-24T22:18:57.062Z,1619302737.062 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.740210 m , settling for 120.000000 min 2021-04-25T00:19:09.873Z,1619309949.873 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T00:19:10.002Z,1619309950.002 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T00:21:40.080Z,1619310100.080 [ESPComponent](FAULT): Timeout while waiting for ESP to connect (elapsed=150.0780) 2021-04-25T00:21:51.906Z,1619310111.906 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T00:21:52.025Z,1619310112.025 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T00:22:30.170Z,1619310150.170 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T00:22:30.992Z,1619310150.992 [ESPComponent](IMPORTANT): [sample #39] Setting cartridge argument to -6 2021-04-25T00:22:30.992Z,1619310150.992 [ESPComponent](IMPORTANT): [sample #39] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T00:22:31.435Z,1619310151.435 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.332473 m in 11.428675 degC water with 12.593360 ug/l chlorophyll fluorescence. 2021-04-25T00:22:31.785Z,1619310151.785 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T00:22:32.189Z,1619310152.189 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T00:22:34.210Z,1619310154.210 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T00:23:13.807Z,1619310193.807 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_WAITING_FOR_PRIMING (@16,archive_bac) 2021-04-25T00:23:15.425Z,1619310195.425 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T00:23:16.230Z,1619310196.230 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T00:26:05.514Z,1619310365.514 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T00:26:05.919Z,1619310365.919 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T01:11:26.134Z,1619313086.134 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_PAUSED (@667.6) 2021-04-25T01:11:26.542Z,1619313086.542 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_PROCESSING (@667.6) 2021-04-25T01:11:28.963Z,1619313088.963 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_WAITING_FOR_PROCESSED (@16,archive_bac) 2021-04-25T01:23:34.182Z,1619313814.182 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_STOPPING (@16,archive_bac) 2021-04-25T01:23:47.106Z,1619313827.106 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T01:23:47.515Z,1619313827.515 [ESPComponent](IMPORTANT): [sample #39] ESP sampling state: S_STOPPED 2021-04-25T01:28:47.286Z,1619314127.286 [ESPComponent](IMPORTANT): [sample #39] ESP sampling sequence completed normally. 2021-04-25T01:28:47.690Z,1619314127.690 [ESPComponent](IMPORTANT): [sample #39] ESP log summary report (3 messages): @17:22:44.41 Selecting Cartridge 16 @18:11:23.23 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @18:11:23.43 Sampled 667.6ml 2021-04-25T01:28:48.159Z,1619314128.159 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 27.000000 count completed. 2021-04-25T01:28:50.187Z,1619314130.187 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 27.000000 count 2021-04-25T01:28:51.817Z,1619314131.817 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-25T01:28:53.005Z,1619314133.005 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T01:28:54.646Z,1619314134.646 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.177205 m 2021-04-25T01:28:56.233Z,1619314136.233 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 12.854400 ug/l 2021-04-25T01:29:01.455Z,1619314141.455 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T01:29:12.762Z,1619314152.762 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T01:29:12.831Z,1619314152.831 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-25T01:29:16.447Z,1619314156.447 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 28.000000 count . Target depth = 5.606523 m . 2021-04-25T01:29:16.845Z,1619314156.845 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-25T01:29:24.078Z,1619314164.078 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T01:29:35.391Z,1619314175.391 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T01:29:39.894Z,1619314179.894 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.528401 m , settling for 5.000000 min 2021-04-25T01:29:46.701Z,1619314186.701 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T01:34:52.617Z,1619314492.617 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T01:34:52.773Z,1619314492.773 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T01:35:31.724Z,1619314531.724 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T01:35:32.541Z,1619314532.541 [ESPComponent](IMPORTANT): [sample #40] Setting cartridge argument to -9 2021-04-25T01:35:32.542Z,1619314532.542 [ESPComponent](IMPORTANT): [sample #40] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T01:35:32.990Z,1619314532.990 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.288848 m in 11.400568 degC water with 12.433200 ug/l chlorophyll fluorescence. 2021-04-25T01:35:33.338Z,1619314533.338 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T01:35:33.741Z,1619314533.741 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T01:35:35.762Z,1619314535.762 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T01:36:16.162Z,1619314576.162 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_WAITING_FOR_PRIMING (@11,daSPR_bac) 2021-04-25T01:36:17.778Z,1619314577.778 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T01:36:18.590Z,1619314578.590 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T01:39:11.506Z,1619314751.506 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T01:39:11.910Z,1619314751.910 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T02:14:50.706Z,1619316890.706 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_PAUSED (@509.9) 2021-04-25T02:14:51.111Z,1619316891.111 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_PROCESSING (@509.9) 2021-04-25T02:14:53.535Z,1619316893.535 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_WAITING_FOR_PROCESSED (@11,daSPR_bac) 2021-04-25T02:19:27.213Z,1619317167.213 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-25T02:19:38.077Z,1619317178.077 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006341 CHAN A1 (24V): -0.000158 CHAN A2 (12V): -0.005041 CHAN A3 (5V): -0.003663 CHAN B0 (3.3V): -0.001104 CHAN B1 (3.15aV): -0.001447 CHAN B2 (3.15bV): -0.001102 CHAN B3 (GND): 0.000271 OPEN: 0.007337 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-25T03:11:20.334Z,1619320280.334 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_STOPPING (@11,daSPR_bac) 2021-04-25T03:11:34.882Z,1619320294.882 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T03:11:35.283Z,1619320295.283 [ESPComponent](IMPORTANT): [sample #40] ESP sampling state: S_STOPPED 2021-04-25T03:16:35.057Z,1619320595.057 [ESPComponent](IMPORTANT): [sample #40] ESP sampling sequence completed normally. 2021-04-25T03:16:35.466Z,1619320595.466 [ESPComponent](IMPORTANT): [sample #40] ESP log summary report (3 messages): @18:35:46.11 Selecting Cartridge 11 @19:14:47.97 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @19:14:48.16 Sampled 509.9ml 2021-04-25T03:16:35.933Z,1619320595.933 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 28.000000 count completed. 2021-04-25T03:16:37.943Z,1619320597.943 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 28.000000 count 2021-04-25T03:16:39.559Z,1619320599.559 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-25T03:16:40.765Z,1619320600.765 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T03:16:42.382Z,1619320602.382 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 7.311285 m 2021-04-25T03:16:44.026Z,1619320604.026 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 9.204000 ug/l 2021-04-25T03:16:49.217Z,1619320609.217 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T03:17:00.530Z,1619320620.530 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T03:17:00.606Z,1619320620.606 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-25T03:17:04.200Z,1619320624.200 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 29.000000 count . Target depth = 5.606523 m . 2021-04-25T03:17:04.625Z,1619320624.625 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-25T03:17:08.674Z,1619320628.674 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.572380 m , settling for 120.000000 min 2021-04-25T03:17:11.841Z,1619320631.841 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T03:17:23.155Z,1619320643.155 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T03:17:34.470Z,1619320654.470 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T05:17:21.518Z,1619327841.518 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T05:17:21.647Z,1619327841.647 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T05:18:00.657Z,1619327880.657 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T05:18:01.468Z,1619327881.468 [ESPComponent](IMPORTANT): [sample #41] Setting cartridge argument to -6 2021-04-25T05:18:01.469Z,1619327881.469 [ESPComponent](IMPORTANT): [sample #41] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T05:18:01.952Z,1619327881.952 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.430573 m in 11.203821 degC water with 10.151440 ug/l chlorophyll fluorescence. 2021-04-25T05:18:02.274Z,1619327882.274 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T05:18:02.678Z,1619327882.678 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T05:18:04.698Z,1619327884.698 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T05:18:44.430Z,1619327924.430 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_WAITING_FOR_PRIMING (@14,archive_bac) 2021-04-25T05:18:46.046Z,1619327926.046 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T05:18:46.854Z,1619327926.854 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T05:21:35.734Z,1619328095.734 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T05:21:36.138Z,1619328096.138 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T06:23:07.926Z,1619331787.926 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_PAUSED (@915.1) 2021-04-25T06:23:08.330Z,1619331788.330 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_PROCESSING (@915.1) 2021-04-25T06:23:10.754Z,1619331790.754 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_WAITING_FOR_PROCESSED (@14,archive_bac) 2021-04-25T06:35:15.950Z,1619332515.950 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_STOPPING (@14,archive_bac) 2021-04-25T06:35:29.288Z,1619332529.288 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T06:35:29.686Z,1619332529.686 [ESPComponent](IMPORTANT): [sample #41] ESP sampling state: S_STOPPED 2021-04-25T06:40:29.458Z,1619332829.458 [ESPComponent](IMPORTANT): [sample #41] ESP sampling sequence completed normally. 2021-04-25T06:40:29.862Z,1619332829.862 [ESPComponent](IMPORTANT): [sample #41] ESP log summary report (3 messages): @22:18:15.15 Selecting Cartridge 14 @23:23:05.14 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @23:23:05.34 Sampled 915.1ml 2021-04-25T06:40:30.410Z,1619332830.410 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 29.000000 count completed. 2021-04-25T06:40:32.441Z,1619332832.441 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 29.000000 count 2021-04-25T06:40:34.038Z,1619332834.038 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-25T06:40:35.234Z,1619332835.234 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T06:40:36.860Z,1619332836.860 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.618780 m 2021-04-25T06:40:38.444Z,1619332838.444 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 7.945601 ug/l 2021-04-25T06:40:43.671Z,1619332843.671 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T06:40:54.983Z,1619332854.983 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T06:40:55.013Z,1619332855.013 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-25T06:40:58.652Z,1619332858.652 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 30.000000 count . Target depth = 5.606523 m . 2021-04-25T06:40:59.058Z,1619332859.058 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-25T06:41:06.313Z,1619332866.313 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T06:41:17.626Z,1619332877.626 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T06:41:28.959Z,1619332888.959 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T06:41:46.750Z,1619332906.750 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.657642 m , settling for 5.000000 min 2021-04-25T06:46:59.506Z,1619333219.506 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T06:46:59.676Z,1619333219.676 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T06:47:37.414Z,1619333257.414 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T06:47:38.225Z,1619333258.225 [ESPComponent](IMPORTANT): [sample #42] Setting cartridge argument to -9 2021-04-25T06:47:38.225Z,1619333258.225 [ESPComponent](IMPORTANT): [sample #42] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T06:47:38.699Z,1619333258.699 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 4.235543 m in 11.116816 degC water with 9.813440 ug/l chlorophyll fluorescence. 2021-04-25T06:47:39.030Z,1619333259.030 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T06:47:39.438Z,1619333259.438 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T06:47:41.454Z,1619333261.454 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T06:48:21.856Z,1619333301.856 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_WAITING_FOR_PRIMING (@9,daSPR_bac) 2021-04-25T06:48:23.470Z,1619333303.470 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T06:48:24.279Z,1619333304.279 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T06:51:16.790Z,1619333476.790 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T06:51:17.194Z,1619333477.194 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T07:51:41.110Z,1619337101.110 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_PAUSED (@885.4) 2021-04-25T07:51:41.519Z,1619337101.519 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_PROCESSING (@885.4) 2021-04-25T07:51:43.938Z,1619337103.938 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_WAITING_FOR_PROCESSED (@9,daSPR_bac) 2021-04-25T08:19:38.292Z,1619338778.292 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-25T08:19:49.177Z,1619338789.177 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006100 CHAN A1 (24V): 0.001864 CHAN A2 (12V): -0.004370 CHAN A3 (5V): -0.002254 CHAN B0 (3.3V): -0.000798 CHAN B1 (3.15aV): -0.001350 CHAN B2 (3.15bV): -0.001112 CHAN B3 (GND): -0.000342 OPEN: 0.007263 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-25T08:52:17.974Z,1619340737.974 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_STOPPING (@9,daSPR_bac) 2021-04-25T08:52:32.518Z,1619340752.518 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T08:52:32.926Z,1619340752.926 [ESPComponent](IMPORTANT): [sample #42] ESP sampling state: S_STOPPED 2021-04-25T08:57:32.710Z,1619341052.710 [ESPComponent](IMPORTANT): [sample #42] ESP sampling sequence completed normally. 2021-04-25T08:57:33.114Z,1619341053.114 [ESPComponent](IMPORTANT): [sample #42] ESP log summary report (4 messages): @23:47:51.76 Selecting Cartridge 9 @00:51:38.30 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @00:51:38.50 Sampled 885.4ml @00:54:20.00 Slide::Error in PROCESSING -- Slide::Error while holding Archive Syringe (ArS) in position 2021-04-25T08:57:33.570Z,1619341053.570 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 30.000000 count completed. 2021-04-25T08:57:35.599Z,1619341055.599 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 30.000000 count 2021-04-25T08:57:37.213Z,1619341057.213 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-25T08:57:38.414Z,1619341058.414 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T08:57:40.038Z,1619341060.038 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 7.339108 m 2021-04-25T08:57:41.681Z,1619341061.681 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.284000 ug/l 2021-04-25T08:57:46.883Z,1619341066.883 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T08:57:58.185Z,1619341078.185 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T08:57:58.232Z,1619341078.232 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-25T08:58:01.877Z,1619341081.877 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 31.000000 count . Target depth = 5.606523 m . 2021-04-25T08:58:02.272Z,1619341082.272 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-25T08:58:09.507Z,1619341089.507 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T08:58:20.810Z,1619341100.810 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T08:58:26.511Z,1619341106.511 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.554878 m , settling for 120.000000 min 2021-04-25T08:58:32.127Z,1619341112.127 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T10:58:39.326Z,1619348319.326 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T10:58:39.436Z,1619348319.436 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T11:01:09.561Z,1619348469.561 [ESPComponent](FAULT): Timeout while waiting for ESP to connect (elapsed=150.1252) 2021-04-25T11:01:21.386Z,1619348481.386 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T11:01:21.518Z,1619348481.518 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T11:01:59.322Z,1619348519.322 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T11:02:00.134Z,1619348520.134 [ESPComponent](IMPORTANT): [sample #43] Setting cartridge argument to -6 2021-04-25T11:02:00.135Z,1619348520.135 [ESPComponent](IMPORTANT): [sample #43] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T11:02:00.610Z,1619348520.610 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.595266 m in 11.250330 degC water with 10.520640 ug/l chlorophyll fluorescence. 2021-04-25T11:02:00.938Z,1619348520.938 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T11:02:01.342Z,1619348521.342 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T11:02:03.361Z,1619348523.361 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T11:02:42.954Z,1619348562.954 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_WAITING_FOR_PRIMING (@12,archive_bac) 2021-04-25T11:02:44.572Z,1619348564.572 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T11:02:45.378Z,1619348565.378 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T11:05:33.851Z,1619348733.851 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T11:05:34.254Z,1619348734.254 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T11:59:19.857Z,1619351959.857 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_PAUSED (@816.2) 2021-04-25T11:59:20.258Z,1619351960.258 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_PROCESSING (@816.2) 2021-04-25T11:59:22.682Z,1619351962.682 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_WAITING_FOR_PROCESSED (@12,archive_bac) 2021-04-25T12:11:27.876Z,1619352687.876 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_STOPPING (@12,archive_bac) 2021-04-25T12:11:41.211Z,1619352701.211 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T12:11:41.610Z,1619352701.610 [ESPComponent](IMPORTANT): [sample #43] ESP sampling state: S_STOPPED 2021-04-25T12:16:41.393Z,1619353001.393 [ESPComponent](IMPORTANT): [sample #43] ESP sampling sequence completed normally. 2021-04-25T12:16:41.798Z,1619353001.798 [ESPComponent](IMPORTANT): [sample #43] ESP log summary report (3 messages): @04:02:13.75 Selecting Cartridge 12 @04:59:17.15 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @04:59:17.34 Sampled 816.2ml 2021-04-25T12:16:42.278Z,1619353002.278 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 31.000000 count completed. 2021-04-25T12:16:44.285Z,1619353004.285 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 31.000000 count 2021-04-25T12:16:45.921Z,1619353005.921 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-25T12:16:47.105Z,1619353007.105 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T12:16:48.727Z,1619353008.727 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.958038 m 2021-04-25T12:16:50.339Z,1619353010.339 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 11.804000 ug/l 2021-04-25T12:16:55.554Z,1619353015.554 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T12:17:06.872Z,1619353026.872 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T12:17:06.933Z,1619353026.933 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-25T12:17:10.526Z,1619353030.526 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 32.000000 count . Target depth = 5.606523 m . 2021-04-25T12:17:10.945Z,1619353030.945 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-25T12:17:18.173Z,1619353038.173 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T12:17:29.489Z,1619353049.489 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T12:17:37.626Z,1619353057.626 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.547250 m , settling for 5.000000 min 2021-04-25T12:17:40.802Z,1619353060.802 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T12:22:50.366Z,1619353370.366 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T12:22:50.474Z,1619353370.474 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T12:25:20.550Z,1619353520.550 [ESPComponent](FAULT): Timeout while waiting for ESP to connect (elapsed=150.0759) 2021-04-25T12:25:32.374Z,1619353532.374 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T12:25:32.495Z,1619353532.495 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T12:26:10.245Z,1619353570.245 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T12:26:11.056Z,1619353571.056 [ESPComponent](IMPORTANT): [sample #44] Setting cartridge argument to -9 2021-04-25T12:26:11.057Z,1619353571.057 [ESPComponent](IMPORTANT): [sample #44] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T12:26:11.547Z,1619353571.547 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.357957 m in 10.855341 degC water with 8.331440 ug/l chlorophyll fluorescence. 2021-04-25T12:26:11.863Z,1619353571.863 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T12:26:12.265Z,1619353572.265 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T12:26:14.291Z,1619353574.291 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T12:26:54.690Z,1619353614.690 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_WAITING_FOR_PRIMING (@7,daSPR_bac) 2021-04-25T12:26:56.305Z,1619353616.305 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T12:26:57.114Z,1619353617.114 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T12:29:51.642Z,1619353791.642 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T12:29:52.046Z,1619353792.046 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T13:27:26.302Z,1619357246.302 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_PAUSED (@834.3) 2021-04-25T13:27:26.707Z,1619357246.707 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_PROCESSING (@834.3) 2021-04-25T13:27:29.134Z,1619357249.134 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_WAITING_FOR_PROCESSED (@7,daSPR_bac) 2021-04-25T14:17:41.820Z,1619360261.820 [BPC1](FAULT): Battery stick #38 (s/n: 0139) reported TERMINATE_DISCHARGE_ALARM. Status code: 0x4AD0. 2021-04-25T14:19:49.181Z,1619360389.181 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-25T14:20:00.073Z,1619360400.073 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006199 CHAN A1 (24V): 0.000351 CHAN A2 (12V): -0.004675 CHAN A3 (5V): -0.002481 CHAN B0 (3.3V): -0.001025 CHAN B1 (3.15aV): -0.000908 CHAN B2 (3.15bV): -0.001237 CHAN B3 (GND): -0.000142 OPEN: 0.007534 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-25T14:24:13.680Z,1619360653.680 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_STOPPING (@7,daSPR_bac) 2021-04-25T14:24:28.223Z,1619360668.223 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T14:24:28.627Z,1619360668.627 [ESPComponent](IMPORTANT): [sample #44] ESP sampling state: S_STOPPED 2021-04-25T14:29:28.394Z,1619360968.394 [ESPComponent](IMPORTANT): [sample #44] ESP sampling sequence completed normally. 2021-04-25T14:29:28.800Z,1619360968.800 [ESPComponent](IMPORTANT): [sample #44] ESP log summary report (3 messages): @05:26:24.69 Selecting Cartridge 7 @06:27:23.91 Cartridge::Sampler::Clogged in FILTERING -- Cartridge clogged @06:27:24.10 Sampled 834.3ml 2021-04-25T14:29:29.352Z,1619360969.352 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 32.000000 count completed. 2021-04-25T14:29:31.380Z,1619360971.380 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 32.000000 count 2021-04-25T14:29:32.965Z,1619360972.965 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-25T14:29:34.165Z,1619360974.165 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T14:29:35.972Z,1619360975.972 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 6.122550 m 2021-04-25T14:29:37.398Z,1619360977.398 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 7.540000 ug/l 2021-04-25T14:29:42.610Z,1619360982.610 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:29:53.927Z,1619360993.927 [DAT](FAULT): Buffer send receipt timeout failure. 2021-04-25T14:29:53.983Z,1619360993.983 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-25T14:29:57.595Z,1619360997.595 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 33.000000 count . Target depth = 5.606523 m . 2021-04-25T14:29:58.018Z,1619360998.018 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-25T14:30:10.086Z,1619361010.086 [DAT](FAULT): failed to enter online mode 2021-04-25T14:30:10.086Z,1619361010.086 [DAT](FAULT): Failure returning to online mode 2021-04-25T14:30:10.899Z,1619361010.899 [DAT](FAULT): LCB fault: Software Overcurrent. 2021-04-25T14:30:13.785Z,1619361013.785 [CommandLine](IMPORTANT): got command ! echo 1 > /dev/loadB2 2021-04-25T14:30:27.504Z,1619361027.504 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.475451 m , settling for 120.000000 min 2021-04-25T14:30:45.244Z,1619361045.244 [DAT](FAULT): failed to enter command mode 2021-04-25T14:30:58.581Z,1619361058.581 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:31:09.482Z,1619361069.482 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:31:20.404Z,1619361080.404 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:31:31.301Z,1619361091.301 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:31:42.612Z,1619361102.612 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:31:53.923Z,1619361113.923 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:32:05.234Z,1619361125.234 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T14:46:48.082Z,1619362008.082 [ElevatorServo](FAULT): Reading outside of valid range:0.000000 2021-04-25T16:30:40.294Z,1619368240.294 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T16:30:40.481Z,1619368240.481 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T16:31:20.313Z,1619368280.313 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T16:31:21.127Z,1619368281.127 [ESPComponent](IMPORTANT): [sample #45] Setting cartridge argument to -6 2021-04-25T16:31:21.127Z,1619368281.127 [ESPComponent](IMPORTANT): [sample #45] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T16:31:21.617Z,1619368281.617 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 7.480466 m in 10.639490 degC water with 7.566000 ug/l chlorophyll fluorescence. 2021-04-25T16:31:21.930Z,1619368281.930 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T16:31:22.334Z,1619368282.334 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T16:31:24.353Z,1619368284.353 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T16:32:03.946Z,1619368323.946 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_WAITING_FOR_PRIMING (@10,archive_bac) 2021-04-25T16:32:05.646Z,1619368325.646 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T16:32:06.371Z,1619368326.371 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T16:34:56.462Z,1619368496.462 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T16:34:56.867Z,1619368496.867 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T17:28:20.469Z,1619371700.469 [ThrusterServo](FAULT): getVelocity uart error serial timeout 2021-04-25T17:28:20.470Z,1619371700.470 [ThrusterServo](FAULT): Thruster uart error: serial timeout 2021-04-25T17:38:40.410Z,1619372320.410 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_PAUSED (@1000.0) 2021-04-25T17:38:40.814Z,1619372320.814 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_PROCESSING (@1000.0) 2021-04-25T17:38:43.238Z,1619372323.238 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_WAITING_FOR_PROCESSED (@10,archive_bac) 2021-04-25T17:50:48.834Z,1619373048.834 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_STOPPING (@10,archive_bac) 2021-04-25T17:51:01.763Z,1619373061.763 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T17:51:02.170Z,1619373062.170 [ESPComponent](IMPORTANT): [sample #45] ESP sampling state: S_STOPPED 2021-04-25T17:56:01.938Z,1619373361.938 [ESPComponent](IMPORTANT): [sample #45] ESP sampling sequence completed normally. 2021-04-25T17:56:02.342Z,1619373362.342 [ESPComponent](IMPORTANT): [sample #45] ESP log summary report (2 messages): @09:31:34.42 Selecting Cartridge 10 @10:38:38.66 Sampled 1000.0ml 2021-04-25T17:56:02.800Z,1619373362.800 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 33.000000 count completed. 2021-04-25T17:56:04.839Z,1619373364.839 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 33.000000 count 2021-04-25T17:56:06.441Z,1619373366.441 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-25T17:56:07.641Z,1619373367.641 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T17:56:09.264Z,1619373369.264 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.573279 m 2021-04-25T17:56:10.886Z,1619373370.886 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 4.638400 ug/l 2021-04-25T17:56:16.099Z,1619373376.099 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T17:56:27.410Z,1619373387.410 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T17:56:27.466Z,1619373387.466 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-25T17:56:31.079Z,1619373391.079 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 34.000000 count . Target depth = 5.606523 m . 2021-04-25T17:56:31.498Z,1619373391.498 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-25T17:56:39.935Z,1619373399.935 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T17:56:53.265Z,1619373413.265 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T17:57:00.188Z,1619373420.188 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.518082 m , settling for 5.000000 min 2021-04-25T18:02:12.938Z,1619373732.938 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T18:02:13.050Z,1619373733.050 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T18:02:51.666Z,1619373771.666 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T18:02:52.476Z,1619373772.476 [ESPComponent](IMPORTANT): [sample #46] Setting cartridge argument to -9 2021-04-25T18:02:52.477Z,1619373772.477 [ESPComponent](IMPORTANT): [sample #46] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T18:02:52.961Z,1619373772.961 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 5.746048 m in 10.568048 degC water with 7.580560 ug/l chlorophyll fluorescence. 2021-04-25T18:02:53.282Z,1619373773.282 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T18:02:53.686Z,1619373773.686 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T18:02:55.706Z,1619373775.706 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T18:03:36.110Z,1619373816.110 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_WAITING_FOR_PRIMING (@6,daSPR_bac) 2021-04-25T18:03:37.726Z,1619373817.726 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T18:03:38.534Z,1619373818.534 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T18:06:31.854Z,1619373991.854 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T18:06:32.262Z,1619373992.262 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T19:12:28.433Z,1619377948.433 [ThrusterServo](FAULT): Invalid EZ Servo response:"" 2021-04-25T19:12:29.164Z,1619377949.164 [ThrusterServo](FAULT): LCB fault: Software Overcurrent. 2021-04-25T19:13:53.122Z,1619378033.122 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_PAUSED (@1000.0) 2021-04-25T19:13:53.526Z,1619378033.526 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_PROCESSING (@1000.0) 2021-04-25T19:13:55.950Z,1619378035.950 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_WAITING_FOR_PROCESSED (@6,daSPR_bac) 2021-04-25T20:11:50.407Z,1619381510.407 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_STOPPING (@6,daSPR_bac) 2021-04-25T20:12:05.759Z,1619381525.759 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T20:12:06.236Z,1619381526.236 [ESPComponent](IMPORTANT): [sample #46] ESP sampling state: S_STOPPED 2021-04-25T20:17:06.414Z,1619381826.414 [ESPComponent](IMPORTANT): [sample #46] ESP sampling sequence completed normally. 2021-04-25T20:17:06.818Z,1619381826.818 [ESPComponent](IMPORTANT): [sample #46] ESP log summary report (3 messages): @11:03:05.47 Selecting Cartridge 6 @12:13:51.07 Sampled 1000.0ml @13:11:55.53 Slide::Error in LRAUV -- Lysis Syringe #1 (LyS1) positionErr at 1056uL (sensed 1031uL) 2021-04-25T20:17:07.288Z,1619381827.288 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 34.000000 count completed. 2021-04-25T20:17:09.324Z,1619381829.324 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 34.000000 count 2021-04-25T20:17:10.933Z,1619381830.933 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -9 count 2021-04-25T20:17:12.126Z,1619381832.126 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T20:17:13.741Z,1619381833.741 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 4.247212 m 2021-04-25T20:17:15.368Z,1619381835.368 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 8.039200 ug/l 2021-04-25T20:17:20.571Z,1619381840.571 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:17:31.943Z,1619381851.943 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -6.000000 count , nan K , 0.000000 m , nan degC , nan m , 120.000000 min . 2021-04-25T20:17:34.310Z,1619381854.310 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:17:35.567Z,1619381855.567 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 35.000000 count . Target depth = 5.606523 m . 2021-04-25T20:17:35.965Z,1619381855.965 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -6.000000 count 2021-04-25T20:17:47.662Z,1619381867.662 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:17:58.557Z,1619381878.557 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:18:11.477Z,1619381891.477 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:18:22.388Z,1619381902.388 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:18:33.297Z,1619381913.297 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:18:42.221Z,1619381922.221 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.668415 m , settling for 120.000000 min 2021-04-25T20:18:44.208Z,1619381924.208 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:18:55.520Z,1619381935.520 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T20:20:00.361Z,1619382000.361 [CBIT](IMPORTANT): Beginning ground fault scan 2021-04-25T20:20:11.144Z,1619382011.144 [CBIT](IMPORTANT): No ground fault detected mA: CHAN A0 (Batt): -0.006425 CHAN A1 (24V): 0.000347 CHAN A2 (12V): -0.005101 CHAN A3 (5V): -0.002464 CHAN B0 (3.3V): -0.001264 CHAN B1 (3.15aV): -0.000857 CHAN B2 (3.15bV): -0.000875 CHAN B3 (GND): 0.000511 OPEN: 0.007389 Full Scale Calc: 4.765 mA, -1.589 mA 2021-04-25T22:18:54.978Z,1619389134.978 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T22:18:55.090Z,1619389135.090 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T22:19:34.561Z,1619389174.561 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T22:19:35.373Z,1619389175.373 [ESPComponent](IMPORTANT): [sample #47] Setting cartridge argument to -6 2021-04-25T22:19:35.374Z,1619389175.374 [ESPComponent](IMPORTANT): [sample #47] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T22:19:35.846Z,1619389175.846 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.787151 m in 11.088190 degC water with 11.329760 ug/l chlorophyll fluorescence. 2021-04-25T22:19:36.178Z,1619389176.178 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T22:19:36.582Z,1619389176.582 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T22:19:38.602Z,1619389178.602 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T22:20:18.194Z,1619389218.194 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_WAITING_FOR_PRIMING (@8,archive_bac) 2021-04-25T22:20:19.810Z,1619389219.810 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_PRIMING (@100.0) 2021-04-25T22:20:20.624Z,1619389220.624 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_WAITING_FOR_FILTERING (@100.0) 2021-04-25T22:23:12.317Z,1619389392.317 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_FILTERING (@0.0) 2021-04-25T22:23:12.722Z,1619389392.722 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_WAITING_FOR_PAUSED (@0.0) 2021-04-25T23:13:35.861Z,1619392415.861 [ThrusterServo](FAULT): getVelocity uart error serial timeout 2021-04-25T23:13:35.861Z,1619392415.861 [ThrusterServo](FAULT): Thruster uart error: serial timeout 2021-04-25T23:26:30.912Z,1619393190.912 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_PAUSED (@1000.0) 2021-04-25T23:26:31.314Z,1619393191.314 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_PROCESSING (@1000.0) 2021-04-25T23:26:33.737Z,1619393193.737 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_WAITING_FOR_PROCESSED (@8,archive_bac) 2021-04-25T23:38:39.330Z,1619393919.330 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_STOPPING (@8,archive_bac) 2021-04-25T23:38:52.259Z,1619393932.259 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T23:38:52.664Z,1619393932.664 [ESPComponent](IMPORTANT): [sample #47] ESP sampling state: S_STOPPED 2021-04-25T23:43:52.443Z,1619394232.443 [ESPComponent](IMPORTANT): [sample #47] ESP sampling sequence completed normally. 2021-04-25T23:43:52.846Z,1619394232.846 [ESPComponent](IMPORTANT): [sample #47] ESP log summary report (2 messages): @15:19:49.25 Selecting Cartridge 8 @16:26:28.64 Sampled 1000.0ml 2021-04-25T23:43:53.299Z,1619394233.299 [IsothermDepthSampling:TakeSamples:FiringOnDepth:TriggerOnDepth:C](IMPORTANT): Sample No. 35.000000 count completed. 2021-04-25T23:43:55.330Z,1619394235.330 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataCntSamples:A](IMPORTANT): CntSamples = 35.000000 count 2021-04-25T23:43:56.979Z,1619394236.979 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPCartridgeType:A](IMPORTANT): Science:ESPComponent.cartridge = -6 count 2021-04-25T23:43:58.142Z,1619394238.142 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataESPSampleVolume:A](IMPORTANT): Science:ESPComponent.sampleVolume = nan ml 2021-04-25T23:43:59.770Z,1619394239.770 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleDep:A](IMPORTANT): Universal:depth = 5.982093 m 2021-04-25T23:44:01.381Z,1619394241.381 [IsothermDepthSampling:SendData:SendDataTo1Modem:SendDataSampleChl:A](IMPORTANT): Universal:mass_concentration_of_chlorophyll_in_sea_water = 12.053600 ug/l 2021-04-25T23:44:06.603Z,1619394246.603 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T23:44:17.971Z,1619394257.971 [IsothermDepthSampling:SetSampleOptions:N](IMPORTANT): Set CartridgeType (if ESP), TempDiffFromPeakChl, DepDiffFromPeakChl, TempDesignated, DepthDesignated, WaitDepthUndulation to -9.000000 count , nan K , 0.000000 m , nan degC , nan m , 5.000000 min . 2021-04-25T23:44:20.749Z,1619394260.749 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T23:44:21.628Z,1619394261.628 [IsothermDepthSampling:TakeSamples:FiringOnDepth:E](IMPORTANT): Taking sample No. 36.000000 count . Target depth = 5.606523 m . 2021-04-25T23:44:21.994Z,1619394261.994 [IsothermDepthSampling:TakeSamples:FiringOnDepth:F](IMPORTANT): If ESP, requesting ESP cartridge type: -9.000000 count 2021-04-25T23:44:44.578Z,1619394284.578 [DAT](FAULT): Ack receipt timeout failure. 2021-04-25T23:44:49.890Z,1619394289.890 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:B](IMPORTANT): At 5.504622 m , settling for 5.000000 min 2021-04-25T23:50:02.617Z,1619394602.617 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T23:50:02.746Z,1619394602.746 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T23:52:32.870Z,1619394752.870 [ESPComponent](FAULT): Timeout while waiting for ESP to connect (elapsed=150.1241) 2021-04-25T23:52:44.698Z,1619394764.698 [ESPComponent](IMPORTANT): Starting PPPD with command:/sbin/pppd xonxoff asyncmap A0000 /dev/ttyTX1 linkname esp noauth local lock 115200 10.89.5.2:10.89.5.3 persist maxfail 0 holdoff 10 lcp-echo-interval 60 lcp-echo-failure 3 proxyarp ktune deflate 12 ms-dns 8.8.8.8 2021-04-25T23:52:44.839Z,1619394764.839 [ESPComponent](IMPORTANT): Waiting for ESP to connect (timeout=150.0000) 2021-04-25T23:53:22.966Z,1619394802.966 [ESPComponent](IMPORTANT): ESP has connected as client 2021-04-25T23:53:23.776Z,1619394803.776 [ESPComponent](IMPORTANT): [sample #48] Setting cartridge argument to -9 2021-04-25T23:53:23.777Z,1619394803.777 [ESPComponent](IMPORTANT): [sample #48] ESP sampling sequence starting. Sampling state: S_WAITING_INITIAL_PROMPT 2021-04-25T23:53:24.250Z,1619394804.250 [IsothermDepthSampling:SampleAtDepth:SampleWrapper:SampleESP:C](IMPORTANT): ESP sampling at 6.239228 m in 11.235926 degC water with 12.648480 ug/l chlorophyll fluorescence. 2021-04-25T23:53:24.582Z,1619394804.582 [ESPComponent](IMPORTANT): [sample #48] ESP sampling state: S_PREPARING_SHOW_LOG 2021-04-25T23:53:24.985Z,1619394804.985 [ESPComponent](IMPORTANT): [sample #48] ESP sampling state: S_PREPARING_SHOW_STATUS 2021-04-25T23:53:27.006Z,1619394807.006 [ESPComponent](IMPORTANT): [sample #48] ESP sampling state: S_LOADING_CARTRIDGE 2021-04-25T23:53:29.430Z,1619394809.430 [ESPComponent](IMPORTANT): [sample #48] ESP sampling state: S_STOPPING 2021-04-25T23:53:31.855Z,1619394811.855 [ESPComponent](IMPORTANT): Syncing ESP logs with command: rsync -azPq --timeout=60 esp@esp:'/var/log/esp/real /var/log/esp/real.out /var/log/esp/real.slot /var/log/esp/*.spr' /LRAUV/ESPlogs & 2021-04-25T23:53:32.258Z,1619394812.258 [ESPComponent](IMPORTANT): [sample #48] ESP sampling state: S_STOPPED 2021-04-25T23:58:32.026Z,1619395112.026 [ESPComponent](CRITICAL): [sample #48] ESP sampling sequence terminated with error: In state S_LOADING_CARTRIDGE got exception while awaiting result for cmd='Cmd.loadCartridge -9' exception=<>. Elapsed time since command issued: 2.4240 2021-04-25T23:58:32.171Z,1619395112.171 [CommandLine](FAULT): Scheduling is paused 2021-04-25T23:58:32.430Z,1619395112.430 [ESPComponent](IMPORTANT): [sample #48] No ESP summary messages. 2021-04-25T23:58:35.522Z,1619395115.522 [MissionManager](IMPORTANT): Started mission Default 2021-04-26T00:10:06.610Z,1619395806.610 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41840, MT Status=1, MTMSN=2998 2021-04-26T00:10:07.383Z,1619395807.383 [CommandLine](IMPORTANT): got command schedule asap "load Science/profile_station.xml;set profile_station.MissionTimeout 18 hour;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.Radius 200 meter;set profile_station.YoYoMinDepth 3 meter" 27n18 1 2.000000 2021-04-26T00:10:07.384Z,1619395807.384 [CommandLine](IMPORTANT): Scheduling command #1 of 2 with id=27n18 2021-04-26T00:10:07.384Z,1619395807.384 [CommandLine](IMPORTANT): Scheduled #125 (#1 of 2 with id='27n18'): "load Science/profile_station.xml;set profile_station.MissionTimeout 18 hour;set profile_station.Lat 36.797 degree;set profile_station.Lon -121.847 degree;set profile_station.Radius 200 meter;set profile_station.YoYoMinDepth 3 meter" ASAP 2021-04-26T00:10:35.432Z,1619395835.432 [NAL9602](IMPORTANT): SBD MO Status=1, MOMSN=41841, MT Status=1, MTMSN=2999 2021-04-26T00:10:36.174Z,1619395836.174 [CommandLine](IMPORTANT): got command schedule asap "set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.85 meter_per_second;set profile_station.MinAltitude 7 meter;set profile_station.MaxDepth 175 meter;run" 27n18 2 2.000000 2021-04-26T00:10:36.175Z,1619395836.175 [CommandLine](IMPORTANT): Scheduling command #2 of 2 with id=27n18 2021-04-26T00:10:36.176Z,1619395836.176 [CommandLine](IMPORTANT): Scheduled #126 (#2 of 2 with id='27n18'): "set profile_station.YoYoMaxDepth 50 meter;set profile_station.Speed 0.85 meter_per_second;set profile_station.MinAltitude 7 meter;set profile_station.MaxDepth 175 meter;run" ASAP 2021-04-26T00:10:36.356Z,1619395836.356 [CommandLine](IMPORTANT): got command load ./Missions/Science/profile_station.xml 2021-04-26T00:10:39.870Z,1619395839.870 [CommandLine](IMPORTANT): Loaded ./Missions/Science/profile_station.xml 2021-04-26T00:11:09.765Z,1619395869.765 [CommandLine](IMPORTANT): got command set profile_station.MissionTimeout 18.000000 hour 2021-04-26T00:11:09.766Z,1619395869.766 [CommandLine](IMPORTANT): got command set profile_station.Lat 36.797001 degree 2021-04-26T00:11:09.767Z,1619395869.767 [CommandLine](IMPORTANT): got command set profile_station.Lon -121.847000 degree 2021-04-26T00:11:09.768Z,1619395869.768 [CommandLine](IMPORTANT): got command set profile_station.Radius 200.000000 meter 2021-04-26T00:11:09.768Z,1619395869.768 [CommandLine](IMPORTANT): got command set profile_station.YoYoMinDepth 3.000000 meter 2021-04-26T00:11:09.794Z,1619395869.794 [CommandLine](IMPORTANT): got command schedule asap "set IsothermDepthSampling.CartridgeType25 -6 count;set IsothermDepthSampling.CartridgeType26 -9 count;set IsothermDepthSampling.CartridgeType27 -6 count;set IsothermDepthSampling.CartridgeType28 -9 count" 2021-04-26T00:11:09.794Z,1619395869.794 [CommandLine](FAULT): Incomplete syntax. Try: help schedule 2021-04-26T00:11:30.916Z,1619395890.916 [CommandLine](IMPORTANT): got command set profile_station.YoYoMaxDepth 50.000000 meter 2021-04-26T00:11:30.946Z,1619395890.946 [CommandLine](IMPORTANT): got command set profile_station.Speed 0.850000 meter_per_second 2021-04-26T00:11:30.946Z,1619395890.946 [CommandLine](IMPORTANT): got command set profile_station.MinAltitude 7.000000 meter 2021-04-26T00:11:30.947Z,1619395890.947 [CommandLine](IMPORTANT): got command set profile_station.MaxDepth 175.000000 meter 2021-04-26T00:11:30.948Z,1619395890.948 [CommandLine](IMPORTANT): got command run 2021-04-26T00:11:30.951Z,1619395890.951 [CommandLine](IMPORTANT): Running 2021-04-26T00:11:31.103Z,1619395891.103 [MissionManager](IMPORTANT): Started mission profile_station 2021-04-26T00:17:37.707Z,1619396257.707 [DataOverHttps](IMPORTANT): SBD MTMSN=20210426T001736 2021-04-26T00:17:45.653Z,1619396265.653 [CommandLine](IMPORTANT): got command stop 2021-04-26T00:17:45.654Z,1619396265.654 [CommandLine](IMPORTANT): Scheduling is paused 2021-04-26T00:17:46.398Z,1619396266.398 [MissionManager](IMPORTANT): Started mission Default 2021-04-26T00:20:24.963Z,1619396424.963 [DataOverHttps](IMPORTANT): SBD MTMSN=20210426T002023