read_time_series_points returns NaN at the last time step for some time series #511
Description
Mesh Python SDK version: 1.8.0
Mesh version: Not sure how to find out (let me know if it is important)
Python version: 3.11.9
Operating System (Windows, Ubuntu, etc.): Windows
When running the following script:
import datetime
import uuid
import sys
import aa_mesh_client
import pandas as pd
def main():
tz = datetime.timezone(datetime.timedelta(hours=1))
config = {}
config['gyvatn'] = 132864
config['holsmoen'] = 3087
config['hartevatn'] = uuid.UUID('22f9398e-b59d-42b7-b4ed-9be8ee745219')
config['botsvatn'] = uuid.UUID('69f843e2-0f67-4134-ac15-271386c37e2b')
s_time = datetime.datetime(2024, 8, 1, 0, tzinfo=tz)
e_time = datetime.datetime(2024, 9, 2, 2, tzinfo=tz)
df = read_mesh_ts_tims_key(config, s_time, e_time)
df.index = df.index + datetime.timedelta(hours=1)
df.index.names = ['normal_time']
print(df)
def read_mesh_ts_tims_key(config, s_time, e_time):
mesh_client = aa_mesh_client.MeshClient.from_device_flow()
con = mesh_client.mesh_connection # type: mesh.Connection
df = pd.DataFrame()
with con.create_session() as session:
for k, v in config.items():
data = session.read_timeseries_points(
target=v,
start_time=s_time,
end_time=e_time
)
tmp_df = data.arrow_table.to_pandas().set_index('utc_time')
df[k] = tmp_df['value']
return df
if __name__ == '__main__':
main()
I get:
gyvatn holsmoen hartevatn botsvatn
normal_time
2024-08-01 01:00:00 10.6439 7.6064 758.3671 532.5551
2024-08-01 02:00:00 10.7240 7.5966 758.3685 532.5612
2024-08-01 03:00:00 10.2795 7.3516 758.3701 532.5656
2024-08-01 04:00:00 10.0172 6.7281 758.3686 532.5698
2024-08-01 05:00:00 10.2586 6.3728 758.3701 532.5687
... ... ... ... ...
2024-09-01 23:00:00 8.9891 7.1370 757.9425 542.9951
2024-09-02 00:00:00 9.1229 6.6700 757.9359 542.9871
2024-09-02 01:00:00 9.5427 6.3853 757.9244 542.9808
2024-09-02 02:00:00 9.4680 6.1964 757.9182 542.9695
2024-09-02 03:00:00 9.1774 NaN NaN NaN
If i increase e_time by one hour, I get
gyvatn holsmoen hartevatn botsvatn
normal_time
2024-08-01 00:00:00 10.9156 7.9770 758.3683 532.5478
2024-08-01 01:00:00 10.6439 7.6064 758.3671 532.5551
2024-08-01 02:00:00 10.7240 7.5966 758.3685 532.5612
2024-08-01 03:00:00 10.2795 7.3516 758.3701 532.5656
2024-08-01 04:00:00 10.0172 6.7281 758.3686 532.5698
... ... ... ... ...
2024-09-02 00:00:00 9.1229 6.6700 757.9359 542.9871
2024-09-02 01:00:00 9.5427 6.3853 757.9244 542.9808
2024-09-02 02:00:00 9.4680 6.1964 757.9182 542.9695
2024-09-02 03:00:00 9.1774 5.6755 757.9091 542.9630
2024-09-02 04:00:00 8.7272 NaN NaN NaN
which goes to show that there are actually no NaNs in the data at these time stamps.
This seems to me like a bug.