Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sync push #2413

Merged
merged 51 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
ce0a672
feat: update to push endpoint in client-shared
tm-ruxandra Oct 29, 2024
238b2e0
feat: add full action sync methods to sync model
tm-ruxandra Oct 29, 2024
ad4d7a2
feat: skeleton and redux plumbing for sync actions
tm-ruxandra Oct 29, 2024
cda2440
refactor: simplify sync data model to include errors in sync records
tm-ruxandra Oct 29, 2024
fccf09f
feat: add selector for detecting sync errors
tm-ruxandra Oct 29, 2024
5910964
feat: output mapping for soil data push
tm-ruxandra Oct 29, 2024
0376dc7
feat: minimal diff logic to find deleted depth intervals
tm-ruxandra Oct 29, 2024
e130d8c
feat: input mapping for push action
tm-ruxandra Oct 29, 2024
1277a4a
chore: update deps
tm-ruxandra Oct 30, 2024
35d0fed
feat: make push input mapping more defensive
tm-ruxandra Oct 30, 2024
f9ff544
refactor: remove dead code
tm-ruxandra Oct 30, 2024
a3938f6
refactor: better method name for getting change records
tm-ruxandra Oct 30, 2024
e7c206f
feat: integrate sync manager logic
tm-ruxandra Oct 30, 2024
ed6098d
refactor: simplify is-unsynced logic
tm-ruxandra Oct 30, 2024
b8f99d9
refactor: remove dead code
tm-ruxandra Oct 30, 2024
0c0fe0a
fix: only use sync manager with offline flag
tm-ruxandra Oct 30, 2024
9de14ff
chore: update deps and push api interaction
tm-ruxandra Nov 1, 2024
00f0cc9
fix: incorrect caching of soil id change values in redux selectors
tm-ruxandra Nov 4, 2024
ec6a1e5
fix: incorrect dependencies on sync manager
tm-ruxandra Nov 4, 2024
47db965
refactor: rename SyncManager to more specific PushManager
tm-ruxandra Nov 4, 2024
b9b29de
fix: push errors should still mark records as synced
tm-ruxandra Nov 4, 2024
7fd03b3
docs: explain data and error fields in sync record model
tm-ruxandra Nov 4, 2024
875f7ee
feat: retry timer for failed pushes
tm-ruxandra Nov 4, 2024
efc69c9
refactor: remove dead code
tm-ruxandra Nov 5, 2024
bfc9460
refactor: add reusable method for identifying sync error records
tm-ruxandra Nov 5, 2024
a8548d5
refactor: better name for selector
tm-ruxandra Nov 5, 2024
9d15365
refactor: more graceful handling of possibly-undefined soil data in p…
tm-ruxandra Nov 5, 2024
e03b925
fix: correctly record initial server state when loading so diff calcu…
tm-ruxandra Nov 5, 2024
e5a949d
refactor: make data generic parameter name more clear by renaming to D
tm-ruxandra Nov 7, 2024
08c77f1
docs: explain usage of ref in PushDispatcher
tm-ruxandra Nov 7, 2024
deb9d1c
fix: selectSyncErrorSites should be selecting from all changes, not u…
tm-ruxandra Nov 7, 2024
35b3e89
docs: explain significance of stable values in memoized soil change s…
tm-ruxandra Nov 7, 2024
7d40a05
feat: test coverage for soil id selectors
tm-ruxandra Nov 7, 2024
22bd0e9
fix: correctly merge unsynced data with new initial data loaded from …
tm-ruxandra Nov 7, 2024
bd6792b
refactor: better name for operation that merges new data with unsynce…
tm-ruxandra Nov 7, 2024
b34288c
refactor: pull revision id system out of sync system, give it dedicat…
tm-ruxandra Nov 7, 2024
1babd83
refactor: rename confusing "change records" terminology to "sync reco…
tm-ruxandra Nov 8, 2024
fa0098e
refactor: make terminology consistent for marking a sync record as mo…
tm-ruxandra Nov 8, 2024
dc38d10
refactor: extract base syncRecord logic from full sync file
tm-ruxandra Nov 8, 2024
d7f488b
refactor: extract sync result logic to its own file
tm-ruxandra Nov 8, 2024
f1fd990
refactor: move entity-level sync operations to syncRecord.ts
tm-ruxandra Nov 8, 2024
d021cae
refactor: better names for remaining sync model files
tm-ruxandra Nov 8, 2024
94714b5
refactor: clean up syncResults internals
tm-ruxandra Nov 8, 2024
c60d788
docs: add link to redux manual in comment about memoized selectors
tm-ruxandra Nov 8, 2024
a0b52df
refactor: further simplify sync filenames
tm-ruxandra Nov 8, 2024
9837bf0
refactor: break apart PushDispatcher for easier testing
tm-ruxandra Nov 8, 2024
beee2a0
refactor: move selectors integration test to store subdirectory
tm-ruxandra Nov 8, 2024
75fd8e3
fix: incorrect key filtering in getDataForRecords
tm-ruxandra Nov 8, 2024
f83d6e1
refactor: simplify sync model further to remove redundant code
tm-ruxandra Nov 11, 2024
11f8928
refactor: extract common logic from PushDispatcher to hooks, add inte…
tm-ruxandra Nov 11, 2024
99542bd
fix: move PushDispatcher back to store directory to avoid import orde…
tm-ruxandra Nov 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 212 additions & 0 deletions dev-client/__tests__/integration/models/soilId/soilIdSelectors.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
/*
* Copyright © 2023-2024 Technology Matters
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see https://www.gnu.org/licenses/.
*/

import {renderSelectorHook} from '@testing/integration/utils';
import {cloneDeep} from 'lodash';

import {initialState as accountInitialState} from 'terraso-client-shared/account/accountSlice';
import {SoilData} from 'terraso-client-shared/soilId/soilIdTypes';

import {
selectSyncErrorSites,
selectUnsyncedSiteIds,
selectUnsyncedSites,
} from 'terraso-mobile-client/model/soilId/soilIdSelectors';
import {
markEntityError,
markEntityModified,
markEntitySynced,
} from 'terraso-mobile-client/model/sync/records';
import {AppState, useSelector} from 'terraso-mobile-client/store';

const appState = (): AppState => {
return {
account: {...accountInitialState},
map: {userLocation: {accuracyM: null, coords: null}},
elevation: {elevationCache: {}},
notifications: {messages: {}},
preferences: {colorWorkflow: 'MANUAL'},
project: {projects: {}},
site: {sites: {}},
soilId: {
matches: {},
projectSettings: {},
soilSync: {},
soilData: {},
status: 'ready',
},
};
};

const soilData = (): SoilData => {
return {
depthIntervalPreset: 'CUSTOM',
depthDependentData: [],
depthIntervals: [],
};
};

describe('selectUnsyncedSites', () => {
test('selects unsynced sites only', () => {
const state = appState();
const now = Date.now();
markEntitySynced(state.soilId.soilSync, 'a', {value: soilData()}, now);
markEntityModified(state.soilId.soilSync, 'b', now);

const selected = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
state,
);

expect(selected).toEqual({
b: {lastModifiedAt: now, revisionId: 1},
});
});

test('returns stable values for input states only', () => {
const stateA = appState();
markEntityModified(stateA.soilId.soilSync, 'a', Date.now());

const selectedA1 = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
stateA,
);
const selectedA2 = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
stateA,
);

const stateB = cloneDeep(stateA);
markEntityModified(stateB.soilId.soilSync, 'b', Date.now());

const selectedB = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
stateB,
);

expect(selectedA1).toBe(selectedA2);
expect(selectedA1).not.toBe(selectedB);
expect(selectedA2).not.toBe(selectedB);
});
});

describe('selectUnsyncedSiteIds', () => {
test('selects unsynced site IDs only, sorted', () => {
const state = appState();
const now = Date.now();
markEntitySynced(state.soilId.soilSync, 'a', {value: soilData()}, now);

markEntityModified(state.soilId.soilSync, 'c', now);
markEntityModified(state.soilId.soilSync, 'b', now);

const selected = renderSelectorHook(
() => useSelector(selectUnsyncedSiteIds),
state,
);

expect(selected).toEqual(['b', 'c']);
});

test('returns stable values for input states', () => {
const stateA = appState();
markEntityModified(stateA.soilId.soilSync, 'a', Date.now());

const selectedA1 = renderSelectorHook(
() => useSelector(selectUnsyncedSiteIds),
stateA,
);
const selectedA2 = renderSelectorHook(
() => useSelector(selectUnsyncedSiteIds),
stateA,
);

const stateB = cloneDeep(stateA);
markEntityModified(stateB.soilId.soilSync, 'b', Date.now());

const selectedB = renderSelectorHook(
() => useSelector(selectUnsyncedSiteIds),
stateB,
);

expect(selectedA1).toBe(selectedA2);
expect(selectedA1).not.toBe(selectedB);
expect(selectedA2).not.toBe(selectedB);
});
});

describe('selectSyncErrorSites', () => {
test('selects sync error sites only', () => {
const state = appState();
const now = Date.now();
markEntitySynced(state.soilId.soilSync, 'a', {value: soilData()}, now);
markEntityError(
state.soilId.soilSync,
'b',
{revisionId: 1, value: 'DOES_NOT_EXIST'},
now,
);

const selected = renderSelectorHook(
() => useSelector(selectSyncErrorSites),
state,
);

expect(selected).toEqual({
b: {
lastSyncedAt: now,
lastSyncedRevisionId: 1,
lastSyncedError: 'DOES_NOT_EXIST',
},
});
});

test('returns stable values for input states', () => {
const stateA = appState();
markEntityError(
stateA.soilId.soilSync,
'a',
{value: 'DOES_NOT_EXIST'},
Date.now(),
);

const selectedA1 = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
stateA,
);
const selectedA2 = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
stateA,
);

const stateB = cloneDeep(stateA);
markEntityError(
stateB.soilId.soilSync,
'b',
{revisionId: 1, value: 'DOES_NOT_EXIST'},
Date.now(),
);

const selectedB = renderSelectorHook(
() => useSelector(selectUnsyncedSites),
stateB,
);

expect(selectedA1).toBe(selectedA2);
expect(selectedA1).not.toBe(selectedB);
expect(selectedA2).not.toBe(selectedB);
});
});
181 changes: 181 additions & 0 deletions dev-client/__tests__/integration/store/sync/PushDispatcher.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
/*
* Copyright © 2024 Technology Matters
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see https://www.gnu.org/licenses/.
*/

import {waitFor} from '@testing-library/react-native';
import {render} from '@testing/integration/utils';

import * as syncHooks from 'terraso-mobile-client/store/sync/hooks/syncHooks';
import {
PUSH_DEBOUNCE_MS,
PUSH_RETRY_INTERVAL_MS,
PushDispatcher,
} from 'terraso-mobile-client/store/sync/PushDispatcher';

jest.mock('terraso-mobile-client/store/sync/hooks/syncHooks', () => {
return {
useDebouncedIsOffline: jest.fn(),
useDebouncedUnsyncedSiteIds: jest.fn(),
useIsLoggedIn: jest.fn(),
usePushDispatch: jest.fn(),
useRetryInterval: jest.fn(),
};
});

describe('PushDispatcher', () => {
let useDebouncedIsOffline = jest.mocked(syncHooks.useDebouncedIsOffline);
let useIsLoggedIn = jest.mocked(syncHooks.useIsLoggedIn);
let useDebouncedUnsyncedSiteIds = jest.mocked(
syncHooks.useDebouncedUnsyncedSiteIds,
);

let dispatchPush = jest.fn();
let usePushDispatch = jest.mocked(syncHooks.usePushDispatch);

let beginRetry = jest.fn();
let endRetry = jest.fn();
let useRetryInterval = jest.mocked(syncHooks.useRetryInterval);

beforeEach(() => {
useDebouncedIsOffline.mockReset();
useIsLoggedIn.mockReset();
useDebouncedUnsyncedSiteIds.mockReset();
useDebouncedUnsyncedSiteIds.mockReset();
useDebouncedUnsyncedSiteIds.mockReset();

dispatchPush.mockReset();
usePushDispatch.mockReset();
usePushDispatch.mockReturnValue(dispatchPush);

beginRetry.mockReset();
endRetry.mockReset();
useRetryInterval.mockReset();
jest.mocked(useRetryInterval).mockReturnValue({
beginRetry: beginRetry,
endRetry: endRetry,
});
});

test('uses correct interval for debounces', async () => {
render(<PushDispatcher />);

expect(useDebouncedIsOffline).toHaveBeenCalledWith(PUSH_DEBOUNCE_MS);
expect(useDebouncedUnsyncedSiteIds).toHaveBeenCalledWith(PUSH_DEBOUNCE_MS);
});

test('uses correct interval for retry', async () => {
render(<PushDispatcher />);

expect(useRetryInterval).toHaveBeenCalledWith(
PUSH_RETRY_INTERVAL_MS,
dispatchPush,
);
});

test('uses correct site IDs for push dispatch', async () => {
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);

render(<PushDispatcher />);

expect(usePushDispatch).toHaveBeenCalledWith(['abcd']);
});

test('does not dispatch or retry by default', async () => {
useIsLoggedIn.mockReturnValue(false);
useDebouncedIsOffline.mockReturnValue(true);
useDebouncedUnsyncedSiteIds.mockReturnValue([]);

render(<PushDispatcher />);

expect(dispatchPush).toHaveBeenCalledTimes(0);
expect(beginRetry).toHaveBeenCalledTimes(0);
expect(endRetry).toHaveBeenCalledTimes(0);
});

test('dispatches an initial push when conditions are met', async () => {
useIsLoggedIn.mockReturnValue(true);
useDebouncedIsOffline.mockReturnValue(false);
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);

dispatchPush.mockResolvedValue({payload: {}});
render(<PushDispatcher />);

expect(dispatchPush).toHaveBeenCalledTimes(1);
expect(beginRetry).toHaveBeenCalledTimes(0);
expect(endRetry).toHaveBeenCalledTimes(0);
});

test('begins retry when push has error', async () => {
useIsLoggedIn.mockReturnValue(true);
useDebouncedIsOffline.mockReturnValue(false);
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);

dispatchPush.mockResolvedValue({payload: {error: 'error'}});
render(<PushDispatcher />);

await waitFor(() => expect(beginRetry).toHaveBeenCalledTimes(1));
});

test('begins retry when push is rejected', async () => {
useIsLoggedIn.mockReturnValue(true);
useDebouncedIsOffline.mockReturnValue(false);
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);

dispatchPush.mockRejectedValue('error');
render(<PushDispatcher />);

await waitFor(() => expect(beginRetry).toHaveBeenCalledTimes(1));
});

test('ends retry when logged-in changes', async () => {
useIsLoggedIn.mockReturnValue(true);
useDebouncedIsOffline.mockReturnValue(false);
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);
dispatchPush.mockRejectedValue('error');
const handle = render(<PushDispatcher />);

useIsLoggedIn.mockReturnValue(false);
handle.rerender(<PushDispatcher />);

await waitFor(() => expect(endRetry).toHaveBeenCalledTimes(1));
});

test('ends retry when online changes', async () => {
useIsLoggedIn.mockReturnValue(true);
useDebouncedIsOffline.mockReturnValue(false);
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);
dispatchPush.mockRejectedValue('error');
const handle = render(<PushDispatcher />);

useDebouncedIsOffline.mockReturnValue(true);
handle.rerender(<PushDispatcher />);

await waitFor(() => expect(endRetry).toHaveBeenCalledTimes(1));
});

test('ends retry when unsynced ids changes', async () => {
useIsLoggedIn.mockReturnValue(true);
useDebouncedIsOffline.mockReturnValue(false);
useDebouncedUnsyncedSiteIds.mockReturnValue(['abcd']);
dispatchPush.mockRejectedValue('error');
const handle = render(<PushDispatcher />);

useDebouncedUnsyncedSiteIds.mockReturnValue([]);
handle.rerender(<PushDispatcher />);

await waitFor(() => expect(endRetry).toHaveBeenCalledTimes(1));
});
});
Loading
Loading