refactor(neon_framework): migrate to neon_storage #2529

Leptopoda · 2024-09-29T11:00:42Z

Signed-off-by: Nikolas Rimikis [email protected]

Signed-off-by: Nikolas Rimikis <[email protected]>

This is temporary and the long term goal is to remove most of these classes as the framework transitions to direct sqlite access. Signed-off-by: Nikolas Rimikis <[email protected]>

Signed-off-by: Nikolas Rimikis <[email protected]>

codecov · 2024-09-29T11:11:10Z

Codecov Report

Attention: Patch coverage is 74.21384% with 41 lines in your changes missing coverage. Please review.

Project coverage is 28.69%. Comparing base (29f6a4b) to head (b7d8a42).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../neon_framework/lib/src/storage/neon_cache_db.dart	0.00%	13 Missing ⚠️
...es/account_repository/lib/src/account_storage.dart	85.52%	11 Missing ⚠️
...s/neon_framework/lib/src/storage/neon_data_db.dart	0.00%	10 Missing ⚠️
...eon_framework/lib/src/storage/storage_manager.dart	0.00%	6 Missing ⚠️
...age/lib/src/storage/sqlite_cookie_persistence.dart	92.30%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2529      +/-   ##
==========================================
+ Coverage   28.66%   28.69%   +0.02%     
==========================================
  Files         366      369       +3     
  Lines      136296   136370      +74     
==========================================
+ Hits        39076    39137      +61     
- Misses      97220    97233      +13

Flag	Coverage Δ		*Carryforward flag
account_repository	`95.92% <87.77%> (-2.85%)`	⬇️
cookie_store	`99.48% <ø> (ø)`		Carriedforward from 29f6a4b
dashboard_app	`96.05% <ø> (ø)`
dynamite	`31.05% <ø> (ø)`		Carriedforward from 29f6a4b
dynamite_end_to_end_test	`61.69% <ø> (ø)`		Carriedforward from 29f6a4b
dynamite_runtime	`85.40% <ø> (ø)`		Carriedforward from 29f6a4b
interceptor_http_client	`97.18% <ø> (ø)`		Carriedforward from 29f6a4b
neon_dashboard	`96.05% <ø> (ø)`		Carriedforward from 29f6a4b
neon_framework	`57.02% <9.37%> (-2.18%)`	⬇️
neon_http_client	`97.50% <ø> (ø)`		Carriedforward from 29f6a4b
neon_notifications	`100.00% <ø> (ø)`		Carriedforward from 29f6a4b
neon_storage	`88.88% <97.29%> (-5.78%)`	⬇️
neon_talk	`99.45% <ø> (ø)`		Carriedforward from 29f6a4b
nextcloud	`24.26% <ø> (ø)`		Carriedforward from 29f6a4b
notifications_app	`97.40% <ø> (ø)`
notifications_push_repository	`98.59% <ø> (ø)`
sort_box	`90.90% <ø> (ø)`		Carriedforward from 29f6a4b
talk_app	`98.94% <ø> (ø)`

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines	Coverage Δ
...ackages/neon_framework/lib/src/blocs/accounts.dart	`0.00% <ø> (ø)`
...ges/neon_framework/lib/src/blocs/first_launch.dart	`0.00% <ø> (ø)`
...n_framework/lib/src/models/app_implementation.dart	`0.00% <ø> (ø)`
...lib/src/settings/utils/settings_export_helper.dart	`100.00% <ø> (ø)`
...neon_framework/lib/src/storage/settings_store.dart	`84.61% <ø> (ø)`
..._framework/lib/src/storage/single_value_store.dart	`100.00% <ø> (ø)`
packages/neon_framework/lib/src/testing/mocks.dart	`91.66% <ø> (-0.65%)`	⬇️
...s/neon_framework/lib/src/utils/global_options.dart	`0.00% <ø> (ø)`
.../neon_framework/lib/src/utils/request_manager.dart	`78.04% <100.00%> (+0.54%)`	⬆️
...account_repository/lib/src/account_repository.dart	`100.00% <100.00%> (ø)`
... and 11 more

... and 2 files with indirect coverage changes

provokateurin

The name of the first commit is a bit confusing. It still uses per Account caching, but simply doesn't use the class anymore.

provokateurin · 2024-09-29T17:16:28Z

packages/neon_framework/packages/account_repository/lib/src/account_storage.dart

+
+  /// Column for the [Credentials.serverURL].
+  @protected
+  static const String serverURL = 'server_urlL';


Suggested change

static const String serverURL = 'server_urlL';

static const String serverURL = 'server_url';

provokateurin · 2024-10-18T06:21:30Z

packages/neon_framework/example/pubspec.lock

-  sqflite:
-    dependency: transitive
-    description:
-      name: sqflite
-      sha256: a43e5a27235518c03ca238e7b4732cf35eabe863a369ceba6cbefa537a66f16d
-      url: "https://pub.dev"
-    source: hosted
-    version: "2.3.3+1"


This is still required for Android/iOS

provokateurin · 2024-10-18T06:25:03Z

packages/neon_framework/packages/neon_storage/test/storage/request_cache_test.dart

+      await database.close();
+    });
+
+    test('throws when unutilized', () async {


Suggested change

test('throws when unutilized', () async {

test('throws when uninitialized', () async {

provokateurin · 2024-10-18T06:26:31Z

packages/neon_framework/packages/neon_storage/test/interfaces/cached_persistence_test.dart

+
+      expect(
+        persistence.getValue('key'),
+        equals('value'),


Suggested change

equals('value'),

'value',

provokateurin · 2024-10-18T06:29:15Z

packages/neon_framework/packages/neon_storage/lib/src/storage/sqlite_persistence.dart

+	PRIMARY KEY("prefix","key"),
+	UNIQUE("key","prefix")


Suggested change

PRIMARY KEY("prefix","key"),

UNIQUE("key","prefix")

PRIMARY KEY("prefix", "key")

PRIMARY KEY is already UNIQUE

provokateurin · 2024-10-18T06:39:46Z

packages/neon_framework/packages/account_repository/lib/src/account_storage.dart

+  /// Column for the [Credentials.serverURL].
+  @protected
+  static const String serverURL = 'server_urlL';
+
+  /// Column for the [Credentials.username].
+  @protected
+  static const String loginName = 'login_name';
+
+  /// Column for the [Credentials.appPassword].
+  @protected
+  static const String appPassword = 'app_password';
+
+  /// Column to store the active account.
+  @protected
+  static const String active = 'active';


Can you pre- or suffix these with column?

provokateurin · 2024-10-18T06:42:09Z

packages/neon_framework/packages/account_repository/lib/src/account_storage.dart

+  "$serverURL"      TEXT NOT NULL,
+  "$loginName"      TEXT NOT NULL,
+  "$appPassword"    TEXT,
+  "$active"         BOOLEAN NOT NULL DEFAULT 0,


We should store this separately as it is not really an information about the account itself.
Also there is no check that two accounts can not be active at the same time.

provokateurin · 2024-10-18T06:42:49Z

packages/neon_framework/packages/account_repository/lib/src/account_storage.dart

+      return credentials.build();
+    } on DatabaseException catch (error, stackTrace) {
+      _log.warning(
+        'Error loading cookies.',


Wrong error message (same for all other catch blocks in this class)

provokateurin · 2024-10-18T06:47:04Z

packages/neon_framework/lib/src/platform/linux.dart

How does it work without the platform specific initialization?

The neon_storage package handles this with a conditional import/export.

neon/packages/neon_framework/packages/neon_storage/lib/src/sqlite/sqlite.dart

Lines 1 to 3 in 5484049

export 'database_factory.dart'

if (dart.library.js_interop) '_browser_database_factory.dart'

if (dart.library.io) '_io_database_factory.dart';

We can not easily run the neon_storage tests on web but I'll verify again that this actually works.

provokateurin · 2024-10-18T06:47:13Z

packages/neon_framework/lib/src/platform/web.dart

provokateurin

It would be great if we could have upgrade testing from old schema versions to the latest schema version. This should also include test data to ensure everything is migrated correctly.

Leptopoda added 5 commits September 29, 2024 12:44

refactor(neon_framework): make request cache account independent

be5fb0d

Signed-off-by: Nikolas Rimikis <[email protected]>

feat(neon_storage): migrate neon storages into the neon_storage library

e3de7c9

This is temporary and the long term goal is to remove most of these classes as the framework transitions to direct sqlite access. Signed-off-by: Nikolas Rimikis <[email protected]>

refactor(neon_framework): migrate to neon_storage

b28dae6

Signed-off-by: Nikolas Rimikis <[email protected]>

refactor(account_repository): manage cookie store in account_repository

fef7677

Signed-off-by: Nikolas Rimikis <[email protected]>

refactor(account_repository): migrate to neon_storage

b7d8a42

Signed-off-by: Nikolas Rimikis <[email protected]>

Leptopoda requested a review from provokateurin September 29, 2024 11:00

Leptopoda changed the title ~~refactor(neon_framework): make request cache account independent~~ @Leptopoda refactor(neon_framework): migrate to neon_storage Sep 29, 2024

Leptopoda changed the title ~~@Leptopoda refactor(neon_framework): migrate to neon_storage~~ refactor(neon_framework): migrate to neon_storage Sep 29, 2024

provokateurin requested changes Oct 18, 2024

View reviewed changes

provokateurin requested changes Oct 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(neon_framework): migrate to neon_storage #2529

refactor(neon_framework): migrate to neon_storage #2529

Leptopoda commented Sep 29, 2024

codecov bot commented Sep 29, 2024

provokateurin left a comment •

edited

Loading

provokateurin Sep 29, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

provokateurin Oct 18, 2024

Leptopoda Oct 21, 2024

provokateurin Oct 18, 2024

provokateurin left a comment

	static const String serverURL = 'server_urlL';
	static const String serverURL = 'server_url';

	test('throws when unutilized', () async {
	test('throws when uninitialized', () async {

	PRIMARY KEY("prefix","key"),
	UNIQUE("key","prefix")
	PRIMARY KEY("prefix", "key")

	export 'database_factory.dart'
	if (dart.library.js_interop) '_browser_database_factory.dart'
	if (dart.library.io) '_io_database_factory.dart';

refactor(neon_framework): migrate to neon_storage #2529

Are you sure you want to change the base?

refactor(neon_framework): migrate to neon_storage #2529

Conversation

Leptopoda commented Sep 29, 2024

codecov bot commented Sep 29, 2024

Codecov Report

provokateurin left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

provokateurin left a comment

Choose a reason for hiding this comment

provokateurin left a comment •

edited

Loading