Restructure backend (#439)

* chore: update dependencies

- Update uuid to ^11.1.0 and add @types/uuid
- Update @autonomys/file-caching to ^1.5.1
- Update related dependencies across packages

* feat: add HTTP range request utility functions

- Add parseByteRange function for parsing Range headers
- Add createPartialStream function for creating range-aware streams
- Add utility functions for handling HTTP 206 partial content responses
- Support for byte-range requests in readable streams

* feat: implement range request support in download services

- Add ByteRange interface for range request parameters
- Implement partial download support in memory cache
- Add range-aware download methods in download service
- Support for caching partial content with byte ranges
- Add sync download functionality with range support

* feat: add HTTP range request support to download controllers

- Update download controller to handle Range headers
- Implement HTTP 206 Partial Content responses
- Add proper Content-Range and Accept-Ranges headers
- Support partial file downloads in object controllers
- Update use cases to handle byte range parameters
- Add range request support to object download endpoints

* test: add comprehensive tests for partial retrieval functionality

- Add tests for byte range parsing and validation
- Test partial content download scenarios
- Verify HTTP 206 response handling
- Add test cases for range request edge cases
- Test memory cache integration with range requests

* refactor: move to separate files related logic

* refactor: enhance download response handling with byte range support

- Introduced isExpectedDocument function to streamline document expectation checks
- Refactored handleDownloadResponseHeaders to utilize setFileResponseHeaders and setFolderResponseHeaders for improved clarity
- Added byte range handling in setFileResponseHeaders for partial content responses
- Updated content disposition and content type settings based on encryption and document expectations

* refactor: reorganize file use cases

* fix(testing): cache populated in async setting

* feat: update auto files gateway version

* refactor: reorganize backend structure and update package references

- Updated package.json to reflect new paths for frontend and download server scripts.
- Refactored import paths in various test files to align with the new directory structure.
- Removed deprecated APIs and reorganized use cases and repositories for better clarity and maintainability.
- Deleted unused files and directories to streamline the codebase.

* refactor: re-structure core functionality placement

* update: bump auto-sdk version

* refactor: update FilesUseCases tests and imports

- Refactored tests for FilesUseCases to use getObjectInformation instead of getMetadata.
- Updated mock implementations to reflect changes in the download blocking logic.
- Cleaned up imports in files.spec.ts and index.ts for better organization and clarity.

* refactor: reorganize backend structure and update package references

- Updated package.json to reflect new paths for frontend and download server scripts.
- Refactored import paths in various test files to align with the new directory structure.
- Removed deprecated APIs and reorganized use cases and repositories for better clarity and maintainability.
- Deleted unused files and directories to streamline the codebase.

* refactor: re-structure core functionality placement

* refactor: clean up imports in files.spec.ts

- Removed unused dbMigration import to streamline the test file and improve clarity.

* fix: update byte range header handling in getByteRange function

- Changed the header used to retrieve the byte range from 'content-range' to 'range' for improved accuracy in handling byte range requests.

* fix: enhance byte range validation in getByteRange function

- Added validation checks to ensure start and end byte values are non-negative and that the start value does not exceed the end value, improving the robustness of byte range handling.

* chore: update package dependencies to version 1.5.11

* chore: update yarn.lock to reflect package versions 1.5.11 for auto-dag-data and file-caching

* feat: improved error management w/ neverthrow lib

* feat: enhance error logging in upload controller

* fix: improve error handling in async download process

* fix: enhance error handling in tagUpload function for folder uploads

* refactor: update logging in subscription controller to use trace level and return subscription value

* fix: improve error handling in async download tests and update tagUpload function for folder uploads

* fix: remove unused mock for error handling in async download tests

* fix: update error handling in dismissDownload function to return ForbiddenError for unauthorized users

* fix: handle result in update progress

Author: clostao

backend/__tests__/e2e/downloads/async.spec.ts

@@ -1,15 +1,21 @@
 import { User, AsyncDownloadStatus, AsyncDownload } from '@auto-drive/models'
 import { createMockUser } from '../../utils/mocks.js'
-import { AsyncDownloadsUseCases } from '../../../src/useCases/asyncDownloads/index.js'
+import { AsyncDownloadsUseCases } from '../../../src/core/downloads/index.js'
 import { uploadFile } from '../../utils/uploads.js'
 import { dbMigration } from '../../utils/dbMigrate.js'
-import { asyncDownloadsRepository } from '../../../src/repositories/asyncDownloads/index.js'
-import { Rabbit } from '../../../src/drivers/rabbit.js'
+import { asyncDownloadsRepository } from '../../../src/infrastructure/repositories/asyncDownloads/index.js'
+import { Rabbit } from '../../../src/infrastructure/drivers/rabbit.js'
 import { jest } from '@jest/globals'
-import { createTask } from '../../../src/services/eventRouter/tasks.js'
-import { ObjectUseCases } from '../../../src/useCases/index.js'
-import { downloadService } from '../../../src/services/download/index.js'
+import { createTask } from '../../../src/infrastructure/eventRouter/tasks.js'
+import { ObjectUseCases } from '../../../src/core/index.js'
+import { downloadService } from '../../../src/infrastructure/services/download/index.js'
 import { Readable } from 'stream'
+import {
+  ForbiddenError,
+  InternalError,
+  ObjectNotFoundError,
+} from '../../../src/errors/index.js'
+import { err } from 'neverthrow'
 
 describe('Async Downloads', () => {
   let user: User
@@ -31,9 +37,12 @@ describe('Async Downloads', () => {
   })
 
   it('should fail if object is not found', async () => {
-    await expect(
-      AsyncDownloadsUseCases.createDownload(user, 'not-found'),
-    ).rejects.toThrow('Object not found')
+    const result = await AsyncDownloadsUseCases.createDownload(
+      user,
+      'not-found',
+    )
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ObjectNotFoundError)
   })
 
   it('should create an async download', async () => {
@@ -45,7 +54,10 @@ describe('Async Downloads', () => {
     })
     const mockPublish = jest.spyOn(Rabbit, 'publish').mockResolvedValue()
 
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
 
     expect(download).toMatchObject({
       id: expect.any(String),
@@ -64,20 +76,23 @@ describe('Async Downloads', () => {
   })
 
   it('should fail if download is not found', async () => {
-    await expect(
-      AsyncDownloadsUseCases.asyncDownload('not-found'),
-    ).rejects.toThrow('Download not found')
+    const result = await AsyncDownloadsUseCases.asyncDownload('not-found')
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ObjectNotFoundError)
   })
 
   it('should fail if object is not found', async () => {
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
     const mockGetMetadata = jest
       .spyOn(ObjectUseCases, 'getMetadata')
-      .mockResolvedValue(undefined)
+      .mockResolvedValue(err(new ObjectNotFoundError('Object not found')))
 
-    await expect(
-      AsyncDownloadsUseCases.asyncDownload(download.id),
-    ).rejects.toThrow('Object not found')
+    const result = await AsyncDownloadsUseCases.asyncDownload(download.id)
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ObjectNotFoundError)
 
     mockGetMetadata.mockRestore()
   })
@@ -104,7 +119,10 @@ describe('Async Downloads', () => {
       }),
     )
 
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
     const doneMock = jest.spyOn(AsyncDownloadsUseCases, 'updateStatus')
 
     await AsyncDownloadsUseCases.asyncDownload(download.id)
@@ -138,18 +156,22 @@ describe('Async Downloads', () => {
       }),
     )
 
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
-    const doneMock = jest.spyOn(AsyncDownloadsUseCases, 'setError')
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
 
-    await expect(
-      AsyncDownloadsUseCases.asyncDownload(download.id),
-    ).rejects.toThrow('Download failed')
+    const result = await AsyncDownloadsUseCases.asyncDownload(download.id)
 
-    expect(doneMock).toHaveBeenCalledWith(download.id, expect.any(String))
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(InternalError)
   })
 
   it('should be dismissed if user dismiss it', async () => {
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
     await AsyncDownloadsUseCases.dismissDownload(user, download.id)
 
     const dismissed = await asyncDownloadsRepository.getDownloadById(
@@ -164,8 +186,14 @@ describe('Async Downloads', () => {
 
   it('should get all downloads for a user', async () => {
     // Create multiple downloads for the same user
-    const download1 = await AsyncDownloadsUseCases.createDownload(user, cid)
-    const download2 = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download1 = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
+    const download2 = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
 
     // Get all downloads for the user
     const downloads = await AsyncDownloadsUseCases.getDownloadsByUser(user)
@@ -181,8 +209,14 @@ describe('Async Downloads', () => {
 
   it('should get all downloads for a user', async () => {
     // Create multiple downloads for the same user
-    const download1 = await AsyncDownloadsUseCases.createDownload(user, cid)
-    const download2 = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download1 = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
+    const download2 = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
 
     await AsyncDownloadsUseCases.dismissDownload(user, download1.id)
 
@@ -197,16 +231,20 @@ describe('Async Downloads', () => {
 
   it('should get a specific download by id', async () => {
     // Create a download
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
 
     // Get the download by id
-    const retrievedDownload = await AsyncDownloadsUseCases.getDownloadById(
+    const retrieveResult = await AsyncDownloadsUseCases.getDownloadById(
       user,
       download.id,
     )
 
     // Verify the retrieved download matches the created one
-    expect(retrievedDownload).toMatchObject({
+    expect(retrieveResult.isOk()).toBe(true)
+    expect(retrieveResult._unsafeUnwrap()).toMatchObject({
       id: download.id,
       cid: download.cid,
       oauthProvider: user.oauthProvider,
@@ -215,22 +253,21 @@ describe('Async Downloads', () => {
     })
   })
 
-  it('should throw error when getting download with invalid id', async () => {
-    // Attempt to get a download with an invalid id
-    await expect(
-      AsyncDownloadsUseCases.getDownloadById(user, 'invalid-id'),
-    ).rejects.toThrow('Download not found')
-  })
-
   it('should throw error when user tries to access another user download', async () => {
     // Create a download
-    const download = await AsyncDownloadsUseCases.createDownload(user, cid)
+    const download = await AsyncDownloadsUseCases.createDownload(
+      user,
+      cid,
+    ).then((e) => e._unsafeUnwrap())
 
     // Create another user
     const anotherUser = createMockUser()
     // Attempt to get the download with another user
-    await expect(
-      AsyncDownloadsUseCases.getDownloadById(anotherUser, download.id),
-    ).rejects.toThrow('User unauthorized')
+    const result = await AsyncDownloadsUseCases.getDownloadById(
+      anotherUser,
+      download.id,
+    )
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ForbiddenError)
   })
 })

backend/__tests__/e2e/objects/nodes.spec.ts

@@ -1,5 +1,5 @@
 import { v4 } from 'uuid'
-import { NodesUseCases, ObjectUseCases } from '../../../src/useCases/index.js'
+import { NodesUseCases, ObjectUseCases } from '../../../src/core/index.js'
 import {
   blake3HashFromCid,
   cidOfNode,
@@ -16,17 +16,17 @@ import {
   metadataRepository,
   Node,
   nodesRepository,
-} from '../../../src/repositories/index.js'
+} from '../../../src/infrastructure/repositories/index.js'
 import {
   ObjectMapping,
   ObjectMappingListEntry,
   TransactionResult,
 } from '@auto-drive/models'
 import { mockRabbitPublish, unmockMethods } from '../../utils/mocks.js'
 import { jest } from '@jest/globals'
-import { EventRouter } from '../../../src/services/eventRouter/index.js'
-import { BlockstoreUseCases } from '../../../src/useCases/uploads/blockstore.js'
-import { MAX_RETRIES } from '../../../src/services/eventRouter/tasks.js'
+import { EventRouter } from '../../../src/infrastructure/eventRouter/index.js'
+import { BlockstoreUseCases } from '../../../src/core/uploads/blockstore.js'
+import { MAX_RETRIES } from '../../../src/infrastructure/eventRouter/tasks.js'
 
 describe('Nodes', () => {
   const id = v4()

backend/__tests__/e2e/objects/object.spec.ts

@@ -1,8 +1,7 @@
 import { UserWithOrganization } from '@auto-drive/models'
-import { AuthManager } from '../../../src/services/auth/index.js'
-import { ObjectUseCases } from '../../../src/useCases/index.js'
+import { AuthManager } from '../../../src/infrastructure/services/auth/index.js'
+import { ObjectUseCases } from '../../../src/core/index.js'
 import { dbMigration } from '../../utils/dbMigrate.js'
-import { PreconditionError } from '../../utils/error.js'
 import {
   createMockUser,
   mockRabbitPublish,
@@ -11,8 +10,9 @@ import {
 import { uploadFile } from '../../utils/uploads.js'
 import { jest } from '@jest/globals'
 import { v4 } from 'uuid'
-import { downloadService } from '../../../src/services/download/index.js'
+import { downloadService } from '../../../src/infrastructure/services/download/index.js'
 import { Readable } from 'stream'
+import { ForbiddenError } from '../../../src/errors/index.js'
 
 describe('Object', () => {
   let user: UserWithOrganization
@@ -42,23 +42,27 @@ describe('Object', () => {
 
   it('for not admin user should not be able to share object', async () => {
     const mockUser = createMockUser()
-    await expect(
-      ObjectUseCases.shareObject(mockUser, fileCid, user.publicId!),
-    ).rejects.toThrow(new Error('User is not an admin of this object'))
+    const result = await ObjectUseCases.shareObject(
+      mockUser,
+      fileCid,
+      user.publicId!,
+    )
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ForbiddenError)
   })
 
   it('User that is not owner should not be able to delete object', async () => {
     const mockUser = createMockUser()
-    await expect(
-      ObjectUseCases.markAsDeleted(mockUser, fileCid),
-    ).rejects.toThrow(new Error('User is not an owner of this object'))
+    const result = await ObjectUseCases.markAsDeleted(mockUser, fileCid)
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ForbiddenError)
   })
 
   it('User that is not owner should not be able to restore object', async () => {
     const mockUser = createMockUser()
-    await expect(
-      ObjectUseCases.restoreObject(mockUser, fileCid),
-    ).rejects.toThrow(new Error('User is not an owner of this object'))
+    const result = await ObjectUseCases.restoreObject(mockUser, fileCid)
+    expect(result.isErr()).toBe(true)
+    expect(result._unsafeUnwrapErr()).toBeInstanceOf(ForbiddenError)
   })
 
   it('isArchived should return false for not archived object', async () => {
@@ -164,8 +168,9 @@ describe('Object', () => {
   })
 
   it('should be able to search object by name', async () => {
-    const metadata = await ObjectUseCases.getMetadata(fileCid)
-    if (!metadata) throw new PreconditionError('Metadata not found')
+    const metadata = await ObjectUseCases.getMetadata(fileCid).then((e) =>
+      e._unsafeUnwrap(),
+    )
 
     const search = await ObjectUseCases.searchMetadataByName(
       metadata.name!,
@@ -179,8 +184,10 @@ describe('Object', () => {
   })
 
   it('should be able to see another user with user scope', async () => {
-    const metadata = await ObjectUseCases.getMetadata(fileCid)
-    if (!metadata) throw new PreconditionError('Metadata not found')
+    const metadata = await ObjectUseCases.getMetadata(fileCid).then((e) =>
+      e._unsafeUnwrap(),
+    )
+
     const mockUser = createMockUser()
     const search = await ObjectUseCases.searchMetadataByName(
       metadata.name!,
@@ -194,8 +201,10 @@ describe('Object', () => {
   })
 
   it('should be able to see another user with global scope', async () => {
-    const metadata = await ObjectUseCases.getMetadata(fileCid)
-    if (!metadata) throw new PreconditionError('Metadata not found')
+    const metadata = await ObjectUseCases.getMetadata(fileCid).then((e) =>
+      e._unsafeUnwrap(),
+    )
+
     const search = await ObjectUseCases.searchMetadataByName(
       metadata.name!,
       5,
@@ -207,8 +216,9 @@ describe('Object', () => {
   })
 
   it('should be able to search object by cid', async () => {
-    const metadata = await ObjectUseCases.getMetadata(fileCid)
-    if (!metadata) throw new PreconditionError('Metadata not found')
+    const metadata = await ObjectUseCases.getMetadata(fileCid).then((e) =>
+      e._unsafeUnwrap(),
+    )
 
     const search = await ObjectUseCases.searchMetadataByCID(fileCid, 5, {
       scope: 'user',
@@ -218,8 +228,9 @@ describe('Object', () => {
   })
 
   it('should be able to search object by name (using common method)', async () => {
-    const metadata = await ObjectUseCases.getMetadata(fileCid)
-    if (!metadata) throw new PreconditionError('Metadata not found')
+    const metadata = await ObjectUseCases.getMetadata(fileCid).then((e) =>
+      e._unsafeUnwrap(),
+    )
 
     const search = await ObjectUseCases.searchByCIDOrName(metadata.name!, 5, {
       scope: 'user',
@@ -234,8 +245,9 @@ describe('Object', () => {
   })
 
   it('should be able to search object by cid (using common method)', async () => {
-    const metadata = await ObjectUseCases.getMetadata(fileCid)
-    if (!metadata) throw new PreconditionError('Metadata not found')
+    const metadata = await ObjectUseCases.getMetadata(fileCid).then((e) =>
+      e._unsafeUnwrap(),
+    )
 
     const search = await ObjectUseCases.searchByCIDOrName(fileCid, 5, {
       scope: 'user',