SyntaxError, "Cannot use import statement outside a module"

[Fossy777]

Hi Disane, everybody,

just today tried to bring up this latest (v2.0.7) as a container on my Synology DS923+ (AMD Ryzen, X64) with absolute minimum parameters but it still fails. As I have no CLI, I use Portainer and Stacks (compose) to deploy (but works like a charm with everything up to now).

Here is how my stack looks like:

version: "3.9"
services:
  docudigger:
    image: ghcr.io/disane87/docudigger
    container_name: Docudigger
    hostname: docudigger
    security_opt:
      - no-new-privileges:true
    user: 1026:100
    environment:
      - TZ=Europe/Berlin
      - AMAZON_USERNAME='<myemail>' \
      - AMAZON_PASSWORD='<mypass>' \
      - AMAZON_TLD='de' \
      - AMAZON_YEAR_FILTER='2025' \
      #- AMAZON_PAGE_FILTER='1' \
      - LOG_LEVEL='info' \
    volumes:
      - /volume1/docker/docudigger:/home/node/docudigger:rw
    restart: on-failure:5

And here are the logs:

[0] docudigger scrape all exited with code 1
[0] (node:15) Warning: SyntaxError
[0] module: @oclif/core@4.0.31
[0] task: findCommand (scrape:all.d)
[0] plugin: @disane-dev/docudigger
[0] root: /home/node/.npm-global/lib/node_modules/@disane-dev/docudigger
[0] message: Cannot use import statement outside a module
[0] See more details with DEBUG=*
[0] (Use `node --trace-warnings ...` to show where the warning was created)
[0] (node:15) Warning: SyntaxError
[0] module: @oclif/core@4.0.31
[0] task: findCommand (scrape:amazon:index.d)
[0] plugin: @disane-dev/docudigger
[0] root: /home/node/.npm-global/lib/node_modules/@disane-dev/docudigger
[0] message: Cannot use import statement outside a module
[0] See more details with DEBUG=*
[0] node:os:365
[0]     throw new ERR_SYSTEM_ERROR(ctx);
[0]           ^
[0] 
[0] SystemError [ERR_SYSTEM_ERROR]: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)
[0]     at userInfo (node:os:365:11)
[0]     at Config._shell (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/config/config.js:588:67)
[0]     at Config.load (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/config/config.js:310:27)
[0]     at async Config.load (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/config/config.js:168:9)
[0]     at async Object.run (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/main.js:62:20) {
[0]   code: 'ERR_SYSTEM_ERROR',
[0]   info: {
[0]     errno: -2,
[0]     code: 'ENOENT',
[0]     message: 'no such file or directory',
[0]     syscall: 'uv_os_get_passwd'
[0]   },
[0]   errno: [Getter/Setter],
[0]   syscall: [Getter/Setter]
[0] }
[0] 
[0] Node.js v20.18.0
[0] docudigger scrape all exited with code 1

Am I missing something or is there something wrong with the container/code?

[Disane87]

Thank you for that issue! That seems to be a problem in the main branch. Could you please test the dev branch?

[Fossy777]

same issue with

version: "3.9"
services:
  docudigger:
    image: ghcr.io/disane87/docudigger:2.0.7-dev.3
    container_name: Docudigger
    hostname: docudigger
    security_opt:
      - no-new-privileges:true
    user: 1026:100
    ...

[Fossy777]

same on my old Linux laptop, by the way (i7), but potentially better options for debugging. May it only work on Windows environments, as I can see in your sample code you use a Windows path name?

The log here:

[0] (node:14) Warning: SyntaxError
[0] module: @oclif/core@4.0.31
[0] task: findCommand (scrape:all.d)
[0] plugin: @disane-dev/docudigger
[0] root: /home/node/.npm-global/lib/node_modules/@disane-dev/docudigger
[0] message: Cannot use import statement outside a module
[0] See more details with DEBUG=*
[0] (Use `node --trace-warnings ...` to show where the warning was created)
[0] (node:14) Warning: SyntaxError
[0] module: @oclif/core@4.0.31
[0] task: findCommand (scrape:amazon:index.d)
[0] plugin: @disane-dev/docudigger
[0] root: /home/node/.npm-global/lib/node_modules/@disane-dev/docudigger
[0] message: Cannot use import statement outside a module
[0] See more details with DEBUG=*
[0] /home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/parse.js:73
[0]         throw new errors_1.FlagInvalidOptionError(flag, input);
[0]               ^
[0] 
[0] FlagInvalidOptionError: Expected --logLevel='info' \ to be one of: trace, debug, info, warn, error
[0] See more help with --help
[0]     at validateOptions (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/parse.js:73:15)
[0]     at Object.valueFunction (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/parse.js:347:75)
[0]     at /home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/parse.js:420:113
[0]     at Array.map (<anonymous>)
[0]     at Parser._flags (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/parse.js:420:14)
[0]     at Parser.parse (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/parse.js:180:93)
[0]     at Object.parse (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/parser/index.js:19:33)
[0]     at All.parse (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/node_modules/@oclif/core/lib/command.js:244:38)
[0]     at async All.initFlags (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/dist/classes/base.class.js:25:33)
[0]     at async All.init (/home/node/.npm-global/lib/node_modules/@disane-dev/docudigger/dist/classes/base.class.js:16:9) {
[0]   code: undefined,
[0]   oclif: { exit: 2 },
[0]   skipOclifErrorHandling: undefined,
[0]   suggestions: undefined,
[0]   parse: {},
[0]   showHelp: false
[0] }
[0] 
[0] Node.js v20.18.0
[0] docudigger scrape all exited with code 1

[Disane87]

It should work on Linux too because I use it on my Unraid server. Unfortunately I messed it up and since that it doesn't seem to work.

The problem is, the codebase is pretty awefull because it went from prototype to prod without further enhancements. I'm currently working on a refactoring with streamlined debugging and building capabilities and some other flexible features but that needs some more work to go.

My plan is to integrate that new system in a new branch and merge it with the dev and master when it has a good starting point. Because of some private things I didn't had some time to publish my work atm.

Because of that refactoring in the pipeline I don't want to debug this stuff anymore because the new system is a much better and convenient approach.

[Fossy777]

never mind, 2.0.4 works :-)

PS: this could also be useful on any customer portals like many insurance companies operate or German Telekom or similar

[Disane87]

never mind, 2.0.4 works :-)

Glad it worked for you!

PS: this could also be useful on any customer portals like many insurance companies operate or German Telekom or similar

I have some plans to outsource the definition of service portals like Vodafone or something else into a json file where you can define the process of scraping pages like loops, extract, get and some other stuff. I'm 90% done with it but some things are missing atm.

If that lands in production its flexible enough to support all variations of pages to scrape documents or only information from them without even touching the source code. Its also possible then to plug in some other pre modelled defintion of pages which will later be shipped by default.

[Fossy777]

Shouted out too quickly: it runs, yes, but it does not work properly. Only ~half of the invoices gets downloaded. I noticed it by chance on a double-check, when I was searching for a specific invoice then in PaperlessNGX and couldn’t find it (and of course, following Murphy‘s law, it’s an important one relevant for tax declaration). At first i thought it could be because for some orders one gets multiple / split invoices, but in this specific example it was even an easy simple case: one separate order, one line item, one invoice. Sad but like that I cannot rely on it. So I guess have to wait for your next major release?

[Disane87]

Yeah that seems reasonable. I had a bug where it doesn't properly stepped through the pages because the index wasn't correct.

Unfortunately you have to wait for the major overhaul.

[gramels]

Hi I would like to use it for my taxes 2024, any eta when it will work again (docker, linux host)?

[Disane87]

I hope I will release a preview this month but please don't pin me down on that because I am currently renovating my house and have some current health problems

[ldehner]

@Disane87 hope you are okay. Are there and News on the Update?

[dosubot]

Hi, @Fossy777. I'm Dosu, and I'm helping the docudigger team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• You reported a SyntaxError with Docudigger v2.0.7 on Synology DS923+ and Linux, related to module imports.
• Disane87 suggested testing the dev branch, but the error persisted.
• Ongoing refactoring is planned, but the release timeline is uncertain due to personal commitments.
• Version 2.0.4 is partially functional, but with incomplete invoice downloads.
• Gramels is interested in using the tool for taxes in 2024, and a preview release is anticipated.

Next Steps:

• Please confirm if this issue is still relevant with the latest version of Docudigger.
• If there is no further activity, the issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!

[gramels]

I am stil interested in a solution.