Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@sentry/astro breaks drizzle-orm in Astro #12912

Closed
3 tasks done
Tracked by #12806
laptou opened this issue Jul 15, 2024 · 32 comments
Closed
3 tasks done
Tracked by #12806

@sentry/astro breaks drizzle-orm in Astro #12912

laptou opened this issue Jul 15, 2024 · 32 comments
Assignees
Labels
Package: astro Issues related to the Sentry Astro SDK

Comments

@laptou
Copy link

laptou commented Jul 15, 2024

Is there an existing issue for this?

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/astro

SDK Version

8.17.0

Framework Version

Astro 4.11.5

Link to Sentry event

No response

SDK Setup/Reproduction Example

https://stackblitz.com/edit/github-yligk2?file=astro.config.mjs

Steps to Reproduce

  1. Create a new blank Astro project.
  2. Install @sentry/astro and drizzle-orm.
  3. Create a src/middleware.ts file.
  4. Import any operator from drizzle-orm:
import { or } from 'drizzle-orm';
  1. Run the server and open the index page.
  2. Receive error: MiddlewareCantBeLoaded because drizzle-orm has no export "or".

Expected Result

Pages should load as normal even if the middleware depends on drizzle-orm.

Actual Result

 astro  v4.11.5 ready in 415 ms

┃ Local    http://localhost:4321/
┃ Network  use --host to expose

24:59:05 watching for file changes...
24:59:15 [ERROR] [MiddlewareCantBeLoaded] An unknown error was thrown while loading your middleware.
  Error reference:
    https://docs.astro.build/en/reference/errors/middleware-cant-be-loaded/
  Stack trace:
    at /home/ibiyemi/projects/personal/rdvz-web/node_modules/astro/dist/core/middleware/loadMiddleware.js:8:24
    [...] See full stack trace in the browser, or rerun with --verbose.
  Caused by:
  [vite] The requested module 'drizzle-orm' does not provide an export named 'or'
    at analyzeImportedModDifference (file:///home/ibiyemi/projects/personal/rdvz-web/node_modules/vite/dist/node/chunks/dep-CzJTQ5q7.js:52533:15)
    at async ssrImport (file:///home/ibiyemi/projects/personal/rdvz-web/node_modules/vite/dist/node/chunks/dep-CzJTQ5q7.js:53392:16)
    at async instantiateModule (file:///home/ibiyemi/projects/personal/rdvz-web/node_modules/vite/dist/node/chunks/dep-CzJTQ5q7.js:53451:5)
@github-actions github-actions bot added the Package: astro Issues related to the Sentry Astro SDK label Jul 15, 2024
@andreiborza andreiborza self-assigned this Jul 15, 2024
@andreiborza
Copy link
Member

andreiborza commented Jul 15, 2024

Hello, thank you for writing in. I've tried reproducing this locally with an astro starter project and the steps outlined here but did not encounter the issue.

 astro  v4.11.5 ready in 86 ms

┃ Local    http://localhost:4321/
┃ Network  use --host to expose

09:38:15 watching for file changes...
[Function: or]

Could you please provide a sample repo or stackblitz?
Taking a look at the provided stackblitz :)

@andreiborza
Copy link
Member

This looks to be some kind of issue with import in the middle and ESM. We are still investigating, but for now to get you unstuck you could create an empty sentry.server.config.js file. You won't get any server-side instrumentation, just client-side but at least it should let you run your app.

@andreiborza andreiborza assigned Lms24 and unassigned andreiborza Jul 15, 2024
@timfish timfish self-assigned this Jul 15, 2024
@timfish
Copy link
Collaborator

timfish commented Jul 15, 2024

I suspect this is the same issue as seen here:
nodejs/import-in-the-middle#139

I've got an open PR to fix this:
nodejs/import-in-the-middle#140

@Lms24
Copy link
Member

Lms24 commented Jul 15, 2024

@timfish lol I just opened another issue in nodejs/import-in-the-middle#141 b/c I missed the connection to nodejs/import-in-the-middle#139 😅

@timfish
Copy link
Collaborator

timfish commented Jul 15, 2024

Well, I do still need to check that they're caused by the same issue!

@Lms24
Copy link
Member

Lms24 commented Jul 15, 2024

General comment to what we found out while debugging this issue: It looks like there are several problems with Astro and OpenTelemetry in in general and even if we get this to run with the current SDK initialization procedure, auto instrumentation is likely not gonna work. I couldn't verify this yet though, so it's just an educated guess.

The core problem is that we need to register the opentelemetry hook as early as possible. In Astro we can't pass the --import <filename> flag to the Astro CLI so we're left with setting the NODE_OPTIONS=--import <filename> env variable.
Doing so however, results in the following error:

> npm run dev
Fragment is not defined
  Stack trace:
    at Function.assign (<anonymous>)
    at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
    at async runCommand (file:///Users/lukas/code/test-projects/gh-sentry-javascript-12912-astro-iitm/node_modules/astro/dist/cli/index.js:130:23)

I'm gonna look a bit into the Astro side of things to see what might be causing this.

@timfish
Copy link
Collaborator

timfish commented Jul 15, 2024

I think the long term solution might be to push frameworks like Astro (and any others where this is an issue) to follow Nextjs and add an instrumentation.js where this sort of configuration is called early!

@timfish
Copy link
Collaborator

timfish commented Jul 15, 2024

Do any of the Astro integration hooks get called before user code is asyncroniously loaded?

As long as we can get Sentry init'ed before the instrumented libraries are loaded via require() or import() we're all good!

@Lms24
Copy link
Member

Lms24 commented Jul 15, 2024

We currently already use the injectScripts function to load the SDK which seems to be too late on the server side. But maybe that's not the only place where we can do this. I'll look into it.

@timfish
Copy link
Collaborator

timfish commented Jul 15, 2024

A fix was release in [email protected] which should fix the 'drizzle-orm' does not provide an export named 'or' issue.

You can either delete your lock file and reinstall or npm/yarn update import-in-the-middle to get the latest version.

@laptou
Copy link
Author

laptou commented Jul 15, 2024

It still doesn't work. I tried opening the StackBlitz and running npm update import-in-the-middle. Here's the output of npm why import-in-the-middle after doing so:

❯ npm why import-in-the-middle
[email protected]
node_modules/import-in-the-middle
  import-in-the-middle@"^1.8.1" from @opentelemetry/[email protected]
  node_modules/@opentelemetry/instrumentation
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-connect
      @opentelemetry/instrumentation-connect@"0.38.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-express
      @opentelemetry/instrumentation-express@"0.41.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-fastify
      @opentelemetry/instrumentation-fastify@"0.38.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-graphql
      @opentelemetry/instrumentation-graphql@"0.42.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-hapi
      @opentelemetry/instrumentation-hapi@"0.40.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"0.52.1" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-http
      @opentelemetry/instrumentation-http@"0.52.1" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-ioredis
      @opentelemetry/instrumentation-ioredis@"0.42.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-koa
      @opentelemetry/instrumentation-koa@"0.42.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-mongodb
      @opentelemetry/instrumentation-mongodb@"0.46.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-mongoose
      @opentelemetry/instrumentation-mongoose@"0.40.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-mysql
      @opentelemetry/instrumentation-mysql@"0.40.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-mysql2
      @opentelemetry/instrumentation-mysql2@"0.40.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-nestjs-core
      @opentelemetry/instrumentation-nestjs-core@"0.39.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-pg
      @opentelemetry/instrumentation-pg@"0.43.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.0" from @opentelemetry/[email protected]
    node_modules/@opentelemetry/instrumentation-redis-4
      @opentelemetry/instrumentation-redis-4@"0.41.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.49 || ^0.50 || ^0.51 || ^0.52.0" from @prisma/[email protected]
    node_modules/@prisma/instrumentation
      @prisma/instrumentation@"5.16.1" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project
    @opentelemetry/instrumentation@"^0.52.1" from @sentry/[email protected]
    node_modules/@sentry/node
      @sentry/node@"8.17.0" from @sentry/[email protected]
      node_modules/@sentry/astro
        @sentry/astro@"^8.17.0" from the root project
    peer @opentelemetry/instrumentation@"^0.52.1" from @sentry/[email protected]
    node_modules/@sentry/opentelemetry
      @sentry/opentelemetry@"8.17.0" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project

[email protected] optional
node_modules/opentelemetry-instrumentation-fetch-node/node_modules/import-in-the-middle
  import-in-the-middle@"1.7.1" from @opentelemetry/[email protected]
  node_modules/opentelemetry-instrumentation-fetch-node/node_modules/@opentelemetry/instrumentation
    @opentelemetry/instrumentation@"^0.46.0" from [email protected]
    node_modules/opentelemetry-instrumentation-fetch-node
      optional opentelemetry-instrumentation-fetch-node@"1.2.3" from @sentry/[email protected]
      node_modules/@sentry/node
        @sentry/node@"8.17.0" from @sentry/[email protected]
        node_modules/@sentry/astro
          @sentry/astro@"^8.17.0" from the root project

But the site still doesn't work. In my own project I tried forcing v1.7.1 to resolve to v1.9.1, but that didn't help.

@Lms24
Copy link
Member

Lms24 commented Jul 16, 2024

Thanks for testing! We also found out that the fix didn't address the drizzle-orm bug. Tim is on it: nodejs/import-in-the-middle#141

@timfish
Copy link
Collaborator

timfish commented Jul 16, 2024

With v8.18.0 of the SDKs just released, there's a new config option which can disable import-in-the-middle for specific libraries:

import * as Sentry from '@sentry/node';

Sentry.init({
  dsn: '__DSN__',
  registerEsmLoaderHooks: { exclude: ['drizzle-orm'] },
});

@laptou
Copy link
Author

laptou commented Jul 17, 2024

what are the implications of doing so?

@timfish
Copy link
Collaborator

timfish commented Jul 17, 2024

This setting will disable import-in-the-middle from making drizzle-orm instrumentable. As far as I understand, Sentry does not instrument drizzle-orm by default, so as long as you are not adding any other otel instrumentations that attempt to instrument this library, there are no implications.

I'm working on some other improvements to import-in-the-middle which should negate the need for this manual configuration in the future!

@RIP21
Copy link

RIP21 commented Jul 20, 2024

@timfish is there a way to set this registerEsmLoaderHooks: { exclude: ['drizzle-orm'] }, in the preload somehow?
As I need to set up beforeSend hook and it requires me to import other modules, so lazy is the only option, I would prefer to use preload with that function.

These other modules are TS files, so it's a bit complicated e.g. I'm using @swc-node/register

node --import @sentry/node/preload --import @swc-node/register/esm-register ./src/main.ts

mydea added a commit that referenced this issue Jul 22, 2024
As mentioned in here:
#12912 (comment)
there is no way today to exclude/include esm modules when preloading
today.

This PR adds the option to pass `registerEsmLoaderHooks` as option to
`preloadOpenTelemetry`, which allows to exclude/include packages there
as well. Users can then write their own custom `preload` script and
configure this there, if wanted.

## Naming

I chose to use the same option naming here than for `init`, although the
semantics are a bit different - here we can't actually disable the
wrapping (because that's the only reason to even call this). We can also
use a different name if we want, but I thought this would maybe be
easier to understand that this is the same thing 🤔
@laptou
Copy link
Author

laptou commented Jul 22, 2024

With v8.18.0 of the SDKs just released, there's a new config option which can disable import-in-the-middle for specific libraries:

How do I use this in Astro? There's no Sentry.init call, just the @sentry/astro plugin doing some magic.

@Lms24
Copy link
Member

Lms24 commented Jul 23, 2024

@laptou you can define a custom Sentry.init call for your server-side setup: https://docs.sentry.io/platforms/javascript/guides/astro/manual-setup/#server-side

@laptou
Copy link
Author

laptou commented Jul 23, 2024 via email

@laptou
Copy link
Author

laptou commented Jul 23, 2024

I just tested it, and even if I exclude drizzle-orm in sentry.server.config.js, Astro still won't serve anything if Sentry is enabled on the server side.

StackBlitz

@mydea
Copy link
Member

mydea commented Jul 24, 2024

hey, can you ensure to update the transitive import-in-the-middle dependency to 1.10.0? This includes some fixes, and it can happen that you don't have the latest version because opentelemetry depends on ^1.8.1.
Then, you can also try to configure exclude: [/drizzle-orm/] (so a regex), this will also match any sub exports of that package.

Does it still fail with these steps?

@laptou
Copy link
Author

laptou commented Jul 24, 2024

Does it still fail with these steps?

Yes. I updated the StackBlitz with an NPM override, and it did not work.

@Lms24
Copy link
Member

Lms24 commented Jul 26, 2024

quick update: I'm a bit busy with other stuff unfortunately. But we asked @timfish (whenever you have some time, Tim) to take another look in the meantime.

@timfish
Copy link
Collaborator

timfish commented Jul 27, 2024

I am working on a solution for this!

@mydea
Copy link
Member

mydea commented Jul 29, 2024

@timfish is there a way to set this registerEsmLoaderHooks: { exclude: ['drizzle-orm'] }, in the preload somehow? As I need to set up beforeSend hook and it requires me to import other modules, so lazy is the only option, I would prefer to use preload with that function.

These other modules are TS files, so it's a bit complicated e.g. I'm using @swc-node/register

node --import @sentry/node/preload --import @swc-node/register/esm-register ./src/main.ts

@RIP21 Since v8.20.0, you can now pass registerEsmLoaderHooks to preloadOpenTelemetry. With this, you can build your own preload file that you can use instead of @sentry/node/preload! See #12998

@RIP21
Copy link

RIP21 commented Jul 29, 2024

@mydea yup. I already used it yesterday. Thanks for the swift fix :)

@laptou
Copy link
Author

laptou commented Aug 7, 2024

@RIP21 I don't understand how you solved the issue. Can you give an example please? I'm not using a preload because I don't have any special hooks, but I would love to use one if it will let me add Sentry back into my application.

@RIP21
Copy link

RIP21 commented Aug 7, 2024

@laptou inused include instead of exclude, including all the dependencies (excluding dev dependencies) I use except Drizzle-orm and it worked. Just doing exclude on Drizzle-orm didn't work out for me.

@timfish @mydea FYI, maybe it's also a problem, but I hope underlying issue will be solved so these include/exclude won't be needed (BTW which issues to subscribe to to track that?)

@mydea
Copy link
Member

mydea commented Aug 8, 2024

We are working on #13139, and will eventually look at using this by default, which will allow you to skip wrapping anything that we are not explicitly instrumenting, without having to define packages manually!

@timfish
Copy link
Collaborator

timfish commented Sep 9, 2024

v8.29.0 of the Node SDK adds the new onlyIncludeInstrumentedModules option to registerEsmLoaderHooks.

import * as Sentry from '@sentry/node';

Sentry.init({
  dsn: '__PUBLIC_DSN__',
  registerEsmLoaderHooks: { onlyIncludeInstrumentedModules: true },
});

When set to true, import-in-the-middle will only wrap ESM modules that are specifically instrumented by OpenTelemetry plugins. This is useful to avoid issues where import-in-the-middle is not compatible with some of your dependencies.

This feature will only work if you Sentry.init() the SDK before the instrumented modules are loaded. This can be achieved via the Node --register and --import CLI flags or by loading your app code via async import() after calling Sentry.init().

Please let me know if this helps solve the import-in-the-middle incompatibility issues you are experiencing!

@AbhiPrasad
Copy link
Member

Closing this for clean-up. If this issue still applies, please re-open. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Package: astro Issues related to the Sentry Astro SDK
Projects
Archived in project
Development

No branches or pull requests

7 participants