Skip to content

feat: An option to disable triggers validation #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

elfenlaid
Copy link
Contributor

The changes introduce an option that disables triggers validation

The motivation behind the option is aimed mainly at test environments. Validation is great, but it works in an asynchronous manner, which gets in the way of sandboxed repos (Ecto.Adapters.SQL.Sandbox.mode(Repo, :manual))

So, when running tests, you can hit the following error:

(DBConnection.OwnershipError) cannot find ownership process for #PID
2025-06-14 16:16:31.087 [error] Task #PID<0.629.0> started from EctoWatch terminating
** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.629.0>.

When using ownership, you must manage connections in one
of the four ways:

* By explicitly checking out a connection
* By explicitly allowing a spawned process
* By running the pool in shared mode
* By using :caller option with allowed process

The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.

The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.

The fourth option requires [caller: pid] to be used when
checking out a connection from the pool. The caller process
should already be allowed on a connection.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

See Ecto.Adapters.SQL.Sandbox docs for more information.
    (ecto_sql 3.12.1) lib/ecto/adapters/sql.ex:1093: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_watch 0.13.2) lib/ecto_watch/watcher_trigger_validator.ex:180: EctoWatch.WatcherTriggerValidator.sql_query/2
    (ecto_watch 0.13.2) lib/ecto_watch/watcher_trigger_validator.ex:147: EctoWatch.WatcherTriggerValidator.find_triggers/1
    (ecto_watch 0.13.2) lib/ecto_watch/watcher_trigger_validator.ex:114: anonymous fn/1 in EctoWatch.WatcherTriggerValidator.triggers_by_repo_mod/0
    (elixir 1.18.4) lib/map.ex:257: Map.do_map/2
    (elixir 1.18.4) lib/map.ex:251: Map.new_from_map/2
    (ecto_watch 0.13.2) lib/ecto_watch/watcher_trigger_validator.ex:38: EctoWatch.WatcherTriggerValidator.run/1
    (elixir 1.18.4) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &EctoWatch.WatcherTriggerValidator.run/1
    Args: [nil]

From my understanding, that happens because the validator tries to return a database connection to the pool, but isn't authorized to do that. Mostly because it was spawned outside test cases.

The validate_triggers? makes it possible to skip validation in tests and avoid the above error altogether.


That said, please let me know if you have any other ideas to avoid the mentioned scenario. I'm more than willing to cooperate on this.

Also, thank you for such a nice library! 🙇

@cheerfulstoic
Copy link
Owner

Thanks for the contribution!

Thinking about this: to it might be clearer about what's going on if the application just didn't add EctoWatch to the supervision tree in these cases? So like:

  children = [
    ... other children ...
    if Mix.env != :test do
      {EctoWatch,
        repo: MyApp.Repo,
        pub_sub: MyApp.PubSub,
        watchers: [
          {User, :inserted},
          {User, :updated},
          {User, :deleted},
          {Package, :inserted},
          {Package, :updated}
        ]}
    else
      :ignore
    end

Or maybe there could be a function:

  def ecto_watch_child_spec do
    if Mix.env != :test do
      {EctoWatch,
        repo: MyApp.Repo,
        pub_sub: MyApp.PubSub,
        watchers: [
          {User, :inserted},
          {User, :updated},
          {User, :deleted},
          {Package, :inserted},
          {Package, :updated}
        ]}
    else
      :ignore
    end
  end

But also, I think sometimes people are going to want to test their ecto_watch logic in some of their tests. They would need to adjust the sandbox logic for those tests, but then that also means that disabling ecto_watch entirely in test mode isn't good and we'd need a more "run time" way 🤔 Thoughts?

@elfenlaid
Copy link
Contributor Author

But also, I think sometimes people are going to want to test their ecto_watch logic in some of their tests. They would need to adjust the sandbox logic for those tests, but then that also means that disabling ecto_watch entirely in test mode isn't good and we'd need a more "run time" way 🤔 Thoughts?

Indeed, I was trying to find a middle ground that will allow ecto_watch to work in both test and non-test environments.

Maybe it's just better to accept that the library won't run out-of-the-box in a default test environment. That said, one can always explicitly run it with start_supervised where needed.
It will still invoke triggers validation, but as long as a database in a shared mode, I think it should be okay.

I'll give it a spin and let you know how it went.

@cheerfulstoic
Copy link
Owner

Maybe it's just better to accept that the library won't run out-of-the-box in a default test environment.

Hrmmm, that made me think, TBH... One goal is for it to "just work" for people... I wonder if perhaps if ecto_watch should always be disabled in MIX_ENV=test since that's the default behavior most people will want for most of their tests. Then there could be a hexdocs guide page suggesting that people should use start_supervised if they want to test their ecto_watch integration. Of course you would want to have that configuration not just be directly in the application.ex file and either be in a function or, probably better, a separate module. Maybe a module with use Supervisor that's responsible for all ecto_watch instances might be a good general practice anyway... 🤔 Thoughts?

@elfenlaid
Copy link
Contributor Author

Hrmmm, that made me think, TBH... One goal is for it to "just work" for people... I wonder if perhaps if ecto_watch should always be disabled in MIX_ENV=test since that's the default behavior most people will want for most of their tests. Then there could be a hexdocs guide page suggesting that people should use start_supervised if they want to test their ecto_watch integration.

That makes sense. Yet, I think the library as close to working out-of-the-box in tests as it gets. The rest of the settings involve some integration work. Depending on the approach you take, maybe https://github.com/ash-project/igniter could manage that integration step automatically.

And quite ironically, the one thing that kinda gets in the way of tests, at least for my project, is that automatic trigger validation. But I'm not sure if it's feasible to toggle it based on Mix.env from the library's side 🤔

Of course you would want to have that configuration not just be directly in the application.ex file and either be in a function or, probably better, a separate module. Maybe a module with use Supervisor that's responsible for all ecto_watch instances might be a good general practice anyway... 🤔 Thoughts?

Indeed, that's the approach I'm taking:

Using a separate module to spin up `EctoWatch`
# config/test.exs
conifg(:myapp, MyApp.RepoWatcher, enabled: false)

defmodule MyApp.RepoWatcher do
  use Supervisor

  def start_link(args) do
    name = Keyword.get(args, :name, __MODULE__)
    enabled? = Keyword.get_lazy(args, :enabled?, &enabled?(name))

    if enabled? do
      Supervisor.start_link(__MODULE__, args, name: __MODULE__)
    else
      :ignore
    end
  end

  def enabled?(module) do
    :myapp
    |> Application.get_env(module, [])
    |> Keyword.get(:enabled, true)
  end

  @impl true
  def init(_) do
    children = [
      {EctoWatch,
       repo: Mindme.Repo,
       pub_sub: Mindme.PubSub,
       watchers: [
         MySchema
       ]}
    ]

    opts = [strategy: :one_for_one]
    Supervisor.init(children, opts)
  end
end

While in tests it looks like this:

defmodule MyApp.DataTest do
  use MyApp.DataCase, async: false

  setup do
    start_supervised({MyApp.RepoWatcher, enabled?: true})
  end
end

That said, I'm yet to try it out for real 😅

@ponychicken
Copy link
Contributor

There is another valid reason to diable the notifications.
Try setting up a PSQL partioned table.
Then when Ecto setups the triggers PSQL will in the background actually set triggers to each child partition table.
Then on next start EctoWatch will complain lengthily about additional triggers because it does not take this into effect.
Its a bit complicated to solve, but until then it would be deinitively good to be able to just silence this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants