Skip to content

SQLite extension loading via sqlx.toml for CLI and query macros #3713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: sqlx-toml
Choose a base branch
from

Conversation

djarb
Copy link

@djarb djarb commented Jan 29, 2025

fixes issue #3330 by allowing the query checking macros to load SQLite extensions specified in sqlx.toml.

Related to that, also allows the CLI to load the SQLite extensions specified in sqlx.toml when running migrations

Copy link
Collaborator

@abonander abonander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very keen in landing this, but besides fixing the build, it does also need some sort of test to prove that it works, and ideally a compile-checked example that shows how to use it. In #3383 I realized I could take out two birds with one stone by adding some example projects showing off the new configuration. I'd suggest doing the same.

@abonander
Copy link
Collaborator

(I'm just now realizing the build failures are from #3383 itself so disregard that.)

@djarb
Copy link
Author

djarb commented Jan 31, 2025

I'll be happy to add an example, but there's a complication: to demonstrate this functionality, it's necessary to load a SQLite extension, and that requires that the example is dependent on having that extension installed. Are you okay with having an example that has a third-party dependency which is not shipped with SQLx? If so, what is your preference for how I structure that?

@abonander
Copy link
Collaborator

Our CI already downloads an extension from https://github.com/nalgeon/sqlean/ for testing:

- run: mkdir /tmp/sqlite3-lib && wget -O /tmp/sqlite3-lib/ipaddr.so https://github.com/nalgeon/sqlean/releases/download/0.15.2/ipaddr.so

One thing is that for Unix platforms, you need to show the user either how to set LD_LIBRARY_PATH or to specify a relative path to the extension.

@djarb djarb requested a review from abonander February 3, 2025 11:06
@djarb
Copy link
Author

djarb commented Feb 12, 2025

Is there anything further that you need from me to get this merged?

@djarb
Copy link
Author

djarb commented Mar 28, 2025

Hello? Just checking in. I'd really appreciate it if this could be merged, or if you could let me know what I need to do to get it to that point.

Copy link
Contributor

@joshka joshka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a maintainer, so take my review as just from an interested party. I found this based on reading the semi-related jiff-sqlx issue at BurntSushi/jiff#272).

Comment on lines +42 to +47
/// UNSTABLE: for use with `sqlx-cli`
///
/// Connect to the database, and instruct the nested driver to
/// read options from the sqlx.toml file as appropriate.
#[doc(hidden)]
pub fn connect_with_config(url: &str) -> BoxFuture<'static, Result<Self, Error>>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be reasonable to make this available to apps using sqlx other than just the cli? What makes it a sqlx-cli only option? Would it make sense to instead make loading the config file default functionality (or perhaps feature gated instead of adding a new method)?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps, but I was working on the principle of least change. The CLI is the only place I needed it. It would not be tagged as pub if that weren't a technical necessity, so marking it unstable and hiding it in the docs is the next best option.

Comment on lines 20 to +22
pub database_url: Url,
pub log_settings: LogSettings,
pub enable_config: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that these are part of the public interface, they should probably all have doc comments.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, but not having a doc for enable_config is consistent with the surrounding code. Documenting all of the options would be a subject for a different pull request.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I'd say that documenting new code is more important than being consistent with undocumented items. enable_config is effectively meaningless as a field name without additional context. Obviously we have it right now, but the docs are for people that come later.

Comment on lines +79 to +80
/// Specify extensions to load.
///
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth adding docs about the file path / LD_LIBRARY_PATH here similar to above?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems redundant to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason that it's not (IMO) is the docs page is where you would generally discover information about how to configure this structure. The reference toml file may never be seen or even thought of as a possible place where there's documentation about this by someone looking for config information.

Doc formatting tweak

Co-authored-by: Josh McKinney <[email protected]>
@abonander abonander force-pushed the sqlx-toml branch 2 times, most recently from 0133038 to 5cac661 Compare May 3, 2025 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants