-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial .NET Aspire integration prototype #585
Conversation
@davidfowl This is a really awesome inner loop experience for folks not using EF Core Migrations, but "databasemodel first" or even just Dapper / raw ADO.NET. Who can help with the warning mentioned above? |
@pksorensen FYI! (and any comments / advice you may have) |
I was able to run and publish the .dacpac! In order to run this locally in VS, I installed Docker Desktop and updated the .sln file to reference the TestProject. I also added some robustness to Extension.cs to avoid some build warnings. |
I was thinking that maybe the database project should be added as a custom resource, not as the built-in project resource, so you would get something like: var builder = DistributedApplication.CreateBuilder(args);
var databaseProject = builder.AddDatabaseProject<Projects.TestProject>("db");
var sql = builder.AddSqlServer("sql")
.AddDatabase("test")
.WithDatabaseProject(databaseProject);
builder.Build().Run(); Then we can further customize the database project to include additional properties relevant for the deployment as well in the future and scope the |
@jmezach Yes, that seems the way to go, and would probably also remove the ASPIRE004 warning |
I'm afraid it won't fix that, since that warning is coming from the fact that I've added a reference to the project. If I remove the It might fix another issue I noticed when I tried to debug my AppHost project. It tried to run and attach a debugger to the database project, which obviously doesn't work. |
I did similar for my database deploy projekt - the problem i ran into was that Aspire do not "create" databases, so the deploy part is responsible of creating the actually database if its missing. This was a gotcha for me that i had to work that out myself. Inspecting the code i dont see anything related to this but maybe bacpac takes care of it |
@pksorensen Thanks for chiming in! The dacpac magic takes care of creating the database and the objects inside it. |
The goal for me was to make it super easy for a new dev on the project to start the project. So stuff i was looking into was also to be able to run the aspire project (F5 experience) where it restores a database from a .bak file to be able to provide initial source data from some "restore point" to make it easier on dev to run/debug issues locally after a project has gone live. (a bit more advanced seed data that is able to fullfill usecases of the project for easier testing) |
@pksorensen You can also add a SQL MERGE script as a post deploy step to a .dacpac |
I've done some more updates on this. First of all I've introduced the Now that I'm seeing this I'm wondering if we should perhaps inverse the relation ship between the SqlServer and the database project. So instead of this: var builder = DistributedApplication.CreateBuilder(args);
var databaseProject = builder.AddDatabaseProject<Projects.TestProject>("db");
var sql = builder.AddSqlServer("sql")
.AddDatabase("test")
.WithDatabaseProject(databaseProject);
builder.Build().Run(); If we instead should do this: var builder = DistributedApplication.CreateBuilder(args);
var sql = builder.AddSqlServer("sql")
.AddDatabase("test");
var databaseProject = builder.AddDatabaseProject<Projects.TestProject>("db");
.DeployTo(sql);
builder.Build().Run(); |
Awesome. Would inverting the relationship change the UX in the dashboard? |
No, I don't think so :). |
Agree on the updated fluent syntax! |
I made that change now. It also makes the lifecycle hook a bit cleaner as now we only have to loop through the database projects, instead of going through all the SqlServerDatabaseResources and then all the annotation on that. Have to say I'm pretty happy with how this has turned out so far and I think it can be a real game changer. |
There are some challenges with finding the .dacpac and ensure the project has been built... Is it possible to use the MsBuild SDK to load the project, or invoke a dotnet build and get some structured output from it? |
I think the DeployTo is interesting. I agree that its easier when reading the code to understand what is going on, but i dont think i seen that in other aspire stuff. Feels like it been mosly "AddReference" and such. I personally however prefer the I am trying to figure out why that "deployTo" does not feel right with me (read, that i dont feel that strongly about it) but i think its something related to that i feel that aspire describes the relationship between resources and not that we are deploying a project onto the database. Deploying is something that is generated from metatada from the relationship metadata in aspire. Senarie could be that i want to use a database project in the database, but i dont want it to "deploy" it. Just thinking out loud, i think this stuff is nice and i can use this as inpiration to my stuff to, thanks @ErikEJ for looping me in. |
I tried using But I'm not sure whether it is worth it. I think we can safely assume that the project has already been build, since you need to add it as a reference to your AppHost project anyway. As for the "magic" strings, I think those can be assumed as well as we're setting these in the Sdk.props anyway. The only thing I can think of that might provide additional benefit is if we could read the deployment related properties from the project file and use them while deploying the |
"Senarie could be that i want to use a database project in the database, but i dont want it to "deploy" it" You can not "use" a database project without publishing (deploying) it to a database. Would PublishTo be easier to understand? (Yeah, naming is hard) |
I guess |
@jmezach So we could read the publish properties from just .xml parsing? |
@jmezach "publish" is also what sqlpackage does |
I think we might be able to get to the deployment properties by just using |
Okay, I gave up and just used MSBuild after all. Now I'm just evaluating the project and getting the Also renamed the |
I will have another play with this this weekend. Looks like the package name should be something that ends with .Aspire.Hosting |
Having looked more into Aspire and the configuration patterns, I think this is more natural (but an opinion from an actual Aspire expert would be helpful) var builder = DistributedApplication.CreateBuilder(args);
var databaseProject = builder.AddDatabaseProject<Projects.TestProject>("db");
builder.AddSqlServer("sql")
.AddDatabase("test")
.WithDatabaseProject(databaseProject);
builder.Build().Run(); var builder = DistributedApplication.CreateBuilder(args);
builder.AddSqlServer("sql")
.AddDatabase("test")
.WithDacpac("myTestDacpac");
builder.Build().Run(); |
One problem I see with that proposal is that in the first case we add a resource to the I think having the "dacpac" as a resource in the model kind of makes sense, as we can also hook up logs and status to that resource. Perhaps we should use the "Data-Tier Application" moniker for this resource, so something like this? Use case - Referenced project: var builder = DistributedApplication.CreateBuilder(args);
var database = builder.AddSqlServer("sql")
.AddDatabase("test");
var databaseProject = builder.AddDataTierApplication<Projects.TestProject>("db");
.PublishTo(database);
builder.Build().Run(); Use case - Local var builder = DistributedApplication.CreateBuilder(args);
var database = builder.AddSqlServer("sql")
.AddDatabase("test");
var databaseProject = builder.AddDataTierApplication("db", "/path/to/MyLocal.dacpac");
.PublishTo(database);
builder.Build().Run(); |
Looks great! |
Did a quick spike on the other overload: using Aspire.Hosting.Lifecycle;
using Microsoft.Build.Evaluation;
using Microsoft.Build.Locator;
using Microsoft.Extensions.Logging;
using Microsoft.SqlServer.Dac;
namespace Aspire.Hosting;
public static class Extensions
{
public static IResourceBuilder<DatabaseProjectResource> AddDataTierApplication<TProject>(this IDistributedApplicationBuilder builder, string name)
where TProject : IProjectMetadata, new()
{
var resource = CreateDatabaseProjectReference(name);
return builder.AddResource(resource)
.WithAnnotation(new TProject());
}
public static IResourceBuilder<DatabaseProjectResource> AddDataTierApplication(this IDistributedApplicationBuilder builder, string name, string path)
{
var resource = CreateDatabaseProjectReference(name);
return builder.AddResource(resource)
.WithAnnotation(new TargetDacpacResourceAnnotation(path));
}
public static IResourceBuilder<DatabaseProjectResource> PublishTo(
this IResourceBuilder<DatabaseProjectResource> builder, IResourceBuilder<SqlServerDatabaseResource> project)
{
builder.ApplicationBuilder.Services.TryAddLifecycleHook<DeployDatabaseProjectLifecycleHook>();
builder.WithAnnotation(new TargetDatabaseResourceAnnotation(project.Resource.Name), ResourceAnnotationMutationBehavior.Replace);
return builder;
}
public static IProjectMetadata? GetProjectMetadata(this DatabaseProjectResource resource)
{
return resource.Annotations.OfType<IProjectMetadata>().SingleOrDefault();
}
public static TargetDacpacResourceAnnotation? GetDacpacMetadata(this DatabaseProjectResource resource)
{
return resource.Annotations.OfType<TargetDacpacResourceAnnotation>().SingleOrDefault();
}
private static DatabaseProjectResource CreateDatabaseProjectReference(string name)
{
MSBuildLocator.RegisterInstance(MSBuildLocator.QueryVisualStudioInstances().OrderByDescending(
instance => instance.Version).First());
var resource = new DatabaseProjectResource(name);
return resource;
}
}
public sealed class DatabaseProjectResource(string name) : Resource(name)
{
public string GetDacpacPath()
{
if (this.GetProjectMetadata() == null)
{
if (this.GetDacpacMetadata() is null)
{
throw new InvalidOperationException("Path to .dacpac file not specified.");
}
return this.GetDacpacMetadata()!.TargetDacpacPath;
}
else
{
var projectPath = this.GetProjectMetadata()!.ProjectPath;
var project = new Project(projectPath);
return project.GetPropertyValue("TargetPath");
}
}
}
public record TargetDacpacResourceAnnotation(string TargetDacpacPath) : IResourceAnnotation
{
} |
Adding this to Sdk.Props gets rid of the ASPIRE0004 error, but unsure if it has any other consequences (I cannot see any obvious): <PropertyGroup>
....
<OutputType>Exe</OutputType>
</PropertyGroup>
|
Signed-off-by: Jonathan Mezach <[email protected]>
Signed-off-by: Jonathan Mezach <[email protected]>
Signed-off-by: Jonathan Mezach <[email protected]>
Made some changes so that we try Also improved the error handling a bit. |
Publishing ready event would be a great addition, so we can support scenarios like Data Api Builder expecting the schema to be present at startup. |
@jmezach Can confirm .sqlprojx support works great! |
Just saw this: https://github.com/CommunityToolkit/Aspire |
@ErikEJ Yeah, saw that too. Looks interesting. Not entirely sure though if it makes sense to move this to the toolkit. Or do you feel like it should be part of it? |
@jmezach There are pros and cons, but it looks like we will gain a lot of visibility in VS https://github.com/CommunityToolkit/Aspire/blob/main/docs/faq.md#finding-community-toolkit-integrations Con: More formal constrains, but does not seem too bad. |
I think the view will also include packages from community according to the FAQ linked above.. |
Full error message when trying to load a classic .sqlproj:
Wonder if we could do some probing given we have the path to the .sqlproj? |
I managed to add support for classic .sqlproj with the following changes: using Aspire.Hosting;
using Aspire.Hosting.ApplicationModel;
using Microsoft.Build.Evaluation;
using Microsoft.Build.Exceptions;
namespace MSBuild.Sdk.SqlProj.Aspire;
public sealed class SqlProjectResource(string name) : Resource(name)
{
public string GetDacpacPath()
{
var projectMetadata = Annotations.OfType<IProjectMetadata>().FirstOrDefault();
if (projectMetadata != null)
{
try
{
var projectPath = projectMetadata.ProjectPath;
using var projectCollection = new ProjectCollection();
var project = projectCollection.LoadProject(projectPath);
// .sqlprojx has a SqlTargetPath property, so try that first
var targetPath = project.GetPropertyValue("SqlTargetPath");
if (string.IsNullOrWhiteSpace(targetPath))
{
targetPath = project.GetPropertyValue("TargetPath");
}
return targetPath;
}
catch (InvalidProjectFileException ex)
{
if (projectMetadata.ProjectPath.EndsWith(".sqlproj", StringComparison.OrdinalIgnoreCase))
{
var dacpacPath = GetSqlprojDacpacPath(projectMetadata.ProjectPath);
if (dacpacPath != null)
{
return dacpacPath;
}
}
throw new InvalidOperationException($"Failed to load SQL Server Database project file {projectMetadata.ProjectPath}.", ex);
}
}
var dacpacMetadata = Annotations.OfType<DacpacMetadataAnnotation>().FirstOrDefault();
if (dacpacMetadata != null)
{
return dacpacMetadata.DacpacPath;
}
throw new InvalidOperationException($"Unable to locate SQL Server Database project package for resource {Name}.");
}
private string? GetSqlprojDacpacPath(string projectPath)
{
var directory = Path.GetDirectoryName(projectPath);
if (directory == null)
{
return null;
}
var searchPath = Path.Combine(directory, "bin\\debug");
var file = FindFile(searchPath);
if (file != null)
{
return file;
}
searchPath = Path.Combine(directory, "bin\\release");
return FindFile(searchPath);
}
private static string? FindFile(string searchPath)
{
if (!Directory.Exists(searchPath))
{
return null;
}
var files = Directory.GetFiles(searchPath, "*.dacpac", SearchOption.AllDirectories)
.Where(f => !f.EndsWith("\\msdb.dacpac", StringComparison.OrdinalIgnoreCase)
&& !f.EndsWith("\\master.dacpac", StringComparison.OrdinalIgnoreCase))
.ToList();
if (files.Count == 1)
{
return files[0];
}
return null;
}
} |
Signed-off-by: Jonathan Mezach <[email protected]>
I've now added publishing of an event when we're done deploying the |
Signed-off-by: Jonathan Mezach <[email protected]>
Signed-off-by: Jonathan Mezach <[email protected]>
Cool, love the redeploy, very useful. And the eventing makes scenarios like Data Api Builder so simple! (In particular when a Data Api Builder hosting component is also available) |
I have just confirmed that project discovery also works with the new .sqlproj projects supported by Azure Data Studio / VS Code |
Not entirely sure what you mean with that ;). |
@jmezach There are currently multiple project types to build .dacpacs. Azure Data Studio and VS Code mssql extension support a project type with a .sqlproj extension that is the same as .sqlprojx (preview) in Visual Studio. The Aspire integration can load this project type. |
Is this a known problem that the package fails if the However it seems to work properly with the Here is the error that I got:
|
@kzryzstof See this: #585 (comment) - do you have a preview SDK installed? Try a global.json file. |
@ErikEJ So I added the global.json file as suggested and chose the 8.0.403 SDK (not a preview). It still failed with the same error :( All my projects are running |
@jmezach has a PR approved in the official Aspire Community Toolkit repo for Aspire 9, which will be published very soon. Suggest you wait for that, the current package will be deprecated very soon. |
We have decided that this integration is better served from the CommunityToolkit for Aspire rather than integrating it here. Thanks to everyone who has tried out this feature. We highly recommend migrating to CommunityToolkit version of this integration. |
As described in #584 we think there is value in providing some kind of integration between MSBuild.Sdk.SqlProj and .NET Aspire which would essentially allow a developer to spin up a SQL Server container and deploy a database project it. This could significantly enhance the inner loop.
This is a very early prototype of what such an integration could look like. A couple of observations:
Extensions.cs
should probably move into a separate project (something likeMSBuild.Sdk.SqlProj.Aspire
maybe?)warning ASPIRE004: '../TestProject/TestProject.csproj' is referenced by an Aspire Host project, but it is not an executable. Did you mean to set IsAspireProjectResource="false"?
. Setting that flag obviously resolves the warning, but then we can't get a hold of the project inProgram.cs
. Perhaps this should be a question to the .NET Aspire team..dacpac
that needs to be deployed. It looks like .NET Aspire only provides us with the path to the.csproj
but no other metadata from MSBuild that we can use (at least not that I've seen).dacpac
I'm using the DacFx package directly, so it is essentially part of the AppHost project. Not sure if that's the right way to go, but at least it works.@ErikEJ @jeffrosenberg Thoughts?