MBrace on .NET core

[FedericoBinda]

Hi

I am trying to make MBrace work in a .NET Core project. However, when I try to start a local thespian cluster I get a 'System.PlatformNotSupportedException'. If I try instead to make it work with an Azure cluster, I get all kinds of compatibility errors when I include the MBrace.Azure.Management package in my Paket dependencies and run 'paket install'.
Is there a way to make any of this work on .NET Core or should I just give up?

[isaacabraham]

@FedericoBinda Unfortunately MBrace is not supported on .NET Core. This is due to changes in the way runtime serialization and appdomains work (@eiriktsarpalis can elaborate more) which meant that a large effort would need to be undertaken to enable this.

It's something I would dearly love to see but there's currently no roadmap for this.

Sorry.

[FedericoBinda]

OK, thanks for the info.

[dsyme]

This is due to changes in the way runtime serialization and appdomains work (@eiriktsarpalis can elaborate more)

I think with .NET Standard 2.0 we could revisit this at some point.

AMong other things I'd like to see us refactor MBrace back into a single solution so we can work on it more easily - at the moment the engineering is too hard with all the different repos and packages to publish. Just making an end-to-end update is very painful.

[isaacabraham]

That would be fantastic. I don't personally have an understanding of the intricacies of porting things like Vagabond to .NET Core (my understanding is that this was the main "blocker" as such).

And yes, porting everything MBrace-ish into a monolithic solution would make life easier (even if some of those bits were distributed as distinct nuget packages).

[mikev]

My team works on three different projects. One is using .net framework, while the other two are .net core projects. Many of the reasons we decided on .net core, including performance and cross-platform support Microsoft enumerates - https://docs.microsoft.com/en-us/dotnet/standard/choosing-core-framework-server.

One .net core project is devoted to big data services, while the other supports an application built using MS's new Razor MVC engine. Both .net core projects could take good advantage of MBrace, In particular we could incorporate Mbrace into the data services project. We're currently blocked from using Mbrace given the lack of support for .net core.

We will definitely use Mbrace when .net core support is provided. +1 to add this feature.

[isaacabraham]

@mikev you're in the same boat as we are here. MBrace on .NET Core would be fantastic.

[eiriktsarpalis]

Main blocker for a .NET Core move AFAICT is lack of a repl that targets netstandard or netcorepp. It removes the primary appeal for using this. AppDomains can be replaced with out-of-proc execution with a little bit of work.

[dsyme]

Main blocker for a .NET Core move AFAICT is lack of a repl that targets netstandard or netcorepp. It removes the primary appeal for using this. AppDomains can be replaced with out-of-proc execution with a little bit of work.

The REPL can be made to work fairly easily - it does actually work today if you pass in masses of explicit references. Or you could build a new one with FSharp.COmpiler.Service - TBH it could even make sense for MBrace or its hypothetical successor to have its own REPL

[dsyme]

Here's example code that launches the FSI.EXE on .NET Core correctly: https://github.com/Microsoft/visualfsharp/blob/master/tests/scripts/fsci.fsx

[mikev]

I've undertaken a first effort to port the code to dotnetcore2.2 using VS 2019. At the least, this helps me to understand the project better. Core and Flow are easily ported. Runtime and Vagabond is the blocker. My assumption is that the entire cluster and clients could all be homogeneous dotnetcore. I'd discontinue using Mono, since dotnetcore is cross-platform between Linux/Windows and I don't see any advantage to supporting dotnetframework.

My project VS2019 forks/branch is here:
Vagabond
https://github.com/mikev/Vagabond/tree/ftr-dotnetcore
MBrace
https://github.com/mikev/MBrace.Core/tree/ftr-dotnetcore

TBH it could even make sense for MBrace or its hypothetical successor to have its own REPL

That sound interesting. I'm open to working on this.

AMong other things I'd like to see us refactor MBrace back into a single solution so we can work on it more easily

Agree with this. Having just one solution would be ideal.

Next steps?

[isaacabraham]

@mikev I've been chatting with several people about this. In my opinion the first thing is sorting Vagabond out - that's the guts of the "magic" of MBrace. @johnazariah mentioned to me last week that he would be interested in having a crack at looking at this.

My understanding is that the MBrace runtime doesn't have anything itself that is specific to net framework (might be wrong here...).

I'd be even inclined to consider a "reboot" of MBrace under a new name that keeps some abstractions (the core cloud { } computation, CloudFlow, CloudStore and some other bits) but strips away some others. Certainly the Azure implementation can be massively simplified with some of the newer services that are out there that weren't available when MBrace was first created.

[eiriktsarpalis]

Certainly the Azure implementation can be massively simplified with some of the newer services that are out there that weren't available when MBrace was first created.

Service bus can almost certainly be replaced with azure queues