breaking change in tower design (0.6 or beyond): first class support for async fn traits

[GlenDC]

Hi, I would like to open this work to start the discussion and start to make progress towards getting tower ready for when async fn in traits and impl return values for traits are allowed.

As I really needed this kind of first class support I have a completely ported version of tower ready at https://github.com/plabayo/tower-async.

• it is meant to unblock me and very opiniated;
• it does not reflect my willingness to work with you on making an actual tower version with such support a reality, I am open to any direction or approach you want me. That might be close to the approach I took or something very different
• tower-async can be serve as a starting point of discussion. And While it can reflect the design we take, I am also fine for it to be nothing more then a starter for this discussion and be completely ignored after that
• Even if we choose to go for a new design, I am more then happy to release a new breaking release of towers (and as many versions as it takes) to try out the ideas. I am more then happy to use that repo as a playground, and I'm actually using it in production, so you would get actual use feedback in my rama project :)

I took a couple of drastic changes:

• first of all the signature changes of Service. But that is obvious as it's the entire idea of this proposal. You can see that trait at: https://github.com/plabayo/tower-async/blob/6680bc9422083d893c42d5c5a0a28293bf10f281/tower-async-service/src/lib.rs#L196-L209
  • besides the async fn support you'll notice that:
    1. I dropped poll_ready: see the FAQ: https://github.com/plabayo/tower-async#faq, happy to discuss more. Also again, just my current POV, happy to change it if this is not shared with maintainers
    2. I changed from &mut self in &self. This is not a requirement and my first ported version did still have &mut, however:
       • I never had a real need for &mut, and making it & reflects that direction and also gives a simpler life
       • it makes it easier to interface with codebases like hyper (v1);
• because of (1) of the previous point I also dropped stuff that rely on poll_ready. e.g. anything related to load balancing and the like. This is on purpose as I didn't have an eed for it, and I think it's out of scope. I have ideas on how we can support it by providing such code but making it that users would integrate it in either the MakeService stuff or as utility code that they would inject themselves in their call functions. Or have services that can pool other services etc etc. But again I didn't have a need for it or desire, so honestly I didn't push any of those ideas further and just dropped it.

How to play with my experimental tower-async version?

The port of tower and tower-http is completely done and can be used now using the tower-async and tower-async-http crates.

You can use tower-async-bridge crate to interface with classic Tower interfaces (to and from).

You can use tower-async-hyper to interface with the "low level" hyper (v1) library.

All crates are published on crates.io: https://crates.io/search?q=tower-async

How is my experience?

Great. It works and I'm using it in production. I'm also working on rama (https://github.com/plabayo/rama) where I'll have a similar production-ready setting ready as open source. But that is still early days.

Seems that all the design works just fine and with the current stability plans it means that tower-async and tower-async-http will be ready for stable rust this year.

Open Problems

As my production use proves I have no issues/problems any longer that block me. There are however none the less still open problems.

Boxed Services is non trivial

Open issue: plabayo/tower-async#10

This can be solved and I might add it as an experiment to tower-async. But the only way I see how for now is using stuff like call(): Send, which... requires nightly. Dyn trait objects with async is not in active progress and not even sure this will be something that in the 2024 edition will be.

I need exactly this nightly feature for my current tower-async-bridge and tower-async-hyper crates. This makes me also fearful and unsure if we can really provide stable support for hyper anytime soon by only adding code to hyper-util. Because in order to implement hyper::service::Service we will always require a boxed Future. This is only possible with the call(): Send nightly feature, which is not stable anytime soon.

The only hope is that someone can help me figure a way out to add support for async fn Tower trait in hyper-util by making use of the service_fn approach that hyper allows. If we can internally within hyper-util use that, then we do not need a boxed future and thus also not the call(): Send syntax. I have a feeling however that this is only possible once the next described problem (rust-lang/rust#114142) is resolved.

Trait Resolving is not flawless (e.g. might not deduce the future is Send)

Open issue: rust-lang/rust#114142

Example: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=df177519275726a7df18045cf90a59a9

You can come in situations where a higher order function cannot correctly deduce that a future of an async fn trait implementation is Send. A work around for this is turbofish notation to explicitly declare the type that is passed in. This is however not always possible and makes usage also more awkward when you run into this.

So far however I've not come into a situation where this work around does not work. If you need it I would suggest to hide it behind a central point as much as possible so that the rest of your code can be as ergonomic as possible.

This problem and its workaround were also documented since 0.1 of tower-async at: https://github.com/plabayo/tower-async#faq.

Can we run everything in Stable?

Yes. However, currently there is no async fn trait support in hyper or hyper-util. So in order to make something like tower-async-hyper work I currently need to be able to define a Future type and that requires me to Box it. This requires me to use the call((): Send notation (however you call it) and this is going to be a nightly only feature for much longer.

The good news is that this shouldn't be needed for an official async-fn ready tower version as we can then just implement official tower support with async fn trait support directly into hyper-util which would allow us to drop the Boxed Future and thus also the need for Nightly Rust.

Open for feedback

To iterate some of the above.

These are all just proposals though. I am open to any feedback, a totally different direction. I can do more experimentation if you want a different direction (completely or partly). Honestly fully open about it all. I just already needed it as for my purposes (where I do deal with plenty of async fn usage in my middlewares it was becoming a pain in the ass to hand roll these futures myself. If everything in Rust was manual futures it would be a lot easier, and I don't mind the work. But given that some stuff (e.g. plenty of tokio utility code) works with async fn that became very awkward and painful very soon.

tower-async is not meant to be permanent or me wanting it this way or no way. It was just unblock myself. A realisation made in the 900th iteration of rama. After struggling a lot with classic tower in many iterations before that.

Prior work

@LucioFranco had apparently already added a case study for Tower in the context of the async fn trait RFC.

Link: https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/tower.html

I didn't know of its existence until I already have version 0.1 of tower-async. I did know about it prior to implementing version 0.2. As far as I can see it came to very similar conclusions as I have.

It also proposes some potential intermediate solutions, but so far I have not had a need for it.

Next Steps

Most important I think is that the maintainers align on a vision of how they see tower. tower-async can be an inspiration for this, and its design can even be taken as is. But even if none of its designs are taken at all, at least it might hopefully teach some lessons on how to to it different in that case.

Either way, this is is how I see it progress after initial discussions and alignments:

• do small experiments with Rust play grounds of core concepts
• once experiments and more discussions lead to an agreed upon vision and "core" design, I can work on version 0.3 of tower-async to actually implement these ideas and also test them in production use.

This can be seen as a typical design iteration loop, where we cycle through as much as needed and in any desired order. tower-async can be seen as a playground for "final" (hopeful) designs that we think are worth testing in production.

The final step would then be porting tower-async back into tower and tower-http.

Once that is complete the ecosystem can also start adopting it, which we could kickstart by providing first class support for it in hyper-util. I would obviously myself also start using tower once again (instead of tower-async) into rama. Other maintainers, such as the ones of axum can also help by migrating to this.

This requires no change of hyper and neither breaks any 1.0 promises in such systems, as we can provide bridge functionality for this new tower design by adding it to hyper-util, similar to hyperium/hyper-util#46

Timeline

A proposed timeline would be get a version of tower out of the door with this in the next months. By then Rust has already stable support for all required features.

[hawkw]

I took a couple of drastic changes:

• first of all the signature changes of Service. But that is obvious as it's the entire idea of this proposal. You can see that trait at: https://github.com/plabayo/tower-async/blob/6680bc9422083d893c42d5c5a0a28293bf10f281/tower-async-service/src/lib.rs#L196-L209

  • besides the async fn support you'll notice that:

    1. I dropped poll_ready: see the FAQ: https://github.com/plabayo/tower-async#faq, happy to discuss more. Also again, just my current POV, happy to change it if this is not shared with maintainers
    2. I changed from &mut self in &self. This is not a requirement and my first ported version did still have &mut, however:
       • I never had a real need for &mut, and making it & reflects that direction and also gives a simpler life
       • it makes it easier to interface with codebases like hyper (v1);

• because of (1) of the previous point I also dropped stuff that rely on poll_ready. e.g. anything related to load balancing and the like. This is on purpose as I didn't have an eed for it, and I think it's out of scope. I have ideas on how we can support it by providing such code but making it that users would integrate it in either the MakeService stuff or as utility code that they would inject themselves in their call functions. Or have services that can pool other services etc etc. But again I didn't have a need for it or desire, so honestly I didn't push any of those ideas further and just dropped it.

I think cutting scope for the sake of experimentation makes sense, but I'd like to push back on this a little bit.

Removing poll_ready and changing the Service::call receiver to &selfis a substantial change from the existing design, and one that makes tower substantially less expressive. In many ways, tower's primary purpose is to provide a shared abstraction in the form of the Service trait that allows a variety of libraries and codebases to interact. I think it's important to maintain the ability to serve as an integration point, and that means that it's important for tower's central abstraction to be able to abstract over as wide a range of functionality as possible.

With the current design, functionality like pooling, routing, and load balancing can all be represented with the Service trait abstraction. A change that makes these things more difficult or impossible to abstract over using tower is kind of a substantial regression in what tower can be used for, and I think we should avoid that. Many users do make use of the code that had to be removed in order to make these changes, and I want to ensure that these users don't have to give up on tower in the future.

This is not to say that I'm not open to making drastic changes to the Service trait, such as removing poll_ready and/or changing the receiver for call to &self. However, I think that if we're going to make those changes, we need to have a clear story for how stuff that tower can currently represent will be represented in the future. In order to convince me that major changes to the Service trait like this are a good idea, I would want to see a proposal that includes an implementation of tower::balance or similar existing code with the new API. Proving that patterns that can currently be represented using tower are still possible with a new design would demonstrate that a potential change isn't just making tower less useful for the projects that currently rely on it.

[GlenDC]

I can stand behind all of that @hawkw.

I'll wait until more feedback from you and others so it's more clear what further experimentation can aid this discussion.

The original reason that I cut out poll_ready is because I had no use for it and many others neither. That said I do know there is use by it by plenty. The other reason I did is because I didn't see a way to combine that with async fn traits. Of course we could have poll_ready just return a poll instead of it being async and keep it.

That said, I do lean more into the camp of providing that functionality as additional utility code that people can integrate in their service calls. But yeah also there I do agree that if we go that road that we indeed need to make sure that it is possible indeed.

[seanmonstar]

I agree with @hawkw that if those things are removed, some sort of design for tower::balance can still exist. We shouldn't make the decision without explicitly pointing out the future of those.

---

@hawkw I do have one idea, that is probably sufficiently deep enough that it could become its own issue, but I'll dangle it here. I noticed that hyper v0.14's client pool was essentially a SvcAsMakeSvc, which would create a SendRequest in a wrapper which after sending once, it would get inserted back into a cache on Drop. I've been working on a design doc for this, I think this is how a generic tower::Pool should be expressed, since I've failed several times to make it work with poll_ready. My question then, is could tower::balance be expressed similarly? Is that the way to do backpressure?

[Dessix]

@seanmonstar:

I noticed that hyper v0.14's client pool was essentially a SvcAsMakeSvc, which would create a SendRequest in a wrapper which after sending once, it would get inserted back into a cache on Drop. I've been working on a design doc for this, I think this is how a generic tower::Pool should be expressed, since I've failed several times to make it work with poll_ready. My question then, is could tower::balance be expressed similarly? Is that the way to do backpressure?

This is what I've been doing in my implementation atop tower_async, which - rather ironically- made it much harder to express due to lifetime and Send requirement expressibility limitations in the current iteration of Async Fns In Traits. This led me to wanting to box the contents for ease of representation, which led to my filing of plabayo/tower-async#10 in the first place. In the end, I used tower's formulation of MakeService after boxing all async-fn-in-trait-based traits to simplify typing.

While MakeService fits the puzzle nicely, it was very difficult (and extremely verbose, even with type aliases) to translate the fully unboxed types to implementations, though I suspect some of that was due to being unable to directly implement for traits thanks to the orphan rule.

[GlenDC]

It is worth pointing out that it's trivial to go from a classic future tower service to the new async fn tower service, but very hard to reverse. That is to say:

pub trait FutureService<Request> {
    type Response;
    type Error;
    type Future: Future<Output = Result<Self::Response, Self::Error>>;

    fn poll_ready(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>;
    fn call(&self, req: Request) -> Self::Future;
}

A FutureService like that can easily be turned into the new kind of service:

pub trait Service<Request> {
    type Response;
    type Error;

    fn call(
        &self,
        req: Request,
    ) -> impl std::future::Future<Output = Result<Self::Response, Self::Error>>;
}

• Service::call can be implemented for FutureService inline as we just can return the future returned by FutureService::call;
• FutureService::poll_ready would be used similar to utilities already provided: `where you await it first and then run call.

This is of course not interesting if we care only about that conversion. But it does mean that middleware at the "beginning" of your stack could require a FutureService and thus do the balancing, and then the later layers of your stack wouldn't have to care about it and can just require a Service. It would also allow any service in the current ecosystem to be easily converted.

That said, even if you want to provide the kind of features that poll_ready had to provide for, I found its contract always very shaky. First of all you enforce a contract on all users while only a subset of them care about it (that is resolved with an idea described in this section).

The bigger issue is that switching from &mut self to &self (as done in my proposal of this GH Ticket, would make that contract not safe against races.

---

That said, I've always believed that the expressiveness of Tower comes from the fact that the Service contract is the most simple you can get. The poll_ready was always a bit of an awkward one, and for most users that really does nothing. Don't have numbers on it to, so I might live in a bubble, but given that Hyper and Axum neither do something with it, tells a lot.

I do mean that if we want to keep that support I'm all for it, but if so it should be by facilitating those users with utility code and proper documentation how it can be easily achieved. This way it would still be there, but totally opt-in. I don't have a specific design proposal on that one, but it would essentially be some kind of "higher order" Service. Similar to MakeService (or perhaps using it directly).

[davidpdrsn]

Boxed Services is non trivial

This can be solved and I might add it as an experiment to tower-async. But the only way I see how for now is using stuff like call(): Send, which... requires nightly. Dyn trait objects with async is not in active progress and not even sure this will be something that in the 2024 edition will be.

That is something I didn't think about during our previous chats on Discord. Not being able to box services is a deal breaker for axum 😞 No boxing would make the Router type grow as you add more services and middleware. axum used to work this way and it caused lots of compile time issues.

[GlenDC]

I get you there. I am for now doing rama without boxing, but I totally get you. If I make a mistake somewhere so that i do not satisfy a bound it gives really long types and pretty hard to read error messages. I can always solve them, but they are painful and I wouldn't wish them to anyone >.<

That said, I'm pretty certain that we can work around this. There are indirect ways to box. And thus use cases like Axum should be possible. Only sad part is that it will require that we can use things like call(): Send in the bridging logic. And I don't think that will be in stable this year. I do think that it will be available in the 2024 edition.

[hawkw]

The fact that there is (currently) no way to introduce bounds like Send, Sync, 'static, or Unpin on the futures generated by async trait functions is a significant limitation of async fns in traits, which, I think, would make a great deal of existing tower code challenging to represent. If users want to tokio::spawn a call() future, for example, they need to require that it's Send + 'static.

It's worth noting that the current design, with an associated future type, can be used with unboxed async blocks using #[feature(type_alias_in_trait)], which i would certainly hope will stabilize sooner than call(): Send or similar return type bounding. For example, we can implement a Service with the current Service trait using an unboxed async block:

#![feature(type_alias_in_trait)]

impl Service<Req> for MyService {
    type Response = Response;
    type Error = Error;
    // using the `type-alias-in-trait` feature, we can return an `async` block future
    // without boxing it.
    type Future = impl Future<Output = Result<Self::Response, Self::Error>>;
   
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        // ...
    }

    fn call(&mut self, request: Req) -> Self::Future {
        // we can still use async-await to implement `call`!
         async move { 
             // do stuff...
         }
    }
}

Downstream code can depend on this Service implementation and add S::Future: Send + 'static or similar trait bounds on the associated Future type freely, and we can add Send, 'static, or other traits to the associated type like:

    type Future = impl Future<Output = Result<Self::Response, Self::Error>> + Send + 'static;

This allows code to introduce additional trait bounds on a Service's call future type freely (such as Send + 'static to make a future spawnable), but doesn't require tower to add those bounds at the trait level and require them for all implementations of Service. This is something that's currently not possible at all using async fn in traits...

[hawkw]

Unfortunately, I think being able to introduce trait bounds on Service::call futures is a pretty hard requirement for a lot of currently-existing code using tower (e.g. any code that spawns a future returned by Service::call on a work-stealing executor like tokio, which...seems like a thing a lot of people probably want to do).

We probably can't seriously consider an async fn-based Service trait until return-type notation (the call(): Send notation discussed above) is available on stable Rust. Unfortunately, the "Send bound problem" is a pretty well known limitation of the current support async fn as trait methods, and the lang team currently recommends not using it at all in libraries whose users may want to use a work-stealing async runtime.

When RTN is available on stable Rust, we'll probably be able to reconsider this.

[hawkw]

Potentially, we could consider other breaking changes to the Service trait prior to the availability of an RTN feature on stable, such as changing the receiver for call to &self, or removing poll_ready. But it might be preferable to wait until RTN is available to make any breaking changes, so that we don't cause a bunch of ecosystem churn by breaking the interface multiple times over the next year or so.

[GlenDC]

To be honest, I didn't consider the #[feature(type_alias_in_trait)] feature, as I didn't know that this was even on a realistic roadmap. Thought it was just some random thing that maybe some day would be a thing.

As much as I have a forbidden love for async fn traits, I honestly think that if type_alias_in_trait becomes stable that there's a future for tower which never really needs async fn traits? Because honestly that solves the entire original problem I have with tower... Which is that it was hard to call async fn stuff from within a tower service.

So given that this wouldn't need any change from tower, we could perhaps instead focus on a tower 0.6 where we do breaking changes as &mut self -> &self and keep tower working with futures.

People like myself could then already use the nightly feature to already allow async fn like implementations where this is needed / desired. And all is well? Or am I seeing things to optimistic / missing potential use cases here?

[hawkw]

People like myself could then already use the nightly feature to already allow async fn like implementations where this is needed / desired. And all is well? Or am I seeing things to optimistic / missing potential use cases here?

In my opinion, this is probably the best approach, since we get to maintain our current flexibility while still allowing use with unboxed async/await when the user is willing to opt-in to nightly. Perhaps we should add documentation explicitly demonstrating use with #![feature(type_alias_in_trait)], so that users are aware that it's an option for their applications.¹

Footnotes

1. We probably want to advise against its use in libraries, since library authors generally want their code to be usable on stable... ↩

[GlenDC]

I agree that this might be the best option indeed. Saves a lot of potential pain as well.

Also agreed that documenting this is a good idea. We can add that footnote indeed, even though seems obvious that nightly features limit your library's use?

I'm curious what others have to say about this change of heart.

[ekanna]

It looks like there is a typo
#![feature(type_alias_in_trait)] should be read as #![feature(type_alias_impl_trait)]

[GlenDC]

@hawkw I made a little playground of an earlier playground I had shared in the hyper discord as a matter of showing how I connect hyper and tower-async. This time I emulated a kind of tower as we describe here.

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=11b0998956ab3ac077ec54a683d5c425

https://gist.github.com/rust-play/11b0998956ab3ac077ec54a683d5c425

FYI @ekanna

According to the rust playground it seems to be neither of those two.
I needed these to make it work on nightly:

#![feature(impl_trait_in_assoc_type)]
#[allow(incomplete_features)]

It works excellent. Very easy to use and can be used as-is in the existing eco system. Regular tower services would work and implementing tower services where you do need opaque async services (e.g. to call async functions) are just as easy to implement. Genius.

Honestly as much as I would like super magical async fn traits, given how bad the ergonomics are for now (and for much longer to come) I think this is the way to go given the Rust status quo. And honestly I am okay with it.

Only breaking changes I did do in the tower::Service of my playground snippet:

• use &self instead of &mut self (I still think this is the way to go, let me know what you think of that);
• give a default implementation for poll_ready. As this is anyway what the majority of services end up doing we might as well do that? It should even be backwards compatible AFAIK?

I am still not too happy with the fact that poll_ready is still in there though. I don't mind keeping it in, as I get for some people it has use. But I still find it a rare duck in the trait. That is, the Service trait is the minimal representation of what a req -> resp service is, except for the poll_ready which makes it no longer the minimal it can be. Would be great if it can perhaps be turned into a separate trait? So where you have the Service trait and then also the PollReady trait or w/e. Just spit balling here. More important thing is that I do think that given it is the minority case (AFAIK) it might as well be also the thing that you somehow opt-in to rather then that everyone has to drag it.

That said... Maybe I am missing enormous potential and usage and benefits of poll_ready into my proxy crate (rama). Can someone tell me for a proxy library that I use to build distortion proxies (think MITM proxies that emulate browsers), and that I keep minimal in resources but scale in k8s with my usage based on resources and other metrics, what use would poll_ready be for me? I am asking this with an open mind, because maybe there is just enormous value I'm missing that would make me less cringe in keeping it in.

And with that I am looking forward to the feedback of all of you and see how we can move forward this discussion. I am thrilled at the future now more then ever, as the approach that @hawkw suggest does seem to work very fine, and would allow me to drop the tower-async crates a lot faster, making it easier for me to also spend more time in helping improve the "real" tower crates.

[davidpdrsn]

In my opinion, this is probably the best approach, since we get to maintain our current flexibility while still allowing use with unboxed async/await when the user is willing to opt-in to nightly. Perhaps we should add documentation explicitly demonstrating use with #![feature(type_alias_in_trait)], so that users are aware that it's an option for their applications.

I agree this seems like an interesting path that's worth considering!

give a default implementation for poll_ready. As this is anyway what the majority of services end up doing we might as well do that? It should even be backwards compatible AFAIK?

I'm not sure that's possible. Ideally a default implement would be self.inner.poll_ready(cx) but we can't do that in the trait definition. I'm not sure returning Poll::Ready(Ok(())) is the right default because if there is just one middleware in the stack that doesn't propagate backpressure then things break. So all middleware should implement poll_ready to propagate which makes it kind of a footgun to have a default impl in the first place imo.

I'm not really sure how I feel about poll_ready honestly 😅 it's sort of a "known unknown" for me since I've never really used it. So I don't feel confident saying we can just remove it, though for my own needs it is in the way.

[GlenDC]

Well put @davidpdrsn , good catch. That would make a default implementation of poll_ready indeed a footgun. Never mind that small side proposal then.

[hlbarber]

Cross-referencing another Service design: #757

[GlenDC]

FYI I did start to experiment with a boxed tower-async Service (behind a nightly feature flag)...
It's a big mess... But not yet concluded if that mess is my fault due to incompetence, or
because it really is just messy by nature due to how immature it is.

You can find the open blocked (by not working) PR at plabayo/tower-async#12

another point for impl associated type (which sadly doesn't have a specific/real plan towards stability, at least not one with an estimated date of arrival)

[GlenDC]

Another plot twist... In the 5th major iteration on the Rama project I moved away from tower-async... First I was planning to go down the route of providing my own Service but bridging it to Tower, as to be tower (classic) compatible...

I then realised however how big of a mess that is and I especially don't like it as it hinges so much on the assumption that I do not want or need to support the poll_ready feature, which I still believe should be opt-in and part of the connection points in once application where connections/requests are being accepted or branched, if desired at all.

The result can be seen at https://github.com/plabayo/rama/blob/985bb11cad8e86e78776d9cbbd521890aaef27da/src/service/svc.rs and so far I'm pretty happy with it:

• it allows me to use stable rust >= 1.75
• It allows me to use async traits using the impl Future which has the benefit that you can easily switch between hand rolled futures and opaque async fn types
• Using the DynTrait design pattern described in an earlier Rust blog post you can also Box it, and thus allow for stuff such as http routers where dynamic dispatching is more appropriate;

The trait looks as follows:

/// A [`Service`] that produces rama services,
/// to serve requests with, be it transport layer requests or application layer requests.
pub trait Service<S, Request>: Send + Sync + 'static {
    /// The type of response returned by the service.
    type Response: Send + 'static;

    /// The type of error returned by the service.
    type Error: Send + Sync + 'static;

    /// Serve a response or error for the given request,
    /// using the given context.
    fn serve(
        &self,
        ctx: Context<S>,
        req: Request,
    ) -> impl Future<Output = Result<Self::Response, Self::Error>> + Send + '_;

    /// Box this service to allow for dynamic dispatch,
    /// only possible if the service is [`Clone`].
    fn boxed(self) -> BoxService<S, Request, Self::Response, Self::Error>
    where
        Self: Clone,
    {
        BoxService {
            inner: Box::new(self),
        }
    }
}

The boxed method is provided and in a future Rust version is no longer needed. But for now it makes it a lot easier to turn any service into a boxed service where needed. Only thing I do not like about it (besides having to do this temporary work around) is that I require my service to be clone in order to make it work and still be able to support cloning it where needed. This because the graph of trait bounds that I couldn't seem to get work optionally. Then again supporting cloning is anyway something I usually do for services in case I do need it, and most of the time I don't.

In this new design (sadly for now completely separated away from the tower ecosystem, which I dearly admire and like) I do not have to clone except for a couple of cases:

1. In order to support my http services to be used as an hyper::service::Service: This only because it requires me to be able to define a type Future associated type, which requires me to box and thus I need to 'move' a cloned version in order to be able to provide such a 'static future...
2. For locations where I accept connections or branch of requests: this is normal and expected, as I do not really like the MakeService pattern and prefer my services to be cloned (analog to forking processes) where I need to branch off;

And that's it. In most places I don't need it at all... And this all thanks to the fact that all services now require to be Send + Sync + 'static. For a generic library as tower this is probably not something ever desired, and once Rust evolved further it is probably also no longer needed anyway, but for me it makes sense as I only ever operate within one setting and that is multithreaded async, so it makes little sense to then pretend to also support something else and make it not possible for me to use it for all my use cases due to limitations.

First I was only requiring it to be Send + 'static but this brought be in trouble and required me to clone a lot more (in order to be able to go across await points). @davidpdrsn seemed to indicate he was planning to move to Sync requirement for all services which I was first sceptical of, but wrongfully so. It's a brilliant solution for the current state of Rust and it was the last puzzle piece to finish this iteration of.

I'm now at a point where I can finally build the more interesting pieces of Rama. And while I'm sad that for now I am moving further in this project without tower, I also realise it is for the best as throughout this year and many iterations of working on the project on and off, I realised that the biggest thing I each time was working around was tower. That might be because of its current design or simply because I want something different in implementation even if similar in spirit.

You can browse the codebase for seeing how it is used and plays out, in case it might inspire the future direction of tower, or perhaps might inspire how to not do it. Either way I stay available on GitHub and Discord for collaboration and elaboration. Happy new year all and take care.

Extra (not important):

• I renamed the method from call to serve, but this is more to prepare for a future where I might be able to be compatible with a future version of Tower again, as to make the two method calls less ambiguous to understand and use;

[WhyNotHugo]

+1 to replacing &mut self with &self in tower_service::Service::call. This requirement has a source of needless complexity for me.

Returning impl Future<Output = Result<Self::Response, Self::Error>> would entail some reduction in flexibility, but it's hard for me too see any realistic impact with this loss in flexibility anyway.

I would not add a + Send requirement as mentioned in one of the comments above; it's perfectly feasible to use tower in a single-thread async context, and this imposes a needless requirement on these applications.

I renamed the method from call to serve, but this is more to prepare for a future where I might be able to be compatible with a future version of Tower again, as to make the two method calls less ambiguous to understand and use;

As someone who's using hyper+tower for clients, this naming doesn't sound right. The new name only makes sense when using tower for servers. Calling my_http_client.serve() to send a request sounds wrong.