[BUG] Scala 2.13 OpenApi generated code causes a StackOverflow (Scala3 generated code does not)

[dwalend]

Tapir version: 1.11.50

Scala version: 2.13.16 (it works on the Scala 3.3.6 side of the cross project build).

Tapir generates this line of code from NiFi's OpenApi spec for Scala 2.13:

  implicit lazy val controllerServiceReferencingComponentDTOTapirSchema: sttp.tapir.Schema[ControllerServiceReferencingComponentDTO] = sttp.tapir.Schema.derived
  implicit lazy val controllerServiceReferencingComponentEntityTapirSchema: sttp.tapir.Schema[ControllerServiceReferencingComponentEntity] = sttp.tapir.Schema.derived

This causes an infinite recursion at class load time:

An exception or error caused a run to abort. 
java.lang.StackOverflowError
	at com.macrohealth.platform.api.nififromopenapi.NifiFromOpenApiSchemas2$.controllerServiceReferencingComponentDTOTapirSchema$lzycompute(NifiFromOpenApiSchemas2.scala:119)
	at com.macrohealth.platform.api.nififromopenapi.NifiFromOpenApiSchemas2$.controllerServiceReferencingComponentDTOTapirSchema(NifiFromOpenApiSchemas2.scala:119)
	at com.macrohealth.platform.api.nififromopenapi.NifiFromOpenApiSchemas2$.controllerServiceReferencingComponentEntityTapirSchema$lzycompute(NifiFromOpenApiSchemas2.scala:120)
	at com.macrohealth.platform.api.nififromopenapi.NifiFromOpenApiSchemas2$.controllerServiceReferencingComponentEntityTapirSchema(NifiFromOpenApiSchemas2.scala:120)

I was able to isolate the problem in a test:

  "Schemas that reference each other" should "not have a circular dependency" in {
    // The lazy val evaluation hits the circular dependency, but defs won't
    val schema1 = NifiFromOpenApiSchemas2.controllerServiceReferencingComponentDTOTapirSchema
    val schema2 = NifiFromOpenApiSchemas2.controllerServiceReferencingComponentEntityTapirSchema

    assert(schema1 != schema2) //if the code gets this far it worked.
  }

The code it generates for Scala3 uses defs instead of lazy vals - and passes that simple test

  implicit def controllerServiceReferencingComponentDTOTapirSchema: sttp.tapir.Schema[ControllerServiceReferencingComponentDTO] = sttp.tapir.Schema.derived
  implicit def controllerServiceReferencingComponentEntityTapirSchema: sttp.tapir.Schema[ControllerServiceReferencingComponentEntity] = sttp.tapir.Schema.derived

I'm generating this code from NiFi's OpenApi spec, after stripping out anything problematic to Tapir, and all the security business. Reproducing that hairball would be a lot of work, and would likely bury the problem. I could try to refine it but I don't know what I might be looking for.

I'm hoping Tapir fixed a bug for Scala3 that is easy to port to Scala2.13.

[adamw]

Maybe @hughsimpson has some insights on this? :)

[dwalend]

I tried hacking the generated code - converting those two lazy vals to defs - or even converting all of them in the schema file - does not prevent the stack overflow.

[hughsimpson]

👀 Thanks for the report, I'll take a look at this

[hughsimpson]

FWIW the reason we generate defs rather than lazy vals for scala 3 was a runtime issue, not a compile-time one (cf this pr)

[dwalend]

It's a runtime issue for this code as well - at class load time.

[hughsimpson]

Ah, sorry, I see that now. There are a few other issues I'm seeing using this openapi spec. I'm gonna ignore the security ones, because I think they're because the security is mis-declared. Also gonna ignore the weird nested #/components/schemas/LongParameter (and similar) schemas and the object query params, because I'm not sure what the correct thing is there (or if the former are valid). But we should be able to handle the missing descriptions, because that's actually pretty common in the wild, and we should also be able to handle the transferFlowFiles response definition. Also */* schemas could probably be handled more nicely somehow...

Anyway, all that aside, I'm seeing your error now

[hughsimpson]

Reduced to this. Given

case class O(n: N)
case class N(o: Option[O])

then this breaks:

  import sttp.tapir.generic.auto._
  implicit lazy val s1: sttp.tapir.Schema[N] = sttp.tapir.Schema.derived
  implicit lazy val s2: sttp.tapir.Schema[O] = sttp.tapir.Schema.derived
  val schema = s1
  val schema2 = s2

and this works:

  import sttp.tapir.generic.auto._
  implicit lazy val s1: sttp.tapir.Schema[N] = sttp.tapir.Schema.derived
  val schema = s1
  val schema2 = sttp.tapir.Schema.derived[O]

but I'm not sure what a good resolution for avoiding this whilst still making the schemas implicitly available would be 🤔

A hack that takes advantage of how scala implicits in 2.13 are visible based on name would be

  val (s1, s2) = {
    implicit lazy val schema1: sttp.tapir.Schema[N] = sttp.tapir.Schema.derived
    val schema2: sttp.tapir.Schema[O] = sttp.tapir.Schema.derived
    (schema1, schema2)
  }
  implicit val schema1: sttp.tapir.Schema[N] = s1
  implicit val schema2: sttp.tapir.Schema[O] = s2

Or, alternatively (and probably a bit easier to codegen):

  implicit lazy val schema1: sttp.tapir.Schema[N] = {
    val schema2 = null // shadow mutually-recursive schema
    sttp.tapir.Schema.derived
  }
  implicit lazy val schema2: sttp.tapir.Schema[O] = {
    val schema1 = null // shadow mutually-recursive schema
    sttp.tapir.Schema.derived
  }

which, obviously, I kinda hate, but should be something we can codegen to fix this. Will get a pr together tomorrow.

[hughsimpson]

@dwalend I have a fix here that generates

  implicit lazy val controllerServiceReferencingComponentDTOTapirSchema: sttp.tapir.Schema[ControllerServiceReferencingComponentDTO] = {
    val controllerServiceReferencingComponentEntityTapirSchema = null // shadow mutually-recursive schema
    sttp.tapir.Schema.derived
  }
  implicit lazy val controllerServiceReferencingComponentEntityTapirSchema: sttp.tapir.Schema[ControllerServiceReferencingComponentEntity] = {
    val controllerServiceReferencingComponentDTOTapirSchema = null // shadow mutually-recursive schema
    sttp.tapir.Schema.derived
  }

-- will wrap it up tomorrow. should be good to go. Also fixes some of those other issues with the NiFi spec I ran into.

[hughsimpson]

@adamw over to you 😄 (ngl, I've actually run into a couple of the issues in that spec myself that I've been working around, and low-key appreciate the nudge to fix them)

[hughsimpson]

@dwalend did the latest release resolve your issue?