com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.22 installation failure in Databricks cluster

[johnlourdu]

Installation of com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.22 library is failing in Databricks cluster due to below exception

Library installation attempted on the driver node of cluster 0212-102108-or3k95yz and failed. Library resolution failed. Error code: ERROR_MAVEN_LIBRARY_RESOLUTION, error message: Library resolution failed because unresolved dependency: net.minidev:json-smart:[1.3.1,2.3]: not found

:: problems summary ::
:::: WARNINGS
module not found: net.minidev#json-smart;[1.3.1,2.3]
==== preferred-maven-central-mirror: tried
https://maven-central.storage-download.googleapis.com/maven2/net/minidev/json-smart/[revision]/json-smart-[revision].pom
[2.5.2]
-- artifact net.minidev#json-smart;[1.3.1,2.3]!json-smart.jar:
https://maven-central.storage-download.googleapis.com/maven2/net/minidev/json-smart/[revision]/json-smart-[revision].jar
[2.5.2]
==== central: tried
https://repo1.maven.org/maven2/net/minidev/json-smart/[revision]/json-smart-[revision].pom
[2.5.2]
-- artifact net.minidev#json-smart;[1.3.1,2.3]!json-smart.jar:
https://repo1.maven.org/maven2/net/minidev/json-smart/[revision]/json-smart-[revision].jar
[2.5.2]
==== spark-packages: tried
https://repos.spark-packages.org/net/minidev/json-smart/[revision]/json-smart-[revision].pom
-- artifact net.minidev#json-smart;[1.3.1,2.3]!json-smart.jar:
https://repos.spark-packages.org/net/minidev/json-smart/[revision]/json-smart-[revision].jar
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: net.minidev#json-smart;[1.3.1,2.3]: not found
::::::::::::::::::::::::::::::::::::::::::::::

[pmocci]

We are experiencing the same issue, we can't find the com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.22 library when searching in the Maven repository

[HooverCz]

You can workaround this (until the issue will be resolved) by filling in the Exclusions field by value "net.minidev:json-smart"

At least my case, the library was then successfully installed and stream processing worked just fine.

[catbatsford-bjss]

We're getting this issue as well; we believe it's linked to this issue which is breaking mavern: netplex/json-smart-v2#240

[Vicferron]

We're getting this issue as well; we believe it's linked to this issue which is breaking mavern: netplex/json-smart-v2#240

Yep I think so, the time and the error seems to match, I've tried several things to install json-smart from another repo but can't.
I'm trying right now to exclude it

[Rexeh]

You can workaround this (until the issue will be resolved) by filling in the Exclusions field by value "net.minidev:json-smart"

At least my case, the library was then successfully installed and stream processing worked just fine.

This is identical to our fix too, confirmed working with 2.4.2 version of the dependency.

This package is no longer maintained so if you're on databricks it might be worth planning to move to the native Kafka connector to connect to your event hub.

[Vicferron]

So,
Isn't Microsoft working on replacing this dependency?
Does excluding json-smart involve anything special?
And what about the change to the Kafka connector?

[catbatsford-bjss]

You can workaround this (until the issue will be resolved) by filling in the Exclusions field by value "net.minidev:json-smart"

At least my case, the library was then successfully installed and stream processing worked just fine.

This works with our non-unity-catalog enabled clusters, but I get the following when I try on our unity-catalog-enabled clusters:

[johnlourdu]

The workaround is to download the jar file and install it from dbfs. This works for me

[Axxeption]

We faced the same issue, the work around by excluding "net.minidev:json-smart" worked. Thanks a lot.
I was wondering why there is no more activity on this code? Is Microsoft recommending another package that is maintained?