Workarounds for Maven Json-smart 2.5.2 Release Breaking Azure Databricks Job Dependencies

A recent Maven library release corrupted json-smart’s central metadata, ended up removing all previous versions of Maven library other than 2.5.2. This results in DBR ivy resolution failures if customers’ job has transitive dependency on previous version of json-smart. The impact to Azure Databricks customers is jobs failure and in some cases the job clusters could fail to start. More details and updates of the issue can be found here: https://github.com/netplex/json-smart-v2/issues/240.  

This also affects google’s maven mirror, which is used by DBR 11+ to resolve maven libraries, and maven central is used as the backup for google’s maven mirror.

To mitigate the issue, you can take the workaround steps to install the customer’s requested library and smart-json separately

Steps 1: Install the customer’s requested library (use azure-eventhubs-spark as an example) but exclude json-smart. 

Step 2: Install net.minidev:json-smart:2.3 (or any other version that was needed)

Workaround for run time,  if you have a preferred Maven mirror that is not affected by this issue, or if you host a private Maven mirror, you can configure your Databricks environment to use it.

• Step:
• Set the Spark configuration parameter spark.databricks.driver.preferredMavenCentralMirrorUrl to your preferred mirror repository URL.
•