Adapt Spark3

[uxian]

Hey, thanks for creating this solution, this package is pretty powerful! However we have migrated to Spark3, is there any plan to adapt Spark3? Thanks!

[cenekskarda]

Hey, thanks for creating this solution, this package is pretty powerful! However we have migrated to Spark3, is there any plan to adapt Spark3? Thanks!

Hi, it's great to hear you like our lib! It is already in progress, but I can't provide you a release date at the moment. We are currently starting to fine tune the performance. We don't have an estimate at the moment, because we were dependent on mllib library, which has been significantly changed in Spark3.