Commit f0235d7
authored
Account for LinkedIn Spark versioning (#551)
### What changes are proposed in this pull request, and why are they necessary?
In `TransportUDFTransformer` we have a function which decides the Scala version to use for a given Spark version. In LinkedIn we have changed the versioning scheme we use internally for Spark 3.5+, where the major version of the internal Spark distribution denotes the full OSS Spark version. E.g. For Spark 3.5.2, this would be 352.x.y (where x and y are internal versions). This PR accounts for this new versioning format.
### How was this patch tested?
Existing unit tests.
Tested the regex separately via regex101
<img width="231" alt="image" src="https://github.com/user-attachments/assets/779eb267-b839-4329-8925-a86900b7467f" />1 parent f6ce2be commit f0235d7
File tree
1 file changed
+2
-2
lines changed- coral-spark/src/main/java/com/linkedin/coral/spark/transformers
1 file changed
+2
-2
lines changedLines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
91 | | - | |
| 91 | + | |
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
| |||
0 commit comments