Skip to content

[FEATURE]: Talend Conversion Missing #1724

Open
@ericbugin

Description

@ericbugin

Is there an existing issue for this?

  • I have searched the existing issues

Category of feature request

Profiler

Problem statement

https://medium.com/towards-data-engineering/migrating-from-talend-to-databricks-lakehouse-with-lakebridge-c25d65359f50

Most of the functions seen in this Medium article to convert Talend are not all present in the cli.py offered in the current version of LakeBridge.

lakebridge discover --source talend --input talend_exports/ --output talend_inventory.json
lakebridge assess --input talend_inventory.json --output assessment_report.json
lakebridge analyze talend --input talend_exports/ --output talend_profile.json
lakebridge translate talend --input talend_exports/ --output dlt_jobs/
lakebridge generate workflows --input dlt_jobs/ --output databricks_jobs/
lakebridge deploy --workspace-url https://<workspace> --token <token> --project databricks_jobs/
lakebridge validate --source-output talend_outputs/ --target-output databricks_outputs/ --output validation_report.json

https://github.com/databrickslabs/lakebridge/blob/2dade0bf4ba1d96f5329f953b14749e79dbce6ba/src/databricks/labs/lakebridge/cli.py

BladeBridge used to support this conversion before Databricks bought them and I was hoping to see this again
https://www.youtube.com/watch?v=Rwx4XrybuIs

Proposed Solution

Enable the missing functions, especially the translate function, for converting Talend ETL jobs to Pyspark as BladeBridge did before.

Additional Context

In the event that I am mistaken, please let me know the proper way of converting Talend ETL jobs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bb converterIssues related to BB converter

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions