Expose configs via pipeline element table

[jogrogan]

Expose configs via pipeline element table

0: Hoptimator> select NAME, CONFIG from "k8s".pipeline_elements;
WARN Object default/TableTrigger/ads-database-pageviews-trigger considered ready by default.
INFO TableTrigger/ads-database-pageviews-trigger is ready.
WARN Object default/FlinkSessionJob/ads-database-audience considered ready by default.
INFO FlinkSessionJob/ads-database-audience is ready.
WARN Object default/SqlJob/test-sqljob considered ready by default.
INFO SqlJob/test-sqljob is ready.
+---------------------------------------------+-------------------------------------------------------------+
|                    NAME                     |                            CONFIG                           |
+---------------------------------------------+-------------------------------------------------------------+
| SqlJob/test-sqljob                          | {"pipeline":"jogrogan-test","key1":"value1","key2":"value2"}|
| FlinkSessionJob/ads-database-audience       | {}                                                          |
| TableTrigger/ads-database-pageviews-trigger | {}                                                          |
+---------------------------------------------+-------------------------------------------------------------+
3 rows selected (0.085 seconds)

Configs should never contain sensitive information or they will be queryable

[ryannedolan]

I wonder why we wouldn't include the whole spec, instead of jut the configs?

[jogrogan]

I wonder why we wouldn't include the whole spec, instead of jut the configs?

Configs just seems more general. We could include the whole spec but it'll be dependent on whatever downstream workflows to parse the spec correctly by type