[Core feature] Add some start time template variable for Spark log links

[Tom-Newton]

Motivation: Why do you think this is important?

When linking to some external logging service its very convenient if it automatically opens the correct time window. Currently this is not really possible for Spark tasks.

Goal: What should the final outcome look like, ideally?

Approximate parity with normal python tasks in terms of the template variables available for forming log links

flyte/flyteplugins/go/tasks/plugins/k8s/kfoperators/common/common_operator.go

Lines 138 to 146 in f5a4ce0

	workerLog, err := logPlugin.GetTaskLogs(tasklog.Input{
	PodName: name + fmt.Sprintf("-worker-%d", workerIndex),
	Namespace: namespace,
	PodRFC3339StartTime: RFC3999StartTime,
	PodRFC3339FinishTime: RFC3999FinishTime,
	PodUnixStartTime: startTime,
	PodUnixFinishTime: finishTime,
	TaskExecutionID: taskExecID,
	})

.

Describe alternatives you've considered

Just provide log links with a really long time window. Unfortunately this can slow down the queries on the logging tool and occasionally leads to timeouts.

Propose: Link/Inline OR Additional context

For spark tasks the log links are created by

flyte/flyteplugins/go/tasks/plugins/k8s/spark/spark.go

Lines 342 to 347 in f5a4ce0

	o, err := p.GetTaskLogs(tasklog.Input{
	PodName: sj.Status.DriverInfo.PodName,
	Namespace: sj.Namespace,
	LogName: "(Driver Logs)",
	TaskExecutionID: taskExecID,
	})

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[demoncoder-crypto]

#6411 Hi check this implementation, let me know if i am on the right track

[Tom-Newton]

#6411 Hi check this implementation, let me know if i am on the right track

That looks good to me. Thanks for working on it 🙂