Support sidecar containers

[Mossaka]

I would want the SpinApp CRD to be able to specify sidecar (Linux) containers that can run in the same pod as the the Spin App.

The main changes are

1. A new field Containers in the spinapp_types.go that defines a list of sidecar containers to be included in the Deployment.

// Containers defines the list of sidecar containers to be included in the deployment.
//
// These containers will not include the main Spin App. They share the Spin App's
// environment variables and volumes.
// +kubebuilder:validation:Optional
Containers []corev1.Container `json:"containers,omitempty"`

2. in spinapp_controller.go, adds containers to the deployment creation.

var containers []corev1.Container
if len(app.Spec.Containers) > 0 {
  for _, c := range app.Spec.Containers {
	  if c.Image == app.Spec.Image {
		  return nil, errors.New("container in app.Spec.Containers must have a different image than Spin App")
	  }
	  if c.Name == "" {
		  return nil, errors.New("container in app.Spec.Containers must have a name")
	  }
	  if c.Name == app.Name {
		  return nil, errors.New("container in app.Spec.Containers must have a different name than the Spin App")
	  }
	  c.Env = append(c.Env, env...)
	  c.VolumeMounts = append(c.VolumeMounts, volumeMounts...)
	  if c.Resources.Limits == nil && c.Resources.Requests == nil {
		  c.Resources = resources
	  }
	  if c.LivenessProbe == nil {
		  c.LivenessProbe = livenessProbe
	  }
	  if c.ReadinessProbe == nil {
		  c.ReadinessProbe = readinessProbe
	  }
	  containers = append(containers, c)
  }
}

I've done a bit prototype and you may check out the diff: main...Mossaka:spin-operator-msk:sidecars

[Mossaka]

Close this issue as we should move the discussion to spinkube/skips#11