tyk-pump

Tyk Pump

Tyk Pump is a pluggable analytics purger to move Analytics generated by your Tyk nodes to any back-end.

Overview of Tyk Pump

More on pump configurations

Introduction

This chart deploys the open source Tyk pump on a Kubernetes cluster using the Helm package manager.

For typical usage, we recommend using following umbrella charts:

• For Tyk Open Source, please use tyk-oss
• For Tyk Hybrid Gateway with Tyk Cloud or MDCB Remote Gateway, please use tyk-data-plane
• For Tyk Self-Managed, please use tyk-stack

Learn more about different deployment options

Prerequisites

• Kubernetes 1.19+
• Helm 3+
• Redis

Installing the Chart

To install the chart from the Helm repository in namespace tyk with the release name tyk-pump:

helm repo add tyk-helm https://helm.tyk.io/public/helm/charts/
helm repo update
helm show values tyk-helm/tyk-pump > values-pump.yaml

Note: Set Redis connection details first. See Configuration below.

helm install tyk-pump tyk-helm/tyk-pump -n tyk --create-namespace -f values-pump.yaml

Uninstalling the Chart

helm uninstall tyk-pump -n tyk

This removes all the Kubernetes components associated with the chart and deletes the release.

Upgrading Chart

helm upgrade tyk-pump tyk-helm/tyk-pump -n tyk

Upgrading from tyk-headless chart

Please see Migration notes in tyk-oss chart

Configuration

See Customizing the Chart Before Installing.

To get all configurable options with detailed comments:

helm show values tyk-helm/tyk-pump > values.yaml

You can update any value in your local values.yaml file and use -f [filename] flag to override default values during installation. Alternatively, you can use --set flag to set it in Tyk installation.

Set Redis connection details (Required)

Redis is Tyk Pump's primary database where it scrapes Tyk Gateway analytics from. You may set global.redis.addr and global.redis.pass with redis connection string and password for Tyk Gateway respectively.

Pump Configurations

Pump	Configuration
Prometheus Pump (Default)	Add the value prometheus to the pump.backend entry, and add connection details for Prometheus under pump.prometheusPump.
Mongo Pump	Add mongo to pump.backend, and add connection details for mongo under global.mongo.
Mongo Selective Pump	Add mongo-selective to pump.backend, and add connection details for mongo under global.mongo.
Mongo Aggregate Pump	Add mongo-aggregate to pump.backend, and add connection details for mongo under global.mongo.
Postgres Pump	Add postgres to pump.backend, and add connection details for postgres under global.postgres.
Postgres Aggregate Pump	Add postgres-aggregate to pump.backend, and add connection details for postgres under global.postgres.
Uptime Pump	Set pump.uptimePumpBackend to mongo or postgres or ""
Hybrid Pump	Add hybrid to pump.backend, and setup global.remoteControlPlane section with the required adresses and tokens
Other Pumps	Add the required environment variables in pump.extraEnvs

Note

For additional information on Tyk Pump configurations, refer to the Setup Dashboard Analytics documentation. To explore the list of supported backends for Tyk Pump, please visit https://tyk.io/docs/tyk-stack/tyk-pump/other-data-stores/.

Prometheus Pump

Add prometheus to pump.backend, and add connection details for prometheus under pump.prometheusPump.

We also support monitoring using Prometheus Operator. All you have to do is set pump.prometheusPump.prometheusOperator.enabled to true. This will create a PodMonitor resource for your Pump instance.

Mongo Pump

If you are using the MongoDB pumps in the tyk-oss installation you will require MongoDB installed for that as well.

To install MongoDB you can use these rather excellent charts provided by Bitnami:

helm install tyk-mongo bitnami/mongodb --version {HELM_CHART_VERSION} --set "replicaSet.enabled=true" -n tyk

(follow notes from the installation output to get connection details and update them in values.yaml file)

Note

Here is list of supported MongoDB versions. Please make sure you are installing mongo helm chart that matches these version.

Note

Important Note regarding MongoDB: This helm chart enables the PodDisruptionBudget for MongoDB with an arbiter replica-count of 1. If you intend to perform system maintenance on the node where the MongoDB pod is running and this maintenance requires for the node to be drained, this action will be prevented due the replica count being 1. Increase the replica count in the helm chart deployment to a minimum of 2 to remedy this issue.

Add following under the global section in values.yaml:

   # Set mongo connection details if you want to configure mongo pump.     
   mongo:
      # The mongoURL value will allow you to set your MongoDB address.
      # Default value: mongodb://mongo.{{ .Release.Namespace }}.svc:27017/tyk_analytics
      # mongoURL: mongodb://mongo.tyk.svc:27017/tyk_analytics
      # If your MongoDB has a password you can add the username and password to the url
      # mongoURL: mongodb://root:[email protected]:27017/tyk_analytics?authSource=admin
      mongoURL: <MongoDB address>
      
     # mongo-go driver is supported for Tyk 5.0.2+.
     # We recommend using the mongo-go driver if you are using MongoDB 4.4.x+.
     # For MongoDB versions prior to 4.4, please use the mgo driver.
     # Since Pump 1.9 the default driver is mongo-go.
      driver: mongo-go

      # Enables SSL for MongoDB connection. MongoDB instance will have to support that.
      # Default value: false
      # useSSL: false

SQL Pump

If you are using the SQL pumps in the tyk-oss installation you will require PostgreSQL installed for that as well.

To install PostgreSQL you can use these rather excellent charts provided by Bitnami:

helm install tyk-postgres bitnami/postgresql --set "auth.database=tyk_analytics" -n tyk

(follow notes from the installation output to get connection details and update them in values.yaml file)

Add following under the global section in values.yaml:

  # Postgres connection string parameters.
  postgres:
    # host corresponds to the host name of postgres
    host: tyk-postgres-postgresql.tyk.svc
    # port corresponds to the port of postgres
    port: 5432
    # user corresponds to the user of postgres
    user: postgres
    # password corresponds to the password of the given postgres user in selected database
    password:
    # database corresponds to the database to be used in postgres
    database: tyk_analytics
    # sslmode corresponds to if postgres runs in sslmode (https)
    sslmode: disable
    # Connection string can also be set using a secret. Provide the name of the secret and key below.
    # connectionStringSecret:
    #   name: ""
    #   keyName: ""

Hybrid Pump

Hybrid Pump can be configured by setting pump.backend to hybrid. Set the connection details of your control plane in global.remoteControlPlane field of values file. Connection details can be stored either directly in the values file or you can store it in k8s secret and provide reference to it in the global.remoteControlPlane.useSecretName field.

  # Set remoteControlPlane connection details if you want to configure hybrid pump.
  remoteControlPlane:
      # useSecretName can be used if you don't want to store plaintext values for remote control plane configurations in
      # the Helm value file and would rather provide the k8s Secret externally.
      # You should set following fields in the secret
      # - orgId - Sets slave_options.rpc_key of Tyk Gateway
      # - userApiKey - Sets slave_options.api_key of Tyk Gateway
      # - groupID - Sets slave_options.group_id of Tyk Gateway
      useSecretName: ""
      # connection string used to connect to an MDCB deployment. For Tyk Cloud users, you can get it from Tyk Cloud Console and retrieve the MDCB connection string.
      connectionString: ""
      # orgID of your dashboard user
      orgId: ""
      # API key of your dashboard user
      userApiKey: ""
      # needed in case you want to have multiple data-planes connected to the same redis instance
      groupID: ""
      # enable/disable ssl
      useSSL: true
      # Disables SSL certificate verification
      sslInsecureSkipVerify: true

  # hybridPump configures Tyk Pump to forward Tyk metrics to a Tyk Control Plane.
  # Please add "hybrid" to .Values.pump.backend in order to enable Hybrid Pump.
  hybridPump: 
    # Send aggregated analytics data to Tyk MDCB
    enableAggregateAnalytics: true
    # Hybrid pump RPC calls timeout in seconds. If not specified, default value will be picked up by Tyk Pump.
    callTimeout: 10
    # Hybrid pump connection pool size. If not specified, default value will be picked up by Tyk Pump.
    poolSize: 5

Uptime Pump

Uptime Pump can be configured by setting pump.uptimePumpBackend in values.yaml file. It supports following values

1. mongo: Used to set mongo pump for uptime analytics. Mongo Pump should be enabled.
2. postgres: Used to set postgres pump for uptime analytics. Postgres Pump should be enabled.
3. empty: Used to disable uptime analytics.

Other Pumps

To set up other backends for pump, refer to this document and add the required environment variables in pump.extraEnvs