SSL errors in log with http_endpoint with tls

[robinpecha]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Hi, Im collecting journald and docker logs and sending it to http_endpoint sink on another vector instance. This connection is encrypted with tls.

SENDER:

sources:
  dockerlogs:
    type: docker_logs
  journald:
    type: journald
sinks:
  https:
    type: http
    inputs: 
      - journald
      - dockerlogs
      # - host_metrics
      # - host_metrics_remap
    uri: "https://vector-receiver.example.com:8888"
    encoding: { codec: "json" }
    tls:
      verify_certificate: true
      ca_file:  "/etc/vector/tls/vector-server.crt"
      crt_file: "/etc/vector/tls/vector-client.crt"
      key_file: "/etc/vector/tls/vector-client.key"
    auth:
      strategy: basic
      user: user
      password: password

RECEIVER:

sources:
   https_endpoint:
    type: http_server
    address: "0.0.0.0:8888"
    encoding: json
    tls:
      enabled: true
      verify_certificate: true
      ca_file:  "/etc/vector/tls/vector-client.crt"
      crt_file: "/etc/vector/tls/vector-server.crt"
      key_file: "/etc/vector/tls/vector-server.key" 
    auth:
      type: basic
      username: user
      password: password

Here is output of vector top.
5 820 000 sucessfully sent logs, 1 error

But log of vector is full of errors like this, each few minutes:

Jan 28 09:06:54 dev-vm-1 vector[1723162]: 2025-01-28T08:06:54.392456Z  WARN sink{component_kind="sink" component_id=https component_type=http}:request{request_id=7935}: vector::sinks::util::retries: Retrying after error. error=Failed to make HTTP(S) request: error trying to connect: error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_s3.c:908:SSL alert number 80 internal_log_rate_limit=true

Configuration

Debian GNU/Linux 12 (bookworm)
Linux dev-vm-1 6.1.0-22-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.94-1 (2024-06-21) x86_64 GNU/Linux

Version

vector 0.43.0 (x86_64-unknown-linux-gnu 88e017f 2024-12-03 22:44:54.502541095)

Debug Output

Example Data

No response

Additional Context

No response

References

No response

[pront]

Hey @robinpecha, this could be a mismatch between Client and Server certificates. Here are some things to try:

• Try disabling verify_certificate to see if it’s purely a trust issue.
• Enable VECTOR_LOG=debug to get more detailed logs.
• Ensure TLS version compatibility