Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replication: Too many open files #23996

Open
suitableZebraCaller opened this issue Dec 20, 2022 · 0 comments · May be fixed by #23997
Open

Replication: Too many open files #23996

suitableZebraCaller opened this issue Dec 20, 2022 · 0 comments · May be fixed by #23997

Comments

@suitableZebraCaller
Copy link
Contributor

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

  1. Create a Replication Stream
  2. Disconnect the remote database so that data is written to replication queue
  3. Write entries to the local database (Potentially greater than ulimit number of entries?)
  4. Reconnect remote

Expected behavior:
Describe what you expected to happen.
No errors/Data should be replicated without issue.

Actual behavior:
Describe What actually happened.
Replication occurs until the influxdb process reaches ulimit.
Logs show it continues to retry. Replication hangs until TCP connections close.
Cycle repeats.

Environment info:

  • System info: Run uname -srm and copy the output here
    Linux 5.15.0-52-generic x86_64
  • InfluxDB version: Run influxd version and copy the output here
    InfluxDB v2.6.0 (git: 24a2b62) build_date: 2022-12-15T18:47:00Z
  • Other relevant environment details: Container runtime, disk info, etc
$ ulimit -n
1024

Logs:
Include snippet of errors in log.

2022-12-20T03:06:51.311605Z error Error in replication stream {"log_id": "0esNoDGW000", "service": "replications", "replication_id": "0a77499ad3a3f000", "error": "Post "http://localhost:8086/api/v2/write?bucket=77afb3a8286e1929&org=b7003017570b8fbd\": dial tcp localhost:8086: socket: too many open files", "retries": 1}

2022-12-20T03:06:51.728569Z info http: Accept error: accept tcp [::]:8086: accept4: too many open files; retrying in 5ms {"log_id": "0esNoDGW000", "service": "http"}

@suitableZebraCaller suitableZebraCaller linked a pull request Dec 20, 2022 that will close this issue
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant