EMFILE: too many open files

[4piu]

Check List

Please check followings before submitting a new issue.

• I have already read Docs page & Troubleshooting page
• I have already searched existing issues and they are not help to me
• I examined error or warning messages and it's difficult to solve
• Using the latest version of Hexo (run hexo version to check)
• Node.js is higher than minimum required version

Expected behavior

Hexo server start

Actual behavior

 foo@local:~/Downloads/tmp$ hexo v
INFO  Validating config
hexo: 5.0.2
hexo-cli: 4.2.0
os: Linux 5.4.0-42-generic linux x64
node: 14.8.0
v8: 8.4.371.19-node.12
uv: 1.38.1
zlib: 1.2.11
brotli: 1.0.7
ares: 1.16.0
modules: 83
nghttp2: 1.41.0
napi: 6
llhttp: 2.0.4
openssl: 1.1.1g
cldr: 37.0
icu: 67.1
tz: 2020a
unicode: 13.0

 foo@local:~/Downloads/tmp$ ulimit -n 1048576

 foo@local:~/Downloads/tmp$ ulimit -n        
1048576

 foo@local:~/Downloads/tmp$ hexo s
INFO  Validating config
INFO  Start processing
FATAL {
  err: Error: EMFILE: too many open files, watch '/home/foo/Downloads/tmp/source/'
      at FSWatcher.<computed> (internal/fs/watchers.js:218:26)
      at Object.watch (fs.js:1474:34)
      at createFsWatchInstance (/home/foo/Downloads/tmp/node_modules/chokidar/lib/nodefs-handler.js:119:15)
      at setFsWatchListener (/home/foo/Downloads/tmp/node_modules/chokidar/lib/nodefs-handler.js:166:15)
      at NodeFsHandler._watchWithNodeFs (/home/foo/Downloads/tmp/node_modules/chokidar/lib/nodefs-handler.js:331:14)
      at NodeFsHandler._handleDir (/home/foo/Downloads/tmp/node_modules/chokidar/lib/nodefs-handler.js:559:19)
      at processTicksAndRejections (internal/process/task_queues.js:93:5)
      at async NodeFsHandler._addToNodeFs (/home/foo/Downloads/tmp/node_modules/chokidar/lib/nodefs-handler.js:608:16)
      at async /home/foo/Downloads/tmp/node_modules/chokidar/index.js:445:21
      at async Promise.all (index 0) {
    errno: -24,
    syscall: 'watch',
    code: 'EMFILE',
    path: '/home/foo/Downloads/tmp/source/',
    filename: '/home/foo/Downloads/tmp/source/'
  }
} Something's wrong. Maybe you can find the solution here: %s https://hexo.io/docs/troubleshooting.html

How to reproduce?

• hexo init tmp
• cd tmp/
• ulimit -n 1048576
• hexo s

Is the problem still there under "Safe mode"?

server command incompatible with --safe

Environment & Settings

Node.js & npm version(node -v && npm -v)

node v14.8.0
npm 6.14.7

Your site _config.yml (Optional)

# Hexo Configuration
## Docs: https://hexo.io/docs/configuration.html
## Source: https://github.com/hexojs/hexo/

# Site
title: Hexo
subtitle: ''
description: ''
keywords:
author: John Doe
language: en
timezone: ''

# URL
## If your site is put in a subdirectory, set url as 'http://yoursite.com/child' and root as '/child/'
url: http://yoursite.com
root: /
permalink: :year/:month/:day/:title/
permalink_defaults:
pretty_urls:
  trailing_index: true # Set to false to remove trailing 'index.html' from permalinks
  trailing_html: true # Set to false to remove trailing '.html' from permalinks

# Directory
source_dir: source
public_dir: public
tag_dir: tags
archive_dir: archives
category_dir: categories
code_dir: downloads/code
i18n_dir: :lang
skip_render:

# Writing
new_post_name: :title.md # File name of new posts
default_layout: post
titlecase: false # Transform title into titlecase
external_link:
  enable: true # Open external links in new tab
  field: site # Apply to the whole site
  exclude: ''
filename_case: 0
render_drafts: false
post_asset_folder: false
relative_link: false
future: true
highlight:
  enable: true
  line_number: true
  auto_detect: false
  tab_replace: ''
  wrap: true
  hljs: false
prismjs:
  enable: false
  preprocess: true
  line_number: true
  tab_replace: ''

# Home page setting
# path: Root path for your blogs index page. (default = '')
# per_page: Posts displayed per page. (0 = disable pagination)
# order_by: Posts order. (Order by date descending by default)
index_generator:
  path: ''
  per_page: 10
  order_by: -date

# Category & Tag
default_category: uncategorized
category_map:
tag_map:

# Metadata elements
## https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta
meta_generator: true

# Date / Time format
## Hexo uses Moment.js to parse and display date
## You can customize the date format as defined in
## http://momentjs.com/docs/#/displaying/format/
date_format: YYYY-MM-DD
time_format: HH:mm:ss
## updated_option supports 'mtime', 'date', 'empty'
updated_option: 'mtime'

# Pagination
## Set per_page to 0 to disable pagination
per_page: 10
pagination_dir: page

# Include / Exclude file(s)
## include:/exclude: options only apply to the 'source/' folder
include:
exclude:
ignore:

# Extensions
## Plugins: https://hexo.io/plugins/
## Themes: https://hexo.io/themes/
theme: landscape

# Deployment
## Docs: https://hexo.io/docs/one-command-deployment
deploy:
  type: ''

Your theme _config.yml (Optional)

# Header
menu:
  Home: /
  Archives: /archives
rss: /atom.xml

# Content
excerpt_link: Read More
fancybox: true

# Sidebar
sidebar: right
widgets:
- category
- tag
- tagcloud
- archive
- recent_posts

# display widgets at the bottom of index pages (pagination == 2)
index_widgets:
# - category
# - tagcloud
# - archive

# widget behavior
archive_type: 'monthly'
show_count: false

# Miscellaneous
google_analytics:
gauges_analytics:
favicon: /favicon.png
twitter:
google_plus:
fb_admins:
fb_app_id:

Hexo and Plugin version(npm ls --depth 0)

[email protected] /home/foo/Downloads/tmp
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

Your package.json package.json

{
  "name": "hexo-site",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "build": "hexo generate",
    "clean": "hexo clean",
    "deploy": "hexo deploy",
    "server": "hexo server"
  },
  "hexo": {
    "version": "5.0.2"
  },
  "dependencies": {
    "hexo": "^5.0.0",
    "hexo-generator-archive": "^1.0.0",
    "hexo-generator-category": "^1.0.0",
    "hexo-generator-index": "^2.0.0",
    "hexo-generator-tag": "^1.0.0",
    "hexo-renderer-ejs": "^1.0.0",
    "hexo-renderer-marked": "^3.0.0",
    "hexo-renderer-stylus": "^2.0.0",
    "hexo-server": "^2.0.0"
  }
}

Others

[curbengh]

Which distro and version are you using?

[4piu]

Which distro and version are you using?

                          ./+o+-       foo@local
                  yyyyy- -yyyyyy+      OS: Ubuntu 20.04 focal
               ://+//////-yyyyyyo      Kernel: x86_64 Linux 5.4.0-42-generic
           .++ .:/++++++/-.+sss/`      Uptime: 2m
         .:++o:  /++++++++/:--:/-      Packages: 2699
        o:+o+:++.`..```.-/oo+++++/     Shell: zsh 5.8
       .:+o:+o/.          `+sssoo+/    Resolution: 1920x1080
  .++/+:+oo+o:`             /sssooo.   DE: KDE 5.68.0 / Plasma 5.18.5
 /+++//+:`oo+o               /::--:.   WM: KWin
 \+/+o+++`o++o               ++////.   GTK Theme: Adwaita-dark [GTK2/3]
  .++.o+++oo+:`             /dddhhh.   Icon Theme: Tela-dark
       .+.o+oo:.          `oddhhhh+    Disk: 1.1T / 1.5T (77%)
        \+.++o+o``-````.:ohdhhhhh+     CPU: Intel Core i7-7700HQ @ 8x 3.8GHz [55.0°C]
         `:o+++ `ohhhhhhhhyo++os:      GPU: GeForce GTX 1060
           .o:`.syhhhhhhh/.oo++o`      RAM: 2673MiB / 15960MiB
               /osyyyyyyo++ooo+++/    
                   ````` +oo+++o\:    
                          `oo++.      

[curbengh]

• hexo init tmp
• cd tmp/
• ulimit -n 1048576
• hexo s

this issue affects the default blog? I tested this in a 3GB RAM VM with much lower ulimit:

$ ulimit -Sn
1024
$ ulimit -Hn
524288

and even in increased ulimit -n 1048576.

[4piu]

Problem mysteriously disappears after reboot multiple times. Can not replicate.

[dimaslanjaka]

need troubleshoot for windows

i tried generate 7k markdown posts.

Also we need separated generator to minimalizing memory usages #5552 (comment)

@SukkaW @uiolee @yoshinorin @4piu