Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] sudden explosion in memory consumption #524

Open
baggepinnen opened this issue Mar 10, 2020 · 16 comments
Open

[BUG] sudden explosion in memory consumption #524

baggepinnen opened this issue Mar 10, 2020 · 16 comments
Labels

Comments

@baggepinnen
Copy link

baggepinnen commented Mar 10, 2020

I have experience a couple of times the behavior shown in the screenshot below

Screenshot from 2020-03-10 17-53-26

It's the julia process suddenly eating up all available ram and swap, and it happens even if I'm not running any julia code for the moment. When this instance of the problem occured, I had julia in Juno running (cycle mode not active) and I was editing code without running anything (I might have been resizing the editor, but it sometimes happens when I'm definitely not resizing anything). When I noticed the mouse pointer getting stuck, I saw the juno striped progress bar indicating that juno was up to something. When the swap is full, the julia process is automatically terminated and I regain control of the computer. I know it's the julia process eating the ram as verified by the system monitor.

This has happened 10+ times now, and in different scripts doing completely different things. Debuginfo below
...
actually, when I asked juno for debuginfo, the julia process restarted and the bug immediately occurred again, crashing the entire Ubuntu login session (thankfully the text I had written in this issue was saved somehow)

# Atom:
Version: 1.44.0
Dev Mode: false
Official Release: true
{
  "http_parser": "2.8.0",
  "node": "10.11.0",
  "v8": "6.9.427.31-electron.0",
  "uv": "1.23.0",
  "zlib": "1.2.11",
  "ares": "1.14.0",
  "modules": "69",
  "nghttp2": "1.33.0",
  "napi": "3",
  "openssl": "1.1.0",
  "electron": "4.2.7",
  "chrome": "69.0.3497.128",
  "icu": "62.2",
  "unicode": "11.0",
  "cldr": "33.1",
  "tz": "2019a"
}
# julia-client:
Version: 0.12.3
Config:
{
  "firstBoot": false,
  "juliaOptions": {
    "bootMode": "Basic",
    "workingDir": "/home/fredrikb/"
  },
  "juliaPath": "/home/fredrikb/julia/julia",
  "juliaSyntaxScopes": [
    "source.julia",
    "source.weave.md",
    "source.weave.latex",
    "source.gfm",
    "source.embedded.julia",
    "meta.function.environment.general.latex",
    "meta.embedded.block.source",
    "text.tex.latex"
  ],
  "remoteOptions": {
    "remoteJulia": "export JULIA_NUM_THREADS=6;/home/fredrikb/julia/julia"
  },
  "uiOptions": {
    "cellDelimiter": [
      "##",
      "#---",
      "#%%",
      "# %%",
      "# #"
    ],
    "docsDisplayMode": "inline",
    "enableMenu": true,
    "errorNotifications": false,
    "highlightCells": false,
    "layouts": {
      "console": {
        "defaultLocation": "center",
        "split": "right"
      },
      "defaultPanes": {
        "documentation": false,
        "plotPane": false,
        "workspace": false
      },
      "plotPane": {
        "split": "up"
      },
      "terminal": {
        "defaultLocation": "center",
        "split": "right"
      },
      "workspace": {
        "defaultLocation": "right",
        "split": "no split"
      }
    }
  }
}


# ink:
Version: 0.12.3
Config:
undefined


# uber-juno:
Version: 0.3.0
Config:
{
  "disable": true
}


# language-julia:
Version: 0.19.2
Config:
undefined


# language-weave:
Version: 0.6.7
Config:
undefined


# indent-detective:
Version: 0.4.0
Config:
undefined


# latex-completions:
Version: 0.3.6
Config:
{
  "disableForSelector": ".nothing"
}


# versioninfo():
Julia Version 1.4.0-rc2.0
Commit b99ed72c95 (2020-02-24 16:51 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4550U CPU @ 1.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, haswell)
Environment:
  JULIA_NUM_THREADS = 2
  JULIA_EDITOR = atom  -a

Status `~/.julia/environments/v1.4/Project.toml`
  [14f7f29c] AMD v0.3.1
  [c52e3926] Atom v0.12.9
  [dbf13d8f] AudioClustering v0.1.0 [`~/.julia/dev/AudioClustering`]
  [70b36510] AutomaticDocstrings v0.1.1 [`~/.julia/dev/AutomaticDocstrings`]
  [aae01518] BandedMatrices v0.14.3
  [6e4b80f9] BenchmarkTools v0.5.0
  [336ed68f] CSV v0.5.26
  [aaaa29a8] Clustering v0.13.5
  [3abffc1c] ControlSystemIdentification v0.2.0 [`../../dev/ControlSystemIdentification`]
  [a6e380b2] ControlSystems v0.5.6 [`~/.julia/dev/ControlSystems`]
  [717857b8] DSP v0.6.3
  [a93c6f00] DataFrames v0.20.2
  [bb1859e0] DefaultArrays v1.0.1
  [0c46a032] DifferentialEquations v6.11.0
  [b4f34e82] Distances v0.8.2
  [31c24e10] Distributions v0.22.5
  [ffbed154] DocStringExtensions v0.8.1
  [f6369f11] ForwardDiff v0.10.9
  [994df76e] ForwardDiff2 v0.2.2
  [28b8d3ca] GR v0.47.0
  [891a1506] GaussianProcesses v0.11.2 [`~/.julia/dev/GaussianProcesses`]
  [c27321d9] Glob v1.2.0
  [775960c6] Grep v0.2.0
  [dbf13d8f] HDBSCAN v0.1.0 [`~/.julia/dev/HDBSCAN`]
  [e91730f6] Hungarian v0.6.0
  [7073ff75] IJulia v1.20.0
  [916415d5] Images v0.22.0
  [e5e0dc1b] Juno v0.8.1
  [26dcc766] LPVSpectral v0.1.5
  [3abffc1c] LTVModels v0.1.0 [`~/.julia/dev/LTVModels`]
  [5078a376] LazyArrays v0.15.0
  [82ba68a1] LazyWAVFiles v1.0.0 [`~/.julia/dev/LazyWAVFiles`]
  [6e84ca90] LengthChannels v1.0.0 [`~/.julia/dev/LengthChannels`]
  [093fc24a] LightGraphs v1.3.1
  [d3d80556] LineSearches v7.0.1
  [d9d29d28] LowLevelParticleFilters v0.4.1 [`~/.julia/dev/LowLevelParticleFilters`]
  [f0e99cf1] MLBase v0.8.0
  [eff96d63] Measurements v2.2.0
  [0987c9cc] MonteCarloMeasurements v0.8.3
  [6f286f6a] MultivariateStats v0.7.0
  [872c559c] NNlib v0.6.6
  [b8a86587] NearestNeighbors v0.4.4
  [429524aa] Optim v0.20.1
  [1dea7af3] OrdinaryDiffEq v5.29.0
  [9b87118b] PackageCompiler v1.1.0
  [2dcacdae] ParallelDataTransfer v0.5.0
  [18e31ff7] Peaks v0.1.0
  [58dd65bb] Plotly v0.3.0
  [91a5bcdd] Plots v0.27.1
  [a725b495] ProximalOperators v0.10.3
  [438e738f] PyCall v1.91.4
  [d330b81b] PyPlot v2.8.2
  [1a8c2f83] Query v0.12.2
  [77f5812f] SingularSpectrumAnalysis v0.3.1 [`~/.julia/dev/SingularSpectrumAnalysis`]
  [8ce77f84] Soss v0.7.0
  [2b0dec9d] SpectralDistances v0.1.0 [`../../dev/SpectralDistances`]
  [90137ffa] StaticArrays v0.12.1
  [2913bbd2] StatsBase v0.32.2
  [f3b207a7] StatsPlots v0.12.0
  [a2db99b7] TextAnalysis v0.7.0 [`~/.julia/dev/TextAnalysis`]
  [dbf13d8f] ThreadTools v0.1.1 [`~/.julia/dev/ThreadTools`]
  [028f657a] TotalLeastSquares v1.4.1 [`~/.julia/dev/TotalLeastSquares`]
  [c4f8c510] UMAP v0.1.5
  [c4a57d5a] UnsafeArrays v0.4.0
  [8149f6b0] WAV v1.0.3


Some more info:

  • It has happened while editing both scripts in module main, and in code files that are part of a loaded package.
  • When it happens, some times it's enough to restart atom. On a few occasions, this has not been enough and the bug reappeared immediately. Restarting the computer makes it run again for a couple of hours or something.
@baggepinnen baggepinnen changed the title [BUG] [BUG] sudden explosion in memory consumption Mar 10, 2020
@baggepinnen
Copy link
Author

This is the file I was editing while it happened
https://gist.github.com/baggepinnen/52e27ec6001b758d5e8ac7c54ee432c1
nothing special in it as far as I can see. I was editing the function in the bottom

@pfitzseb
Copy link
Member

I still didn't have any luck reproducing this. My best bet is that CSTParser and/or CodeTools run into infinite recursion and crash everthing, but not sure what we can do about that...

@baggepinnen
Copy link
Author

baggepinnen commented Mar 23, 2020

Can I disable one of them and see if the problem goes away? It has messed up 4 sessions today alone, so I'm eager to help getting it resolved.

@pfitzseb
Copy link
Member

pkg> dev Atom and apply

diff --git a/src/completions.jl b/src/completions.jl
index 397a457..2830fa5 100644
--- a/src/completions.jl
+++ b/src/completions.jl
@@ -72,7 +72,7 @@ function basecompletionadapter(

   # initialize suggestions with local completions so that they show up first
   prefix = line[replace]
-  comps = if force || !isempty(prefix)
+  comps = if false # force || !isempty(prefix)
     filter!(let p = prefix
       c -> startswith(c[:text], p)
     end, localcompletions(context, row, column, prefix))
diff --git a/src/outline.jl b/src/outline.jl
index 496d6e1..a695a68 100644
--- a/src/outline.jl
+++ b/src/outline.jl
@@ -6,11 +6,11 @@ handle("updateeditor") do data
         updateSymbols || true
     ] = data

-    return try
-        todict.(updateeditor(text, mod, path, updateSymbols))
-    catch err
+    # return try
+    #     todict.(updateeditor(text, mod, path, updateSymbols))
+    # catch err
         []
-    end
+    # end
 end

 # NOTE: update outline and symbols cache all in one go

to disable local completions and the outline. You should be able to use something like

diff --git a/src/modules.jl b/src/modules.jl
index 166c9c3..1dbaa55 100644
--- a/src/modules.jl
+++ b/src/modules.jl
@@ -285,34 +285,40 @@ end
 const MAIN_MODULE_LOCATION = Ref{Tuple{String, Int}}(moduledefinition(Main))

 handle("module") do data
-  main, sub = modulenames(data, cursor(data))
-
-  mod = CodeTools.getmodule(main)
-  smod = CodeTools.getmodule(mod, sub)
-
-  if main == "Main" && sub == ""
-    MAIN_MODULE_LOCATION[] = get!(data, "path", ""), data["row"]
-  end
-
-  loaded_mods = copy(Base.loaded_modules_array())
-  if main == "Main"
-    filter!(m -> string(m) == sub, loaded_mods)
-    if !isempty(loaded_mods)
-      return Dict(
-               :main => string(loaded_mods[1]),
-               :sub  => "",
-               :inactive => false,
-               :subInactive => false
-             )
-    end
-  end
-
+  # main, sub = modulenames(data, cursor(data))
+  #
+  # mod = CodeTools.getmodule(main)
+  # smod = CodeTools.getmodule(mod, sub)
+  #
+  # if main == "Main" && sub == ""
+  #   MAIN_MODULE_LOCATION[] = get!(data, "path", ""), data["row"]
+  # end
+  #
+  # loaded_mods = copy(Base.loaded_modules_array())
+  # if main == "Main"
+  #   filter!(m -> string(m) == sub, loaded_mods)
+  #   if !isempty(loaded_mods)
+  #     return Dict(
+  #              :main => string(loaded_mods[1]),
+  #              :sub  => "",
+  #              :inactive => false,
+  #              :subInactive => false
+  #            )
+  #   end
+  # end
+  #
+  # return Dict(
+  #          :main => main,
+  #          :sub  => sub,
+  #          :inactive => (mod==nothing),
+  #          :subInactive => smod==nothing
+  #        )
   return Dict(
-           :sub  => sub,
-           :main => main,
-           :inactive => (mod==nothing),
-           :subInactive => smod==nothing
-           :sub  => sub,
-         )
+    :main => "Main",
+    :sub  => "",
+    :inactive => false,
+    :subInactive => false
-           :inactive => (mod==nothing),
-           :subInactive => smod==nothing
+  )
-         )
 end
+    :main => "Main",


+    :sub  => "",
+    :inactive => false,
@@ -321,9 +327,10 @@ find all modules
 =#
+    :subInactive => false
+  )

 handle("allmodules") do
 end
-  sort!([string(m) for m in CodeTools.allchildren(Main)])

+  # sort!([string(m) for m in CodeTools.allchildren(Main)])
+  []
 end

 handle("ismodule") do mod

-  return CodeTools.getthing(mod) isa Module || mod in string.(Base.loaded_modules_array())
+  return false # CodeTools.getthing(mod) isa Module || mod in string.(Base.loaded_modules_array())
 end
@@ -321,9 +327,10 @@ find all modules
 =#

 handle("allmodules") do
-  sort!([string(m) for m in CodeTools.allchildren(Main)])
+  # sort!([string(m) for m in CodeTools.allchildren(Main)])
+  []
 end

 handle("ismodule") do mod
-  return CodeTools.getthing(mod) isa Module || mod in string.(Base.loaded_modules_array())
+  return false # CodeTools.getthing(mod) isa Module || mod in string.(Base.loaded_modules_array())
 end

to disable most CodeTools features.

Try the first patch first though, the other stuff was in Juno for a long time.

@baggepinnen
Copy link
Author

Done! I'll report back in a while when I know if it works or not. (And if I don't, it might mean that the problem has been solved and I've forgotten about it ^^)

@baggepinnen
Copy link
Author

I can confirm that it does not solve the problem of the REPL suddenly freezes and stops responding while evaluation in the editor keeps working.

@pfitzseb
Copy link
Member

Ok, but the memory leak issue didn't happen again?

@baggepinnen
Copy link
Author

So far it has not appeared which is a good sign. I would give it a few more days though to be sure.

@aviatesk
Copy link
Member

well, I am VERY MUCH interested in for what script you had encontered the memory leak.
Every file that was being open at that time can be to blame, so I would appreciate if you report them when you see the symptom again.

@pfitzseb
Copy link
Member

Any file that's open and any file that's reachable from any of those, yes...

@aviatesk
Copy link
Member

😓

@baggepinnen
Copy link
Author

I have used Juno extensively a couple of days now without seeing the explosion of memory use, it seems likely that the first patch @pfitzseb provided above resolved the issue. I did not apply the second patch.

@aviatesk
Copy link
Member

@baggepinnen hmm... if possible, please tell us the precise timing or the exact script you're editing when you encounter this issue again (after removing the first patch).
Well, that means you again accept the danger of the annoying sudden memory explosion for our sake, so, only if you want to help us fix this bug.

@aviatesk aviatesk pinned this issue Mar 27, 2020
@BeastyBlacksmith
Copy link

BeastyBlacksmith commented Mar 30, 2020

Just had the same issue. Only noticed after reading this issue since on macOS part of RAM is reserved, so the OS is still responsive, but the Juno progress bar never finishes. Killing julia and restart usually solves the problem.
I had two project folders open and 5 files plus the plot pane.

@aviatesk
Copy link
Member

aviatesk commented Mar 30, 2020

Well, we can't do nothing additional unless you can share us your scripts.
But even looking at your scripts might not help so much. I will take another look and refactor on our static code analyzer in the coming months and hope we can ged rid of this bug while the refactor.

@baggepinnen
Copy link
Author

baggepinnen commented Apr 17, 2020

I kind of forgot about this issue, the patch

diff --git a/src/completions.jl b/src/completions.jl
index 397a457..2830fa5 100644

suggested by @pfitzseb appears to have more or less completely removed most Juno bugs for me, including exploding memory consumption and the editor-eval getting stuck. I'm not completely sure what I deactivate by applying the patch, but if the patched behavior could be made optional it would be great.

I did provide a gist with the code I was running in this comment #524 (comment)
but the problem appeared in many different contexts with different packages so I kind of don't think it was related to what code I was running. In fact, once it happened when the julia process was terminated and I executed the atom command Julia client: debug info, which started the julia process after which the problem occurred immediately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants