Out of memory at web loading result.json

[sadmaenv]

info

OS: 5.10.0 aarch64 GNU/Linux
Python: Python 3.10.6 64bit
RAM: 32GB
Version: VizTracer 1.0.4
trace file size: 3.0 GB ( file content is secret)

expect

slice result.json into small piece to load

log

viztracer start.py

vizviewer result.json

web log

UI: http://localhost:9001/v47.0-02b2414d5

RuntimeError: unreachable @ perfetto::base::AlignedAlloc(unsigned long, unsigned long)
 - perfetto::base::AlignedAlloc(unsigned long, unsigned long) (wasm://wasm/0242ba1a:wasm-function[525]:0x3c530)
 - perfetto::base::CircularQueue<perfetto::trace_processor::TraceSorter::TimestampedEvent>::ChangeCapacity(unsigned long) (wasm://wasm/0242ba1a:wasm-function[2954]:0x217653)
 - perfetto::base::CircularQueue<perfetto::trace_processor::TraceSorter::TimestampedEvent>::Grow(unsigned long) (wasm://wasm/0242ba1a:wasm-function[2952]:0x2172c3)
 - perfetto::trace_processor::TraceSorter::PushJsonValue(long long, std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char> >) (wasm://wasm/0242ba1a:wasm-function[4043]:0x2a1964)
 - perfetto::trace_processor::JsonTraceTokenizer::HandleTraceEvent(char const*, char const*, char const**) (wasm://wasm/0242ba1a:wasm-function[4040]:0x2a0d0f)
 - perfetto::trace_processor::JsonTraceTokenizer::ParseInternal(char const*, char const*, char const**) (wasm://wasm/0242ba1a:wasm-function[4038]:0x2a046e)
 - perfetto::trace_processor::JsonTraceTokenizer::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[4036]:0x2a0046)
 - perfetto::trace_processor::ForwardingTraceParser::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[1483]:0x157d10)
 - perfetto::trace_processor::TraceProcessorStorageImpl::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[1579]:0x15f677)
 - perfetto::trace_processor::TraceProcessorImpl::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[241]:0x14865)
 -  (stderr: )
 - Loading trace 67.11 MB (25.9 MB/s)Loading trace 134.22 MB (28.6 MB/s)Loading trace 201.33 MB (29.9 MB/s)Loading trace 268.44 MB (30.5 MB/s)Loading trace 335.54 MB (30.9 MB/s)Loading trace 402.65 MB (31.0 MB/s)Loading trace 469.76 MB (31.4 MB/s)Loading trace 536.87 MB (31.5 MB/s)Loading trace 603.98 MB (31.6 MB/s)Loading trace 671.09 MB (31.7 MB/s)Loading trace 738.20 MB (31.7 MB/s)Loading trace 805.31 MB (31.7 MB/s)Loading trace 872.42 MB (31.8 MB/s)Loading trace 939.52 MB (31.8 MB/s)Loading trace 1006.63 MB (31.9 MB/s)Loading trace 1073.74 MB (31.9 MB/s)Loading trace 1140.85 MB (32.0 MB/s)Loading trace 1207.96 MB (32.0 MB/s)Loading trace 1275.07 MB (32.0 MB/s)Loading trace 1342.18 MB (32.1 MB/s)Loading trace 1409.29 MB (32.0 MB/s)Loading trace 1476.40 MB (32.1 MB/s)Loading trace 1543.50 MB (32.2 MB/s)Loading trace 1610.61 MB (32.1 MB/s)Loading trace 1677.72 MB (32.2 MB/s)Loading trace 1744.83 MB (32.2 MB/s)Loading trace 1811.94 MB (32.2 MB/s)Loading trace 1879.05 MB (32.2 MB/s)Loading trace 1946.16 MB (32.2 MB/s)Loading trace 2013.27 MB (32.2 MB/s)Loading trace 2080.37 MB (32.2 MB/s)Loading trace 2147.48 MB (32.2 MB/s)Loading trace 2214.59 MB (32.2 MB/s)Loading trace 2281.70 MB (32.2 MB/s)Loading trace 2348.81 MB (32.2 MB/s)Loading trace 2415.92 MB (32.3 MB/s)Loading trace 2483.03 MB (32.3 MB/s)Loading trace 2550.14 MB (32.3 MB/s)Loading trace 2617.25 MB (32.3 MB/s)Loading trace 2684.35 MB (32.3 MB/s)Loading trace 2751.46 MB (32.3 MB/s)Loading trace 2818.57 MB (32.3 MB/s)Loading trace 2885.68 MB (32.3 MB/s)Loading trace 2952.79 MB (32.4 MB/s)Loading trace 3019.90 MB (32.4 MB/s)Loading trace 3087.01 MB (32.4 MB/s)[097.085]            utils.cc:328 PERFETTO_CHECK(res) (errno: 48, Out of memory)
 -  (-----BEGIN PERFETTO PRE-CRASH LOG-----)
 -  (trace_uuid: 00000000-0000-0000-f054-f6d3777)
 - [097.085]            utils.cc:328 PERFETTO_CHECK(res) (errno: 48, Out of memory)
 -  (-----END PERFETTO PRE-CRASH LOG-----)

Trace: not available (URL). Provide repro steps.
UA: Mozilla/5.0

refer

#115

viztracer --minimize_memory start.py

still 3.0 GB, err log looks same

UI: http://localhost:9001/v47.0-02b2414d5

RuntimeError: unreachable @ perfetto::base::AlignedAlloc(unsigned long, unsigned long)
 - perfetto::base::AlignedAlloc(unsigned long, unsigned long) (wasm://wasm/0242ba1a:wasm-function[525]:0x3c530)
 - perfetto::base::CircularQueue<perfetto::trace_processor::TraceSorter::TimestampedEvent>::ChangeCapacity(unsigned long) (wasm://wasm/0242ba1a:wasm-function[2954]:0x217653)
 - perfetto::base::CircularQueue<perfetto::trace_processor::TraceSorter::TimestampedEvent>::Grow(unsigned long) (wasm://wasm/0242ba1a:wasm-function[2952]:0x2172c3)
 - perfetto::trace_processor::TraceSorter::PushJsonValue(long long, std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char> >) (wasm://wasm/0242ba1a:wasm-function[4043]:0x2a1964)
 - perfetto::trace_processor::JsonTraceTokenizer::HandleTraceEvent(char const*, char const*, char const**) (wasm://wasm/0242ba1a:wasm-function[4040]:0x2a0d0f)
 - perfetto::trace_processor::JsonTraceTokenizer::ParseInternal(char const*, char const*, char const**) (wasm://wasm/0242ba1a:wasm-function[4038]:0x2a046e)
 - perfetto::trace_processor::JsonTraceTokenizer::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[4036]:0x2a0046)
 - perfetto::trace_processor::ForwardingTraceParser::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[1483]:0x157d10)
 - perfetto::trace_processor::TraceProcessorStorageImpl::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[1579]:0x15f677)
 - perfetto::trace_processor::TraceProcessorImpl::Parse(perfetto::trace_processor::TraceBlobView) (wasm://wasm/0242ba1a:wasm-function[241]:0x14865)
 -  (stderr: )
 - Loading trace 67.11 MB (25.2 MB/s)Loading trace 134.22 MB (28.2 MB/s)Loading trace 201.33 MB (29.8 MB/s)Loading trace 268.44 MB (30.3 MB/s)Loading trace 335.54 MB (30.7 MB/s)Loading trace 402.65 MB (30.9 MB/s)Loading trace 469.76 MB (31.1 MB/s)Loading trace 536.87 MB (31.2 MB/s)Loading trace 603.98 MB (31.3 MB/s)Loading trace 671.09 MB (31.4 MB/s)Loading trace 738.20 MB (31.5 MB/s)Loading trace 805.31 MB (31.6 MB/s)Loading trace 872.42 MB (31.6 MB/s)Loading trace 939.52 MB (31.6 MB/s)Loading trace 1006.63 MB (31.6 MB/s)Loading trace 1073.74 MB (31.5 MB/s)Loading trace 1140.85 MB (31.6 MB/s)Loading trace 1207.96 MB (31.6 MB/s)Loading trace 1275.07 MB (31.7 MB/s)Loading trace 1342.18 MB (31.7 MB/s)Loading trace 1409.29 MB (31.8 MB/s)Loading trace 1476.40 MB (31.8 MB/s)Loading trace 1543.50 MB (31.9 MB/s)Loading trace 1610.61 MB (31.9 MB/s)Loading trace 1677.72 MB (31.9 MB/s)Loading trace 1744.83 MB (31.9 MB/s)Loading trace 1811.94 MB (31.9 MB/s)Loading trace 1879.05 MB (32.0 MB/s)Loading trace 1946.16 MB (32.0 MB/s)Loading trace 2013.27 MB (32.0 MB/s)Loading trace 2080.37 MB (32.1 MB/s)Loading trace 2147.48 MB (32.1 MB/s)Loading trace 2214.59 MB (32.1 MB/s)Loading trace 2281.70 MB (32.1 MB/s)Loading trace 2348.81 MB (32.1 MB/s)Loading trace 2415.92 MB (32.2 MB/s)Loading trace 2483.03 MB (32.2 MB/s)Loading trace 2550.14 MB (32.1 MB/s)Loading trace 2617.25 MB (32.2 MB/s)Loading trace 2684.35 MB (32.2 MB/s)Loading trace 2751.46 MB (32.2 MB/s)Loading trace 2818.57 MB (32.2 MB/s)Loading trace 2885.68 MB (32.2 MB/s)Loading trace 2952.79 MB (32.2 MB/s)Loading trace 3019.90 MB (32.2 MB/s)Loading trace 3087.01 MB (32.2 MB/s)[098.764]            utils.cc:328 PERFETTO_CHECK(res) (errno: 48, Out of memory)
 -  (-----BEGIN PERFETTO PRE-CRASH LOG-----)
 -  (trace_uuid: 00000000-0000-0000-51ab-3d34842)
 - [098.764]            utils.cc:328 PERFETTO_CHECK(res) (errno: 48, Out of memory)
 -  (-----END PERFETTO PRE-CRASH LOG-----)

Trace: http://localhost:9001/localtrace
UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36
Referrer: 

[gaogaotiantian]

It's impossible to slice result.json to smaller pieces to read - first of all, json is not a format that supports that. Most importantly, we are using Perfetto as the front-end and it needs to load the whole thing. Your browser has a maximum memory limit for each process so a 3GB trace might be simply too large.

However, if your RAM is enough (maybe 32+?), you can try vizviewer --use_external_processor which will use a separate process to load the trace and serve as the SQL server for the browser. The problem is it won't be able to show the source code anymore.