Skip to content

Commit b4e35a6

Browse files
committed
clean code base
1 parent d951b37 commit b4e35a6

15 files changed

+24
-1216
lines changed

docs/performance.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,29 @@
1+
# Extremely Fast
2+
3+
`csvtojson` takes care of performance and optimise for Node.js apps.
4+
5+
[This project](https://github.com/Keyang/csvbench) shows `csvtojson` is about **4 - 6 times** faster than other popular csv parsing libraries.
6+
17
# Performance Optimisation
28

39
Version 1.1.0 is much faster than versions before.
410

5-
Below test is parsing a 300K records csv
11+
Below test is parsing a 300k records csv on a 4 core machine
612

7-
Time | workNum = 1 | workNum = 2 (fork)* | workNum = 3 | workNum = 4
13+
Time | workerNum = 1 | workerNum = 2 (fork)* | workerNum = 3 | workerNum = 4
814
--- | --- | --- | --- | ---
915
**1.0.3** | 11.806s | 15.945s | 8.611s | 8.314s
1016
**1.1.0** | 9.707s | 10.065s | 5.955s | 4.563s
1117

12-
*when workNum=2, it only creates 1 extra worker to unblock main process. It is reasonalbe it is slightly slower than workerNum=1 (just use main process).
18+
*when workerNum=2, it only creates 1 extra worker to unblock main process. It is reasonalbe it is slightly slower than workerNum=1 (just use main process).
1319

1420
The result shows V1.1.0 has about 30% - 50% performance boost.
1521

1622
# CPU usage leverage
1723

1824
Below is CPU core usages for v1.1.0 when running the test:
1925

20-
Core | workNum = 1 | workNum = 2| workNum = 3 | workNum = 4
26+
Core | workerNum = 1 | workerNum = 2| workerNum = 3 | workerNum = 4
2127
--- | --- | --- | --- | ---
2228
Core 1 (Main) | 100% | 25% | 60% | 90%
2329
Core 2 | N/A | 80% | 70% | 70%

libs/core/CSVLine_bk.js

Lines changed: 0 additions & 100 deletions
This file was deleted.

libs/core/Converter.js

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,8 @@
11
var util = require("util");
22
var Transform = require("stream").Transform;
3-
var Readable = require("stream").Readable;
4-
var Result = require("./Result");
53
var os = require("os");
64
var eol = os.EOL;
75
// var Processor = require("./Processor.js");
8-
var utils = require("./utils.js");
9-
var async = require("async");
106
var defParam=require("./defParam");
117
var csvline=require("./csvline");
128
var fileline=require("./fileline");

0 commit comments

Comments
 (0)