initial release

.gitattributes

@@ -0,0 +1,2 @@
+# Treat search indices as binary; don't try to merge them as text
+**/search.json -diff

.gitignore

@@ -0,0 +1,116 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+lerna-debug.log*
+
+# Diagnostic reports (https://nodejs.org/api/report.html)
+report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
+
+# Runtime data
+pids
+*.pid
+*.seed
+*.pid.lock
+
+# Directory for instrumented libs generated by jscoverage/JSCover
+lib-cov
+
+# Coverage directory used by tools like istanbul
+coverage
+*.lcov
+
+# nyc test coverage
+.nyc_output
+
+# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
+.grunt
+
+# Bower dependency directory (https://bower.io/)
+bower_components
+
+# node-waf configuration
+.lock-wscript
+
+# Compiled binary addons (https://nodejs.org/api/addons.html)
+build/Release
+
+# Dependency directories
+node_modules/
+jspm_packages/
+
+# Snowpack dependency directory (https://snowpack.dev/)
+web_modules/
+
+# TypeScript cache
+*.tsbuildinfo
+
+# Optional npm cache directory
+.npm
+
+# Optional eslint cache
+.eslintcache
+
+# Microbundle cache
+.rpt2_cache/
+.rts2_cache_cjs/
+.rts2_cache_es/
+.rts2_cache_umd/
+
+# Optional REPL history
+.node_repl_history
+
+# Output of 'npm pack'
+*.tgz
+
+# Yarn Integrity file
+.yarn-integrity
+
+# dotenv environment variables file
+.env
+.env.test
+
+# parcel-bundler cache (https://parceljs.org/)
+.cache
+.parcel-cache
+
+# Next.js build output
+.next
+out
+
+# Nuxt.js build / generate output
+.nuxt
+dist
+
+# Gatsby files
+.cache/
+# Comment in the public line in if your project uses Gatsby and not Next.js
+# https://nextjs.org/blog/next-9-1#public-directory-support
+# public
+
+# vuepress build output
+.vuepress/dist
+
+# Serverless directories
+.serverless/
+
+# FuseBox cache
+.fusebox/
+
+# DynamoDB Local files
+.dynamodb/
+
+# TernJS port file
+.tern-port
+
+# Stores VSCode versions used for testing VSCode extensions
+.vscode-test
+
+# yarn v2
+.yarn/cache
+.yarn/unplugged
+.yarn/build-state.yml
+.yarn/install-state.gz
+.pnp.*

.nvmrc

@@ -0,0 +1,4 @@
+14
+
+# Sets which Node version to use
+# https://github.com/nvm-sh/nvm

.prettierignore

@@ -0,0 +1 @@
+**/search.json

.prettierrc

@@ -0,0 +1,11 @@
+{
+  "tabWidth": 2,
+  "semi": true,
+  "trailingComma": "all",
+  "overrides": [{
+    "files": ["README.md"],
+    "options": {
+      "printWidth": 120
+    }
+  }]
+}

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 Alex Krolick
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,206 @@
+# search-notes
+
+A search index and more for your markdown notes.
+
+```sh
+search-notes highlands              basic text search
+search-notes name:wallace           search yaml frontmatter
+search-notes "tag:royalty britain"  multiple terms
+search-notes "brit*"                prefix search
+search-notes "scatlond~1"           fuzzy search
+search-notes "tags:stuart -france"  exclude terms
+search-notes "tags:stuart +france"  boolean AND
+search-notes "britain^2 france^1"   boost terms
+```
+
+```
+% search-notes "scot* +tags:stuart" -d ./test/notes
+Search took 0.017599736 seconds
+┌─────────┬─────────────────────┬─────────┬────────────────────────────────────────────┐
+│ (index) │        File         │  Score  │                    Hits                    │
+├─────────┼─────────────────────┼─────────┼────────────────────────────────────────────┤
+│    0    │ 'prince-charlie.md' │ '0.221' │ '"scotland" (body, tags), "stuart" (tags)' │
+│    1    │   'queen-anne.md'   │ '0.189' │    '"scotland" (body), "stuart" (tags)'    │
+└─────────┴─────────────────────┴─────────┴────────────────────────────────────────────┘
+```
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [Install](#install)
+- [Features](#features)
+  - [Metadata search](#metadata-search)
+  - [Tag search](#tag-search)
+- [CLI](#cli)
+  - [Caching the search index](#caching-the-search-index)
+    - [Without Cache](#without-cache)
+    - [With Cache](#with-cache)
+    - [Using the --write-cache (-w) flag](#using-the---write-cache--w-flag)
+- [TODO](#todo)
+- [License](#license)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Install
+
+Zero-install: use npx to download and run
+
+```
+npx search-notes query
+```
+
+Global:
+
+```
+npm install -g local-notes
+search-notes query
+```
+
+## Features
+
+### Metadata search
+
+Parses YML frontmatter into structured data you can query.
+
+```md
+---
+title: Scottish History
+year: 2020
+---
+
+# Scottish History
+
+This is the body of the post.
+You can query it using fuzzy search.
+```
+
+```
+search-notes "title:scottish year:202* past~1"
+```
+
+### Tag search
+
+```md
+Hashtags are #indexed so you can #query them
+```
+
+```sh
+search-notes tag:query
+```
+
+## CLI
+
+```
+search-notes --help
+```
+
+```text
+search-notes [query]
+
+Search for notes using structured data and full text index, with fuzzy matching.
+
+Positionals:
+  query  text to search for                                                                        [string] [default: ""]
+
+Options:
+  --directory, -d    directory to search in                                                        [string] [default: ""]
+  --cache, -c        cached search index to use (ignored if file doesn't exist)         [string] [default: "search.json"]
+  --write-cache, -w  create or update search index cache file                                  [boolean] [default: false]
+  --explain          show relevance score and other details for results                         [boolean] [default: true]
+  --version          Show version number                                                                        [boolean]
+  --help             Show help                                                                                  [boolean]
+
+Examples:
+  search-notes highlands               basic text search
+  search-notes name:wallace            search yaml frontmatter
+  search-notes "tags:royalty britain"  multiple terms
+  search-notes "brit*"                 prefix search
+  search-notes "scatlond~1"            fuzzy search
+  search-notes "scatlonz~2"            fuzzier search
+  search-notes "tags:stuart -france"   negate term
+  search-notes "tags:stuart +france"   boolean AND
+  search-notes "britain^2 france^1"    boost term relevance
+  search-notes -w                      re-index folder and save cache to disk
+  search-notes -c index.json query     specify index cache file
+```
+
+### Caching the search index
+
+Writing the search index to disk will significantly speed up subsequent queries.
+The downside is you need to remember to update it when you files change.
+In the future there might be a filesystem watch mode that can do this for you.
+
+If you want to stop using the cache, just delete the file.
+
+#### Without Cache
+
+```
+% search-notes "scot* tags:stuart" -d ./test/notes
+Search took 0.018347418 seconds
+┌─────────┬──────────────────────┬─────────┬────────────────────────────────────────────┐
+│ (index) │         File         │  Score  │                    Hits                    │
+├─────────┼──────────────────────┼─────────┼────────────────────────────────────────────┤
+│    0    │ 'william-wallace.md' │ '1.181' │            '"scottish" (body)'             │
+│    1    │ 'prince-charlie.md'  │ '0.221' │ '"scotland" (body, tags), "stuart" (tags)' │
+│    2    │   'queen-anne.md'    │ '0.189' │    '"scotland" (body), "stuart" (tags)'    │
+└─────────┴──────────────────────┴─────────┴────────────────────────────────────────────┘
+```
+
+#### With Cache
+
+```
+% search-notes "scot* tags:stuart" -d ./test/notes
+Search took 0.001248612 seconds
+┌─────────┬──────────────────────┬─────────┬────────────────────────────────────────────┐
+│ (index) │         File         │  Score  │                    Hits                    │
+├─────────┼──────────────────────┼─────────┼────────────────────────────────────────────┤
+│    0    │ 'william-wallace.md' │ '1.181' │            '"scottish" (body)'             │
+│    1    │ 'prince-charlie.md'  │ '0.221' │ '"scotland" (body, tags), "stuart" (tags)' │
+│    2    │   'queen-anne.md'    │ '0.189' │    '"scotland" (body), "stuart" (tags)'    │
+└─────────┴──────────────────────┴─────────┴────────────────────────────────────────────┘
+```
+
+#### Using the --write-cache (-w) flag
+
+You can invoke `search-notes -w` with no search option to index the folder and write the index to disk.
+
+```
+% search-notes -d ./test/notes -w
+Updated index file in 0.02012653 seconds
+Search took 0.021347405 seconds
+┌─────────┬──────────────────────┬─────────┬──────┐
+│ (index) │         File         │  Score  │ Hits │
+├─────────┼──────────────────────┼─────────┼──────┤
+│    0    │ 'prince-charlie.md'  │ '0.000' │  ''  │
+│    1    │   'queen-anne.md'    │ '0.000' │  ''  │
+│    2    │ 'william-wallace.md' │ '0.000' │  ''  │
+└─────────┴──────────────────────┴─────────┴──────┘
+```
+
+## TODO
+
+- [x] index/reindex command
+- [x] search command
+- [ ] index nested folders
+- [ ] more sensible search defaults (see elasticlunr)
+- [ ] more output formatting options
+- [ ] index links between notes (`"linksTo:somewhere.md linkedFrom:elsewhere.md"`)
+- [ ] print snippet of file around hits (like `grep -n`)
+- [ ] add more remarkable plugins out of the box (LaTeX formula rendering, etc)
+- [ ] extract core modules from CLI, to enable re-use
+- [ ] background process for watching file changes and reindexing
+- [ ] server for viewing & browsing rendered notes, with search embed
+- [ ] incremental index update (not supported by lunr)
+- [ ] command to set up git to treat index file as binary (see .gitattributes)
+- [ ] webcomponent for embedding search in markdown
+- [ ] numeric data types for metadata ("rating > 4") _hard_
+- [ ] backlink visualizer (node graph)
+  - [ ] static output
+  - [ ] web page output
+  - [ ] filter-then-display
+- [ ] package as a binary instead of nodejs library
+  - [ ] electron app that acts as a container for background processes and wraps CLI
+
+## License
+
+[MIT](./LICENSE)

lib/cli.js

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+
+require('.')