Skip to content

Latest commit

 

History

History
59 lines (44 loc) · 1.64 KB

0176-cl-ltsv.org

File metadata and controls

59 lines (44 loc) · 1.64 KB
Raw

cl-ltsv

This system implements an LTSV logs parser and serializer. LTSV is based on TSV format but each field has a name. This lets us easily add new fields and to process logs in a manageable way:

POFTHEDAY> (cl-ltsv:alist-ltsv '(("message" . "Hello world!")
                                 ("request_id" . 100500)))
"message:Hello world!	request_id:100500"

POFTHEDAY> (cl-ltsv:parse-line *)
(("message" . "Hello world!")
 ("request_id" . "100500"))

LTSV is based on TSV format which has some escaping rules for tabs, newlines and backslashes, but LTSV FAQ says forget about escaping, we don’t need it for our access logs.

I think this decision makes LTSV unusable for general-purpose logs.

For example, if you have newlines or tabs in the logged value, a log will be broken:

POFTHEDAY> (concatenate 'string
                        "Hello"
                        '(#\Newline #\Tab)
                        "World!")
"Hello
	World!"

;; This call should produce a single line,
;; replacing a newline with \n and tab with \t:
POFTHEDAY> (cl-ltsv:alist-ltsv (list (cons "message" *)))
"message:Hello
	World!"

;; Parsing does not process escaped symbols either:

POFTHEDAY> (cl-ltsv:parse-line "message:Hello\\n\\tWorld!")
(("message" . "Hello\\n\\tWorld!"))

That is all I have for today. Probably tomorrow we’ll catch a more interesting library.