Skip to content

Commit 5c2822b

Browse files
author
Phil Shafer
committed
Merge branch 'develop'
2 parents ac837e4 + 4142308 commit 5c2822b

File tree

4 files changed

+370
-86
lines changed

4 files changed

+370
-86
lines changed

‎libxo/Makefile.am‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ man5_files = \
7474
xo_format.5
7575

7676
man7_files = \
77+
libxo-csv.7 \
7778
xo_options.7
7879

7980
man_MANS = ${man3_files} ${man5_files} ${man7_files}

‎libxo/libxo-csv.7‎

Lines changed: 274 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,274 @@
1+
.\" #
2+
.\" # Copyright (c) 2021, Juniper Networks, Inc.
3+
.\" # All rights reserved.
4+
.\" # This SOFTWARE is licensed under the LICENSE provided in the
5+
.\" # ../Copyright file. By downloading, installing, copying, or
6+
.\" # using the SOFTWARE, you agree to be bound by the terms of that
7+
.\" # LICENSE.
8+
.\" # Phil Shafer, May 2021
9+
.\"
10+
.Dd May 13, 2021
11+
.Dt LIBXO-CSV 7
12+
.Os
13+
.Sh NAME
14+
.Nm --libxo encoder=csv[+options]
15+
.Nd a CVS encoder for libxo\-based commands
16+
.Sh DESCRIPTION
17+
The
18+
.Nm libxo
19+
library supports a "pluggable" encoder mechanism, and ships with an
20+
encoder for CSV (comma separated values) files. The encoder allows
21+
paths and fields to be selected out of the output contents:
22+
.Bd -literal -offset indent
23+
% df --libxo @csv
24+
name,total-blocks,used-blocks,available-blocks,used-percent,mounted-on
25+
zroot/ROOT/default,3825984331,29376725,3796607605,1,/
26+
devfs,1,1,0,100,/dev
27+
zroot/usr/home,3808301289,11693684,3796607605,0,/usr/home
28+
zroot/var/audit,3796607806,201,3796607605,0,/var/audit
29+
...
30+
% df --libxo @csv+leafs=name.used-percent
31+
name,used-percent
32+
zroot/ROOT/default,1
33+
devfs,100
34+
zroot/usr/home,0
35+
zroot/var/audit,0
36+
...
37+
% df --libxo @csv+leafs=available-blocks+no-header /
38+
3796607605
39+
.Ed
40+
contains software to generate four "built-in"
41+
formats: text, XML, JSON, and HTML.
42+
These formats are common and useful, but there are other common and
43+
useful formats that users will want, and including them all in the
44+
libxo software would be difficult and cumbersome.
45+
.Pp
46+
To allow support for additional encodings, libxo includes a
47+
"pluggable" extension mechanism for dynamically loading new encoders.
48+
.Nm libxo -based
49+
applications can automatically use any installed encoder.
50+
.Pp
51+
Use the "encoder=XXX" option to access encoders. The following
52+
example uses the "cbor" encoder, saving the output into a file:
53+
.Bd -literal -offset indent
54+
df --libxo encoder=cbor > df-output.cbor
55+
.Ed
56+
.Pp
57+
Encoders can support specific options that can be accessed by
58+
following the encoder name with a colon (':') or a plus sign ('+') and
59+
one of more options, separated by the same character:
60+
.Bd -literal -offset indent
61+
df --libxo encoder=csv+path=filesystem+leaf=name+no-header
62+
df --libxo encoder=csv:path=filesystem:leaf=name:no-header
63+
.Ed
64+
.Pp
65+
These examples instructs libxo to load the "csv" encoder and pass the
66+
following options:
67+
.Bd -literal -offset indent
68+
path=filesystem
69+
leaf=name
70+
no-header
71+
.Ed
72+
.Pp
73+
Each of these option is interpreted by the encoder, and all such
74+
options names and semantics are specific to the particular encoder.
75+
Refer to the intended encoder for documentation on its options.
76+
.Pp
77+
The string "@" can be used in place of the string "encoder=".
78+
.Bd -literal -offset indent
79+
df --libxo @csv:no-header
80+
.Ed
81+
.Sh The CSV (Comma Separated Values) Encoder
82+
.Nm libxo
83+
ships with a custom encoder for "CSV" files, a common format for
84+
comma separated values. The output of the CSV encoder can be loaded
85+
directly into spreadsheets or similar applications.
86+
.Pp
87+
A standard for CSV files is provided in RFC 4180, but since the
88+
format predates that standard by decades, there are many minor
89+
differences in CSV file consumers and their expectations. The CSV
90+
encoder has a number of options to tailor output to those
91+
expectations.
92+
.Pp
93+
Consider the following XML:
94+
.Bd -literal -offset indent
95+
% list-items --libxo xml,pretty
96+
<top>
97+
<data test="value">
98+
<item test2="value2">
99+
<sku test3="value3" key="key">GRO-000-415</sku>
100+
<name key="key">gum</name>
101+
<sold>1412</sold>
102+
<in-stock>54</in-stock>
103+
<on-order>10</on-order>
104+
</item>
105+
<item>
106+
<sku test3="value3" key="key">HRD-000-212</sku>
107+
<name key="key">rope</name>
108+
<sold>85</sold>
109+
<in-stock>4</in-stock>
110+
<on-order>2</on-order>
111+
</item>
112+
<item>
113+
<sku test3="value3" key="key">HRD-000-517</sku>
114+
<name key="key">ladder</name>
115+
<sold>0</sold>
116+
<in-stock>2</in-stock>
117+
<on-order>1</on-order>
118+
</item>
119+
</data>
120+
</top>
121+
.Ed
122+
.Pp
123+
This output is a list of `instances` (named "item"), each containing a
124+
set of `leafs` ("sku", "name", etc).
125+
.Pp
126+
The CSV encoder will emit the leaf values in this output as `fields`
127+
inside a CSV `record`, which is a line containing a set of
128+
comma-separated values:
129+
.Bd -literal -offset indent
130+
% list-items --libxo encoder=csv
131+
sku,name,sold,in-stock,on-order
132+
GRO-000-415,gum,1412,54,10
133+
HRD-000-212,rope,85,4,2
134+
HRD-000-517,ladder,0,2,1
135+
.Ed
136+
.Pp
137+
Be aware that since the CSV encoder looks for data instances, when
138+
used with
139+
.Nm xo ,
140+
the `--instance` option will be needed:
141+
.Bd -literal -offset indent
142+
% xo --libxo encoder=csv --instance foo 'The {:product} is {:status}\n' stereo "in route"
143+
product,status
144+
stereo,in route
145+
.Ed
146+
.Sh The "path" Option
147+
By default, the CSV encoder will attempt to emit any list instance
148+
generated by the application.
149+
In some cases, this may be unacceptable, and a specific list may be
150+
desired.
151+
.Pp
152+
Use the "path" option to limit the processing of output to a specific
153+
hierarchy. The path should be one or more names of containers or
154+
lists.
155+
.Pp
156+
For example, if the "list-items" application generates other lists,
157+
the user can give "path=top/data/item" as a path:
158+
.Bd -literal -offset indent
159+
% list-items --libxo encoder=csv:path=top/data/item
160+
sku,name,sold,in-stock,on-order
161+
GRO-000-415,gum,1412,54,10
162+
HRD-000-212,rope,85,4,2
163+
HRD-000-517,ladder,0,2,1
164+
.Ed
165+
.Pp
166+
Paths are "relative", meaning they need not be a complete set
167+
of names to the list. This means that "path=item" may be sufficient
168+
for the above example.
169+
.Sh The "leafs" Option
170+
The CSV encoding requires that all lines of output have the same
171+
number of fields with the same order. In contrast, XML and JSON allow
172+
any order (though libxo forces key leafs to appear before other
173+
leafs).
174+
.Pp
175+
To maintain a consistent set of fields inside the CSV file, the same
176+
set of leafs must be selected from each list item. By default, the
177+
CSV encoder records the set of leafs that appear in the first list
178+
instance it processes, and extract only those leafs from future
179+
instances. If the first instance is missing a leaf that is desired by
180+
the consumer, the "leaf" option can be used to ensure that an empty
181+
value is recorded for instances that lack a particular leaf.
182+
.Pp
183+
The "leafs" option can also be used to exclude leafs, limiting the
184+
output to only those leafs provided.
185+
.Pp
186+
In addition, the order of the output fields follows the order in which
187+
the leafs are listed. "leafs=one.two" and "leafs=two.one" give
188+
distinct output.
189+
.Pp
190+
So the "leafs" option can be used to expand, limit, and order the set
191+
of leafs.
192+
.Pp
193+
The value of the leafs option should be one or more leaf names,
194+
separated by a period ("."):
195+
.Bd -literal -offset indent
196+
% list-items --libxo encoder=csv:leafs=sku.on-order
197+
sku,on-order
198+
GRO-000-415,10
199+
HRD-000-212,2
200+
HRD-000-517,1
201+
% list-items -libxo encoder=csv:leafs=on-order.sku
202+
on-order,sku
203+
10,GRO-000-415
204+
2,HRD-000-212
205+
1,HRD-000-517
206+
.Ed
207+
.Pp
208+
Note that since libxo uses terminology from YANG (:RFC:`7950`), the
209+
data modeling language for NETCONF (:RFC:`6241`), which uses "leafs"
210+
as the plural form of "leaf". libxo follows that convention.
211+
.Sh The "no-header" Option
212+
CSV files typical begin with a line that defines the fields included
213+
in that file, in an attempt to make the contents self-defining:
214+
.Bd -literal -offset indent
215+
sku,name,sold,in-stock,on-order
216+
GRO-000-415,gum,1412,54,10
217+
HRD-000-212,rope,85,4,2
218+
HRD-000-517,ladder,0,2,1
219+
.Ed
220+
.Pp
221+
There is no reliable mechanism for determining whether this header
222+
line is included, so the consumer must make an assumption.
223+
.Pp
224+
The csv encoder defaults to producing the header line, but the
225+
"no-header" option can be included to avoid the header line.
226+
.Sh The "no-quotes" Option
227+
RFC 4180 specifies that fields containing spaces should be quoted, but
228+
many CSV consumers do not handle quotes. The "no-quotes" option
229+
instruct the CSV encoder to avoid the use of quotes.
230+
.Sh The "dos" Option
231+
RFC 4180 defines the end-of-line marker as a carriage return
232+
followed by a newline. This "CRLF" convention dates from the distant
233+
past, but its use was anchored in the 1980s by the `DOS` operating
234+
system.
235+
.Pp
236+
The CSV encoder defaults to using the standard Unix end-of-line
237+
marker, a simple newline. Use the "dos" option to use the `CRLF`
238+
convention.
239+
.Sh Option Handling
240+
The handling of command-line options is complex, since there are three
241+
hierarchies in use, but the rules are:
242+
.Bl -bullet
243+
.It
244+
commas separate
245+
.Nm libxo
246+
options
247+
.Bd -literal -ofset indent
248+
\-\-libxo json,pretty,warn
249+
.Ed
250+
.It
251+
pluses separate encoder options
252+
.Bd -literal -ofset indent
253+
\-\-libxo @csv+dos+no-header,warn
254+
.Ed
255+
.It
256+
periods separate tag names
257+
.Bd -literal -ofset indent
258+
\-\-libxo @csv+leafs=name.used.free+dos,warn
259+
.El
260+
.Sh SEE ALSO
261+
.Xr libxo 3 ,
262+
.Xr xo_options 7
263+
.Sh HISTORY
264+
The
265+
.Nm libxo
266+
library first appeared in
267+
.Fx 11.0 .
268+
The CSV encoder first appeared in
269+
.Fx 13.0 .
270+
.Sh AUTHORS
271+
.Nm libxo
272+
was written by
273+
.An Phil Shafer Aq Mt [email protected] .
274+

‎libxo/libxo.3‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,95 @@ suited for terminal output and HTML is suited for display in a web
8686
browser (see
8787
.Xr xohtml 1 ).
8888
.Pp
89+
.Nm libxo
90+
uses command line options to trigger rendering behavior.
91+
The following options are recognised:
92+
.Pp
93+
.Bl -tag -width "--libxo"
94+
.It
95+
\-\^\-libxo <options>
96+
.It
97+
\-\^\-libxo=<options>
98+
.It
99+
\-\^\-libxo:<brief-options>
100+
.El
101+
.Pp
102+
Options is a comma-separated list of tokens that correspond to output
103+
styles, flags, or features:
104+
.Pp
105+
.Bl -tag -width "12345678"
106+
.It Sy "Token Action"
107+
.It Dv dtrt
108+
Enable "Do The Right Thing" mode
109+
.It Dv html
110+
Emit HTML output
111+
.It Dv indent=xx
112+
Set the indentation level
113+
.It Dv info
114+
Add info attributes (HTML)
115+
.It Dv json
116+
Emit JSON output
117+
.It Dv keys
118+
Emit the key attribute for keys (XML)
119+
.It Dv log-gettext
120+
Log (via stderr) each
121+
.Xr gettext 3
122+
string lookup
123+
.It Dv log-syslog
124+
Log (via stderr) each syslog message (via
125+
.Xr xo_syslog 3 )
126+
.It Dv no-humanize
127+
Ignore the {h:} modifier (TEXT, HTML)
128+
.It Dv no-locale
129+
Do not initialize the locale setting
130+
.It Dv no-retain
131+
Prevent retaining formatting information
132+
.It Dv no-top
133+
Do not emit a top set of braces (JSON)
134+
.It Dv not-first
135+
Pretend the 1st output item was not 1st (JSON)
136+
.It Dv pretty
137+
Emit pretty-printed output
138+
.It Dv retain
139+
Force retaining formatting information
140+
.It Dv text
141+
Emit TEXT output
142+
.It Dv underscores
143+
Replace XML-friendly "-"s with JSON friendly "_"s e
144+
.It Dv units
145+
Add the 'units' (XML) or 'data-units (HTML) attribute
146+
.It Dv warn
147+
Emit warnings when libxo detects bad calls
148+
.It Dv warn-xml
149+
Emit warnings in XML
150+
.It Dv xml
151+
Emit XML output
152+
.It Dv xpath
153+
Add XPath expressions (HTML)
154+
.El
155+
.Pp
156+
The
157+
.Dq brief-options
158+
are single letter commands, designed for those with
159+
too little patience to use real tokens.
160+
No comma separator is used.
161+
.Bl -column "i<num>"
162+
.It Sy "Token Action"
163+
.It "H " "Enable HTML output (XO_STYLE_HTML)"
164+
.It "I " "Enable info output (XOF_INFO)"
165+
.It "i<num> " "Indent by <number>"
166+
.It "J " "Enable JSON output (XO_STYLE_JSON)"
167+
.It "P " "Enable pretty-printed output (XOF_PRETTY)"
168+
.It "T " "Enable text output (XO_STYLE_TEXT)"
169+
.It "W " "Enable warnings (XOF_WARN)"
170+
.It "X " "Enable XML output (XO_STYLE_XML)"
171+
.It "x " "Enable XPath data (XOF_XPATH)"
172+
.El
173+
.Pp
174+
Refer to
175+
.Xr xo_options 7
176+
for additional option information.
177+
.Pp
89178
The
90179
.Nm
91180
library allows an application to generate text, XML, JSON,
@@ -291,6 +380,7 @@ Instructs
291380
to use an alternative set of low-level output functions.
292381
.El
293382
.Sh SEE ALSO
383+
.Xr libxo-csv 7,
294384
.Xr xo 1 ,
295385
.Xr xolint 1 ,
296386
.Xr xo_attr 3 ,
@@ -303,6 +393,7 @@ to use an alternative set of low-level output functions.
303393
.Xr xo_no_setlocale 3 ,
304394
.Xr xo_open_container 3 ,
305395
.Xr xo_open_list 3 ,
396+
.Xr xo_options 7,
306397
.Xr xo_parse_args 3 ,
307398
.Xr xo_set_allocator 3 ,
308399
.Xr xo_set_flags 3 ,

0 commit comments

Comments
 (0)