Reference manual updates for 4.7.0

Author: dmbaturin

site/reference-manual.md

@@ -218,6 +218,21 @@ The value is a <term>Jingoo</term> template. There are following variables in th
 * `source_file_base_name` — name of the input file without extensions (like `cat`).
 * `target_dir` — output directory path.
 
+## Character encoding
+
+By default, soupault assumes that all pages are stored in UTF-8.
+However, if your website uses a different encoding and you have reasons to keep it that way,
+you can configure soupault to use it.
+
+```toml
+[settings]
+  page_character_encoding = 'utf-8'
+```
+
+The following encodings are supported: `ascii`, `iso-8859-1`, `windows-1251`, `windows-1252`, `utf-8`,
+`utf-16`, `utf-16le`, `utf-16be`, `utf-32le`, `utf-32be`, and `ebcdic`.
+You can write those options in either upper or lower case (e.g., `UTF-16LE`, `UTF-16le`, and `utf-16le`
+are equally acceptable).
 
 ## Page processing
 
@@ -652,6 +667,14 @@ Since soupault 4.0.0, you can also specify index sorting options on a per-view b
   strict_sort = false
 ```
 
+Since 4.7.0, it is possible to limit the number of displayed items with:
+
+```toml
+[index.views.main-page-news]
+  # Display up to ten entries on the main page
+  max_items = 10
+```
+
 ### Interaction with widgets
 
 Soupault first inserts rendered index data, then runs widgets. This is to allow widgets to modify HTML generated by index processors.
@@ -726,6 +749,13 @@ or write pages to disk during that first pass.
 Then it will make a second pass to generate actual pages as usual, except the `index_entry` variable will contain the complete
 site index even for plugins running on content pages.
 
+To help plugins and hooks determine if index data for a page is guaranteed to be available or not yet,
+there is a variable named `soupault_pass`.
+
+* 0, if `index_first = false`
+* 1, if `index_first = true` and it's the first (index-only) pass.
+* 2, if `index_first = true` and it's the second (full page build) pass.
+
 ### Exporting metadata to JSON
 
 If built-in functionality is not enough, you can export the site index data to a JSON file
@@ -1603,7 +1633,7 @@ Plugins have access to the following global variables:
   <dt>config</dt>
   <dd>A table with widget config options.</dd>
   <dt>soupault_config</dt>
-  <dd>The global soupault config (deserialized contents of <code>soupault.conf</code>).</dd>
+  <dd>The global soupault config (deserialized contents of <code>soupault.toml</code>).</dd>
   <dt>site_index</dt>
   <dd>Site index data structure.</dd>
   <dt>index_entry</dt>
@@ -1612,15 +1642,22 @@ Plugins have access to the following global variables:
   <dd>Convenience variables for the corresponding config options.</dd>
   <dt>persistent_data</dt>
   <dd>A table for values supposed to be persistent between different plugin runs.</dd>
+  <dt>global_data</dt>
+  <dd>A table for values shared between all plugins and hooks.</dd>
+  <dt>soupault_pass</dt>
+  <dd>
+    The number of the website build pass <a href="#making-index-data-available-to-every-page">two-pass workflow</a>.
+    Always 0 if <code>index.index_first = false</code>, otherwise 1 on the first pass and 2 on the second pass.
+  </dd>
 </dl>
 
 <h4 id="plugin-persistent-data">Persistent data</h4>
 
-All of these variables _except for `persistent_data`_ are injected into the interpreter environment every time a plugin is executed.
+All of these variables _except for `persistent_data` and `global_data`_ are injected into the interpreter environment every time a plugin is executed.
 If you modify their values, it will only affect the instance of the plugin that is currently running. When soupault finishes processing the current page
 and moves on to a new page, the plugin will start in a clean environment.
 
-The `persistent_data` variable is an exception. On soupault startup, its value is set to an empty table.
+The `persistent_data` and `global_data` variables are exception. On soupault startup, their value is set to an empty table.
 When a plugin finishes running, soupault will retrieve it from the Lua interpreter state and pass it to the next plugin run.
 This can be used to avoid running some expensive calculations more than once, or for gathering data from all pages.
 
@@ -1936,12 +1973,21 @@ using this function.
 
 ##### Convenience functions
 
-###### <function>HTML.unwrap(element)</function>
+###### <function>HTML.wrap(node, elem)</function> (since 4.7.0)
+
+Wraps `node` in `elem`.
+
+###### <function>HTML.unwrap(element)</function> (since 4.7.0)
 
 Removes the element and inserts its former children in its place.
 For example, `<div> <p>Test!</p> </div>` will remove the outer `<div>`
 and leave just `<p>Test!</p>`.
 
+###### <function>HTML.swap(l, r)</function>
+
+Swaps two elements in the element tree. Element nodes `l` and `r`, obviously,
+must belong to the same element tree.
+
 ###### <function>HTML.get_heading_level(element)</function>
 
 For elements whose tag name matches `<h[1-9]>` pattern, returns the heading level.
@@ -2425,6 +2471,27 @@ Same as `YAML.from_string`, but returns `nil` on parse errors. Parse error messa
 
 </module>
 
+<module name="CSV"> (since 4.7.0)
+
+##### <function>CSV.from_string(str)</function>
+
+Parses CSV data and returns it as a list (i.e., an int-indexed table) of lists.
+
+##### <function>CSV.unsafe_from_string(str)</function>
+
+Like `CSV.from_string` but returns `nil` on errors instead or raising an exception.
+
+##### <function>CSV.to_list_of_tables(csv_data)</function>
+
+Converts CSV data returned by `CSV.from_string` into a list of string-indexed tables for easy rendering in templates.
+The data variable must have at least two rows, the first row is assumed to be the header
+and used for field names.
+
+All rows must have the same number of columns. If a row is malformed, this function raises an exception.
+There is no unsafe equivalent of it in soupault.
+
+</module>
+
 <module name="Date">
 
 ##### <function>Date.now_timestamp()</function>
@@ -2631,8 +2698,8 @@ Page processing hooks allow Lua code to run between processing steps of replace
 They have access to the same API functions as plugins, but their execution environments
 are somewhat different and vary between hooks.
 
-Hooks are configured in the `hooks` table. The following hooks are supported as of soupault 4.2.0:
-`pre-parse`, `pre-process`, `post-index`, `render`, `save`, `post-save`.
+Hooks are configured in the `hooks` table. The following hooks are supported as of soupault 4.7.0:
+`pre-parse`, `pre-process`, `post-index`, `render`, `save`, `post-save`, and `post-build`.
 
 Like widget subtables, hook subtables can contain arbitrary options.
 
@@ -2657,6 +2724,8 @@ It has the following variables in its environment:
 * `soupault_config` — the complete soupault config.
 * `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
 * `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
+* `global_data` — the global shared data table.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
 
 For example, this website uses a pre-parse hook to globally replace the `$SOUPAULT_RELEASE$` string
 with the latest soupault release version from custom options.
@@ -2685,6 +2754,8 @@ It has the following variables in its environment:
 * `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
 * `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
 * `build_dir` — the output directory from `settings.build_dir` or the `--build-dir` option if present.
+* `global_data` — the global shared data table.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
 
 Soupault takes back the following variables:
 
@@ -2743,6 +2814,8 @@ It has the following variables in its environment:
 * `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
 * `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
 * `build_dir` — the output directory from `settings.build_dir` or the `--build-dir` option if present.
+* `global_data` — the global shared data table.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
 
 The following variables are taken back by soupault:
 
@@ -2782,6 +2855,8 @@ It has the following variables in its environment:
 * `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
 * `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
 * `build_dir` — the output directory from `settings.build_dir` or the `--build-dir` option if present.
+* `global_data` — the global shared data table.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
 
 Soupault takes back the following variables:
 
@@ -2812,6 +2887,8 @@ It has the following variables in its environment:
 * `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
 * `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
 * `build_dir` — the output directory from `settings.build_dir` or the `--build-dir` option if present.
+* `global_data` — the global shared data table.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
 
 For a trivial example, here’s how to just write the HTML to the default output file:
 
@@ -2836,6 +2913,8 @@ It has the following variables in its environment:
 * `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
 * `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
 * `build_dir` — the output directory from `settings.build_dir` or the `--build-dir` option if present.
+* `global_data` — the global shared data table.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
 
 For a trivial example, here’s how to output generated file size using `du` (on UNIX-like systems):
 
@@ -2844,6 +2923,33 @@ For a trivial example, here’s how to output generated file size using `du` (on
   lua_source = '''
     page_size = Sys.get_program_output(format("du -h %s", target_file))
     Log.info(format("Page size: %s", page_size))
+  '''
+```
+
+<h3 id="hooks-post-build">Post-build</h3>
+
+The `post-build` hook runs after all pages are processed and soupault is about to terminate.
+It runs once rather than for every page, so it does not have page-specific variables in its environment
+(like `page_file` or `target_file` — they only make sense for hooks that run on every page).
+
+It has the following variables in its environment:
+
+* `config` (aka `hook_config`) — the hook config table.
+* `soupault_config` — the complete soupault config.
+* `force` — true when soupault is called with `--force` option, plugins are free to interpret it.
+* `site_dir` — the value from `settings.site_dir` or the `--site-dir` option if present.
+* `build_dir` — the output directory from `settings.build_dir` or the `--build-dir` option if present.
+* `soupault_pass` — the number of the pass in the [two-pass workflow](#making-index-data-available-to-every-page).
+* `global_data` — the global shared data table.
+
+Example:
+
+```toml
+[hooks.post-build]
+  lua_source = '''
+    Log.debug("The final state of globao_data:")
+    Log.debug(JSON.to_string(global_data))
+  '''
 ```
 
 ## Glossary