(PDB-5739) Reduce factsets "stable||volatile" merging

documentation/api/query/v4/fact-contents.markdown

@@ -16,10 +16,11 @@ canonical: "/puppetdb/latest/api/query/v4/fact-contents.html"
 [factsets]: ./factsets.markdown
 [nodes]: ./nodes.markdown
 
-You can query fact information with greater power by using the `/fact-contents`
-endpoint. This endpoint provides the capability to descend into structured
-facts and query tree nodes deep within this data by using the concept of paths
-and values.
+The `/fact-contents` endpoint provides selective access to factset
+subtrees via `fact path`s.  Note that the `inventory` endpoint will
+often provide more flexible and efficient access to the same
+information.
+
 
 ## Paths and Values
 Structured facts can be thought of as trees. For example,

documentation/api/query/v4/facts.markdown

@@ -16,9 +16,11 @@ canonical: "/puppetdb/latest/api/query/v4/facts.html"
 [fact-contents]: ./fact-contents.markdown
 [nodes]: ./nodes.markdown
 
-You can query facts by making an HTTP request to the `/facts` endpoint.
-
-In Puppet's world, you only interact with facts from one node at a time, so any given fact consists of only a **fact name** and a **value.** But because PuppetDB interacts with a whole population of nodes, each PuppetDB fact also includes a **certname** and an **environment.**
+The `/facts` endpoint provides access to a represntation of node
+factsets where a result is returned for each top-level key in the
+node's structured factset.  Note that the `inventory` endpoint will
+often provide more flexible and efficient access to the same
+information.
 
 ## `/pdb/query/v4/facts`
 
@@ -37,8 +39,8 @@ See [the AST query language page][ast].
 
 ### Query fields
 
-* `name` (string): the name of the fact.
-* `value` (string, numeric, Boolean): the value of the fact.
+* `name` (string): the top-level name of the fact.
+* `value` (json): the value of the fact.
 * `certname` (string): the node associated with the fact.
 * `environment` (string): the environment associated with the fact.
 

documentation/api/query/v4/factsets.markdown

@@ -18,9 +18,11 @@ canonical: "/puppetdb/latest/api/query/v4/factsets.html"
 [producers]: ./producers.markdown
 [nodes]: ./nodes.markdown
 
-You can query factsets by making an HTTP request to the `/factsets` endpoint.
-
-A factset is the set of all facts for a single certname.
+The `/factsets` endpoint provides access to a represntation of node
+factsets where each result includes the structured facts for a node
+broken down into a vector of top-level key/value pairs.  Note that the
+`inventory` endpoint will often provide more flexible and efficient
+access to the same information.
 
 ## `/pdb/query/v4/factsets`
 

documentation/api/query/v4/inventory.markdown

@@ -20,9 +20,9 @@ canonical: "/puppetdb/latest/api/query/v4/inventory.html"
 [query]: query.markdown
 [ast]: ./ast.markdown
 
-The `/inventory` endpoint enables an alternative query syntax for digging into
-structured facts, and can be used instead of the `facts`, `fact-contents`, and
-`factsets` endpoints for most fact-related queries.
+The `/inventory` endpoint provides an alternate and potentially more
+efficient way to access structured facts as compared to the `facts`,
+`fact-contents`, and `factsets` endpoints.
 
 ## `/pdb/query/v4/inventory`
 

src/puppetlabs/puppetdb/command.clj

@@ -901,8 +901,9 @@
     (let [msg (trs "Unable to shut down delayed command scheduler, requesting server shutdown")]
       (log/error msg)
       (shut-down-after-command-scheduler-unresponsive
+       ;; This fails request-shutdown schema validation during command-test
        #(request-shutdown {:puppetlabs.trapperkeeper.core/exit
-                           {:status 2 :messages [msg *err*]}})))))
+                           {:status 2 :messages [[msg *err*]]}})))))
 
 (def delay-pool-size
   "Thread count for delaying messages for retry, and broadcast pool

src/puppetlabs/puppetdb/jdbc.clj

@@ -37,8 +37,9 @@
   (and (instance? ExceptionInfo ex)
        (let [{:keys [rollback handling]} (ex-data ex)]
          (and rollback
+              (instance? SQLException handling)
               (= (sql-state :admin-shutdown)
-                 (some-> handling .getSQLState))))))
+                 (.getSQLState handling))))))
 
 (defmacro with-db-connection [spec & body]
   `(sql/with-db-connection [db# ~spec]
@@ -435,18 +436,6 @@
       (.setType "text[]")
       (.setValue (str \{ (str/join \, quoted) \})))))
 
-;; Q: move/replace?
-(defn create-json-path-extraction
-  "Given a base json field and a path of keys to traverse, construct the proper
-  SQL query of the form base->'key'->'key2'..."
-  [field path]
-   (str field
-        (when (seq path)
-          (->> (map single-quote path)
-               (str/join "->")
-               ;; prefix for the first arrow in field->'key1'...
-               (str "->")))))
-
 ;; TODO this is no longer used by the main query engine (engine.clj), but it is
 ;; still used by the legacy query engines and should be removed once it is
 ;; handled by honeysql everywhere

src/puppetlabs/puppetdb/query.clj

@@ -385,17 +385,19 @@
                      FROM (SELECT certname,
                                   producer_timestamp,
                                   environment_id,
-                                  (jsonb_each((stable||volatile))).*
-                           FROM factsets) fs
+                                  facts.*
+                           FROM factsets,
+                           lateral (select * from jsonb_each(stable)
+                                    union all select * from jsonb_each(volatile)) as facts) fs
                            LEFT JOIN environments env ON fs.environment_id = env.id
                      ) AS facts
                     WHERE %s" (column-map->sql fact-columns) where)]
     (apply vector sql params)))
 
 (defn certname-names-query [active]
   (if active
-    "SELECT name FROM certnames WHERE deactivated IS NULL AND expired IS NULL"
-    "SELECT name FROM certnames WHERE deactivated IS NOT NULL OR expired IS NOT NULL") )
+    "SELECT certname FROM certnames_status WHERE deactivated IS NULL AND expired IS NULL"
+    "SELECT certname FROM certnames_status WHERE deactivated IS NOT NULL OR expired IS NOT NULL"))
 
 (defn compile-resource-equality
   "Compile an = operator for a resource query. `path` represents the field