Skip to content

Robust :and parser, add :andn #1182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 107 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2493,7 +2493,7 @@ Schemas can be used to parse values using `m/parse` and `m/parser`:
; :value "Hello, world of data"}}]}}}]}}}
```

Parsing returns tagged values for `:orn`, `:catn`, `:altn` and `:multi`.
Parsing returns tagged values for `:orn`, `:catn`, `:altn`, `:andn` and `:multi`.

```clojure
(def Multi
Expand All @@ -2508,6 +2508,87 @@ Parsing returns tagged values for `:orn`, `:catn`, `:altn` and `:multi`.
; => #malli.core.Tag{:key :malli.core/default, :value {:type "sized", :size 1}}
```

### Parsing `:and`

The `:and` schema combines multiple schemas for the same value and yet only returns the results of parsing one of them.
Which schema is used for parsing is usually chosen automatically.

```clojure
(m/parse [:and [:orn [:left :int] [:right :int]] [:fn number?]] 1)
; => #malli.core.Tag{:key :left, :value 1}
(m/parse [:and [:fn number?] [:orn [:left :int] [:right :int]]] 1)
; => #malli.core.Tag{:key :left, :value 1}
```

The error `:malli.core/and-schema-multiple-transforming-parsers` is thrown if the transforming
parser cannot be picked automatically. This usually means that multiple conjuncts
will transform their input or a false-positive has occurred because the underlying schema
does not implement `malli.core/ParserInfo`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about defaulting to the first transforming parser? That's kind of what we do for json-schema generation etc. Or do you think it would trip up users?

Copy link
Collaborator Author

@frenchy64 frenchy64 May 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this and it broke existing tests. You'd think first parser would be the obvious choice, but I also found ones where last parser was the correct choice.

Instead, I concentrated on more accurate static analysis of parsers to reduce the frequency of manual overrides. I think it was enough to support all the schemas in the current tests automatically.

For example, there was a schema like [:and [:map ...] <transforming-parser-schema>] in the tests somewhere. By improving the detection of :map (that it's only transforming if any children are), we could automatically and intelligently pick the intended transforming parser.

I also had an eye on future robustness. I thought we should help users from accidentally masking their own parsers.

Copy link
Collaborator Author

@frenchy64 frenchy64 May 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After writing #1182 (comment) maybe a default handler should be provided so the user can handle 3rd-party schemas that need an explicit transforming child. This would be a tool to avoid dependency hell.

e.g.,

(m/parser S {::default-parser-info-handler
             (fn [s opts]
               (when (<question> s)
                 {:parse/transforming-child <decision>})})

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a good workaround as well.


```clojure
(m/parser [:and [:map [:left [:orn [:one :int]]]] [:map [:right [:orn [:one :int]]]]])
; Execution error (ExceptionInfo) at malli.core/-exception (core.cljc:189).
; :malli.core/and-schema-multiple-transforming-parsers
```

There are several ways to resolve this.

If you know a single conjunct should exclusively parse the input, use the `:parse` property
to identify the conjunct by index.
To opt-out of parsing any further levels of this schema, use the `:parse :none` property.

```clojure
(m/parse [:and {:parse 0}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is reserving the top-level :parse key for a specific purpose. Also, "parse" on its own is not very descriptive. We propose using a :parse/ ns (just like :gen/ for example). How about :parse/transforming-child-index or :parse/index or :parse/child?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I'll see if a good name comes to me, but other than its length I like :parse/transforming-child-index.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:parse/transforming-child seems ok.

:parse/transforming-child :none
:parse/transforming-child 0
:parse/transforming-child 4

Further possible extensions like :last seem to work with this name.

[:map [:left [:orn [:one :int]]]]
[:map [:right [:orn [:one :int]]]]]
{:left 1 :right 1})
; => {:left #malli.core.Tag{:key :one, :value 1}, :right 1}

(m/parse [:and {:parse 1}
[:map [:left [:orn [:one :int]]]]
[:map [:right [:orn [:one :int]]]]]
{:left 1 :right 1})
; => {:left 1, :right #malli.core.Tag{:key :one, :value 1}}

(m/parse [:and {:parse :none}
[:map [:left [:orn [:one :int]]]]
[:map [:right [:orn [:one :int]]]]]
{:left 1 :right 1})
; => {:left 1, :right 1}
```

To parse all conjuncts, you must migrate the schema to `:andn`. This involves tagging each conjunct
with syntax like `:orn` and `:map`. The results of parsing `:andn` will be wrapped in `Tags` with
an entry for parsing the original value with each conjunct.

Only the left-most child will be unparsed, useful if you plan to modify the results of parsing.

```clojure
(def Paired+Flat
[:andn
[:paired [:* [:catn [:name :string] [:id :int]]]]
[:flat [:vector [:orn [:name :string] [:id :int]]]]])

(m/parse Paired+Flat ["x" 1 "y" 2])
; => #malli.core.Tags{:values
; {:paired [#malli.core.Tags{:values {:name "x", :id 1}}
; #malli.core.Tags{:values {:name "y", :id 2}}],
; :flat [#malli.core.Tag{:key :name, :value "x"}
; #malli.core.Tag{:key :id, :value 1}
; #malli.core.Tag{:key :name, :value "y"}
; #malli.core.Tag{:key :id, :value 2}]}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, this behaviour makes sense, I get it


(as-> ["x" 1 "y" 2] $
(m/parse Paired+Flat $)
(update $ :values
(fn [{:keys [flat paired] :as res}]
;; remove other :andn results like :flat when transforming
{:paired (map-indexed (fn [i p] (update-in p [:values :id] * (+ 2 i) (count flat)))
(rseq paired))}))
(m/unparse Paired+Flat $))
["y" 16 "x" 12]
```

## Unparsing values

The inverse of parsing, using `m/unparse` and `m/unparser`:
Expand All @@ -2521,15 +2602,37 @@ The inverse of parsing, using `m/unparse` and `m/unparser`:
; [:p "Hello, world of data"]]
```

Tags are mapped back to the original schema to be unparsed.

```clojure
(m/unparse [:orn [:name :string] [:id :int]]
(m/tagged :name "x"))
(m/unparse [:orn [:left :string] [:right :int]]
(m/tag :left "x"))
; => "x"

(m/unparse [:* [:catn [:name :string] [:id :int]]]
(m/unparse [:orn [:left :string] [:right :int]]
(m/tag :left 1))
; => ::m/invalid
```

Unparsing can be used to update complex values via an associative interface.

```clojure
(def FlatPairs [:* [:catn [:name :string] [:id :int]]])

(m/parse FlatPairs ["x" 1 "y" 2])
; => [#malli.core.Tags{:values {:name "x", :id 1}}
; #malli.core.Tags{:values {:name "y", :id 2}}]

(m/unparse FlatPairs
[(m/tags {:name "x" :id 1})
(m/tags {:name "y" :id 2})])
; => ["x" 1 "y" 2]

(->> ["x" 1 "y" 2 "z" 3]
(m/parse FlatPairs)
(mapv (fn [tags] (-> tags (update :values #(-> % (update :name str "_") (update :id * 2))))))
(m/unparse FlatPairs))
; => ["x_" 2 "y_" 4 "z_" 6]
```

## Serializable functions
Expand Down
1 change: 1 addition & 0 deletions src/malli/clj_kondo.cljc
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
(defmethod accept :not= [_ _ _ _] :any) ;;??

(defmethod accept :and [_ _ _ _] :any) ;;??
(defmethod accept :andn [_ _ _ _] :any) ;;??
(defmethod accept :or [_ _ _ _] :any) ;;??
(defmethod accept :orn [_ _ _ _] :any) ;;??
(defmethod accept :not [_ _ _ _] :any) ;;??
Expand Down
Loading