Bring back `apply` for `argmove`?

[gilch]

My first attempt at implementing partition looked like this:

(defn partition [n coll]
  (->> coll (iter) (,) (* n) (apply zip)))

This doesn't work anymore because we removed apply. I wasn't sure if it was redundant or not at the time. But let's try it with the new syntax.

=> (defn partition-v1 [n coll]
...   (->> coll iter , (* n) (zip #*)))
   ...
LexException: Ran into a RPAREN where it wasn't expected.

Not good. apply still has uses, apparently. I'm not sure how well I liked the old version, but it feels like something should replace it, since #*/#** can't always do it.

You can sort of work around this with xi

=> (require [hy.extra.anaphoric [xi]])
None
=> (defn partition-v2 [n coll]
...   (-> coll iter , (* n) ((xi zip #* x1))))

def partition_v2(n, coll):
    return (lambda x1: zip(*x1))(((iter(coll),) * n))
None

But this adds a useless lambda in the compilation. apply didn't require an extra call like this. Maybe some kind of macro could work.

[gilch]

Here's a candidate apply macro.

(defmacro apply [expr args &optional [kwargs {}]]
  `(~@(if (isinstance expr HyExpression)
        expr
        [expr])
     #* ~args
     #** ~kwargs))

Python's distinction between args and kwargs made the old apply awkward compared to the usual Lisp version.

I like this candidate better than our old one. hylang/hy#891.

It still works like before:

=> (apply print [4 5 6] {'sep "::"})
from hy import HySymbol
print(*[4, 5, 6], None={HySymbol('sep'): '::',})
4::5::6

But now the final dict is optional, and you can partially apply things before unpacking.

=> (setv args [4 5 6])
args = [4, 5, 6]
None
=> (apply (print 1 2 :sep "::") args)
print(1, 2, *args, sep='::', None={})
1::2::4::5::6

My original partition would work with this apply. It still doesn't work well with -> though, since it's not usually the function you want inserted. Maybe the solution is a another version with different ordering.

[gilch]

Maybe the solution is a another [apply] version with different ordering.

On second thought, you can use a ->> anywhere in a -> to thread in the tail for a single form, while threading first for everything else.

=> (-> (range 10) iter , (* 2) (->> (apply zip)) list)

list(zip(*((iter(range(10)),) * 2), None={}))
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]

You can also use as-> this way to thread in any position you like.

=> (-> (range 10) iter , (* 2) (as-> it (apply zip it)) list)

it = ((iter(range(10)),) * 2)
it = zip(*it, None={})
list(it)
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]

But this isn't really required for the candidate apply, even if you're trying to apply to a dict, since you can also thread a dict in the tail position by providing an empty list -- (-> ... (->> (apply foo [])).

It's certainly usable like this, but I'm not sure if a different ordering would be better.

[gilch]

I just realized that the as-> macro works even without apply.

=> (-> (range 10) iter , (* 2) (as-> it (zip #* it)) list)

it = ((iter(range(10)),) * 2)
it = zip(*it)
list(it)
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]

The compilation is not as nice as with (-> (range 10) iter , (* 2) (->> (apply zip)) list), because it leaks the it, but the fact that it works now makes this issue seem much less urgent. But I wonder if there's a way to tweak the compiler to make (->> ... (foo #*)) work.

We might also want a shadow apply for use with higher-order functions, like map and comp. Our #*/#** syntax can't do this. (A quick xi would suffice, but that's true of a lot of core functions.) A possible implementation--

(defn apply [f args &optional [kwargs {}]]
  (f #* args #** kwargs))

This lacks the partial syntax of the macro, which is not possible for a function, but if you're using HOF anyway, you could just use partial on the f before apply gets it. This version seems useful, but again, I'm not sure if something else would be better. Good design might require more thought about use cases. If that were a common requirement, a higher-order version might be better:

(defn apply [f &rest args &kwargs kwargs]
  (fn [args2 &optional [kwargs2 {}]]
    (f #* (chain args args2)
       #** (doto (.copy kwargs)
                 (.update kwargs2)))))

[Kodiologist]

But I wonder if there's a way to tweak the compiler to make (->> ... (foo #*)) work.

Possibly, using more compiler peeking into child forms. The thing is that #* x parses to a single form (unpack_iterable x), which (on Python 3) compiles to a single node of type ast.Starred. (f a #* b c) compiles to a function call with 3 arguments, not 4. That's why the parser requires a term after the #*. What you'd want is to have #* parse to its own object, like a new HyModel type called HyIterableUnpack, and then make it the compiler's job to join HyIterableUnpack with a following form into an ast.Starred in the cases where Python allows this.

[gilch]

a new HyModel type ... the compiler's job to join

Exactly what I was thinking.

[Kodiologist]

My guess from Matthew's original example here and from this Stack Overflow question is that the remaining need for apply arises from argmove macros like ->. So I'm going to reconstrue this issue as a request to add a feature to argmove macros that provides something like apply.