[proposal] Design syntax like Haskell's where

[stylewarning]

The Lisp tradition of stacking bindings top-down is reasonably well understood, but in some cases, it's a distraction from the meat of the program. I think we should give an option to put bindings after an expression. Maybe we can use PROGN?

(progn
  (start-gui x)
  (clean-up y)
  (do-laundry clothes)
 :where 
  (start-gui x) =
    (open-blah ...)
  (clean-up y) =
    ...
  (do-laundry z) =
    ...
  clothes = ...)

A big negative with this example is that it's just very different than the rest of the syntax of Coalton.

(let ((foo ...)
      (bar ...))
  ...)

(let x = ...)
(let y = ...)
...

Anyway, something to consider.

[macrologist]

(progn 
    (start-gui x)
    (clean-up y)
    (do-laundry clothes)
    (where 
       (start-gui x) = (open-blah ...)
       (clean-up y) = ....
        (do-laundry clothes) = ....
        x = ...
        y = ...
        clothes = ...))

This would, perhaps in an unwelcome way, permit multiple where forms throughout a prog body.

(define (cook-dinner meal) 
    (let gathered = (gather-ingredients ingredients)
    (where 
         ingredients = (internet:lookup-ingredients meal)
         (gather-ingredients xs) = (grocery:order xs credit-card))
    (prep-kitchen gathered)
    (where 
        (prep-kitchen xs) = ...)
   ...)

[stylewarning]

@macrologist What are your thoughts on the idea more broadly?

[macrologist]

@macrologist What are your thoughts on the idea more broadly?

@stylewarning

I'm generally in favor of it. I like seeing the intended effect of a computation first. However, I'm even more in favor of conventions, idioms, and syntactic support that disfavors long function bodies.

Because Coalton auto-curries applications, I think there is slightly less of a benefit to this kind of thing than there would be in, say, Common Lisp.

In CL, crowding the top of a defun with a salad of flet, lables, let* does tend to distract from the "buisiness end" of the function. As such, in CL something like

(defun-where cool  (zs) 
       (loop :for z : in zs :do (do-cool-stuff 10 z))
        (where-labels 
             ((do-cool-stuff (y x) ...))))

actually seems useful.

But in Coalton, we might as well just

(define (do-cool-stuff y x) ...)

(define (cool zs) 
    (for-each! (do-cool-stuff 10) (into-iter zs)))

or something like that.

But, again, Im generally in favor of it.

E.g, being able to do this would be nice

(define (cool zs) 
    (duplicate-map coolest zs)
    :where 
       (duplicate-map f xs) = (double (map f xs))
       (double ys) = (<> ys ys)
       (coolest z) = (do-cool-stuff 10 z))

[garlic0x1]

I don't like this idea since it allows two different shapes to a function and can take more time to understand how to read it, and I don't see any real benefit.

If the top of a function is crowded with let bindings maybe the user should define auxiliary functions instead.

[macrologist]

If the top of a function is crowded with let bindings maybe the user should define auxiliary functions instead.

I'm sympathetic to this critique too. In general I prefer small function bodies with few binding and "parceling" forms. What I think such a proposal has going for it is that, when somebody decides to define a gaggle of auxiliaries in the body of their function, this syntax makes it easier to see what the auxiliaries are meant to do before having to wade through all of their messy definitions.

In short, I think it makes code easier to read and write under those circumstances when somebody has already decided that module-wise sibling definitions are not the right choice.

[macrologist]

I don't like this idea since it allows two different shapes to a function and can take more time to understand how to read it, and I don't see any real benefit.

What if the shape were announced:

(define (cool-beans beans)
  (where-let                           ; <- announces the syntax, watch for ...
    (report-beans beans)
    (twice-cooled xs)

    :where                              ; ... this, you knew to expect it.
    (twice-cooled xs) = (twice (map cool beans))
    (twice xs) = (<> xs xs)
    (cool x) = (cool-by 10 x)))

[stylewarning]

I'm sympathetic to this critique too. In general I prefer small function bodies with few binding and "parceling" forms. What I think such a proposal has going for it is that, when somebody decides to define a gaggle of auxiliaries in the body of their function, this syntax makes it easier to see what the auxiliaries are meant to do before having to wade through all of their messy definitions.

I read a lot of Lisp code that ends up being 4–5 useless utilities with no practical independent value above the "real" function. It feels good we have more functions, but to me it's like filling our vocabulary with words that only make sense in context.

Haskell code uses this where syntax absolutely pervasively, and it usually leads to (imho) very readable code when the where definitions are named well. Random non-cherry-picked (literally clicked on random articles, if it included where I copy-pasted) examples from Rosetta Code:

comb :: Int -> [a] -> [[a]]
comb m xs = combsBySize xs !! m
  where
    combsBySize = foldr f ([[]] : repeat [])
    f x next =
      zipWith
        (<>)
        (fmap (x :) <$> ([] : next))
        next

--

average :: [PixelRGB8] -> PixelRGB8
average pixels = PixelRGB8 (avg red) (avg green) (avg blue)
  where
    len   = toInteger $ length pixels
    avg c = fromIntegral $ (sum $ map (toInteger . c) pixels) `div` len

--

step rule a = listArray (l,r) res
  where (l,r) = bounds a
        res = [rule (a!r)     (a!l) (a!(l+1)) ] ++
              [rule (a!(i-1)) (a!i) (a!(i+1)) | i <- [l+1..r-1] ] ++
              [rule (a!(r-1)) (a!r) (a!l)     ]

--

fft [] = []
fft [x] = [x]
fft xs = zipWith (+) ys ts ++ zipWith (-) ys ts
    where n = length xs
          ys = fft evens
          zs = fft odds 
          (evens, odds) = split xs
          split [] = ([], [])
          split [x] = ([x], [])
          split (x:y:xs) = (x:xt, y:yt) where (xt, yt) = split xs
          ts = zipWith (\z k -> exp' k n * z) zs [0..]
          exp' k n = cis $ -2 * pi * (fromIntegral k) / (fromIntegral n)

--

isColorful :: Int -> Bool
isColorful n
  | n < 0 = error "Only non-negative integers are allowed"
  | n == 0 || n == 1 = True
  | 0 `elem` digits = False
  | 1 `elem` digits = False
  | not (distinct digits) = False
  | not (distinct subpros) = False
  | otherwise = True
  where
    digits = map digitToInt (show n)
    subpros =  [(prods !! i) `div` (prods !! j) |j <- [0..length(digits)], i <- [(j+1)..length(digits)]]
    prods = scanl (*) 1 digits

Contrast with Lisp-style let:

convexHull
  :: (Floating a, Ord a)
  => [[a]] -> [[a]]
convexHull points =
  let o = minimum points
      presorted = sortBy (compareFrom o) (filter (/= o) points)
      collinears = groupBy (((EQ ==) .) . compareFrom o) presorted
      outmost = maximumBy (comparing (distanceFrom o)) <$> collinears
  in dropConcavities [o] outmost

--

findCycleLength :: Eq a => [a] -> Maybe Int
findCycleLength [] = Nothing
findCycleLength (x:xs) =
  let loop _ _ _ [] = Nothing
      loop pow lam x (y:ys)
        | x == y     = Just lam
        | pow == lam = loop (2*pow) 1 y ys
        | otherwise  = loop pow (1+lam) x ys
  in loop 1 1 x xs

One reason this works very well in Haskell is because it's lazy. In Coalton, eagerness means the bindings would be evaluated first, which means the program's execution flow is no longer left-to-right, top-to-bottom. (The problem may be theoretical, and if not, is somewhat alleviated if all bindings are required to be functions.)

[garlic0x1]

It is appealing that this might allow you to look at the first few lines of a function to understand what it does without looking at the following bindings, but I am dubious about adding another way to bind things with syntax that isn't very "lispy" and that doesn't provide much practical value.

[garlic0x1]

I would rather have scheme-style lexical define, which could maybe even be SCC'd from the bottom of a function body or something idk

[macrologist]

I would rather have scheme-style lexical define, which could maybe even be SCC'd from the bottom of a function body or something idk

I like it

(coalton-toplevel

  (define (cool-beans beans)
    (report-hot-beans beans)
    (double-cool beans)

    (define (double-cool xs)
      (double (map cool xs)))

    (define (double xs) (<> xs xs))

    (define (cool x) (cool-by 10 x))))

Though maybe a different token could be chosen instead of define

local or something? which could be nested.

Or alternatively, we could just make a new let = like form called local or where

(coalton-toplevel

  (define (cool-beans beans)
    (report-hot-beans beans)
    (double-cool beans)

    (local (double-cool xs) = (double (map cool xs)))
    (local (double xs) = (<> xs xs))
    (local (cool x) = (cool-by 10 x))))

[garlic0x1]

I would rather have scheme-style lexical define, which could maybe even be SCC'd from the bottom of a function body or something idk

I like it

(coalton-toplevel

  (define (cool-beans beans)
    (report-hot-beans beans)
    (double-cool beans)

    (define (double-cool xs)
      (double (map cool xs)))

    (define (double xs) (<> xs xs))

    (define (cool x) (cool-by 10 x))))

Though maybe a different token could be chosen instead of define

local or something? which could be nested.

Or alternatively, we could just make a new let = like form called local or where

(coalton-toplevel

  (define (cool-beans beans)
    (report-hot-beans beans)
    (double-cool beans)

    (local (double-cool xs) = (double (map cool xs)))
    (local (double xs) = (<> xs xs))
    (local (cool x) = (cool-by 10 x))))

Yes this is a good idea, since users might want to grep "define" for toplevel things. I still don't see myself using it much in a bottom-to-top manner but this example makes me less dubious.

[karlosz]

I think where works in Haskell in a way that it wouldn't work well in Coalton for the following reasons (with a lead-in into how to solve the same problem differently in a more Lispy way):

• In my view, the main (semantic) reason why it doesn't work well in Coalton, as @stylewarning already mentioned, is that Coalton is applicative. That's probably also why ML doesn't have the syntax. It is way too confusing to reorder the control flow with where statements, since in applicative languages, evaluation order generally read top down when computations are sequenced. This is also the reason why internal define doesn't seem as bad at the end as when where bindings are at the end. This is just because internal defines are usually for functions where evaluation order is not important with respect to the main body of the program. Haskell doesn't have this problem because things happen lazily, and it reads more like math where there's no such thing as evaluation order, you just describe values in terms of other values, and it matches the intended information structure of the sentence you are trying to say better. That's what makes certain sentence orders more natural than others, both in natural language and in programming, and why people sometimes add quantifiers at the end in English (e.g. x equals x for all x), when in formal logic all quantifiers go at the beginning. If you think about it, CL also allows defun of functions to go after other defuns which reference it, but nothing similar for values. (again, applicative). This is foreshadowing for the solution at the end of the next point.
• A secondary, syntactic reason, is that it's very weird and unlispy to have syntax that's not nested in a lexical scoping fashion, especially when the bindings are at the end, and especially with keyword syntax. I am already very skeptical of the existing 'flat' let, but probably mostly because it has weird keyword syntax with the = symbol. I understand the intent is for indentation purposes, but usually the main reason that indentation grows for me in CL is that different binding constructs like destructuring-bind or multiple-value-bind can't participate in let, and that conditionals also introduce more nesting. I think one binding construct to rule them all, i.e. let with pattern matching, and when-let type macros make the problem much better already. However, there is one exception for the 'flat' construct thing, which I think would be allowable and even good because it mirrors the syntax of such 'flat' bindings at the top-level, that is, Scheme-like internal define. I'll elaborate this construct below because I think a more relaxed version of the Scheme internal define solves all our problems and is the only 'flat' construction needed (indeed, top-level define also behaves flatly) and also solves the where problem with information structure in the cases where order of evaluation doesn't matter, and even supercedes the need for let =.

I was surprised to see Scheme-like internal define not included in Coalton at first. It matches what happens at the top-level, if you add it with one key difference: in Scheme, you can only do internal defines before anything else happens in the function body. I think this is unnecessarily restrictive. It's important to not allow out of order definitions of values with respect to their uses, due to the lazy/applicative thing, but you could still have define appear interleaved with values forms as long as definitions and intended references are in the right order, just like at the top-level. And for functions, you don't need to make sure definitions are before uses at all, just like at the top-level. You can put them all at the end if you want just like with where in Haskell if that suits the intended information structure better, or put them in some other interleaving. Again, this is because that's no problem in the applicative world. And this would work great in Coalton, unlike in Scheme, because we have static typing to deduce what is a function or not, similar to how define is treated at the top-level. And I don't buy needing a separate name for internal define: you can grep for indentation as well, and in more cases than not you want to grep without caring about internal-ness or not, which you can't do as easily with a different name as restricting by indentation with just one name.

tl;dr summary: Don't add a where construction, especially not for values. Instead, add internal define (but please not another name), where functions can go anywhere, and values have to respect definition before use order, but potentially interleaved with other forms, exactly like at the top-level. And Coalton's static typing makes this possible, unlike in Scheme. And as a bonus, the current flat let syntax would be superseded by the internal define, and I don't think it looks great anyway, and plain let should be used otherwise.

[karlosz]

Addendum: What happens with:

(define (my-cool-fun x)
  (your-cool-fun (+ x 1))

  (define (your-cool-fun x)
     (+ x x))

  (your-cool-fun (+ x 1))
  (define (your-cool-fun y)
     (+ x x))))

There is only one sensible option I think:
Ban internal define of the same name. This makes sense because why have two functions of the same name in the same compilation unit??

It illustrates that we will have ambiguity with functions which are defined out of order with the other forms if we allow multiple of the same name. (Scheme doesn't have this problem because all internal defines appear at the beginning and thus we can do the normal thing of taking the last binding.) Luckily, there's pretty much zero reason to having multiple bindings of the same name with internal define.

[macrologist]

The danger, from a code-thats-nice-to-look-at perspective, with internal defines is (a) that they encourage labels-esque function body bloat, and (b) they're difficult to scope at-a-glance.

When considering a feature like this - a purely cosmetic one that adds no new algorithmically expressive capabilities to the language, but only stylistically expressive possibilities - questions of code-thats-nice-to-look-at-ness are of principle importance, imo.

(People, I find, systematically undervalue code-thats-nice-to-look-at-ness in the design of their programming languages.)

Regarding danger (a):

One of the advantages, for what its worth, of a (where (f x) = (+ x x)) syntax is that it discourages the definition of unwieldy body-like expressions on the RHS of the =.

It is not that you cannot do

(where (f x) = (progn 
                            (id-like-to-do-this-with x)
                            (then-this-with x)
                            (and-at-last x))

but you have to really decide if you can live with yourself when choose to.

And danger (b):

I find inner defines in scheme to be a major turn off and I never use them in scheme code. I find code that uses them heavily to be difficult to read and is likely to be a bit uncertain of itself vis-a-vis design. When I see a function body including a dozen inner defines (as with a dozen labels bindings) I know the function will be hard (or at least obnoxious) to debug and to maintain. For these reasons, I would discourage the inclusion of nested defines into the language.

Codicil to danger (b): As an old person, I very much prefer for visual distinctiveness to clue me in to the environment of a binding - whether it is local or global or other.

[stylewarning]

I agree with much of the above. I also think that limiting syntax is a great way to promote readability. So for instance, (where ...) might be designed in such a way it can only show up at the bottom of a define, and nowhere else.

(As an aside, this isn't a definite feature I want to implement, it's more that I wanted to discuss it.)

[Jason94]

My take on this is that I like using Coalton syntactically as close to Common Lisp style as possible. For me, one of the reasons I like Coalton over something like Haskell or OCAML is the simpler LISP syntax that's easier to read. I also might use macros more heavily than most when writing coalton (citation needed), and I could see that as Coalton gets more features like this, writing macros could become harder and less intuitive overtime.

(The existing (let x = (foo ...)) syntax also fits into this category, so it might be fine.)