Proposal: Mapped names

[sbergen]

Problem

Currently, named subjects are restrictive in the sense that they can not be used in the following scenario using named processes (usually with supervision):

• Process 1 is running code that has a function subscribe(source: Source1, subject: Subject(Msg1))
• Process 2 is running code that has a function subscribe(source: Srouce2, subject: Subject(Msg2))
• Process 3 wants to subscribe to both with a named subject.

While with unnamed subjects, we can do

let subject1 = process.new_subject()
let subject2 = process.new_subject()

mod1.subscribe(source1, subject1)
mod2.subscribe(source2, subject2)

process.new_selector()
|> process.select_map(subject1, map_msg_1)
|> process.select_map(subject1, map_msg_2)

the same does not work with named subjects.

Proposal

Instead, I'm proposing a mechanism to create a set of names that have a similar data mapping built into them. Here is a test from my proof of concept branch:

pub fn mapped_name_test() {
  let #(name1, name2, compound_name) =
    process.new_mapped_names2("test", int.to_string, float.to_string)
  let sync = process.new_subject()

  process.spawn(fn() {
    assert process.register(process.self(), compound_name) == Ok(Nil)
    process.send(sync, False)

    let subject = process.named_subject(compound_name)

    // Expect the mapped messages
    assert process.receive(subject, 100) == Ok("1")
    assert process.receive(subject, 100) == Ok("2.0")
    assert process.receive(subject, 100) == Ok("3")
    process.send(sync, True)
  })

  // Wait for name to be registered
  assert process.receive(sync, 10) == Ok(False)

  // Send the messages
  process.send(process.named_subject(name1), 1)
  process.send(process.named_subject(name2), 2.0)
  process.send(process.named_subject(compound_name), "3")

  assert process.receive(sync, 100) == Ok(True)
}

Consequences

Names are no longer always atoms

This would inherently have the consequence of process names no longer being atoms, when created like this, but rather carry some data along with them. I could not think of a way to have only the "combining" name carry extra data. My current PoC takes the opposite approach and only adds data to the "sending" names, but has the downside that the mapping function (which might close over data) is sent alongside each message.

Another approach that could work instead, is tagging the messages that require mapping with an atom or integer, and store a map of mapping functions in the "combining" name, so that it would need to be moved to the receiving process only once. However, this would add more complexity to the implementation (and one map lookup per receive).

The third option is to do the mapping in the sending process, but I don't like moving the potential for panicking to the sender. However, it would make the solution more lightweight.

I considered some other options also, but didn't find anything I'd like more than these three.

Multiple new_mapped_namesX functions

Given Gleam doesn't have variadics, I couldn't think of a type safe way to do this, without having to write multiple arities of a function like this.

Questions

Did I miss some scenario where this would not work? Is doing this well too complex to justify the benefit? Does this look like a proposal worth pursuing further? If so, are the following principles something we should stick to, or are some of them negotiable?

1. The mapping function should be run on the receiving process.
2. We should not be sending functions alongside the messages, as they could cause copying of closed over data.

[lpil]

Your point about named subjects being less powerful than regular subjects is a good one! This is a problem. It'd be good to work out what pattern or new functionality could be used to resolve this.

This specific design seems unsound to me. It depends on receive being used with the "compound" name, but there's nothing in the type system to enforce that. If receive is used with one of the mapped names then the type will not match the actual runtime value.

[sbergen]

This specific design seems unsound to me. It depends on receive being used with the "compound" name, but there's nothing in the type system to enforce that. If receive is used with one of the mapped names then the type will not match the actual runtime value.

That's actually not true (but it's not very obvious), as in that case {Name, Map} does not match Name (as Name is implicitly pinned), but the Name pattern matches instead, leading to skipping the mapping operation. My implementation was a pretty low-effort PoC to explore the idea, so we should probably make things more explicit if this was to be properly implemented.

However, I wouldn't be surprised if I missed something else similar, and my thoughts were that since we already have panics for "Cannot receive with a subject owned by another process" and "Sending to unregistered name", we could follow the same pattern here, by adding checks where necessary. I tried to draw some inspiration from how tokio channels in Rust use separate senders and receivers for channels (with thread-count enforcement using Send, Sync, and linear types), but failed to apply any ideas from there with Gleam's type system. I don't have format proof, but given how any value can be copied between processes, it feels like it's impossible to make something that would be completely safe at runtime.

[lpil]

Ah, thank you! In the case they're subjects that appear to be broken as they can be sent to but not received from, even though the API and the type system says they can be received using. Adding a footgun where you always fail to select messages seems like a DX issue to me.

Stepping back a bit, why is it that the message sending code cannot be given a message constructing function, similar to how what gleam/erlang/process.call etc do?

[sbergen]

Ah, thank you! In the case they're subjects that appear to be broken as they can be sent to but not received from, even though the API and the type system says they can be received using. Adding a footgun where you always fail to select messages seems like a DX issue to me.

Sorry, I probably didn't communicate that very clearly: With the current design you can receive on any of the subjects, and it works as expected.

Stepping back a bit, why is it that the message sending code cannot be given a message constructing function, similar to how what gleam/erlang/process.call etc do?

The use cases I see for this would include passing the Name or Subject to some code that doesn't know how to do the mapping. For example, if I had two packages that are sending messages, and I'd want to receive them in one single process running code in a third package.

You are right that if all functions receiving the Subject also include a mapping function as an argument, then what I'm suggesting doesn't provide any new functionality. However, I see a few potential issues with this

1. Potential panics in the mapping now happen in the sending process. I'm not very experienced with the Erlang/OTP failure philosophy, so I might be off here, but it would feel more appropriate to me for the receiving process to panic. (This was one of my questions in the original post.)
2. It's easy to forget to include the mapping function in the API, as it's not strictly necessary when not using named subjects. This could be a skill issue on my side, but it also doesn't feel like the friendliest API design for gleam/erlang to push that on the developer.
3. It makes for more complicated APIs. However, I'm not sure if moving the complexity elsewhere helps overall.

That all does make me wonder if having something that is closer to a dual of Selector would be more appropriate, something like

let subject: Subject(String) = process.named_subject(some_name)
let sender: Sender(Int) = process.map_send(subject, int.to_string)

It would also IMO make it more obvious when something is intended only for sending, not receiving, as sometimes a Subject can be slightly ambiguous. I don't know how feasible it would be to run the mapping function in the receiving process with this design though, without trying to do a similar proof of concept first.