[IDEA] Pass startup parameters via URL hash

[saqimtiaz]

It is desirable to be able to pass startup parameters to TiddlyWiki as key value pairs on startup via the URL hash, such as index.html##readonly:yes#layout:mylayout

These parameters could be passed as a blob of JSON to action tiddlers tagged with a new system tag and would then be evaluated one after the other in tag order.

Potential use cases include:

• setting the wiki to read only: readonly:yes
• changing layout based on a parameter: layout:mylayout
• changing the content displayed based on a parameter studentid=9341
• customizing permalink/permaview handing by moving the logic into wikitext: open:HelloThere:HelloThere%20%5B%5BQuick%20Start%5D%5D

[saqimtiaz]

@Jermolene @CrossEye some thoughts on implementation and potential caveats follow. I believe there are still gaps to address in the proposed design and welcome discussion on this.

Implementation ideas:

• We will need to introduce a new system tag, for example $:/tags/StartupParameterActions, for now referred to as parameter handlers.
• we would likely use existing mechanisms (widget.invokeActionsByTag) to evaluate tiddlers thus tagged, therefore the tag order would also be execution order.
  • A UI in the control panel - similar to the one we have for cascades - would allow introspection by end users.
• Currently all URL hashes in which the hash starts with a # character itself - that is there is a double octothorp character ##HelloThere - are ignored. Thus this can be used as the trigger for parsing the hash as key value pairs seperated by #.
  • Example: index.html##readonly:yes#layout:mylayout#open:HelloThere:HelloThere%20%5B%5BQuick%20Start%5D%5D
• The default behaviour without the extra leading # character would be as present, that is to parse the hash as a permalink and display the relevant tiddlers. However, this processing and logic would be moved into wikitext allowing it be customizable, and would be equivalent to using ##open:<permalink-here>.
  • As such this could be an alternative to the change in permalink handling explored in Simplify permalinks #7990
  • This also aligns with the long term developmental goal of TiddlyWiki to move as much of the operational logic into wikitext as possible.
  • The actions for the open: handler would also need to be invoked at other times than startup based on the values of $:/config/Navigation/UpdateAddressBar and $:/config/Navigation/UpdateHistory

Open questions and other thoughts:

• Do we required that each parameter handler tiddler registers what parameter it handles and is only invoked when that parameter exists in the hash? A pattern that we already use is a condition field that specifies a filter that must return a result in order for the actions to be invoked. Or do we just pass every paramter handler all the params as JSON and let them implement their own conditional logic? The appeal of the latter is that a single tiddler can handle multiple parameters.
• It would be possible in this proposed implementation for there to be multiple handlers for the same key, is this likely to be a problem and does it need mitigation? We could potentially require that each parameter handler tiddler is titled with a given prefix, followed by the parameter name to avoid this: $:/core/parameter-handlers/open
• The usual design pattern we have in TiddlyWiki for actions invoked at a given time is to use tagged tiddlers. However, the more I mull this over the more I feel that in this case we need to register/track tiddlers not by a single tag but based on which hash parameter they handle, so that we can invoke the appropriate action tiddler when needed.
• passing more complex data as a key value pair would require URI encoding it in the hash and then decoding it in the corresponding actions.
• Sample parameter handler tiddler:

title: $:/paramhandlers/readonly
tags: $:/tags/StartupParameterActions
condition: (optional filter that needs to return a result in order for the actions to be executed)

<%if [<params>jsonget[readonly]match[yes]] %>
<$action-setfield $tiddler="$:/readOnly" text="yes"/>
<%endif%>

or alternatively:

title: $:/paramhandlers/readonly
tags: $:/tags/StartupParameterActions
params: readonly <!-- list of parameters that this tiddler can handle -->

<%if [<param>match[yes]] %>
<$action-setfield $tiddler="$:/readOnly" text="yes"/>
<%endif%>

[CrossEye]

Great stuff @saqimtiaz!

1. The biggest gap I see here is that this is all about URL decoding only. I would expect handlers to also need to encode parts of the URL. readonly:yes sounds simple enough, but we might also have foo:%7B%22bar%22%3A1%2C%22baz%22%3A%22yes%22%2C%22qux%22%3A%5B1%2C2%2C3%5D%7D (URL-equivalent to foo:{"bar":1,"baz":"yes","qux":[1,2,3]}.

   But that leads to the question of who is responsible for URL en/decoding of the values for each key? My first thought is that it would be the system that does so; the implementers of the handlers receive and return plain text; they would be responsible for JSON en/decoding of their payload, though, if that is required. But I haven't thought it through entirely.

   Another consideration is that we need to somehow pair the two sides of this interface/protocol. In JS, we'd simply say that your object needed to have these two methods; it would be more formalized in a strongly-typed language. What do we have in TW to say, "You need to supply both encode and decode methods with these behaviors"?

   we would likely use existing mechanisms (widget.invokeActionsByTag) to evaluate tiddlers thus tagged

   That doesn't leave much room for both encoding and decoding, does it? Would they have to be separately configured? That would worry me.

2. You ask

   Do we required that each parameter handler tiddler registers what parameter it handles and is only invoked when that parameter exists in the hash? [ ... ] Or do we just pass every paramter handler all the params as JSON and let them implement their own conditional logic?

   The latter would be my preference. For instance, I might write a handler that acts one way if there is a readonly parameter with value yes, and a different way if the value is not yes OR if the parameter is not supplied. We wouldn't be able to manage that scenario if we only call the handler when readonly is supplied.

   Moreover, we might want a handler to deal with multiple parameters. Perhaps we want to let a readonly handler and a layout hander do their jobs, but if we happen to have readonly: yes and layout: xyz, then we want some special handling.

3. Is there a fallback mechanism? Does a handler have to report if it successfully handled the request? Is there a way to claim a parameter, to say, "I handled readonly; don't pass it to others"? My first thought is no, but do we need to consider it?

4. Might we consider a second level, at least for ##open handlers? I would love it if the API let me write implementations of en/decodeTarget and en/decodeList and something generic was responsible for figuring out what to do with the story river based on those results, or for deriving the list sent to these functions based on the story river. So we might want pluggable implementation similar to my Simplified Permalinks, or a slug-based one, or who knows what else? (I think of this as similar to how we can use the View Template Body Cascade to choose our own handler, but if we choose the default handler, we can still use View Templates to configure how the default handler behaves.)

Probably more later...

But thank you for putting in this effort!

[saqimtiaz]

@CrossEye

4. Might we consider a second level, at least for ##open handlers? I would love it if the API let me write implementations of en/decodeTarget and en/decodeList and something generic was responsible for figuring out what to do with the story river based on those results, or for deriving the list sent to these functions based on the story river. So we might want pluggable implementation similar to my Simplified Permalinks, or a slug-based one, or who knows what else? (I think of this as similar to how we can use the View Template Body Cascade to choose our own handler, but if we choose the default handler, we can still use View Templates to configure how the default handler behaves.)

I think this makes perfect sense. I am thinking in terms of two different wikitext action procedures:

• one that uses a specified core global function (example tf.decode.permaview) to decode a permaview/permalink and opens the resultant list of tiddlers in the story.
• the other uses another specified global core function (tf.encode.permaview) to set the permaview/permalink based on what is in the story.

We will also need this for updating the hash in response to changes in the story.
Overriding these two global functions would allow the implementation of a different permalink format, the reserved names of the these two core functions and procedures is how we would pair them.

Re: 1. you are correct that my proposal above entirely concerns itself with URL decoding. This is in part due to the fact that we don't really have an established practice for pairing encoders and decoders (or functions/procedures). One could imagine another set of handlers that are encoders and making the permalink/permaview mechanism extensible so that it is possible to create a button that when clicked updates the hash to represent a specified combination of key value pairs according to the specified parameters. However, outside of permaview/permalink handling I am unsure how much demand there would be for such a facility so I am thinking of it as stretch goal or eventual improvement.

In terms of URL decoding (and encoding) of the values of each key, I agree that it would be convenient to have the system do that if we can come up with an elegant implementation.

2. You ask
> Do we required that each parameter handler tiddler registers what parameter it handles and is only invoked when that parameter exists in the hash? [ ... ] Or do we just pass every paramter handler all the params as JSON and let them implement their own conditional logic?

The latter would be my preference.  For instance, I might write a handler that acts one way if there is a `readonly` parameter with value `yes`, and a different way if the value is not `yes` **OR** if the parameter is not supplied.  We wouldn't be able to manage that scenario if we only call the handler when `readonly` is supplied.

I concur.

3. Is there a fallback mechanism? Does a handler have to report if it successfully handled the request? Is there a way to claim a parameter, to say, "I handled readonly; don't pass it to others"? My first thought is no, but do we need to consider it?

I could well imagine scenarios where it could be useful to allow a handler to say a parameter has been handled and needs no further processing by other handlers. Part of the challenge here is that we don't presently have a means for action strings to return values to the executing JavaScript, and neither do I see an elegant way of implementing the same in wikitext at present.

The first possibility that comes to mind is one where the handlers get nested within each other as transclusions for execution, requiring the presence of a slot with a specified name inside each action string, so that they can redefine the variables available to the next handler by wrapping a let widget around the transclusion slot. However, this kind of nested transclusion is likely to be brittle as it would rely on the users placing the slots in the right place in their actions.

[CrossEye]

I am thinking in terms of two different wikitext action procedures:

• one that uses a specified core global function (example tf.decode.permaview) to decode a permaview/permalink and opens the resultant list of tiddlers in the story.
• the other uses another specified global core function (tf.encode.permaview) to set the permaview/permalink based on what is in the story.

There's another wrinkle. I don't know how easy it is in TW for a function to return a data structure that is not a String or a list of Strings.

In JS, we would do something like:

decodePermaview() //=> {target: 'Bar', tiddlers: ['Foo', 'Bar', 'Baz Qux']}

I haven't seen anything similar to that in TW.

While we could insist that the first return element is distinguished, where the equivalent of the above would be

decodePermaview() //=> ['Bar', 'Foo', 'Bar', 'Baz Qux']

that, to my mind is very sloppy API design, and rather fragile.

The only other thing that I can come up with feels odd, but seems like it would work. Instead of returning our data structure, we store it in a temp tiddler, with something like

<$action-setfield $tiddler="$:/temp/tw/permaview" target="Bar", tiddlers="Foo Bar [[Baz Qux]]" />

The alternative is to double the API surface. I've suggested this in the thread above, with some variants of get/setTarget, get/setTiddlers.

None of these options seem great. Do you have any other ideas?

---

we don't really have an established practice for pairing encoders and decoders (or functions/procedures).

... and further, I at least, can't easily imagine a way to do this that feels idiomatic.

But you seem more sanguine about this than I feel, or that's how I read this:

However, outside of permaview/permalink handling I am unsure how much demand there would be for such a facility so I am thinking of it as stretch goal or eventual improvement.

Are you saying that this pairing of URL encoders and decoders is a stretch goal or that making a general pairing mechanism is one? If the former, then I strongly disagree. Requiring users to configure two different subsystems to handle encoding and decoding would be ugly and likely error-prone.

---

The first possibility that comes to mind is one where the handlers get nested within each other as transclusions for execution, requiring the presence of a slot with a specified name inside each action string, so that they can redefine the variables available to the next handler by wrapping a let widget around the transclusion slot.

That's going to take several re-reads, more than I have time for right now. I don't know much about the slot mechanism, so this is gobbledygook at the moment. I will try again tomorrow.

[saqimtiaz]

There's another wrinkle. I don't know how easy it is in TW for a function to return a data structure that is not a String or a list of Strings.

Hmm. The only options I can think of here add some complexity to the functions.

TW v5.4.0 will introduce multi-valued variables (lists). Perhaps we could insist that the first item is the target and the second is a title-list encoded string representing the tiddlers to open. It has been a while since I looked at the multi-valued variables so we need to make sure that a writing a function that returns the correct structure isn't too complicated. Similarly, we could also insist that the operators return JSON.

However, outside of permaview/permalink handling I am unsure how much demand there would be for such a facility so I am thinking of it as stretch goal or eventual improvement.

Are you saying that this pairing of URL encoders and decoders is a stretch goal or that making a general pairing mechanism is one? If the former, then I strongly disagree. Requiring users to configure two different subsystems to handle encoding and decoding would be ugly and likely error-prone.

I have been thinking of URL encoders outside of the one for the open key to be a stretch goal. It would be helpful here if we could think about use cases or user stories where they would be used, and where the pairing would be critical. Can you think of any?

For the open key, the core needs to know how to encode and decode the key value. However apart from this I can only think of scenarios like a plugin that introduces a layout key and provides a button that when clicked copies to the keyboard the URL to use to open the wiki in a specified layout. How would you want to enforce the pairing here and what value would it impart?

The first possibility that comes to mind is one where the handlers get nested within each other as transclusions for execution, requiring the presence of a slot with a specified name inside each action string, so that they can redefine the variables available to the next handler by wrapping a let widget around the transclusion slot.

That's going to take several re-reads, more than I have time for right now. I don't know much about the slot mechanism, so this is gobbledygook at the moment. I will try again tomorrow.

I honestly would not recommend spending time on it as its incredibly brittle technique and complex to implement, however if you are curious here is an example though it might take perusing the entire thread to understand it.

[CrossEye]

It has been a while since I looked at the multi-valued variables so we need to make sure that a writing a function that returns the correct structure isn't too complicated.

Yes, that is a real possibility we can investigate.

Similarly, we could also insist that the operators return JSON.

My experience with JSON in TW has been meh. But I suppose this would work.

I have been thinking of URL encoders outside of the one for the open key to be a stretch goal. It would be helpful here if we could think about use cases or user stories where they would be used, and where the pairing would be critical. Can you think of any?

Interesting. I feel that it's essential that encoders and decoders are (or at least can be) paired. You've mentioned readonly and layout. I feel that both of these have to do with important state of the wiki. If I want my wiki to track readonly, then when I click a permalink/permaview button, the generated URL should include that current readonly state. If I want it to track layout, then when I click one, it should include the current layout.

The alternative would seem quite weird to me: If I arrive at https://my-wiki.example.com/##readonly:yes&open:Bar:Foo;Bar;Baz+Qux, then open and close some tiddlers and hit the permaview button, I would be very surprised if the URL it generates is for a read-write wiki. I would expect something like https://my-wiki.example.com/##readonly:yes&open:Corge;Foo;Grault;Bar;Corge;Waldo. And similarly with layout. If I arrive with a url-specified layout, it would be quite disconcerting if the permaview result opens in a different layout... unless I've changed layout.

I honestly would not recommend spending time on it as its incredibly brittle technique and complex to implement, however if you are curious #7340 (comment) is an example though it might take perusing the entire thread to understand it.

I'll read it when I have a little time. Even if it turns out not to be relevant, I do like to stretch my wings.

[CrossEye]

Additional thoughts:

• My "Simplify Permalinks" suggestions had one really useful feature. They are almost entirely backward compatible. Except in one obscure case, if we were to add this idea to tiddlywiki.com, the permalinks/views generated by the current code would continue to work correctly, even though newly generated permas would use the new style.

  I'm not sure we could insist, but it would be nice to at least strongly suggest that other ##open handlers try to do the same. If nothing else, this would make for smoother updates.

• I don't know if you were thinking this, but I've just realized that while we should handle ##prop1:value1&...&open:Bar:Foo;Bar;Baz;+Qux and #Bar:Foo;Bar;Baz+Qux ¹, it would probably be useful to also handle #Bar:Foo;Bar;Baz;+Qux#prop1:value1&... (Note the additional # in the middle.) That latter might be the preferred output format, as it feels like any other fragment identifier on the web, only with a weird #... tacked on. It simply feels cleaner.

Footnotes

1. I'm probably just going to use my simplified permalinks in examples in these discussions, as they're easier to read, but the same is true for #Bar%3AFoo%20Bar%20%5B%5BBaz%20%2B%20Qux%5D%5D ↩