✅ Deploy Preview for tiddlywiki-previews ready!

Name	Link
🔨 Latest commit	f41ed1c
🔍 Latest deploy log	https://app.netlify.com/projects/tiddlywiki-previews/deploys/6973396ff82f21000835e68f
😎 Deploy Preview	https://deploy-preview-9595--tiddlywiki-previews.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Confirmed: Jermolene has already signed the Contributor License Agreement (see contributing.md)

📊 Build Size Comparison: empty.html

Diff: ⬆️ Increase: +2.3 KB

✅ Change Note Status

All change notes are properly formatted and validated!

📝 $:/changenotes/5.4.0/#9595

Type: enhancement | Category: filters
Release: 5.4.0

Easier avoidance of deduplication in filters

🔗 #9595

👥 Contributors: Jermolene

It would be nice to have a shortcut to toggle the same behaviour at the level of a filter expression.
From an earlier brainstorm: filter pragmas
::dedupe:no [tag[Welcome]] :map[...]

It would be nice to have a shortcut to toggle the same behaviour at the level of a filter expression. From an earlier brainstorm: filter pragmas ::dedupe:no [tag[Welcome]] :map[...]

Yes, I think something like that would be very useful. Would a more general syntax such as that would obviate the need for this extension to the "subfilter" operator? Presumably the effects of the pragma would be inherited by a subsequent "subfilter" operator. I think that would work for #8702. I will investigate further.

I note in passing that there is a precedent for using a suffix to control deduplication this approach in the "enlist" operator.

Yes, I think something like that would be very useful. Would a more general syntax such as that would obviate the need for this extension to the "subfilter" operator?

I believe so and would in many cases be easier than needing to declare the filter expression as a variable and then go through the subfilter operator.

I note in passing that there is a precedent for using a suffix to control deduplication this approach in the "enlist" operator.

Agreed and I would not have any objections to also extending the subfilter operator in this manner, though we may consider doing the same for the filter filter operator. If we go down this route, we should consider though if this should be a suffix, or a second operand to the operator. A second operand would allow the deduplication to be controlled based on the value of an indirect operand.

Agreed and I would not have any objections to also extending the subfilter operator in this manner, though we may consider doing the same for the filter filter operator.

It sounds like we're happy to merge this PR and then later develop the pragma approach. On that basis I'd like to go ahead with this PR to enable us to defer the policy and architecture work needed to get that right.

If we go down this route, we should consider though if this should be a suffix, or a second operand to the operator. A second operand would allow the deduplication to be controlled based on the value of an indirect operand.

In a way that boat has sailed because we have so many existing operators with suffixes. Perhaps we need a genesis operator that allows any operator to be invoked, with the suffixes passed as parameters.

I believe so and would in many cases be easier than needing to declare the filter expression as a variable and then go through the subfilter operator.

Those usecases would indeed be better served by the proposed pragma syntax. The case for this PR is that there are plenty of situations where the subfilter/filter is already in a variable (or accessible by transclusion) and so the pragma syntax would be more unwieldy.

Another point to consider is that the default filter run prefix is not inherited by nested subfilter/filter operators. I suspect that fixing that would be equivalent to implementing the pragma approach.

It sounds like we're happy to merge this PR and then later develop the pragma approach. On that basis I'd like to go ahead with this PR to enable us to defer the policy and architecture work needed to get that right.

Agreed.

I note in passing that there is a precedent for using a suffix to control deduplication this approach in the "enlist" operator.

The enlist operator uses the suffix raw to disable deduplication. all is probably more semantic. I wonder if we should allow all in enlist as well as being equivalent to the existing raw suffix in the interest of consistency.

Another point to consider is that the default filter run prefix is not inherited by nested subfilter/filter operators. I suspect that fixing that would be equivalent to implementing the pragma approach.

An important distinction is that a pragma would apply to all filter runs in a filter expression, whereas filter run prefixes only apply to invididual filter runs (and potentially in the future their nested subfilter/filter operators).

Widgets that accept filter expressions for $names and $values type of attributes would benefit from deduplication being disabled when their filters are evaluated.

For example:

Relevant widgets:

• genesis
• setmultiplevariables
• action-sendmessage
• action-setmultiplefields

Widgets that accept filter expressions for $names and $values type of attributes would benefit from deduplication being disabled when their filters are evaluated.

Interesting. I think it would be backwards compatible apart from situation where deduplication hides another bug. There might be a small usability issue in having to remember that filters behave subtly differently in different situations.

I've committed the update in 5f03d1e but I am worried that it is a change that does not require users to explicitly opt-in to the alternate behaviour. It moves this PR a little closer to our abandoned idea of changing the default default filter run prefix.

I've committed the update in 5f03d1e but I am worried that it is a change that does not require users to explicitly opt-in to the alternate behaviour. It moves this PR a little closer to our abandoned idea of changing the default default filter run prefix.

I think it helps that these are advanced widgets for which one always has to use deduplication. As long as we update the documentation for the widgets to clarify that for v5.4.0 these widget attributes are not deduplicated, I don't expect this to be problematic. The deduplication in these widget attributes is a common pitfall and came up yet again just last week.