Consider to switch crypto to Web API

[DoumanAsh]

Hello, I'ld like to propose idea for changing crypto implementation used by tiddlywiki
At the current moment sjcl is used but it s likely to be no longer maintained and supported
And while it provides adequate implementation (probably?) it is never going to be performing great on big wikis (small size doesn't really matter and likely to perform better due to overhead of WebCrypto API calls)

While using standard API is great, there are two major issues with it:

• Async - Obviously it cannot be drop in replacement and code would need refactoring
• 'Secure' HTTPS requirement - In their infinite wisdom people responsible for WebCrypto API believed that you MUST not use this API when page is served over plain HTTP. Therefore this will break encryption for people who serves their wiki without TLS (you can still open wiki as file and use crypto API)

On other hand there are some advantages:

• Being async current UI could be changed to make decrypt/encrypt operation more than just freeze of screen
• Depending on browser WebCrypto API would provide significant performance boost on large inputs
• It is going to be maintained for significant time (I suspect a lot of crypto libraries written in pure JS would get less and less attention over time)

I prepared minimal wrapper to provide more usable API as first step (it is using AES_GCM-256 therefore not compatible with current encryption)
master...DoumanAsh:TiddlyWiki5:improve_crypto

But I'd like to hear opinion of everyone if it is worth pursing in general
If there is no interest for crypto implementation to be improved I would consider just keeping local version for my personal use instead.

P.s.
API in question: https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto
Node.js provides compatibility layer too

[Jermolene]

Thank you @DoumanAsh. An upgrade path from sjcl.js to the WebCrypto API would be useful.

Would it be possible to use WebCrypto to decrypt wikis that were encrypted with sjcl.js?

I do have concerns about whether we could maintain backwards compatibility with an asynchronous API. For example, the <$encrypt> widget is used by the default saving templates to encrypt the store area, and yet widget rendering is constrained to be synchronous. If we only had an async crypto API then I think we'd have to redesign the way that encryption is implemented in TiddlyWiki.

[DoumanAsh]

Would it be possible to use WebCrypto to decrypt wikis that were encrypted with sjcl.js?

According to developer.mozilla.org WebCrypto API supports only follows AES algorithms: CTR, CBC and GCM
Unfortunately SJCL uses CCM as default option in convenience API
https://github.com/bitwiseshiftleft/sjcl/blob/master/core/convenience.js#L14

So even if we could adjust output format, it is impossible to support CCM with WebCrypto API and unlikely to be even supported as it is not friendly to streaming operations (which is likely the reason for not including it)
Normally it is recommended to use GCM instead of CCM, but they are not compatible.

I do have concerns about whether we could maintain backwards compatibility with an asynchronous API

Yes, as I mentioned async API would require re-work of internals due to current state of affairs relying on sjcl.js synchronous behavior
So introducing crypto API would require not only breaking change but also re-design of code.

Both are pretty big issues I believe hence I raised this topic.
It would be possible to make this change backward compatible by retaining sjcl.js inside code base as fallback to decrypt old wikis and/or provide synchronous API, but this kinda defeats one of the benefits of reducing code size (by removing sjcl.js)

But async API cannot be worked around unfortunately and requires significant changes as you mentioned

[Jermolene]

Thank you @DoumanAsh. Over at #7329 I've done some work on running wasm code within TiddlyWiki; I wonder if an alternative might be to adopt a modern, standard C library for crypto, and compile it to wasm. Presumably we'd have the same problem with CCM, but we could offer a plugin containing SJCL that could be used by people importing an old wiki into a new one.

[DoumanAsh]

Yes I also consider that, albeit WASM loading requires some async work at startup, but I guess it could be somewhat worked around
I'll need to take a look at how sqlite loads WASM module.

If it is WASM there is more options to enable CCM so I think it is more likely to be possible to retain compatibility without need of separate SJCL plugin.
Although it would have much more complex code as using WASM would require some shim code.

[btheado]

FYI, TiddlyPWA encrypts all user tiddlers and uses a highly size optimized wasm version of argon for its encryption. Maybe it can spark further ideas.

From https://val.packett.cool/blog/tiddlypwa/:

This whole argon2ian thing you might’ve seen was indeed created for TiddlyPWA! I’ve started out using PBKDF2 because it’s available in the Web Crypto API, but eventually decided to upgrade to a modern password hash rather than cranking out iteration counts. I wasn’t satisfied with the state of existing Argon2 WASM+JS wrappers, so I set out to make my own, code-golfed to the tiniest size possible. The yak shaving stack got even deeper during that subproject. Also, I have used the very new Compression Streams API to be able to bundle the Wasm binary as compressed while not having to bunlde a decompressor

Here is the tiddler for loading the wasm argon: https://codeberg.org/valpackett/tiddlypwa/src/branch/trunk/plugins/tiddlypwa/argon2ian.js and here is the code which uses it: https://codeberg.org/valpackett/tiddlypwa/src/branch/trunk/plugins/tiddlypwa/main.js#L681

[Jermolene]

Thank you @btheado, I had forgotten but that is very encouraging