Add support for encrypting / decrypting the chunks + API Keys

[tchaton]

🚀 Feature

Motivation

Add support for encrypting & decrypting the chunks to enable users to consume the data outside anywhere safely.

The data would be decrypted in RAM.

Pitch

Alternatives

Additional context

[bhimrazy]

Hi @tchaton,

I'm interested in learning more about this feature and exploring how I might be able to contribute.
Thanks for your help!

[tchaton]

Hey @bhimrazy. Go for it.

[tchaton]

Hey @bhimrazy,

I think we could have something like this.

optimize(

	encryption=Entryption(password="...", level="sample|chunk")

And we could offer other encryption format such as RSA, etc..

Additionally, we could support the following:

StreamingDataset(
	...,
	api_key="...",
)

Where the api key enables to generate temporary credentials to fetch the chunks.

[bhimrazy]

Thank you, @tchaton, for providing the examples. They are extremely helpful.

For the symmetric encryption option, I am considering using Fernet (symmetric encryption).

What are your thoughts on using Fernet for this purpose or do you have any other library recommendations for symmetric encryption?

[bhimrazy]

Hi @tchaton,

Just checking if we need to move the 'API Keys' part from this issue to a new one, as #219 got closed for handling the encryption/decryption.