Excel unnecessarily caches custom function metadata JSON, not respecting cache-control header sent by the server

[wh1t3cAt1k]

I feel like we need to ask it here because we really look bad in front of our customers every time we update the add-in code to a new version.

Our add-in is distributed through AppSource.

We would like to get to the bottom of how exactly add-in assets are:

• Cached by the host - do they respect the Cache-Control headers sent by our server? If not, why not?.. Are there any additional pitfalls?
• How the update process works when a new version is available in the store?

I was not able to find this in open documentation, but this now seems a crucial piece of knowledge given the severity of the problems we're seeing.

The flow we have been using over the last couple of years was like this:

• If just the JS / custom function metadata JSON were changed, we deliver the update directly to our production server where the add-in is served from.
• If manifest was changed in any way, we resubmit the manifest through the partner centre.

This flow, historically, worked out pretty well for us so that we didn't need to go through lengthy re-review process every time we delivered a hotfix to our code.

Frankly, it makes sense to expect it to work because our server sends "must-revalidate" headers with all JS/HTML/JSON assets and we assumed Excel would respect them.

Lately, however, (in the past few months) we have started experiencing crazy issues every time we pushed out a new version of add-in to production without resubmitting the manifest.

• Infinite update loops in Excel Online, where one would see a prompt like "A newer version of the add-in is available" - one would click "Update" and nothing would happen (the add-in then becomes completely unusable because of repeated prompts popping up, unless removed and reinstalled from the office store)
• "Integrity check" errors - see Is this an error on the Office.js side? #2773 , I am quite certain it comes from Office.js or some other infrastructure layer (could it be the code responsible for asset caching on Excel for Mac? Could you please ping the relevant guys from there?)
• Even crashes, sometimes: 🔥🐛 Crash on the attached file in Excel for Windows Desktop #2776

I have a gut feeling that all those are linked to the fact that the flow we're using has some hidden pitfalls and we would like to understand what they are, and why.

We definitely would like to avoid the process of having to maintain several parallel versions of assets and go through store resubmission on every small change we make, but if this is unavoidable, I want to at least confirm this.

Velixo internal tracking item

[JHJ-MS]

Hi @wh1t3cAt1k ,

1. The following steps you mentioned to update a Custom Functions Addin are exactly what we suggested.

If just the JS / custom function metadata JSON were changed, we deliver the update directly to our production server where the add-in is served from.
If manifest was changed in any way, we resubmit the manifest through the partner centre.

2. Regarding to these two questions:

Cached by the host - do they respect the Cache-Control headers sent by our server? If not, why not?.. Are there any additional pitfalls?

Yes, Excel would respect them.

How the update process works when a new version is available in the store?

It will fetch the latest JSON metadata file and Js file according to the latest manifest.

3. Regarding this issue:

Lately, however, (in the past few months) we have started experiencing crazy issues every time we pushed out a new version of add-in to production without resubmitting the manifest.

Infinite update loops in Excel Online, where one would see a prompt like "A newer version of the add-in is available" - one would click "Update" and nothing would happen (the add-in then becomes completely unusable because of repeated prompts popping up, unless removed and reinstalled from the office store)

Only the manifest updated will cause the "A newer version of the add-in is available" prompt to pop up, the other JS/JSON file change will not cause it. Guess it may be related to your early manifest change. It may be a bug and we are connecting our partner team to take a look at this issue.

Thanks.

[wh1t3cAt1k]

@JHJ-MS thanks for the clarifications. This just in, however, could you guys check just in case?:

We have updated the bundle (and the metadata JSON file) because MS provides a new TEXTSPLIT function, and we had another one with the same name, but different signature.

So, we renamed the function to TEXTSPLIT -> VXTEXTSPLIT and updated the JSON + JS bundle without resubmitting the manifest.

The report states that they opened Excel for Mac, where the store version of the add-in was installed, but the change did not come through. Clearing the cache through the personality menu did not help either:

As per your message above, clearing the cache and reopening of Excel should not be required, especially since after cache clearing the users lose their add-in settings stored in the local storage :(

We do send all proper cache-control headers for our custom functions metadata JSON and the JS bundle, asking the client to always revalidate.

Maybe it's a Mac-specific issue? But I'd like to get to the bottom of it as it seems linked to other weird post-update problems for us...

[JHJ-MS]

Hi @wh1t3cAt1k,
1.regarding your issue On Excel Online

Infinite update loops in Excel Online, where one would see a prompt like "A newer version of the add-in is available" - one would click "Update" and nothing would happen (the add-in then becomes completely unusable because of repeated prompts popping up

If you can still repro this issue could you please provide a Fiddler trace for us?

2.Regarding the issue you mentioned in your comment, Could I confirm with you whether your addin is using Shared Runtime?
If your add-in is using Shared Runtime, would you please first go through the comments of this issue to see whether it is helpful?

[wynntee]

@JHJ-MS , in my experience Excel does NOT respect cache-control for the JSON metadata file. It respects for the other files (HTML and JS) but definitely not JSON.

In fact, I raised this suggestion at Microsoft 365 Developer Platform and some users agree. It needs more attention from MS.

[wh1t3cAt1k]

@JHJ-MS correct, we are using the shared runtime.

However the issue you linked indeed makes an impression that Excel does not respect cache control headers for metadata, as @wynntee mentioned in his last message.

Moreover, from what I understand, the client in the linked issue was eventually forced down the manifest resubmission route on every update. This is very inconvenient to say the least as it will require people

a) to version and store on the servers multiple versions of metadata JSON and potentially other assets
b) to resubmit the manifest on every minor change.

I am sure you will agree with me: what's even the point of caching the metadata JSON file? 😊 Compared to the HTML/JS bundle itself (which does respect cache control headers!), the metadata JSON is a very small asset, with a must-revalidate header it takes a trivial time to check and redownload if needed!

Since above we established that it should not be the case:

and since the change on Excel side is probably easy - I suggest to treat this as a bug and fix because it's severely detrimental to our user experience... I.e. once we deliver an update, we would like people to see the result right away after relaunching Excel, and not after some non-deterministic amount of time, in which period people will see inconsistent results and custom functions.

Right now we have to ask them to uninstall and reinstall the add-in from the store, which means they lose all settings from the add-in local storage - and we're getting unwarranted support request volume, too :(

Please help us out with this guys!

[JHJ-MS]

Hi @madhavagrawal17 , from your comment in this link and @wh1t3cAt1k and @wynntee 's comment above, it seems that When using Shared Runtime, Excel does NOT respect "must-revalidate" headers for the JSON metadata file. Could you provide more info on why we do NOT respect cache-control for the JSON metadata file like other files (HTML and JS)? Is it a bug?

Hi @wh1t3cAt1k ,for the repeated prompts popping up the issue on WAC, if you can still repro this issue could you please provide a Fiddler trace for us? Or would you please provide your add-in name so that we can repro it on our machine?