LENOVO_MT_30GS_BU_Think_FM_ThinkStation P3 Tower fails to update

[joakim-tjernlund]

The machine has FW 0.1.94 and fwupdmgr update wants to update to 0.1.108
This all work fine until rebooting, then BIOS does not do anything w.r.t update, it just does a normal boot.

Tried to update to 0.1.97, the first Production BIOS, too. No change there either.

Is this maybe connected to that the machines we just got has some BIOS pre release?

Tried fwupd 1.9.23 and 1.9.24

@mrhpearson , please advise

[mrhpearson]

I don't have one of these - but I can see 3K+ downloads of the 1.97 update and a bunch of success reports....so it should work.

On reboot is nothing displayed at all?

Have you checked the boot variables (F12 during early boot) to see if there is a Linux-Firmware-Updater and what happens if you manually choose to run that?

[joakim-tjernlund]

I don't have one of these - but I can see 3K+ downloads of the 1.97 update and a bunch of success reports....so it should work.

On reboot is nothing displayed at all?

Nothing.

Have you checked the boot variables (F12 during early boot) to see if there is a Linux-Firmware-Updater and what happens if you manually choose to run that?

Machine is in operation now, can't just play around with it.

What about the non production BIOS it was delivered with? Could it be that this old BIOS cannot update itself?

[joakim-tjernlund]

I see a difference in model number, we have a few 30GS001SMT since before and these update well.
This new one is 30GS00BKMT

[joakim-tjernlund]

One more potential diff, I see the path to the update is:
/boot/EFI/UpdateCapsule/fwupd-f8fe29c6-a9a8-4ff7-86e7-aa8502c858f1.cap

Don't recall seeing the UpdateCapsule before, was something more like:
/boot/EFI/gentoo/fw/fwupd-f8fe29c6-a9a8-4ff7-86e7-aa8502c858f1.cap

[joakim-tjernlund]

Yet an detail, after trying to update I can see that the fwupd-f8fe29c6-a9a8-4ff7-86e7-aa8502c858f1.cap file is gone
under /boot/EFI/UpdateCapsule so it seems BIOS tried but failed early on and delete the update.

[mrhpearson]

I've created internal ticket LO-3314 to ask for guidance from the FW team for next steps.

It looks like it's using Capsule on Disk for update - but that should be fine.

I don't think it will matter, but is it possible to boot a Ubuntu or Fedora live USB and try the update from there? Just because Gentoo is less used and won't have been tested by the FW team before releasing the FW. I know it shouldn't make a difference....but stranger things have happened.

[joakim-tjernlund]

Testing some other live USB distro is not easy since the machines are in operation already.

Also, I have almost the same model but 30GS001SMT, rather than 30GS00BKMT which runs the
same BIOS and that one I have updated successfully.

We employ PXE/image install here which is the same for all Gentoo machines so we run the same OS, including fwupd.

To me this looks like the machines were delivered with the wrong BIOS, non production version, and that
BIOS is buggy w.r.t FW update.

[mrhpearson]

Ack - checking with FW team. Worried about you having pre-production FW installed - that is very strange.

[joakim-tjernlund]

Any updates?
I find odd that the newer variants uses Capsule when the somewhat older does not. Both machines install the same BIOS FW.

[mrhpearson]

Some clarification from the FW team:

1.94 (5EA) is a production BIOS. It is for RPL-S refresh CPU production.
We have not released it on the web because it has compatibility issues on previous systems (13th RPL-S). These issues are resolved in the 61A BIOS.

They have had another report of an update issue, but so far have been unable to reproduce it on their system so are still debugging.
Note - it is golden week holiday in China this week, so I'm not expecting any updates until next week.

For Capsules (I assume you mean Capsule on Disk here) It might be worth trying to disable that - see https://fwupd.github.io/libfwupdplugin/fwupd.conf.html#uefi_capsule-parameters (set DisableCapsuleUpdateOnDisk=true in fwupd.conf).
If that fixes it I can highlight to the FW team that they have a CoD issue.

Mark

[joakim-tjernlund]

DisableCapsuleUpdateOnDisk=true worked!

[mrhpearson]

Awesome - thanks for confirming.
I'll let the FW team know they've got a CoD issue to look into.

[joakim-tjernlund]

Awesome - thanks for confirming. I'll let the FW team know they've got a CoD issue to look into.

Yeah, also obvious they did not test upgrade on Linux before releasing it.

[hughsie]

Is the bug "we don't want to advertise CoD in OsIndicationsSupported" or "CoD should work but does not"?

[joakim-tjernlund]

Is the bug "we don't want to advertise CoD in OsIndicationsSupported" or "CoD should work but does not"?

I would guess the first one. We have new and a bit older machines of the same make and uses the same FW
and the older one does not use CoD but the newer ones does.

[mrhpearson]

If BIOS is advertising CoD it should work. P3 is a new enough platform that I'd expect it to be fine - it works well on other platforms (I haven't seen a CoD issue in quite a while now). We'll have to wait for confirmation from the FW team next week though.

Yeah, also obvious they did not test upgrade on Linux before releasing it.

That's not true or fair. They would have tested before release - it's a requirement of the Lenovo FW release process. They just didn't hit this issue for some reason.
Side note - LVFS also requires a signed test report uploaded before the FW can be released from embargo. There are no shortcuts :)

Issues happen, and I understand the frustration. We'll have a better idea of the root cause once they've evaluated it.

[joakim-tjernlund]

Any news?
There are more of these machines on order so it would be nice with a fix before they get here.

[mrhpearson]

They are confirming that they are looking to disable CoD support in the BIOS. I don't have an ETA for when it will be released - but this process usually takes quite a few weeks.
Recommend going ahead with CoD disabled in fwupd.conf as a workaround until the fixed BIOS is available.

[mrhpearson]

ETA was given as end of December.