You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
microcode-20200609 Release, at least 06-4e-03, hangs user's system #31
Per reports from https://bugs.launchpad.net/ubuntu/+source/intel-microcode/+bug/1882890, sig 0x000406e3, pf_mask 0xc0, 2020-04-27, rev 0x00dc, size 104448 hangs user's system in a similar way as #24 does but to different cpu. It's also reported the same file of a previous revision microcode: sig=0x406e3, pf=0x80, revision=0xd6 works just fine.
The differences between 0xd6 and 0xdc were quite minimal, to work around an SGX issue.
Does disabling SGX in the BIOS work around the hang using 0xdc?
Disabling SGX does not seems to help: SGX was initially on "Software controlled", so changed it to "Disabled", but laptop still hangs when booting the new initrd image.
On Wed, Jun 10, 2020 at 06:51:03AM -0700, Paulo J. S. Silva wrote:
Same thing on my system with a similar CPU. I had to uninstall the intel-microcode package to boot.
On Wed, Jun 10, 2020 at 07:55:50AM -0700, Paulo J. S. Silva wrote:
I used the Ubuntu package intel-microcode (3.20200609.0ubuntu0.20.04.0). I don't have it in my system anymore to check out. From the description of the package in https://launchpad.net/ubuntu/+source/intel-microcode/3.20200609.0ubuntu0.20.04.0 I guess 0x00d6. Do you want me to reinstall it and check?
3.20200609.0ubuntu0.20.04.0 has revision 0xdc of 06-4e-03 microcode file,
3.20191115.1ubuntu3 has 0xd6.
So it is 0xdc (I found the reference in the middle of the description). As I said I can test what ever you want. Just point me to the file and instructions on how to apply it without an Ubuntu package. Thanks!
Did you also update the Linux kernel? It would be nice to know if the regression is related to the microcode update itself, or to the combination of the two...
It looks from my /var/log/apt/history.log that it also update the linux-image package to 5.4.0-37. This is the kernel I am using right now, it did not need to uninstall it. I only uninstalled intel-microcode and its dependencies (meta packages that install the latest kernel and friends). Both were unattended upgrades (because they were marked as a security update).
Would it trouble you too much to test the intel-microcode package with an older kernel? With an old kernel, the microcode enables SRDBS mitigation in transparent mode, it is supposed to "just work" even if it might end up being enabled in a configuration that does not need it...
This ensures the issue is not being caused by some bug in the new kernel code.
If you do test with an older kernel, please look in /proc/cpuinfo to ensure you are indeed running microcode revision 0xdc, i.e. the new one for your processor.
You mean try to use a mainline kernel? What version? I can use mainline PPA to do it easily. How do I install the Intel microcode without a deb package. Just point me to the docs, I can manage myself.
On Wed, Jun 10, 2020 at 08:04:22AM -0700, Paulo J. S. Silva wrote:
So it is 0xdc (I found the reference in the middle of the description). As I said I can test what ever you want. Just point me to the file and instructions on how to apply it without an Ubuntu package.
Problem Statement:
Intel has identified an issue when performing OS patch loading of MCU version 0xE2 on SKL R0 (506e3) and SKL D0 (406e3) systems when the existing Microcode Update (MCU) version is earlier than 0x80
Issue description:
When an OS loads the latest MCU patches on SKL R0 (506e3) and SKL D0 (406e3), may lead to unexpected failures in the following conditions:
• The system has a BIOS containing MCU version earlier than 0x80 (MCU < 0x80)
• The user tries to load via OS load mechanism a new MCU version 0xD8 or greater
Workaround:
Update affected systems to a BIOS containing MCU version 0x80 or greater.
@whpenner: thanks, however we do need some extra information to act upon:
Will Intel somehow be able to work around this issue in a future public update of this microcode ? If so, do you have any idea of when it will be out?
If the answer to (1) above is "no, it is not possible", will Intel handle the required patches for the Linux microcode update driver to blacklist the unsupported updates, or do we need to do it ourselves? Because, well, a lot of vendors are not going to issue updates and a lot of users will not apply them anyway. So we need to avoid the lockups and we need to tell the user they need a firmware update if they require the fixes in the new microcode.
If the answer to (1) above is "yes, now that Intel know where the problem is, there is a way to avoid it and Intel will update the microcode to avoid the issue", then we (distros) can remove the problematic update meanwhile, without the need for kernel patches, etc...
If a bios update is not available from the motherboard manufacturer, I would highly recommend looking into UBU and patch the microcode into a bios update directly... Good luck expecting support from motherboard manufacturers... Last bios update for my server board was 4 years ago at this point (Thanks SuperMicro)
@naitoshedo thanks for pointing me in that direction. Now everything's fine:
$ grep 'stepping\|model\|microcode' /proc/cpuinfo
model : 78
model name : Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
stepping : 3
microcode : 0xe2
model : 78
model name : Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
stepping : 3
microcode : 0xe2
model : 78
model name : Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
stepping : 3
microcode : 0xe2
model : 78
model name : Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
stepping : 3
microcode : 0xe2
The above should be resolved with MCU revision ids of 0xe2 or higher for CPUID 406e3 (06-4e-03) and 506e3 (06-5e-03). MCUs released yesterday included revision id 0xea for these products.
Like in #31 (comment) the latest release also fails in the same way (no change to the bios): Linux debian 5.10.0-7-amd64 #1 SMP Debian 5.10.40-1
intel-microcode_3.20210608.1_amd64.deb
according to release notes that is version=0xea
@JanZerebecki what microcode revision is reported when the kernel is loaded with dis_ucode_ldr parameter? Does early update to 0xd6 plus late update to 0xea work?
@jirireischig may I ask to check if 0xe2/0xea microcode revision helps you (in case you still have the problematic (<0x80) microcode revision applied by firmware/BIOS), by chance?
@whpenner, could you confirm if the latest revision (0xea) is supposed to work around the problem stated in #31 (comment) ?
It would be really nice to be able to ship the microcode update without worrying about severely outdated BIOSes that cannot be fixed due to the usual motherboard-vendor-won't-issue-updated-firmware + EOL issues.
Yes, this should resolve the issue called out in #31. This MCU was updated to work with either the older MCU (older than revision ID 0x80) and the newer MCUs. I would not recommend an early load of an affected MCU and then load of the latest MCU. The results may be unpredictable. Updating from a BIOS loaded MCU to the latest MCU directly is the recommended method.
Reacted by Henrique de Moraes Holschuh and Florent Dutheil
Thank you. Most Linux distros do exactly that: update directly from BIOS to latest MCU (via an early update), that's why we request a reboot to apply the microcode updates.
It is important data for anyone doing late loads, though. Thank you for this information
Activity
andyhhp commentedon Jun 10, 2020
The differences between
0xd6and0xdcwere quite minimal, to work around an SGX issue.Does disabling SGX in the BIOS work around the hang using
0xdc?eguaj commentedon Jun 10, 2020
As requested by @vicamo, here is the output from
lscpu:It's on a Lenovo T460 laptop.
eguaj commentedon Jun 10, 2020
Disabling SGX does not seems to help: SGX was initially on "Software controlled", so changed it to "Disabled", but laptop still hangs when booting the new initrd image.
andyhhp commentedon Jun 10, 2020
Thanks for testing. I think this needs Intel to investigate now
pjssilva commentedon Jun 10, 2020
Same thing on my system with a similar CPU. I had to uninstall the intel-microcode package to boot. here is the output of
lscpu:esyr-rh commentedon Jun 10, 2020
pjssilva commentedon Jun 10, 2020
I used the Ubuntu package intel-microcode (3.20200609.0ubuntu0.20.04.0). I don't have it in my system anymore to check out. From the description of the package in https://launchpad.net/ubuntu/+source/intel-microcode/3.20200609.0ubuntu0.20.04.0 I guess 0x00d6. Do you want me to reinstall it and check?
esyr-rh commentedon Jun 10, 2020
pjssilva commentedon Jun 10, 2020
So it is 0xdc (I found the reference in the middle of the description). As I said I can test what ever you want. Just point me to the file and instructions on how to apply it without an Ubuntu package. Thanks!
hmh commentedon Jun 10, 2020
Did you also update the Linux kernel? It would be nice to know if the regression is related to the microcode update itself, or to the combination of the two...
pjssilva commentedon Jun 10, 2020
It looks from my /var/log/apt/history.log that it also update the linux-image package to 5.4.0-37. This is the kernel I am using right now, it did not need to uninstall it. I only uninstalled intel-microcode and its dependencies (meta packages that install the latest kernel and friends). Both were unattended upgrades (because they were marked as a security update).
hmh commentedon Jun 10, 2020
Would it trouble you too much to test the intel-microcode package with an older kernel? With an old kernel, the microcode enables SRDBS mitigation in transparent mode, it is supposed to "just work" even if it might end up being enabled in a configuration that does not need it...
This ensures the issue is not being caused by some bug in the new kernel code.
hmh commentedon Jun 10, 2020
If you do test with an older kernel, please look in /proc/cpuinfo to ensure you are indeed running microcode revision 0xdc, i.e. the new one for your processor.
pjssilva commentedon Jun 10, 2020
You mean try to use a mainline kernel? What version? I can use mainline PPA to do it easily. How do I install the Intel microcode without a deb package. Just point me to the docs, I can manage myself.
esyr-rh commentedon Jun 10, 2020
76 remaining items
hmh commentedon Jan 18, 2021
@whpenner: thanks, however we do need some extra information to act upon:
If the answer to (1) above is "yes, now that Intel know where the problem is, there is a way to avoid it and Intel will update the microcode to avoid the issue", then we (distros) can remove the problematic update meanwhile, without the need for kernel patches, etc...
dbischof90 commentedon Jan 19, 2021
So a recommended fix would be to update BIOS if possible and then attempt to use the new microcode package?
hmh commentedon Jan 19, 2021
@dbischof90: yes.
naitoshedo commentedon Jan 21, 2021
If a bios update is not available from the motherboard manufacturer, I would highly recommend looking into UBU and patch the microcode into a bios update directly... Good luck expecting support from motherboard manufacturers... Last bios update for my server board was 4 years ago at this point (Thanks SuperMicro)
ZerBea commentedon Jan 21, 2021
@naitoshedo thanks for pointing me in that direction. Now everything's fine:
esyr-rh commentedon Jun 8, 2021
New revision 0xea of 06-4e-03/06-5e-03 microcode files has been published as part of microcode-20210608 release, it may be worth trying out.
Shados commentedon Jun 9, 2021
The
0xearevision in microcode-20210608 boots OK on mysig=0x406e3device (an Asus UX305CA with an m3-6Y30).whpenner commentedon Jun 9, 2021
The above should be resolved with MCU revision ids of 0xe2 or higher for CPUID 406e3 (06-4e-03) and 506e3 (06-5e-03). MCUs released yesterday included revision id 0xea for these products.
fdutheil commentedon Jun 15, 2021
As already stated, the issue was fixed on a previous MCU version for my hardware (see previous comments), this
0xeaversion boots fine too here:JanZerebecki commentedon Jun 15, 2021
Like in #31 (comment) the latest release also fails in the same way (no change to the bios):
Linux debian 5.10.0-7-amd64 #1 SMP Debian 5.10.40-1intel-microcode_3.20210608.1_amd64.deb
according to release notes that is version=0xea
esyr-rh commentedon Jun 15, 2021
@JanZerebecki what microcode revision is reported when the kernel is loaded with
dis_ucode_ldrparameter? Does early update to0xd6plus late update to0xeawork?esyr-rh commentedon Jun 17, 2021
@jirireischig may I ask to check if 0xe2/0xea microcode revision helps you (in case you still have the problematic (<0x80) microcode revision applied by firmware/BIOS), by chance?
hmh commentedon Jun 23, 2021
@whpenner, could you confirm if the latest revision (0xea) is supposed to work around the problem stated in #31 (comment) ?
It would be really nice to be able to ship the microcode update without worrying about severely outdated BIOSes that cannot be fixed due to the usual motherboard-vendor-won't-issue-updated-firmware + EOL issues.
whpenner commentedon Jul 8, 2021
Yes, this should resolve the issue called out in #31. This MCU was updated to work with either the older MCU (older than revision ID 0x80) and the newer MCUs. I would not recommend an early load of an affected MCU and then load of the latest MCU. The results may be unpredictable. Updating from a BIOS loaded MCU to the latest MCU directly is the recommended method.
hmh commentedon Jul 8, 2021
Thank you. Most Linux distros do exactly that: update directly from BIOS to latest MCU (via an early update), that's why we request a reboot to apply the microcode updates.
It is important data for anyone doing late loads, though. Thank you for this information