Dasharo
XMP #1 profile does not boot in combination with some DDR5 configurations
Device
MSI Z790-P DDR5, MSI Z690-A DDR5
Dasharo version
v1.1.3 / v0.9.1
Affected component(s) or functionality
Memory configuration menu
Brief summary
XMP #1 profile on the two DDR5 test platforms is not stable and causes the platform to become bricked.
How reproducible
100%
How to reproduce
Switch memory profile from JEDEC (default) to XMP1 on either of the two DDR5 testing platforms Reboot
Expected behavior
Platform boots, memory gets trained at the frequencies and timings specified in XMP1 profile.
Actual behavior
Platform does not boot, recovery via FlashBIOS is required.
Screenshots
No response
Additional context
Tested configurations:
- MSI Z690-A, Core i5-13600K, 4x Crucial CT8G48C40U5 DDR5 4800 MT/s CL40
- MSI Z790-P, Core i5-13600K, 4x Kingston KF556C40BB-8 DDR5 5600 MT/s CL40
Solutions you've tried
No response
I picked profile 1 by accident, with Z790-P DDR5
FlashBIOS doesn't seem to work for me, it detects the USB device and turn on the system, but it never completes the task.
Same issue on Z690-A DDR5 v1.1.3 (built from source) with Kingston KF552C40BBK2-64 kit, XMP profile 1 bricks it. XMP profile 2 seems to work, but I'll have to run some tests overnight to verify stability. Also the memory profile settings are stored in the SPI flash since resetting CMOS doesn't help?
After more experimenting XMP profile 2 also doesn't work. I couldn't get it to boot again, but I haven't tried that profile on the original MSI BIOS. USB FlashBIOS recovery works for me, have you used MBR FAT32 formatted USB2 drive @renehoj?
4x Crucial CT8G48C40U5 DDR5 4800 MT/s CL40
There is something VERY wrong here. According to Crucial that module seems to be just standard JEDEC with no XMP support whatsover: https://www.crucial.com/memory/ddr5/ct8g48c40u5 This should be possible to verify in Windows with a tool like Thaiphoon Burner, too.
So if the modules don't have XMP profiles, then what the hell it is trying to set? I think miczyg said than the FSP is supposed to check if the profiles are valid, but it seems that there is no fallback in case than the profile doesn't even exists. Same issue with my DDR4 system and setting XMP Profile 3 (Which is DDR5 only, doesn't exist in DDR4), an option that should actually be removed cause the only thing it does is to brick the system. So I assume than the FSP doesn't automatically falls back to JEDEC profile if XMP is not found.
@6502mos I tried 3 different USB devices, I tried Windows formatting, Rufus formatting, and making the filesystem on Linux, I tried the Dasharo and MSI firmware... and taking out the battery, cmos resetting, and taking out the memory and PCI devices.
At this point, I kinda gave up trying to recover the board without using an eprom programmer, and I can't really afford to spend an endless amount of time trying to fix it, so I decided to cut my losses and just order a new board.
@renehoj Hi, where are you from? I'll buy the bricked motherboard from you if you are from Europe and can offer me a good price. :smiley:
I picked profile 1 by accident, with Z790-P DDR5
FlashBIOS doesn't seem to work for me, it detects the USB device and turn on the system, but it never completes the task.
@renehoj I have verified the FlashBIOS works correctly on our Z790-P DDR5.
Could you explain in detail the whole process of updating? Starting with v0.9.0, what you did, what settings you had before update, what peripherals you had connected. The board revision may also make a difference, so would be great if you could give us such info.
The system is 13900K, 64 GB memory, nvidia 1060 + nvidia 4060 GPUs, and VIA chip PCI to USB card.
I installed 0.9.0 with DTS iPXE, and then reflashed 0.9.0 using FlashBIOS to be able to disable ME.
The only changed I can remember doing to 0.9.0, disabling ME with HAP, and disabling the watchdog timer.
I installed 0.9.1 with DTS iPXE, using the disable security features on next boot option.
I had the system running for a few hours.
After enabling XMP 1 I was not able to boot, power on was just a black screen with the fans running. I couldn't turn off the system using the power button, the system with turn off and 1 sec later turn back on, I had to cut mains power to turn the system off.
Took out the 4060 and reset cmos, turned the system back and waited 20-30 sec no change. Took out the battery, no change. Took out the rest of the hardware including memory, no change. Tried flashing with FlashBIOS, which didn't work.
FlashBIOS does run, it only turns on the system is a correctly formatted USB is inserted, but the diode on the motherboard just keeps blinking. I have one USB stick with RW diode, and it does also blink, but only 1 blink around every 1 sec, as if FlashBIOS fails to read the device.
I tried multiple USB devices that I know all are working, and they all do the same thing, so I'm confident it's not the USB device not working.
There was an user that bricked with Z690-A 1.1.3 and couldn't reflash Dasharo via FlashBIOS but got it to detect MSI BIOS. No idea if he managed to recover since he didn't posted again, and no idea why it worked that way, but is the only other thing I can think that you could test: https://github.com/Dasharo/dasharo-issues/issues/684#issuecomment-1905318606
If you can confirm the PCB Revision of your board it could also help: https://docs.dasharo.com/unified/msi/recovery/#smbios-unique-data-recovery
A dump for post mortem could also be useful, but you seem to not have a reprogrammer at hand. You should really get a CH341A with 1.8V adapter, a 30 U$D or so tool that can save you from any further SNAFU. I find painful to have an emergency spend of about 200 U$D on another board plus time dissambling and reassembling again.
I tried both MSI and Dasharo firmware, I couldn't flash any of them.
The version on the silkscreen is v2.1
I do have an eprom programmer, I simply don't have the time to deal with it right now, and buying a new board was a quick and guaranteed solution. I do think the board can be fixed by flashing the firmware with a programmer, and I don't mind having an extra DDR5 backup board, so the cost is whatever.
FlashBIOS does run, it only turns on the system is a correctly formatted USB is inserted, but the diode on the motherboard just keeps blinking. I have one USB stick with RW diode, and it does also blink, but only 1 blink around every 1 sec, as if FlashBIOS fails to read the device.
If the FlashBIOS turns the system on, it means the binary got accepted. But 1 blink per second does not seem to be a good sign. When the platform is turned on in FLashBISO mode, the blinks are faster, like 2 blinks per second. Also it takes at least 5 minutes to flash...
I had no issues flashing the new motherboard.
I tested the XMP P2 and P3, both fail in the same way as P1.
The power button does a blink two times followed by a ~2 sec delay pattern, and it starts instantly at power on, it's the same for both P2 and P3.
I tried booting the system with only 2x16 GB memory, didn't change anything, I didn't try only using 1 memory module.
I'm using 4x16 GB GSKill RipJaws S5 (Intel XMP 3.0) DDR5
Memory Device
Array Handle: 0x0009
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 16 GB
Form Factor: DIMM
Set: None
Locator: Channel-0-DIMM-0
Bank Locator: BANK 0
Type: DDR5
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 4800 MT/s
Manufacturer: GSkill
Serial Number: ec5f0ef6
Asset Tag: Channel-0-DIMM-0-AssetTag
Part Number: F5-6000J3238F16G
Rank: 1
Configured Memory Speed: 4000 MT/s
Minimum Voltage: 1.1 V
Maximum Voltage: 1.1 V
Configured Voltage: 1.1 V
I tested the XMP P2 and P3, both fail in the same way as P1.
The power button does a blink two times followed by a ~2 sec delay pattern, and it starts instantly at power on, it's the same for both P2 and P3.
Did FlashBIOS worked this time?
I'm using 4x16 GB GSKill RipJaws S5 (Intel XMP 3.0) DDR5
GSkill (And some other manufacturers) website specifications only mentions the highest rated XMP profile. One of the things why I tend to like Kingston these days is because they include in their datasheets what are the other profiles settings. To know what XMP2/XMP3 are (Or whenever their exists) you will have to use Windows Software like Thaiphoon Burner. Or alternatively, dumping the SPD, uploading it and letting me to check them. It may also be possible that you're hitting the other bug that it doesn't POST with a nonexisting XMP profile.
Also, is next to impossible that you will ever get XMP1 working. DDR5 2 DPC @ 6000 MHz without manual fine tuning is not easy. It will also most likely require increased Memory Controller voltage, which is currently far from implemented.
Yes, the new board works as expected.
I tried decoding the SPD, but I don't think it's currently possible using Linux. DDR5 dimms don't use the ee1004 driver, and I don't think there is a driver for the SPD5118 eeprom.
Even if it isn't possible to run the memory at 6000Mhz, there is a big gap between 4000 and 6000, I would think 4800 should be possible, and that would still be a 20% improvement.
At an absolute minimum, the feature should critically crash the system, especially considering it's not marked as experimental.
Even if it isn't possible to run the memory at 6000Mhz, there is a big gap between 4000 and 6000, I would think 4800 should be possible, and that would still be a 20% improvement.
XMP doesn't work like that. The main JEDEC profile can be used to calculate other lower clocked speeds and timmings for compatibility because it is part of the standard, but XMP profiles are exact. You can have more XMP profiles that represent those lower points, but you can't extrapolate them from one. And this is notwithstanding than for DDR5 they're intended for 1 DPC and not 2.
XMP should absolutely be flagged experimental. There is a limited number of cases where it can work out of the box, but without any kind of manual controls if it doesn't work, then you can do nothing to tune it.
Speed: 4800 MT/s
Configured Memory Speed: 4000 MT/s
Does that not mean the memory clocked lower than the speed it's supposed to be running, I assume it should be possible to use the default speed without using XMP.
Does that not mean the memory clocked lower than the speed it's supposed to be running, I assume it should be possible to the default speed without using XMP.
4800 MHz is the default JEDEC profile, and it is downclocking to 4000 MHz (Which as I said before, it is possible to calculate lower clocked points/timmings). That is because how populating DDR5 affects default clock speed enforced by the FSP based on default Processor specifications: https://edc.intel.com/content/www/br/pt/design/products/platforms/details/raptor-lake-s/13th-generation-core-processors-datasheet-volume-1-of-2/008/processor-sku-support-matrix/
Intel® Core™ 14th Generation Processors DDR5 1 DPC - S Refresh : UDIMM 1R: 5600 2R: 5600 2 DPC - S Refresh UDIMM: 1 DIMM - 4400 2 DIMMs 1R - 4000 2 DIMMs 2R - 3600
Note that DPC (DIMMs per Channel) naming in the datasheet seems to be wrong because it fits the definition of SPC (Slots per Channel), as any motherboard with 4 slots and only 1 DPC populated is effectively limited to 4400 MHz (Same as Alder Lake-S. Both 13th and 14th gen Raptor Lake-S raised 1 SPC to 5600 MHz, but the rest remained about the same). So basically, out-of-the-box 5600 MHz support is almost a meme since you would need a 2 slots board, which are rare. I was going to straight up link this post from MSI Forum but seems to be wrong since there is no 5200 MHz 2 DPC mentioned at all in the datasheet. No idea from where they got that number from. So basically, it is "working as intended". Naturally, if manual controls gets implemented it should be possible to force highest JEDEC ignoring Memory Controller concerns, but that is already overclocking.
The CL36 4800 and 5600 presets work just fine, I don't really understand why it should be impossible to do the same with Dasharo.
Memory Device
Array Handle: 0x003A
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 16 GB
Form Factor: DIMM
Set: None
Locator: Controller0-DIMMA1
Bank Locator: BANK 0
Type: DDR5
Type Detail: Synchronous
Speed: 4800 MT/s
Manufacturer: G Skill Intl
Serial Number: EC5F0EF6
Asset Tag: 9876543210
Part Number: F5-6000J3238F16G
Rank: 1
Configured Memory Speed: 4800 MT/s
Minimum Voltage: 1.1 V
Maximum Voltage: 1.35 V
Configured Voltage: 1.35 V
Memory Technology: DRAM
Memory Operating Mode Capability: Volatile memory
Firmware Version: Not Specified
Module Manufacturer ID: Bank 5, Hex 0xCD
Module Product ID: Unknown
Memory Subsystem Controller Manufacturer ID: Unknown
Memory Subsystem Controller Product ID: Unknown
Non-Volatile Size: None
Volatile Size: 16 GB
Cache Size: None
Logical Size: None
Memory Device
Array Handle: 0x003A
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 16 GB
Form Factor: DIMM
Set: None
Locator: Controller0-DIMMA1
Bank Locator: BANK 0
Type: DDR5
Type Detail: Synchronous
Speed: 5600 MT/s
Manufacturer: G Skill Intl
Serial Number: EC5F0EF6
Asset Tag: 9876543210
Part Number: F5-6000J3238F16G
Rank: 1
Configured Memory Speed: 5600 MT/s
Minimum Voltage: 1.35 V
Maximum Voltage: 1.35 V
Configured Voltage: 1.35 V
Memory Technology: DRAM
Memory Operating Mode Capability: Volatile memory
Firmware Version: Not Specified
Module Manufacturer ID: Bank 5, Hex 0xCD
Module Product ID: Unknown
Memory Subsystem Controller Manufacturer ID: Unknown
Memory Subsystem Controller Product ID: Unknown
Non-Volatile Size: None
Volatile Size: 16 GB
Cache Size: None
Logical Size: None
The CL36 4800 and 5600 presets work just fine, I don't really understand why it should be impossible to do the same with Dasharo.
Because you're mostly limited to do what the Intel FSP allows you to do. By default the FSP automatically configures RAM clock speeds based on what I said above. The FSP also allows you to select XMP Profiles, which is the capability that was exposed now (SpdProfileSelected on Alder Lake Integration Guide), but as you see, that is rather limited if not outright broken without significantly more work because the FSP XMP autoconfiguration pretty much sucks (For example, I tested on DDR4 and it seems to not apply the XMP profile DRAM Voltage). Fixing FSP XMP autoconfiguration can be done but is tricky. You need the FSP source code (Which is available under NDA), but this isn't very open source friendly because you have to distribute your own FSP binary blob, so you can't use Intel public FSP any longer.
The FSP has a lot of options (Perhaps too many - I don't recognize more than half of them with what you get on MSI BIOS) that should allow to manually configure most settings and give a reasonable level of control, but no test has been done in whenever it is viable to go fully manual by exposing Voltages, Ratio, BClk, and the known Timmings, which may be preferable to auto XMP given what we known from giving it a go. Heck, right now I'm thinking about whenever it would be a good idea that selecting a XMP Profile instead of using SpdProfileSelected "maps" the clock speed/timmings/voltage to the other specific settings. But this has another problem: You can't decode on your own DDR5 XMP Profiles because the XMP specification isn't public, so it would require some reverse engineering.
I'm not using the XMP profile, it only has one 6000MHz XMP profile.
It just the 4800 and 5600 presets for the memory type, with XMP disabled, there is a dropdown menu where you can select the preset by CL value.
So basically, out-of-the-box 5600 MHz support is almost a meme since you would need a 2 slots board, which are rare.
@zirblazer, are you saying that 2 sticks DDR5 package shouldn't actually work at 5600MHz or above even on MSI firmware? I know that's what the docs say, but my impression was that it's not that unusual.
Brief summary
XMP https://github.com/Dasharo/dasharo-issues/issues/1 profile on the two DDR5 test platforms is not stable and causes the platform to become bricked.
How reproducible
100%
Weird, it's either < 100% or something has changed in the hardware. I remember that testing profile switching was harder on DDR5 which I attributed to longer training time, but I definitely saw it working at least on MSI Z790-P. The code for the boards is in different files, so I had to test both boards and we only have DDR5 variant of MSI Z790-P.
@zirblazer, are you saying that 2 sticks DDR5 package shouldn't actually work at 5600MHz or above even on MSI firmware? I know that's what the docs say, but my impression was that it's not that unusual.
Since I don't have a DDR5 board I can't test myself how MSI BIOS behaves by default with 1 or 2 DPCs (4 modules) populated, but if MSI was defaulting to 5600, they are overclocking/out-of-spec out-of-the-box (Which they actually do with things like Power Limits). If you follow Intel specifications to the letter (Which seems to be the case of FSP autoconfiguring), if you insert a 4800+ MHz JEDEC module it should downclock to 4400 MHz (2 SPC Motherboard with 1 DPC populated).
The "presets" renehoj mentions would be pretty much to force a module to work @ 4800 / 5600 ignoring Memory Controller guidelines, even if the modules themselves were using JEDEC standard Timming and Voltage at those speeds. Suppose that you have four 4800 MHz JEDEC modules (The Crucials on the lab build fits that category, they shouldn't have XMP profiles at all) and want to force them to run at 4800 MHz even if that means overclocking the Memory Controller, but the FSP default profile selection doesn't seem to allow to do that and instead falls down to 4000. XMP profiles have tighter Timmings and higher Voltages that JEDEC standard, but otherwise should willfully ignore Memory Controller specifications since selecting XMP assumes that you're overclocking anyways. Still, Intel (At least on DDR4) decides to downgrade the Gear setting by setting Gear 2 instead of 1 to halve the clock speed of the Memory Controller...
If you check this article you will find about halfway a photo of a BIOS from a high end MSI Z690 board with DRAM Frequency settings that goes like this: MT/s = (Multiplier * Ref Clock * Other value I don't know) and Gear 2/4. I know that Multiplier and RefClk are in Alder Lake Integration Guide (I don't understand the 1.00 vs 1.33 value of the photo, unless it is RefClk, but then what the 100 is?):
UINT8 Ratio Offset 76 DDR Frequency ratio, to multiply by 133 or 100 MHz depending on RefClk. 0 = Auto UINT8 RefClk Offset 112 Selects the DDR base reference clock. 0x01 = 100MHz, 0x00 = 133MHz
Note that I'm not sure if MSI Multiplier is twice the value of Ratio or not since DDR5 runs at half the MT/s (Which we usually call MHz). Supposing that you could manually set them to test forcing 4800 MT/s (RefClk 133 MHz, Ratio 18 or 36 for 2400 MHz / 4800 MT/s). Would it work? What would the Timmings default to?
I was using this MSI feature Memory Try It!, but it's pretty much the same thing. You just select the CL and the firmware detects the specific memory model and loads the correct profile, the end result is the same, but the default values are better tuned.
(I don't understand the 1.00 vs 1.33 value of the photo, unless it is RefClk, but then what the 100 is?):
I think they just decided to express memory reference clock as (CPU reference clock = 100MHz) x (1.0 or 1.33).
Supposing that you could manually set them to test forcing 4800 MT/s (RefClk 133 MHz, Ratio 18 or 36 for 2400 MHz / 4800 MT/s). Would it work? What would the Timmings default to?
Trying it out might be the only way to know. I imagine that sources might just write the value into some register without commenting how hardware interprets it.
4800 MHz is the default JEDEC profile, and it is downclocking to 4000 MHz (Which as I said before, it is possible to calculate lower clocked points/timmings). That is because how populating DDR5 affects default clock speed enforced by the FSP based on default Processor specifications: https://edc.intel.com/content/www/br/pt/design/products/platforms/details/raptor-lake-s/13th-generation-core-processors-datasheet-volume-1-of-2/008/processor-sku-support-matrix/
Intel® Core™ 14th Generation Processors DDR5 1 DPC - S Refresh : UDIMM 1R: 5600 2R: 5600 2 DPC - S Refresh UDIMM: 1 DIMM - 4400 2 DIMMs 1R - 4000 2 DIMMs 2R - 3600
I just tried compiling Dasharo with DdrFreqLimit = 5600, now the memory is running 4800Mhz, using 4 DIMMs R2 128 GB.
Didn't change any other options.
Handle 0x000A, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0009
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 32 GB
Form Factor: DIMM
Set: None
Locator: Channel-0-DIMM-0
Bank Locator: BANK 0
Type: DDR5
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 4800 MT/s
Manufacturer: Corsair
Serial Number: 00000000
Asset Tag: Channel-0-DIMM-0-AssetTag
Part Number: CMK128GX5M4B5600C40
Rank: 2
Configured Memory Speed: 4800 MT/s
Minimum Voltage: 1.1 V
Maximum Voltage: 1.1 V
Configured Voltage: 1.1 V
same issue on v1.1.4-rc2 MSI PRO Z690-A DDR4 (RPL 14th)
fun fact: as for DDR5 one, it initialy stops but after restarting pikvm, everything goes back to normal and XMP1 is selected
I have tried on MSI PRO Z790-P WIFI DDR5 (RPL 13th) with msi_ms7e06_v0.9.2_ddr5-rc3 and issue still persists.
I just wanted to add another example of XMP working on MSI Z690-A DDR4 v1.1.3 previously and now no longer posts with any XMP profile on v1.1.4
Thanks for sharing this. I had previously XMP enabled on v1.1.3 for a few days. I guess I'll refrain on enabling it w/ v1.1.4 until further info becomes available.
