rcraid-dkms
rcraid-dkms copied to clipboard
thopiekar
NVMe RAID support
Hi! Does it suppose to work with NVMe RAID? My attempt to install it failed. If doesn't - are there any chances to add such support?
This is a duplicate of #7. TL;DR is that the author of this software is not able to support NVMe RAID at this time.
I can also personally confirm that NVMe RAID is not supported. :-(
Finally, bought myself a NVMe drive. My drive is also not managed by RAIDxpert, but have the same on Windows, too. (Dual-boot) Therefore, I wouldn't blame it on AMD and their driver, since Windows behaves the same. So in my opinion the issue comes from somewhere below OS layer - BIOS level. I'll try to get in contact with ASUS and ask there for help based on my Windows installation (which should be better supported). Hope it will resolve the issue on my Linux installation, too.
To have the NVMe drives managed by RAIDXpert, there is a setting somewhere in UEFI to enable NVMe RAID(it isn't enabled by default). The system has to boot via UEFI with no Legacy BIOS options enabled. I am currently not at my x399 system to go thru the options and look for the specific settings, but i do remember having to enable RAID support for NVMe(Asrock Phantom Gaming 6 mobo). However, even after having created arrays with NVMe drives, they still won't show any sort of array in linux. linux just sees them as 2 seperate block devices.
Yea, there's an option on my asus x570 board specifically to enable NVMe RAID. Then it's still fake RAID handled in software. You'll see two devices during Windows installation until you load AMD's fake raid driver, (actually using the drivers included on the CD like it's 2004), then it's just one device.
Can confirm your problem here. I had to upgrade my Windows drivers and my NVMe appeared as "Array 6". However, going back to Linux and blacklisting ahci and nvme still didn't get me the drive. "Array 6" was still listed, but marked as offline.
I'm going to write an support ticket tomorrow. I see nothing in the driver that mentions NVMe support. On the other hand there are documents already, which mention NVMe support on Linux. So I have the following theories:
- AMD missed to publish the driver
- AMD is testing the driver with partners or internally still
But it is promising.. Just found a PDF with the latest instructions.
There must be a driver 🤔
But it is promising.. Just found a PDF with the latest instructions.
There must be a driver 🤔
this blacklisting of the linux nvme driver is information i was not aware of in my prior attempts. later today i will make a backup of my $HOME directory and experiment with this boot option and the live kubuntu installer. i will update this post when i know something.
Yes, this is new and needs to be done for the new driver I think. You will see no difference now. When looking at the sources you will see no nvme implementation. ;)
Some news: First I sent a support ticket just some days before. I was a bit too busy recently. Anyway, got a reply from the support telling me that they don't provide updated drivers.
However, in the meantime I made some research and found binary drivers distributed by Lenovo for their workstations for Redhat Linux. I'm going to confront them with this fact and question them whether they think there are second class clients, or what?!
Alternatively we can experiment using the same version as Redhat Linux and try to load the blob, but hope this won't be needed to waste time on.
@ducphuc Took a look over your repo. Can you tell me where you've got the sources from? I see some indications that NVMe support is included there. Can you confirm this?
Others: The driver is 9.3.0 :scream:
I got the sources from the RAIDXpert2 application install for RHEL, Ubuntu, and Debian which came with my Lenovo ThinkStation P620. It broke with the changes in Linux kernel 5.6 so I patched it with fixes in this repo.
Yes, I can confirm that NVMe support is included. I have my system booting from a 4 x Samsung 980 PRO PCIe 4.0 NVMe SSD RAID0 array using a PCIe x16 Gen 4 AIC. I get over 21GB/s sequential reads and over 18GB/s sequential writes.
From the AMD RAIDXpert2 User Guide:
RC-8.1 and RC8.2 branches support only SATA. RC-9.1 and above release branches support SATA, M.2 NVMe SSD and HHHL NVMe SSD
does this mean that support will be coming in the near future? will i finally be able to use the hardware i paid for?
edit: just in case: the drivers and sources can be found here https://pcsupport.lenovo.com/us/en/products/workstations/thinkstation-p-series-workstations/thinkstation-p620/downloads/driver-list/component?name=RAID%2FSAS%20HBA%20Controllers,%20Backplanes,%20Storage%20Expanders%2FSwitches
i attempted to do a patch and compile myself using the patch file from Martin Karl Weber's github repo, but it fails to patch 3 files and fails to compile
Will verify the files tomorrow and test the driver myself. Great findings! I was looking for the sources but only saw the binaries so far. Many thanks!
PS: You didn't merge the code correctly. Your overwrote Martin's patches and reapplied them later. Will prepare something cleaner once I got it working here.
Will verify the files tomorrow and test the driver myself. Great findings! I was looking for the sources but only saw the binaries so far. Many thanks!
PS: You didn't merge the code correctly. Your overwrote Martin's patches and reapplied them later. Will prepare something cleaner once I got it working here.
That sounds AWESOME!!!
edit: do u have a staging ppa available that i can also help test before you push it to the stable ppa?
@thopiekar If you referring to my fork with not merging correctly then you're right. The last commit wasn't supposed to be pushed so I reverted it. I was re-applying the unofficial patches(Martin's patches) to see what the differences were. Sorry about the confusion.
What I did was just copy the official sources over the sources in the repo then applied the fix to get Linux Kernel 5.6 working. What I am trying to do is get DKMS working and using what you have in this repo to get started. I am not planning to submit a PR nor planning to keep my repo around once you saw the official sources were available. I will just be using the OEM vendor supplied packages. I am just testing some unsupported distros. The problems I am facing is from the platform being just released and the AMD support for Linux is rough around the edges still. I suspect things will be different now that AMD is trying to get into the workstation market.
Alright. From what I saw in the latest sources, it seems that AMD took over the source code. At least there are copyright tags stating that AMD owns the code beginning from 2020. Hope they are going to support us better now 🤞
Support will come from the OEM partners at least. AMD RAID is currently certified and supported on a Lenovo ThinkStation P620 with Ubuntu 20.04 or RHEL 8.2. This workstation is a great product and I highly recommend it. I'll be submitting a bug report that AMD RAID needs to be fixed for Linux 5.6 kernels with them.
Lenovo has made some big announcements regarding both Linux and AMD(ThreadRipper Pro workstations) these last few months. The work they are doing to bring support to Linux is amazing. For example, they are requiring their component suppliers, such as the fingerprint readers, provide Linux drivers or they will use one that will. My next laptop will definitely be a Lenovo ThinkPad with Linux preinstalled.
- Lenovo Brings Linux® Certification to ThinkPad and ThinkStation Workstation Portfolio, Easing Deployment for Developers & Data Scientists
- Lenovo Launches Linux-Ready ThinkPad and ThinkStation PCs Preinstalled with Ubuntu
- Lenovo ThinkStation P620: World’s First AMD Ryzen™ Threadripper™ PRO Workstation
After migrating the old to the new code and getting the driver running, I got messages like "license level to low to run Array X". However, I have/had no complaints about missing licenses or permissions on Windows.
After about a week of fixing and a messed up hard drive (yeah good timing -.-' ), I gave up with AMD RaidXpress. The Windows and Linux drivers are not fitting at all. If I set in my BIOS NVMe mode to "normal" (I would expect it to be not managed by RaidXpress) it won't appear on Windows at all, but on Linux I can access it from 8.1.0 when not blacklisting nvme. When using 8.1.0 on Linux and 9.3.0 on Windows plus I lost my hard drive with non-critical data for Linux (trash store for building Android firmware or whatever else) I had a "biting" effect. I not only lost the specific drive but also the relevant data. So I lost again days to use recovery tools to find my partitions on the drive(s).
I just don't have time for further experiments and crossed fingers. Regarding the licensing errors reported by the driver I doubt the driver is ready for everyone now. I don't trust this software as it is now.
That's not a good experience. I feel your pain. It was a struggle for me to get things working like I wanted too. It seems the firmware version needs be 9.3.x as well to support NVMe RAID. My AMR RAID firmware is 9.3. I would check with your motherboard vendor for updated firmware. The firmware writes metadata onto the drives. The firmware will also store information about the drives in it's memory. The manual states that the drive will not be visible to the operating systems when deleted from an array. I experienced this myself and is by design. You need to add it an array. Clearing the metadata from the drive and/or from firmware would get it back. There are some quirks like this and others like needing to delete a drive from an array(the automatic legacy one created for it) before you can add it to an array is shown in AMD RAID reviews and many forums. It is not the easiest thing to work with at all. But once you get it working, 21 GB/s is very nice to have.
Also the manual states specifically that dual booting is not supported. It jacks up my system too. Two separate legacy RAID controllers are created for a drive. It would flip flop which one is working and visible between Windows and Linux.
Hello folks,
any news about the NVMe AMD RAID support? I have the same issue discussed here and I would like to fix it on my Ubuntu 20.10
Is a fix on the roadmap or does an alternative solution exist?
Thanks, Mark
I also found a driver
raid_linux_driver_930_00276.zipnamed that says:
can you inform us of where you found this driver?...the problem we've had so far with the one provided by lenovo is that it is only licensed for use on lenovo linux systems, and nvme raid support was not enabled by the lenovo drivers on non-lenovo hardware(for example: i had no success with my Phantom Gaming 6 motherboard).
May be I have found a version of AMD drivers that support NVMe RAID. In this case, on next days I will create a PR for this repository, but please @Fuzzo give to us the link of driver you found because if it's newer than mine I will use it.
The drivers can be downloaded at this page:
Even if they are present in the x370 page, the RAID subsystem should be the same also for other chipsets.
EDITED DUE TO WRONG FILE: i uploaded the right one in comments below, sorry
The drivers can be downloaded at this page: https://www.amd.com/en/support/chipsets/amd-socket-am4/x370 Direct download link is: https://www2.ati.com/drivers/raid_linux_driver_8_01_00_039_public.zip
Even if they are present in the x370 page, the RAID subsystem should be the same also for other chipsets.
the driver version (8.0.1), released 3/27/2017 offers no nvme raid support. i've already gone round and round with that version, as has most everyone using this repo.
most of us have given up on AMD actually offering support for this hardware. Myself, i've moved on to a hardware based HBA (Highpoint 7505 nvme raid controller) which DOES offer Linux support and currently have 3x 4TB Sabrent Rocket Q nvmes housed in it.
raid_linux_driver_930_00276.zip
This is the right file with NVME support: it contains also the file referred by the table screenshot I put above. I'm really sorry but i made a mess with the various releases in my archive :(
@MarkLTZ please use this to update your pull request