a ssd sata drive in a cd-drive caddy converter is a good way to get a boot drive as well
Comment on Help Request - Proxmox Partitions/Boot - CloverBootLoader
scrapeus@feddit.de 1 year ago
Hey, i have had the same trouble on an DL380 G9. Those bioses don’t support booting from PCIe at all. My server can’t even boot from drives from the Raid controller in IT-Mode.
I would suggest, by proxmox being a hypervisor, to just install proxmox on a single SATA disk and try to boot from there. This is what I have done in the end.
You can then use your NVMes as storage pool. Also you bifurcation can always also be a problem when trying to boot from those devices.
I would also as a last call try to disable bifurcation and see if one drive will show up. Maybe then you could use 2 real PCIe slots with cheap m2 to PCIe adapters.
BlueEther@no.lastname.nz 1 year ago
Ac5000@lemm.ee 1 year ago
The server has 24x 2.5" bays. I have an old SSD drive that I figure I could use as a last resort to be the Proxmox boot drive and then just use the NVMe’s as storage.
I was just hoping to have the Proxmox install/configuration in the NVMe RAID1 just for some minor safety in case a drive dies. From what I’ve read, this should be possible. I’m just lacking the knowledge to know what I’ve done wrong. (Mostly my lack of understanding the blkid results.)
fuggadihere@lemmy.world 1 year ago
I was about to suggest scrapeus’ solution but only try boomg off a single nvme, since the zfs mirror adds another layer of complexity. At least you will be able to rule the zfs mirror root out of the equation.
Ac5000@lemm.ee 1 year ago
Prior to doing the Proxmox install, and prior to the PCIe bifurcation, I still was unable to see the drives directly in iDRAC/bios. What I’ve read online is Dell does this for “reasons” and they happen to sell an add-on card to let you directly access NVMe from PCIe.
While I’m not ruling out the ZFS mirror issue, I don’t think it’s the cause of my problem considering both Clover and the Proxmox install debug can see the drives/partitions. I just don’t understand partition/device/boot structures and processes enough to make sense of what I’m seeing in the blkid/preboot results.
Trying to find information about it online just gets me bad guides about making partitions. The Linux docs for blkid and fdisk also don’t seem to have the explanation of the results, just the arguments for the commands.
Ac5000@lemm.ee 1 year ago
I don’t think the bifurcation is causing me issues. Before I enabled it, I wasn’t able to see the drives from iDRAC/Bios. From what I’ve been able to research, this is expected and Dell sells the “solution” to booting directly from them. (Add-in card that’s pretty pricey…)
I do have an old SATA SSD that I’m considering slotting into one of the bays and using to boot. But I see that as a “last resort” option. I was hoping to have a bit of redundancy with the Proxmox install/configuration itself.
I feel that there’s a solution to the current setup and I just lack the knowledge to fix it. Everything I’ve been able to find points to my current setup being able to work. I’m just being hindered by not understanding partition/device/boot structure.
From what I understand, and what I saw during the Proxmox installation, if I can get past whatever part of the POST/boot process is preventing seeing the drives directly, I can use Clover to bootload from there. I’ve been able to boot into Clover just fine, and it was able to “see” the drives and partitions. I just don’t know which one should hold the Proxmox boot and if I’ve configured the Clover config correct.
30021190@lemmy.cloud.aboutcher.co.uk 1 year ago
They actually do but it can only be a HPE supported BootROM… anything non-HPE is ignored (weirdly, some Intel and Broadcom cards PXE boot without the HPE firmware but not all).
Most of these boards have internal USB and internal SD slots which you can boot from with any media, intact HPE sell a USB SD card raid adaptor for the usb slot. So I would recommend using SD card for this…
scrapeus@feddit.de 1 year ago
I wouldn’t suggest usb or sd-cards with proxmox due to its constant logging. You will fry them really quick unfortunately. Had that problem with NVMes.
For litterly anything else I would also suggest SD-Cards.
Ac5000@lemm.ee 1 year ago
The original plan is to use an SD card with Clover in read-only mode to bootload Proxmox running on the NVMe drives. (Read-only to prevent frying the SD card) This server has a built in SD Card slot Dell calls “vFlash” that you can actually remotely partition and configure. That’s where I was going to put the final configuration of Clover.
How fast/often is Proxmox writing logging? It’s concerning that you say you had this fry some NVMes since that’s what I’m trying to do here. Is this a setting that you can adjust?
scrapeus@feddit.de 1 year ago
The problem is more with zfs on consumer grade NVMes. I have/had problems in that configuration due to the bigger sector sizes. Proxmox itself does do frequent writes, but I don’t know how often exactly. I know that my problems went away with not using zfs.
reddit.com/…/zfs_extremely_high_ssd_wearout_seemi…
Ac5000@lemm.ee 1 year ago
From what I’ve read online, Dell does something similar. There’s some sort of card/add-on that can enable directly seeing and booting from PCIe but they are costly.
This server has the internal USB and a build in SD slot accessible from the rear. (There’s also a dual card option like you mention for redundancy.)
My plan was to get Clover working with USB, then use the vFlash SD slot to hold the Clover bootloader in read-only mode. This would hopefully prevent the SD card from dying quickly.
30021190@lemmy.cloud.aboutcher.co.uk 1 year ago
Why not just clone the boot partition after an install and then change the mount points in the fstab?
Or even just install grub or pxelinux onto the SD that then just directs you somewhere else.
Clover always felt very unstable and messy.