robber

As an alternative to mounting full /dev/dri, there are symlinks under /dev/dri/by-path/ which you can use. IIUC they are linked by the used PCI slot, hence should be as stable as the physical order of the cards.

I use this pattern in a compose file:

...
devices:
  - source: /dev/dri/by-path/pci-XXX-render
    target: /dev/dri/renderD128
    permissions: rwm

In case you are unaway, frieds don’t let friends use ollama. The llama.cpp has become pretty much equally easy to use, plus since you use Docker there’s no need to compile anything.

Welcome.

en.wikipedia.org/wiki/David_Heinemeier_Hansson#Co…

The service starts as it should right? The errors appear only on request, right?

You use podman-compose with the Podman backend?

And there are no volumes mounted, just to be sure?

No expert but don’t the logs look like the requests are coming through?

Yeah right? At least a layer of VPN on top.

It really depends on how much you enjoy to set things up for yourself and how much it hurts you to give up control over your data with managed solutions.

If you want to do it yourself, I recommend taking a look at ZFS and its RAIDZ configurations, snapshots and replication capabilities. It’s probably the most solid setup you will achieve, but possibly also a bit complicated to wrap your head around at first.

But there are a ton of options as beautifully represented by all the comments.