The closest thing I’ve found to Plexamp (on Android) is Symphonium. I only pointed it to my Plex server, but it offers support to so many other services. It also works perfect in Android Audio. It does cost $4, but it’s honestly worth that and then some.

But I totally get how great Plexamp is. I use it every day.