has anyone pushed Raspberry Pi 5 to the point of reliable transcoding for a household of 2-3 people?

People definitely have and you can physically do that now… but from experience, you’ll likely be happier doing almost anything else. I wouldn’t consider it if you want those 2-3 streams to occasionally occur simultaneously. Although, reminder that the best plan is to consider ways where you don’t need to transcode.