Let me just answer this part

Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?

Yes absolutely, and you don’t even need an extremely powerful machine. Basic text generation will work on a macbook pro. If the hardware prices were at normal levels, buying a machine for 2-3k USD/EUR and self-hosting powerful models would be feasible. Right now the same hardware will run you about 10k, that why I don’t think it’s practical at this moment.

However training is where the real cost lies. It’s pretty much impossible to train a model from 0 on anything resembling a personal machine. First you need all the data, measuring in millions of terabytes. And then you need to train the model on all of that.

On a side note - I once tried training a GPT2 replica on my laptop with about 3GB of random text. The estimated time was 3 months. That’s the level of requirements we’re talking about.