@bioinformatician_next_door@kafeneio.social @ariadne The mathematical theory and code libraries behind typical LLMs used in coding are produced by Big Tech and are inscrutable black boxes to the vast majority of people. They typically require proprietary binary blobs, the trained weights, that ultimately derive from stolen works without maintaining appropriate citation and credit structures. Meanwhile, they demand enormous hardware outllays---almost surely for NVIDIA hardware---relative to other methods and consume significant amounts of additional electric power, doubling the power consumption of the typical workstation in the typical use cases. The deskilling, slop production, befuddlement of self awareness, and other such effects are just as serious in self-hosted LLMs as they are in their hosted counterparts.

LLMs as a technology class are very problematic and should be avoided, in my view, till we've collectively had time to audit all their many problems and developed proven mititgations. I believe it's best to think of them as proprietary and closed source, and reason from there.