Comment on Are there any decent GPT-detection tools that can be run locally?
ggtdbz@lemmy.dbzer0.com 6 days agoI think I should have been more clear, this is exactly what I’m asking about. I’m somewhat surprised by the reaction this post got, this seems like a very normal thing to want to host.
Doesn’t help that some people here are replying as if I was asking to locally host the “trick” that is feeding a chatbot text and asking it whether it’s machine-generated. Ideally the software I think I’m looking for would be something that has a bank of LLM models and can kind of do some sort of statistical magic to see how likely a block of tokens is to be generated by them. Would probably need to have quantized models just to make it run at a reasonable speed. So it would, for example, feed the first x tokens in, take stock of how the probability table looks for the next token, compare it to the actual next token in the block, and so on.
Maybe this is already a thing and I just don’t know the jargon for it. I’m pretty sure I’m more informed about how these transformer algorithms work than the average user of them, but only just.
null_dot@lemmy.dbzer0.com 6 days ago
Sorry I’m still not really sure what you’re asking for.
I use Open Web UI, which is the worst name ever, but it’s a web ui for interacting with chat format gen AI models.
You can install that locally and point it at any of the models hosted remotely by an inference provider.
So you host the UI but someone else is doing the GPU intensive “inference”.
There seems to be some models for t his task available on huggingface like this one:
huggingface.co/…/roberta-base-ai-text-detection-v…
The difficulty may be finding a model which is hosted by an inference provider. Most of the models available on huggingface are just the binary model which you can download and run locally. The popular ones are hosted by inference providers so you can just point a query at their API and get a response.
As an aside, it’s possible or likely that you know more about how Gen AI works than I do, but I think this type of “probability table for the next token” is from the earlier generations. Or, this type of probability inference might be a foundational concept, but there’s a lot more sophistication layered on top now. I genuinely don’t know. I’m super interested in these technologies but there’s a lot to learn.