sounds like salesforce for a twitch setting. cool use case, must make fun moments when he mentions such things.
Comment on What exactly is a self-hosted small LLM actually good for (<= 3B)
CrayonDevourer@lemmy.world 4 days ago
Currently I’ve been using a local AI (a couple different kinds) to first - take the audio from a Twitch stream; so that I have context about the conversation, convert it to text, and then use a second AI; an LLM – to read twitch chat and store ‘facts’ about specific users so that they can be referenced quickly for a streamer who has ADHD in order to be more personable.
That way, the guy can ask User X how their mothers surgery went. Or he can remember that User K has a birthday coming up. Or remember that User G’s son just got a PS5 for Christmas, and wants a specific game.
Hadowenkiroast@piefed.social 4 days ago
jlow@discuss.tchncs.de 4 days ago
Esp. if the LLM just hallucinates 50% of the “facts” a about the users 👌
CrayonDevourer@lemmy.world 4 days ago
That hasn’t been a problem at all for the 200+ users it’s tracking so far for about 4 months.
I don’t know a human that could ever keep up with this kind of thing. People just think he’s super personable, but in reality he’s not. He’s just got a really cool tool to use.
He’s managed some really good numbers because being that personal with people brings them back and keeps them chatting. He’ll be pushing for partner after streaming for only a year.
catty@lemmy.world 4 days ago
Surely none of that uses a small LLM <= 3B?
CrayonDevourer@lemmy.world 4 days ago
Yes. The small LLM isn’t retrieving data, it’s just understanding context of text enough to know what “Facts” need to be written to a file.
catty@lemmy.world 3 days ago
Some questions and because you don’t actually understand, also, the answers.
- what does the LLM understand the context of, (other user’s data owned by Twitch)
- How is the LLM fed that data? (You store it and feed it to the LLM)
- Do you use Twitch’s data and its users data through an AI without their consent? (Most likely, yes)
- Do you have consent from the users to store ‘facts’ about them (You’re pissy, so obviously not)
- Are you then storing that processed data? (Yes, you are, written to a file)
- Is the purpose this data processing commercial (Yes, it is, designed to increase viewer count for the user of this system - and before you retort “OMG it helps twitch too”… Uhm no, Twitch has the viewers if not watching him, watching someone else)
I mean yeah, it’s a use case, but own up to the fact that you’re wrong. Or be pissy. I don’t care.
CrayonDevourer@lemmy.world 3 days ago
So this wasn’t a post actually asking what a small LLM was good for, it was just an opportunity you could use to dump on LLM usage I take it.
shnizmuffin@lemmy.inbutts.lol 4 days ago
Hey, you’re treating that data with the respect it demands, right? And you definitely collected consent from those chat participants before you Hoover’d up their [re-reads example] extremely Personal Identification Information AND Personal Health Information, right? Because if you didn’t, you’re in violation of a bunch of laws and the Twitch TOS.
CrayonDevourer@lemmy.world 4 days ago
If I say my name is Doo doo head, in a public park, and someone happens to overhear it - they can do with that information whatever they want. Same thing.
aksdb@lemmy.world 4 days ago
Quite arrogant after you just constructed a faulty comparison.
That’s absolutely not the same thing. Overhearing something that is in the background is fundamentally different from actively recording everything going on in a public space. You film yourself or some performance in a park and someone happens to be in the background? No problem. You build a system to identify everyone in the park and collect recordings of their conversations? Absolutely a problem, depending on the jurisdiction. The intent of the recording(s) and the reasonable expectations of the people recorded are factored in in many jurisdictions, and being in public doesn’t automatically entail consent to being recorded.
See for example www.freedomforum.org/recording-in-public/
(And just to clarify: I am not arguing against your explanation of Twitch’s TOS, only against the bad comparison you brought.)
kattfisk@lemmy.dbzer0.com 3 days ago
You’re both getting side-tracked by this discussion of recording. The recording is likely legal in most places.
It’s the processing of that unstructured data to extract and store personal information that is problematic. At that point you go from simply recording a conversation of which you are a part, to processing and storing people’s personal data without their knowledge, consent, or expectation.
CrayonDevourer@lemmy.world 3 days ago
Literally not. The police use this right now to record your location and time seen using license plates all over the nation - with private corporations providing the service.
shnizmuffin@lemmy.inbutts.lol 2 days ago
Let’s take a look at the Developer Agreement that you cited:
This very clearly states that you are disallowed from retaining chat logs for the general purpose of collecting information about Twitch’s end users.
You said that you, “store ‘facts’ about specific users so that they can be referenced quickly,” but then later in a different thread state, “I’m not storing their data. I’m feeding it to an LLM which infers things and storing that data.” You’re retrieving information about specific users at a later time. You’ve built a database of structureless PII from chat logs. You’ve chosen to store the data as inferences, which makes it a bad database, but still a database.
I have questions:
When your streamer mentions something deeply personal, like, “how their mothers surgery went,” that your tool helped them remember, do they disclose that your tool was involved in that transaction? When the viewer gets weirded out and asks your streamer to not mention that again, or forget it entirely, do you have a way to remove that information from your database and a way to prove it’s been deleted? When other people in chat think it’s gross, and ask to opt-out, can you even do it?
Regarding FrostyTools: I don’t think it’s storing the chat logs for a later time. They don’t have a data retention section in their TOS or Privacy Policy that isn’t related to the streamer. (As in, they hold on to the streamer’s Twitch account and some other information for billing, authentication, etc.) I think it’s taking the chat logs only for as long as it needs to output a response and then deleting it. Also, this excerpt from the FrostyTools TOS made me chuckle:
This leads me to believe that you can violate the Twitch TOS quoted above using FrostyTools. It is apparent that FrostyTools has positioned itself as an application that creates User Generated Content (like Photoshop or Word).
CrayonDevourer@lemmy.world 2 days ago
I’m not storing chat logs.
Not creating any kind of public database either. It’s a private tool.
Again - Not storing chat logs. They are processed for information and that information inferred.
catty@lemmy.world 3 days ago
Doesn’t Twitch own all data that is written and their TOS will state something like you can’t store data yourself locally.
CrayonDevourer@lemmy.world 3 days ago
I’m not storing their data. I’m feeding it to an LLM which infers things and storing that data.
carl_dungeon@lemmy.world 4 days ago
Most US states are single party consent. recordinglaw.com/…/one-party-consent-states/
interdimensionalmeme@lemmy.ml 4 days ago
There is no expectation of privacy in public spaces. Participants to these streams which are open to all do not have a prohibition on repeating what they have heard.
kattfisk@lemmy.dbzer0.com 3 days ago
Repeating what they heard is very different from automatically processing the chat to harvest personal information about the participants.
Just because some data is publicly available doesn’t mean all processing of that data is legal and moral.
carl_dungeon@lemmy.world 3 days ago
Right and what I was saying was even if it wasnt “public”, single party consent means the person recording can be that single party- so still a non-issue.