WalnutLum
@WalnutLum@lemmy.ml
- Comment on AMD captures 28.7% market share in desktops 3 days ago:
The fact they pulled ROCM support for older cards boggles the mind.
- Comment on The Great Migration to Bluesky Gives Me Hope for the Future of the Internet 1 week ago:
Bluesky is not great, but it’s at least (for now) a better platform than X and the AT protocol is actually very well written. (For instance having a moderation service separated from the service that provides the posts I think is a hands-down better way to handle it than most ActivityPub servers having their admins handle all incoming and outgoing moderation)
Bluesky federation is just now getting started so it’ll be interesting to see if it goes anywhere/where it goes.
- Comment on The Great Migration to Bluesky Gives Me Hope for the Future of the Internet 1 week ago:
Pretty sure a significant portion of nazis are also moving to bluesky
- Comment on Google open-sources compact Japanese Gemma AI model optimized for local use on mobile devices. 3 weeks ago:
Anyone from the local llama communities experienced with Gemma models?
I’ve heard good things but I only use Mistral because it’s proved the most versatile.
- Comment on Linus Torvalds reckons AI is ‘90% marketing and 10% reality’ 3 weeks ago:
I’ve been curious about google coral, but their memory is so tiny I’m not sure what kinds of models you can run on them
- Comment on Bluesky’s upcoming premium plan won’t give paid users special treatment 4 weeks ago:
If nothing else the atproto is pretty great, we’re starting to see a proper federated net start opening up around it.
- Comment on The Death of the Junior Developer 4 weeks ago:
Anyone else remember when people were making expert systems with scheme and saying that was the end of doctors etc?
- Comment on X's idiocy is doing wonders for Bluesky. 4 weeks ago:
I could be mistaken too, this has all only recently become interoperable so there’s some growing pains
- Comment on X's idiocy is doing wonders for Bluesky. 4 weeks ago:
Isn’t that what Whitewind is doing?
- Comment on X's idiocy is doing wonders for Bluesky. 4 weeks ago:
Yes! Actually.
The full atproto up and running with bluesky is only in the last month or so, so people are finally starting to trickle out and set up their own services and hosts.
It’s actually very promising and hopeful.
- Comment on X's idiocy is doing wonders for Bluesky. 4 weeks ago:
This doesn’t seem to be that big an issue as PDS’s can just directly communicate with one-another like how ActivityPub works.
- Comment on Meta has suspended several Threads and Instagram accounts that track the private jets of celebrities such as Mark Zuckerberg, Elon Musk, Jeff Bezos, Kim Kardashian, and Donald Trump 4 weeks ago:
I wouldn’t lump bluesky in the same pile as threads anymore, the atprotocol is fully up and running and slowly but surely individually hosted data servers are trickling out and away to their own services.
There’s even new services running completely independent of bluesky running on atproto now: whtwnd.com/about
There’s a really good write-up on how atproto federation works here: whtwnd.com/alexia.bsky.cyrneko.eu/3l727v7zlis2i
- Comment on Using AI generated code will make you a bad programmer. 4 weeks ago:
If you treat an AI like anything other than the rubber duck in Rubber Duck Programming you’re using it wrong.
- Comment on Arch Linux and Valve Collaboration 1 month ago:
I think Steam does have enough influence to be able to pull a sizable chunk of users away from windows.
- Comment on Ubisoft's Board is Launching an Investigation Into The Company Struggles 1 month ago:
It seems like a very polarizing game, you either really enjoy it or not at all.
- Comment on Ubisoft's Board is Launching an Investigation Into The Company Struggles 1 month ago:
I love the division 1 and 2 but the first game had some MAJOR bullet soak issues for the first half-year of the game’s lifetime.
Massive always does good work despite Ubisoft, in my opinion.
- Comment on ChatGPT is changing the way we write. Here’s how – and why it’s a problem. 1 month ago:
Biggest issue I see is that these LLMs tend to repeat themselves after a surprisingly short number of times (unless they’re sufficiently bloated like ChatGPT).
If you ask any of the users of Sillytavern or RisuAI they’ll tell you that these things have a long tail of not being very creative.
- Comment on the strange new future of story-driven PC gaming 1 month ago:
I don’t think it’ll solve the problem. Ask anyone in the sillytavern subreddit and they’ll tell you LLMs tend to repeat the same dialogue a lot (look up the “shivers up/down their spine” meme)
- Comment on Elon Musk destroys astronomy 2 months ago:
Looking forward to when Europe and China also launch their own satellite internet constellations
- Comment on Pressure washing 2 months ago:
The key is to get one where you can widen the spray and squeegee everything off.
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
You said open source. Open source is a type of licensure.
The entire point of licensure is legal pedantry.
And as far as your metaphor is concerned, pre-trained models are closer to pre-compiled binaries, which are expressly not considered Open Source according to the OSD.
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
From the approach section:
A Transformer sequence-to-sequence model is trained on various speech processing tasks, including multilingual speech recognition, speech translation, spoken language identification, and voice activity detection. These tasks are jointly represented as a sequence of tokens to be predicted by the decoder, allowing a single model to replace many stages of a traditional speech-processing pipeline. The multitask training format uses a set of special tokens that serve as task specifiers or classification targets.
This is not sufficient data information to recreate the model.
From the training data section:
The models are trained on 680,000 hours of audio and the corresponding transcripts collected from the internet. 65% of this data (or 438,000 hours) represents English-language audio and matched English transcripts, roughly 18% (or 126,000 hours) represents non-English audio and English transcripts, while the final 17% (or 117,000 hours) represents non-English audio and the corresponding transcript. This non-English data represents 98 different languages. As discussed in the accompanying paper, we see that performance on transcription in a given language is directly correlated with the amount of training data we employ in that language.
This is also insufficient data information and links to the paper itself for that data information.
Additionally, model cards =/= data cards. It’s an important distinction in AI training.
There are guides on how to Finetune the model yourself: huggingface.co/blog/fine-tune-whisper
Fine-tuning is not re-creating the model. This is an important distinction.
The OSAID has a pretty simple checklist for the OSAID definition: opensource.org/…/the-open-source-ai-definition-ch…
To go through the list of materials required to fit the OSAID:
Datasets Available under OSD-compliant license
Whisper does not provide the datasets.
Research paper Available under OSD-compliant license
The research paper is available, but does not fit an OSD-compliant license.
Technical report Available under OSD-compliant license
Whisper does not provide the technical report.
Data card Available under OSD-compliant license
Whisper provides the model card, but not the data card.
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
Oh and for the OSAID part, the only issue stopping Whisper from being considered open source as per the OSAID is that the information on the training data is published through arxiv, so using the data as written could present licensing issues.
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
The problem with just shipping AI model weights is that they run up against the issue of point 2 of the OSD:
The program must include source code, and must allow distribution in source code as well as compiled form. Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost, preferably downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed.
AI models can’t be distributed purely as source because they are pre-trained. It’s the same as distributing pre-compiled binaries.
It’s the entire reason the OSAID exists:
- The OSD doesn’t fit because it requires you distribute the source code in a non-preprocessed manner.
- AIs can’t necessarily distribute the training data alongside the code that trains the model, so in order to help bridge the gap the OSI made the OSAID - as long as you fully document the way you trained the model so that somebody that has access to the training data you used can make a mostly similar set of weights, you fall within the OSAID
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
Whisper’s code and model weights are released under the MIT License. See LICENSE for further details. So that definitely meets the Open Source Definition on your first link.
Model weights by themselves do not qualify as “open source”, as the OSAID qualifies. Weights are not source.
Additional WER/CER metrics corresponding to the other models and datasets can be found in Appendix D.1, D.2, and D.4 of the paper, as well as the BLEU (Bilingual Evaluation Understudy) scores for translation in Appendix D.3.
This is not training data. These are testing metrics.
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
Those aren’t open source, neither by the OSI’s Open Source Definition nor by the OSI’s Open Source AI Definition.
The important part for the latter being a published listing of all the training data. (Trainers don’t have to provide the data, but they must provide at least a way to recreate the model given the same inputs).
Data information: Sufficiently detailed information about the data used to train the system, so that a skilled person can recreate a substantially equivalent system using the same or similar data. Data information shall be made available with licenses that comply with the Open Source Definition.
They are model-available if anything.
- Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates 2 months ago:
Those aren’t open source, neither by the OSI’s Open Source Definition nor by the OSI’s Open Source AI Definition.
The important part for the latter being a published listing of all the training data. (Trainers don’t have to provide the data, but they must provide at least a way to recreate the model given the same inputs).
They are model-available if anything.
- Comment on Bumble 2 months ago:
I think I remember a site like that existing in the 2010s, where you had to apply to join and it only let in equal numbers of genders.
It was the 2010s so the waiting list for dudes joining was way longer than the one for women. It was like trying to get in a dance club.
- Comment on Peloton announces $95 “used equipment activation fee” 2 months ago:
How do you do this with a tablet? Can you buy like a wheel sensor or something?
- Comment on Nvidia is ditching dedicated G-Sync modules to push back against FreeSync’s ubiquity 2 months ago:
Unless something has changed recently you still have to submit builds to Nvidia to have them train the DLSS kernel for you, so FSR is substantially easier to integrate.
