Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates

• QuadratureSurfer@lemmy.world ⁨2⁩ ⁨months⁩ ago

  I don’t understand. What’s missing from the code, model, and weights provided to make this “open source” by the definition of your first link? it seems to meet all of those requirements.

  As for the OSAID, the exact training dataset is not required, per your quote, they just need to provide enough information that someone else could train the model using a “similar dataset”.

  source

  • WalnutLum@lemmy.ml ⁨2⁩ ⁨months⁩ ago

    Oh and for the OSAID part, the only issue stopping Whisper from being considered open source as per the OSAID is that the information on the training data is published through arxiv, so using the data as written could present licensing issues.

    source

    • QuadratureSurfer@lemmy.world ⁨2⁩ ⁨months⁩ ago

      Ok, but the most important part of that research paper is published on the github repository, which explains how to provide audio data and text data to recreate any STT model in the same way that they have done.

      See the “Approach” section of the github repository: github.com/openai/whisper?tab=readme-ov-file#appr…

      And the Traning Data section of their github: github.com/openai/whisper/blob/…/model-card.md#tr…

      With this you don’t really need to use the paper hosted on arxiv, you have enough information on how to train/modify the model.

      There are guides on how to Finetune the model yourself: huggingface.co/blog/fine-tune-whisper

      Which, from what I understand on the link to the OSAID, is exactly what they are asking for. The ability to retrain/finetune a model fits this definition very well:

      The preferred form of making modifications to a machine-learning system is:

      • Data information […]
      • Code […]
      • Weights […]

      All 3 of those have been provided.

      source

      • WalnutLum@lemmy.ml ⁨2⁩ ⁨months⁩ ago

        From the approach section:

        A Transformer sequence-to-sequence model is trained on various speech processing tasks, including multilingual speech recognition, speech translation, spoken language identification, and voice activity detection. These tasks are jointly represented as a sequence of tokens to be predicted by the decoder, allowing a single model to replace many stages of a traditional speech-processing pipeline. The multitask training format uses a set of special tokens that serve as task specifiers or classification targets.

        This is not sufficient data information to recreate the model.

        From the training data section:

        The models are trained on 680,000 hours of audio and the corresponding transcripts collected from the internet. 65% of this data (or 438,000 hours) represents English-language audio and matched English transcripts, roughly 18% (or 126,000 hours) represents non-English audio and English transcripts, while the final 17% (or 117,000 hours) represents non-English audio and the corresponding transcript. This non-English data represents 98 different languages. As discussed in the accompanying paper, we see that performance on transcription in a given language is directly correlated with the amount of training data we employ in that language.

        This is also insufficient data information and links to the paper itself for that data information.

        Additionally, model cards =/= data cards. It’s an important distinction in AI training.

        There are guides on how to Finetune the model yourself: huggingface.co/blog/fine-tune-whisper

        Fine-tuning is not re-creating the model. This is an important distinction.

        The OSAID has a pretty simple checklist for the OSAID definition: opensource.org/…/the-open-source-ai-definition-ch…

        To go through the list of materials required to fit the OSAID:

        Datasets Available under OSD-compliant license

        Whisper does not provide the datasets.

        Research paper Available under OSD-compliant license

        The research paper is available, but does not fit an OSD-compliant license.

        Technical report Available under OSD-compliant license

        Whisper does not provide the technical report.

        Data card Available under OSD-compliant license

        Whisper provides the model card, but not the data card.

        source
  • WalnutLum@lemmy.ml ⁨2⁩ ⁨months⁩ ago

    The problem with just shipping AI model weights is that they run up against the issue of point 2 of the OSD:

    The program must include source code, and must allow distribution in source code as well as compiled form. Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost, preferably downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed.

    AI models can’t be distributed purely as source because they are pre-trained. It’s the same as distributing pre-compiled binaries.

    It’s the entire reason the OSAID exists:

    1. The OSD doesn’t fit because it requires you distribute the source code in a non-preprocessed manner.
    2. AIs can’t necessarily distribute the training data alongside the code that trains the model, so in order to help bridge the gap the OSI made the OSAID - as long as you fully document the way you trained the model so that somebody that has access to the training data you used can make a mostly similar set of weights, you fall within the OSAID

    source