Comment on I'm gonna die on this hill or die trying

themeatbridge@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

• DarrinBrunner@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  So… Why don’t I see double spaces after your periods? Test. For. Double. Spaces.

  I still double space after a period, because fuck you, it is easier to read. But as a bonus, it helped me prove that something I wrote wasn’t AI. You literally cannot get an AI to add double spaces after a period. It will say “Yeah, OK, I can do that” and then spit out a paragraph without it. Give it a try, it’s pretty funny.

  source

  • dual_sport_dork@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    Web browsers collapse whitespace by default which means that sans any trickery or   deliberately   using    nonbreaking    spaces   causes any amount of spaces between words to be reduced into one. Since apparently every single thing in the modern world is displayed via some kind of encapsulated little browser engine nowadays, the majority of double spaces left in the universe that are not already firmly nailed down into print now appear as singles. And thus the convention is almost totally lost.

    source

    • Redjard@lemmy.dbzer0.com ⁨3⁩ ⁨weeks⁩ ago

      This seems to match up with some quick tests I did just now, on the pseudonyminized chatbot interface of duckduckgo:

      tests

      Image Image

      source

      • SGforce@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago

        Tokenization can make it difficult for them.

        Image

        The word chunks often contain a space because it’s efficient. I would think an extra space would stand out. Writing it back should be easier, assuming there is a dedicated “space” token like other punctuation tokens, there must be.

        Hard mode would be asking it how many spaces there are in your sentence. I don’t think they’d figure it out unless their own list of tokens and a description is trained into them specifically.

        source
  • Karyoplasma@discuss.tchncs.de ⁨3⁩ ⁨weeks⁩ ago

    You can force the double spaces.  Like this.

    source
  • thesystemisdown@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    Double spaces after periods can create “rivers.” This makes text more difficult to read for those with dyslexia. Whatever is used as a text editor is probably stripping them out for accessibility reasons. I suppose double spaces made sense with monospaced fonts.

    apastyle.apa.org/…/typography#myth4

    source

    • FishFace@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      HTML rendering collapses whitespace; it has nothing to do with accessibility. I would like to see the research on double-spacing causing rivers, because I’ve only ever noticed them in justified text where I would expect the renderer to be inserting extra space after a full stop compared between words within sentence anyway.

      I’ve seen a lot of dubious legibility claims when it comes to typography including:

      1. serif is more legible
      2. sans-serif is more legible
      3. comic sans is more legible for people with dyslexia

      and so on.

      source
• CodeInvasion@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

  This is because spaces typically are encoded by model tokenizers.

  In many cases it would be redundant to show spaces, so tokenizers collapse them down to no spaces at all. Instead the model reads tokens as if the spaces never existed.

  For example it might output: thequickbrownfoxjumpsoverthelazydog

  Except it would actually be a list of numbers like: [1, 256, 6273, 7836, 1922, 2244, 3245, 256, 6734, 1176, 2]

  Then the tokenizer decodes this and adds the spaces because they are assumed to be there. The tokenizer has no knowledge of your request, and the model output typically does not include spaces, hencr your output sentence will not have double spaces.

  source

  • Redjard@lemmy.dbzer0.com ⁨3⁩ ⁨weeks⁩ ago

    I’d expect tokenizers to include spaces in tokens. You get words constructed from multiple tokens, so can’t really insert spaces based on them. And too much information doesn’t work well when spaces are stripped.

    In my tests plenty of llms are also capable of seeing and using double spaces when accessed with the right interface.

    source

    • CodeInvasion@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

      The tokenizer is capable of decoding spaceless tokens into compound words following a set of rules referred to as a grammar in Natural Language Processing (NLP). I do LLM research and have spent an uncomfortable amount of time staring at the encoded outputs of most tokenizers when debugging. Normally spaces are not included.

      There is of course a token for spaces in special circumstances, but I don’t know exactly how each tokenizer implements those spaces. So it does make sense that some models would be capable of the behavior you find in your tests, but that appears to be an emergent behavior, which is very interesting to see it work successfully.

      I intended for my original comment to convey the idea that it’s not surprising that LLMs might fail at following the instructions to include spaces since it normally doesn’t see spaces except in special circumstances. Similar to how it’s unsurprising that LLMs are bad at numerical operations because of how the use Markov Chain probability to each next token, one at a time.

      source

      • Redjard@lemmy.dbzer0.com ⁨3⁩ ⁨weeks⁩ ago

        Yeah, I would expect it to be hard, similar to asking an llm to substitiute all letters e with an a. Which I’m sure they struggle with but manage to perform it too.

        In this context though it’s a bit misleading explaining the observed behavior of op with that though, since it implies it is due to that fundamental nature of llms when in practice all models I have tested fundamentally had the ability.

        It does seem that llms simply don’t use double spaces (or I have not noticed them doing it anywhere yet), but if you trained or just systemprompted them differently they could easily start to. So it isn’t a very stable method for non-ai identification.

        source
• 4am@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago

  LLMs can’t count because they’re not brains. Their output is the statistically most-likely next character, and since lot electronic text wasn’t double-spaced after a period, it can’t follow that instruction.

  source