Comment on I'm gonna die on this hill or die trying
dual_sport_dork@lemmy.world 3 weeks agoWeb browsers collapse whitespace by default which means that sans any trickery or deliberately using nonbreaking spaces causes any amount of spaces between words to be reduced into one. Since apparently every single thing in the modern world is displayed via some kind of encapsulated little browser engine nowadays, the majority of double spaces left in the universe that are not already firmly nailed down into print now appear as singles. And thus the convention is almost totally lost.
Redjard@lemmy.dbzer0.com 3 weeks ago
This seems to match up with some quick tests I did just now, on the pseudonyminized chatbot interface of duckduckgo:
tests
Image Image
SGforce@lemmy.ca 3 weeks ago
Tokenization can make it difficult for them.
Image
The word chunks often contain a space because it’s efficient. I would think an extra space would stand out. Writing it back should be easier, assuming there is a dedicated “space” token like other punctuation tokens, there must be.
Hard mode would be asking it how many spaces there are in your sentence. I don’t think they’d figure it out unless their own list of tokens and a description is trained into them specifically.