Our company rolled out a new and innovative internal LLM. It’s intended to help with coding tasks and find internal documentation.

(Which really is just a wrapper over copilot).

When looking form-documentation, it fails harder than pasting the same text into the sesrch bar.

I don’t find LLMs helpful for coding since they’re wrong so often. But after encouragement I decided to try my hand at using it to help me debug a small 12 line bash script.

Whenever I posted examples it would fail silently. I assumed it wasnt sanitizing inputs so I tried a few other methods of wrapping the text. It would still faik silently.

Eventually, a half hour later I decided to just Google it. After I resolved my own problem it messaged me, 45+ minutes later to say “an error occurred trying to handle your request”.

These things suck so bad they can’t even error out effectively.