Oh yeah, same here except with a self-hosted LLM. I had a log file with thousands of warnings and errors coming from several components. Major refactor of a codebase in the cleanup phase. I wanted to have those sorted by severity, component, and exception (if present). Nothing fancy.

So, hoping I could get a quick solution, I passed it to the LLM. It returned an error. Turns out that a 14 megabyte text file exceeds the context size. That server with several datacenter GPUs sure looks like a great investment now.

So I just threw together a script that applied a few regexes. That worked, no surprise.