Is what you’re doing taking a large amount of text and asking the LLM to convert it into a smaller amount of text? Then it’s probably going to be great at it. If you’re asking it to convert into a roughly equal amount of text it will be so-so. If you’re asking it to create more text than you gave it, forget about it.
Saw this via Simon Willison. It very much rings true, but I think it does rub against AI’s genuine uses as a cliche machine — “you fill out the expected details” is a helpful starting point for many tasks, and a helpful endpoint for at least some, particularly if you really do want the boilerplate, or if you’ve added a sufficiently improv-grade prompt.
Apr 23, 2025