LLMs reward expertise(seangoedecke.com)
1383 points by MaxMussio 71 days ago | 564 comments
tl;dr: Domain expertise dramatically improves LLM output quality: Terence Tao's ChatGPT conversation about the Jacobian Conjecture shows how an expert can steer the model into concise, technical mode, push back on wrong answers, and extract insights a novice couldn't. The key skill isn't prompt engineering tricks but actually understanding the subject well enough to identify weird outputs, suggest alternatives, and ask specific questions. This suggests humans remain the bottleneck for many tasks—the knowledge is already in the model, but it takes expertise to pull it out.
HN Discussion:
  • ↑LLMs amplify the user's existing knowledge and expertise like a mirror
  • ↑Signalling expertise in prompts meaningfully improves LLM output quality
  • ↑LLMs accelerate work but dangerously mislead those without domain knowledge
  • ↓Counter-examples show simple non-expert prompts also yield strong results
  • ~The claim matches experience but needs formal study to rule out bias