Observations.

Observations.
Every week I get about 20 projects/products/teams coming in and telling me what they did/plan to do and so on. What I usually notice:
1/ Decks, all the materials, blurbs, emails are made through AI. You can straight up see that people didn’t read what they themselves made, decks are a hundred-fucking pages, vibe-coded frontends—you know, with that Claude Design aftertaste (that’s when you ask it to make it pretty please). And they all expect that someone will dig into this and study it and, what’s more, give them money?
2/ Most projects don’t know evals even exist. An agent to “plug in some thing” or we’ll improve your agent for “some thing.” But how to understand that the proposed approach is better than the basic “just feed everything to ChatGPT,” no clue. At least go look at benchmarks and scoring systems or something.
3/ Well even if there are zero evals and benches, then you should be paying for that. Otherwise it’s really hard to understand what to do and how to evaluate any of it.
4/ A ton of “slop,” meaning I spend 2–3 minutes and see factual/numerical errors in the materials. Like they write market size and two numbers multiplied give the wrong result. And there’s just a ton of that. No desire to read and talk to a person who didn’t even check what they wrote.
5/ Everyone asks for millions, and I don’t understand why. Everything that was done was done with a $200/month subscription. Why do you need millions?
6/ People don’t know the basics. Yeah yeah, the difference between a harness and a model, what metrics mean what, don’t know about recall etc. in search tasks. What is there even to talk about then?
It adds up to a strong feeling that they don’t even write the texts and answers to questions themselves. And that’s sad, and that’s exactly why it’s gotten harder to do anything if you don’t think for yourself, but if you’ve got your own head on your shoulders then now is the best time, of course!