No, even in the absolute best case scenario, the LLM analysis is a trailing indicator. There’s no way that it indicates current views, just possibly an indication of past views.
Personally I think this entire line of thinking (“silicon sampling”) is dangerous af.
Yeah, I’m not saying a tool akin to LLMs can’t be used as part of a suite of software workflows for parsing through and analyzing large datasets (seems rather obvious to say that), but forgoing the real work of live data gathering and statistics evaluation in order to do a sort of “vibe polling” sounds extremely off to me.
I agree, which is why I find the results they got interesting, the fact that the initial study was able to, arguably quite correctly (well, debatable if it was correct, as I pointed out their results are not the easiest to evaluate), predict real results is pretty impressive.
No, even in the absolute best case scenario, the LLM analysis is a trailing indicator. There’s no way that it indicates current views, just possibly an indication of past views.
Personally I think this entire line of thinking (“silicon sampling”) is dangerous af.
That’s a good point, although I imagine a dedicated company could refine a model using more recently sampled general data to improve the recency.
Yeah, I’m not saying a tool akin to LLMs can’t be used as part of a suite of software workflows for parsing through and analyzing large datasets (seems rather obvious to say that), but forgoing the real work of live data gathering and statistics evaluation in order to do a sort of “vibe polling” sounds extremely off to me.
I agree, which is why I find the results they got interesting, the fact that the initial study was able to, arguably quite correctly (well, debatable if it was correct, as I pointed out their results are not the easiest to evaluate), predict real results is pretty impressive.