How to find limitations in a research paper

The limitations section is a starting list, written by people who still want you to believe the paper. Your job is to find the constraints that actually bound the claim — including the ones given a polite sentence.

Read their list, then ignore the tone

Authors often admit the safe limitations: small sample, single site, self-report. Note them. Then look at whether the headline still talks as if those limits were minor. A paper that “could not randomize” and still says “X causes Y” has a rhetoric problem, not just a sample-size problem.

Five places limitations hide

  1. Eligibility. Who never entered. A heart-failure trial that excludes low ejection fraction is answering a different question than the title.
  2. Missing comparison. No baseline, a weak baseline, or a baseline from a different year. The result can be true and still uninformative.
  3. Measurement. Proxy outcomes (clicks, grades, biomarkers) standing in for the thing you care about (learning, recovery, understanding).
  4. Time and setting. One hospital, one model version, one semester. Generalization is a claim; it needs a sentence.
  5. Analyst degrees of freedom. Many measures, one starred figure. If the primary outcome was not named up front, you are looking at a highlight reel.

A short search pattern

In the PDF, search for “limit,” “cannot,” “may not,” “caution,” “small,” “selected,” “self.” Read those hits in context. Then search for the outcome name and see whether a secondary measure is doing the rhetorical work in the discussion.

Claims that outrun the table

Put the abstract’s last two sentences next to the results table. If the abstract uses “improves” and the table shows a tiny, noisy difference on a surrogate, the limitation is not hidden — it is in the verb. Write the verb you would allow: “associated with,” “in this sample,” “on this benchmark.”

Do not ask a model “what are the limitations of this paper?” and paste the answer into a review. Ask “where do the authors discuss generalizability?” and then open that page. PDF Lens Paper Chat can find the neighborhood; you still have to judge the gap.

What “enough” looks like

You can name: the claim as written, one limitation the authors listed, one they underplayed, and whether you would still cite the paper for your question. That is a complete reading of the risk, not a complete demolition.

Understand methodology · Compare papers · All guides · AI for research papers