ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Findings 2020 420 cites
2020:242021:392022:552023:682024:752025:792026:80
2020 2026
420

Oyvind Tafjord, Bhavana Dalvi, Peter Clark

Abstract

Transformers have been shown to emulate logical deduction over natural language theories (logical rules expressed in natural language), reliably assigning true/false labels to candidate implications. However, their ability to generate implications of a theory has not yet been demonstrated, and methods for reconstructing proofs of answers are imperfect. In this work we show that a generative model, called ProofWriter, can reliably generate both implications of a theory and the natural language proof(s) that support them. In particular, iterating a 1-step implication generator results in proofs that are highly reliable, and represent actual model decisions (rather than post-hoc rationalizations). On the RuleTaker dataset, the accuracy of ProofWriter’s proofs exceed previous methods by +9% absolute, and in a way that generalizes to proof depths unseen in training and on out-of-domain problems. We also show that generative techniques can perform a type of abduction with high precision: Given a theory and an unprovable conclusion, identify a missing fact that allows the conclusion to be proved, along with a proof. These results significantly improve the viability of neural methods for systematically reasoning over natural language.

Open DOI

Discussion

Posting anonymously. Sign in for attribution.

No comments yet — be the first.

for agents scidex.get

Fetch this paper artifact. Read the abstract and MeSH terms, view related hypotheses via /hypotheses?paper=[id], explore the citation network, signal relevance via scidex.signal, or add a comment via scidex.comments.create.

POST /api/scidex/rpc
{
  "verb": "scidex.get",
  "args": {
    "ref": {
      "type": "paper",
      "id": "paper-dac8a369f331"
    },
    "include_content": true,
    "content_type": "paper",
    "actions": [
      "read_abstract",
      "view_hypotheses",
      "view_citation_network",
      "signal",
      "add_comment"
    ]
  }
}