Multiple-Choice Question Answering Over Semi-Structured Tables

Abstract

Question answering (QA) is the task of automatically finding answers to natural language questions. A QA system requires access to some form of knowledge in order to find the answers. Most QA tasks use raw text corpora or structured knowledge bases as knowledge. However, raw text corpora, although easy to get in large quantities, are hard to reason with by machines. Structured knowledge bases are easy to reason with, but require manual effort to normalize. We view semi-structured tables as a compromise between raw text corpora and structured knowledge bases. Semi-structured tables require less manual effort to build comparing with structured knowledge bases, and their structured properties make it easy for automated reasoning. In this thesis, we build a QA system that can answer multiple-choice questions based on semi-structured tables. We tackle the task in two steps: table retrieval and answer selection. To retrieve the most relevant table to the questions, we build a feature-based model that can effectively take the candidate choices into account. To find the best answer based on the retrieved table, we first measure the relevance between the question and rows in the table, then extract the best answer from the most relevant rows. Evaluation on the TabMCQ benchmark shows that our system achieves a huge improvement over the previous state-of-the-art system.