Professor of Political Science, and Chair of the Political Science Concentration for Division of Computational and Data Sciences, Jacob Montgomery, has co-authored a new article in the journal Political Science Research and Methods. The article, written in collaboration with former WashU post-doc and current faculty member at Ohio State University, Ju Yeon (Julia) Park, is titled, "Toward a framework for creating trustworthy measures with supervised machine learning for text."
The article looks at supervised learning and the inconsistencies in reporting practices and validation standards, and introduces a new framework to address those inconsistencies.
Read the abstract below and the full article on the journal's website.
Abstract:
Supervised learning is increasingly used in social science research to quantify abstract concepts in textual data. However, a review of recent studies reveals inconsistencies in reporting practices and validation standards. To address this issue, we propose a framework that systematically outlines the process of transforming text into a quantitative measure, emphasizing key reporting decisions at each stage. Clear and comprehensive validation is crucial, enabling readers to critically evaluate both the methodology and the resulting measure. To illustrate our framework, we develop and validate a measure assessing the tone of questions posed to nominees during U.S. Senate confirmation hearings. This study contributes to the growing literature advocating for transparency and rigor in applying machine learning methods within computational social sciences.
