Vitaly Meursault

Can innovations in NLP help us study innovation?

Our paper compares the performance of four easy-to-implement patent similarity measures on two novel validation tasks. We consider cosine similarity measures based on TF-IDF, Doc2Vec, Universal Sentence Encoder and Sentence-BERT representations. The first validation task asks which measure can best identify whether a pair of patents are involved in a patent interference case, meaning the patent office has deemed them to have potentially identical overlapping claims. Sentence-BERT similarity performs best (by a large margin) at identifying interfering patent pairs. TF-IDF comes in second place, showing that more recent models do not always perform better. The second validation task involves human annotators determining which of the two candidate patents is more similar to a focal patent. Human annotators are much more likely to agree with Sentence-BERT than the TF-IDF-based similarity measure. Our paper provides evidence that new developments in NLP methods can improve innovation research. Our results also highlight the importance of subjecting these methods to domain-specific validation tasks. This has implications for future research that uses text-based patent similarity measures to study the creation and propagation of new technologies, as existing research tends to focus on a single text-based measure (typically cosine similarity of TF-IDF or Doc2Vec representations) with limited internal validation.

Bio: Vitaly Meursault uses machine learning to study questions in economics and finance. His current research involves text analysis of patents, corporate earnings calls and corporate loan documents. He also does work at the intersection of algorithmic fairness and credit scoring.