teaching
SURVMETH 622: Text Analysis and Large Language Models (LLMs)
Course Overview
As a guest lecturer for the University of Michigan’s SURVMETH 622, I teach sessions on Text Analysis and Large Language Models (LLMs). This course covers foundational and advanced topics in computational text analysis, equipping students with the skills to analyze unstructured text data through quantitative methods.
My lecture covers:
Word Representation Techniques: From traditional Bag of Words models to advanced word embeddings like Word2Vec, we explore how words can be numerically represented for analysis. Students learn to appreciate the nuances of vector space models, including Term Frequency-Inverse Document Frequency (TF-IDF) and the limitations of static representations.
Large Language Models (LLMs): We dive into the architecture and applications of LLMs such as BERT, GPT, and recent advancements in transformer models. I guide students through the principles of attention mechanisms, bidirectional processing, and how LLMs excel at capturing context and meaning in text.
Application: Stance Detection: We explore real-world applications of text analysis by focusing on stance detection—analyzing opinion, position, or perspective in text. Using social media as an example, I demonstrate how stance detection offers deeper insights than traditional sentiment analysis, revealing attitudes toward complex topics like public policy and social issues.
Key Topics and Techniques
Bag of Words and Document-Term Matrix: Introduction to simple word frequency models, highlighting their strengths and limitations in capturing semantic meaning.
Word2Vec and Word Embeddings: Examination of the Continuous Bag-of-Words and Skip-gram models, along with advanced techniques such as subsampling and negative sampling.
Transformer Architectures and Self-Attention: Explanation of transformer-based models (BERT, GPT) and their mechanisms for contextual understanding, long-distance dependencies, and bidirectional encoding.
LLMs in Text Analysis: Practical usage of LLMs in tasks like classification, summarization, and stance detection, with an emphasis on the role of prompt engineering and the emerging importance of few-shot and zero-shot learning.