Discussion 2: Text Representation and Tokenization

Contents

Discussion 2: Text Representation and Tokenization#

Session Plan#

  1. Housekeeping — How is HW0 going? Questions or issues?

  2. Exam Prep Material — New Exam Preparation page added to the textbook with study guidance and topic outlines

  3. HW1 Released — Homework 1: Lopez-Lira, Tang (2023) is now available

  4. Structure of News — Exploring how financial news is structured and what makes it useful for NLP

  5. Words to Tokens — From raw text to model input: tokenization in practice

  6. Embedding Analogies — Hands-on exploration of word embeddings and analogies

  7. HW1 Review: Headline Coverage and Model Performance — When you don’t have enough headlines, strategy performance suffers. We’ll look at how to diagnose this by running a local model on both a smaller sample and the fuller sample and comparing results. See the Replication Results notebook for the full comparison.

Topics#

  • Structure of Financial News — What makes news text useful for quantitative analysis and how it is typically structured

  • AI Copilots Intro — Getting started with Claude Code and Cursor for AI-assisted development

  • Words to Tokens — How tokenizers convert text into the numerical representations that models consume

  • Embedding Analogies — Exploring semantic relationships captured by word embeddings

Contents