# Homework 4: Agentic RAG Trace Audit

- **GitHub Classroom Repository:** https://classroom.github.com/a/g-MJxi-2
- **Due Date:** Wednesday, May 13, 2026 at 9:30 AM (Chicago time)

To get started with this homework, accept the GitHub Classroom assignment and clone the repository found above. **All detailed instructions, deliverables, and grading criteria are in the homework repository README.**

This assignment uses Matt Stockton's guest-speaker repository, [agentic-rag-edgar-demo](https://github.com/MattStockton/agentic-rag-edgar-demo), to study **agentic retrieval, tool choice, and trace evaluation** on SEC 10-K filings. Where HW3 measured whether RAG improved factual extraction against Compustat ground truth, HW4 asks a different question: when an LLM is allowed to choose retrieval tools at runtime, what does the trace show about the strategy it used, the tools it needed, and the places where simpler methods win?

The deliverable is short: one trace inventory, one evidence-based audit of a single phase, one small patch with a before/after trace comparison, and a brief reflection plus a golden-set entry. Grading is manual — there is no autograded check on push, so review the "What To Submit" section in the homework README yourself before the deadline.

## Prerequisites

- Python 3.11+ (Matt's repo pins 3.12)
- An LLM API key (OpenAI by default; OpenRouter or Anthropic acceptable with a documented setup-time provider/model swap applied uniformly across all four phase scripts)
- An SEC identity string for EDGAR requests
- ~700 MB of local disk for cached filings, the `BAAI/bge-base-en-v1.5` embedding model, chunks, and embeddings

See the homework README for setup, the four phase scripts, the trace-helpers CLI, and the full submission checklist.