Data Engineering

Last time, we talked about preparing the data for our question generating system. To recall, this was the rough plan: Fetch and prepare Wikipedia data (see more here) Prompt an LLM for a question based on passed-in Wikipedia article (the focus of today’s post) There’s a wide range of different LLM services available online, but for our purposes we’re going to use something that’s open-source and that we can run locally on our machine - ollama....

I’ve always loved quizzes and trivia, and, when I was a kid, one of my favorite games was MindMaze. For those of you who aren’t familiar, MindMaze was a trivia game that was published as a part of Microsoft’s Encarta, a digital multimedia encyclopedia. Ever since Wikipedia launched though, encyclopedias like Encarta became less popular, since they were published in fixed, yearly iterations, were published on CDs, and, well, they cost money....