Introduction Engaging quizzes with well-written questions and distractors are one of the most effective ways to help people learn and retain knowledge. However, manually creating a large volume of high-quality quiz questions is a tedious and time-consuming process, even for subject matter experts. Recent advancements in large language models (LLMs) open up an exciting possibility: what if we could automatically generate complete quizzes, including realistic incorrect answer choices, simply by ingesting a corpus of training material into an LLM?

Previously, we saw the various components required to build a RAG LLM system. In that article, we discussed both how a RAG system comes together and the most common services available to fill each component of that system. In this article, we’ll focus solely on using Databricks to build out RAG. Why Databricks? We’re fans of the platform. We find it one of the most complete, powerful, and useful data platforms available in the cloud.

Large language models (LLMs) have introduced a whole new world of opportunities. Suddenly, the interface between machines and human communication has broken down, and ideas that were recently untenable have become almost trivially simple. It’s an exciting time, for sure. Real success, however, depends on wrapping this new technology with software that makes it truly useful. One pattern that has arisen to make that happen is Retrieval Augmented Generation (RAG).

A picture is worth a thousand words - Fred R. Barnard In the rapidly changing world of LLMs, chatbots, and RAGs (Retrieval-Augmented Generation), most of the focus has been on plain text data. However, a wealth of information is available only in image form. What does something look like? How do elements of a diagram fit together? How is the food pyramid laid out? How do two algorithm outputs vary in visual quality?