Skip to content

Classics with RAG: Analyzing The Master and Margarita

  • #Projects
Read time: 53 minutes

This is my first experience working with large language models (LLMs) and Retrieval-Augmented Generation (RAG) techniques. I’m excited to explore how these technologies can be applied to literary analysis, starting with the iconic novel The Master and Margarita by Mikhail Bulgakov.

When I started this project, my plan was to experiment with simple prompt sets to see how well the model could handle straightforward questions about the text. I also aimed to dive deeper by challenging the model with more complex queries to test its understanding of the novel's themes and characters. My goal was to gain hands-on experience with LLMs and RAG, and to demonstrate their potential in analyzing and interpreting classical literature.

Through this project, I hoped to uncover how these advanced technologies can enhance our understanding of literary works and explore new ways of engaging with classic texts.

Let's go!

In this code, I begin by setting up the environment for my project, including importing necessary libraries and configuring API keys. The script imports modules from LangChain to handle document loading, text splitting, and vector storage. It also sets the OpenAI API key for using OpenAI’s services. The load_documents function is used to load markdown files from a specified directory, which will then be processed for analysis. This setup prepares the data for further indexing and retrieval.

from langchain_community.document_loaders import DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain.prompts import ChatPromptTemplate
from langchain_openai import OpenAIEmbeddings
from langchain.evaluation import load_evaluator
from langchain_openai import ChatOpenAI
import os

os.environ['OPENAI_API_KEY'] = 'key'

DATA_PATH = 'data'
CHROMA_PATH = 'chroma'

def load_documents():
    loader = DirectoryLoader(DATA_PATH, glob='*.md')
    documents = loader.load()
    return documents
documents = load_documents()

In the code below, I utilize the RecursiveCharacterTextSplitter from LangChain to divide the loaded documents into manageable chunks. The text splitter is configured to create chunks of 1000 characters with a 300-character overlap, ensuring that relevant context is preserved between chunks. This process enhances the ability to analyze and retrieve specific parts of the text efficiently.

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=300,
    length_function=len,
    add_start_index=True,
)
chunks = text_splitter.split_documents(documents)
print(f'Split {len(documents)} documents into {len(chunks)} chunks.')

The output:

Split 1 documents into 1417 chunks.

Let's take a look at one of the chunks:

document = chunks[713]
print(document.page_content)
print(document.metadata)

Below is the output:

'Sure nobody knows,' the same trashy voice came from the study. The
binomial theorem, you might think! He's going to die in nine months,
next February, of liver cancer, in the clinic of the First Moscow State
University, in ward number four.'

The barman's face turned yellow.

'Nine months ...' Woland calculated pensively. 'Two hundred and
forty-nine thousand ... rounding it off that comes to twenty-seven
thousand a month ... Not a lot, but enough for a modest life ... Plus
those gold pieces ...'

'He won't get to realize the gold pieces,' the same voice mixed in,
turning the barman's heart to ice. 'On Andrei Fokich's demise, the house
will immediately be torn down, and the gold will be sent to the State
Bank.'
{'source': 'data\\Master and Margarita Bulgakov.md', 'start_index': 497052}

In the next code, I create a new vector database from the processed document chunks using the Chroma class from LangChain. This involves initializing the database with the document chunks and using OpenAI embeddings to convert the text into vector representations. The persist_directory parameter specifies where to store the database, enabling efficient storage and retrieval of the text data. This setup allows for scalable and fast retrieval of information for further analysis.

#Create a new DB from the documents.
db = Chroma.from_documents(
    chunks, OpenAIEmbeddings(), persist_directory=CHROMA_PATH
)
print(f"Saved {len(chunks)} chunks to {CHROMA_PATH}.")

The output:

Saved 1417 chunks to chroma3.

To better understand how text embeddings work and how semantic distances are calculated, I start with the word "apple." By examining how this term is represented in vector space, I can gain insight into the underlying mechanics of the embedding process. Specifically, I’ll compare the semantic distance between "apple" and two other words, "orange" and "iphone," to see how closely related or distinct these terms are based on their embeddings.

This initial test with the word "apple" helps illustrate how the embedding model differentiates between concepts and quantifies their semantic similarity. It serves as a foundation for more detailed analysis, allowing me to grasp the nuances of text representation and how these embeddings can be applied to a larger and more complex dataset, such as the novel The Master and Margarita.

embedding_function = OpenAIEmbeddings()
vector = embedding_function.embed_query('apple')
print(len(vector))

The output is below:

1536

Let's now examine how closely the words 'orange' and 'iPhone' are related to the word 'apple':

evaluator = load_evaluator('pairwise_embedding_distance')

#Run an avaluation
x = evaluator.evaluate_string_pairs(prediction='apple', prediction_b='orange')
print(x)
x1 = evaluator.evaluate_string_pairs(prediction='apple', prediction_b='iphone')
print(x1)

The output is below:

{'score': 0.1354698831743597}
{'score': 0.09710853291781452}

These scores indicate that, according to the embedding model, "apple" is somewhat closer in semantic space to "iphone" than to "orange." This suggests that the model perceives "apple" and "iphone" as more related, likely due to their shared context as consumer electronics, whereas "orange" is semantically more distant, being a different category altogether.

But let's get back to our novel!

In this code, I set up the vector database using Chroma and the OpenAIEmbeddings for converting text into vectors. The database is configured to use the directory where the vector data is stored. This setup allows me to efficiently manage and search the text data based on its embeddings, making it easier to perform further analysis.

# Prepare the DB.
embedding_function = OpenAIEmbeddings()
db = Chroma(persist_directory=CHROMA_PATH, embedding_function=embedding_function)

I prefer to strart from easy questions like this one:

query_text = 'Who are the main characters of the book The Master and Margarita?'

In the code below the similarity_search_with_relevance_scores method will find and rank the top 3 most relevant document chunks based on how closely they match the query. This helps in retrieving specific information related to the main characters of the novel, allowing for focused and relevant insights.

# Search the DB.
results = db.similarity_search_with_relevance_scores(query_text, k=3)
results

Here is the output:

[(Document(page_content="At his death, Bulgakov left [The Master and Margarita]{.italic} in a\nslightly unfinished state. It contains, for instance, certain\ninconsistencies --- two versions of the 'departure' of the master and\nMargarita, two versions of Yeshua's entry into Yershalaim, two names for\nYeshua's native town. His final revisions, undertaken in October of\n1939, broke off near the start of Book Two. Later he dictated some\nadditions to his wife, Elena Sergeevna, notably the opening paragraph of\nChapter 32 ('Gods, my gods! How sad the evening earth!'). Shortly after\nhis death in 1940, Elena Sergeevna made a new typescript of the novel.\nIn 1963, she prepared another typescript for publication, which differs\nslightly from her 1940 text. This 1963 text was published by\n[Moskva]{.italic} in November 1966 and January 1967. However, the\neditors of the magazine made cuts in it amounting to some sixty typed\npages. These cut portions immediately appeared in [samizdat]{.italic}", metadata={'source': 'data\\Master and Margarita Bulgakov.md', 'start_index': 40234}),
  0.8085636614137585),
 (Document(page_content='[Penguin Books Ltd, Registered Offices: 80 Strand, London WC2R oRL,\nEngland]{.calibre15}\n\n[First published as ]{.calibre15}[[Master i\nMargarita]{.italic}]{.calibre15}[ in serial form in\n]{.calibre15}[[Moskva,]{.italic}]{.calibre15}[ 1966-7 ]{.calibre15}\\\n[This translation published in Penguin Books 1997]{.calibre15}\n\n\\\n\\\n\\\n\n[Text copyright © Mikhail Bulgakov, 1966, 1967]{.calibre15}\n\n[Translation, Further Reading and Notes copyright © Richard Pevear and\n]{.calibre15}\\\n[Larissa Volokhonsky, 1997 ]{.calibre15}\\\n[Introduction copyright © Richard Pevear, 1997 ]{.calibre15}\\\n[All rights reserved]{.calibre15}\n\n[eISBN : 978-1-440-67408-2]{.calibre15}\n\n[Set in 10/12pt Monotype Garamond ]{.calibre15}\\', metadata={'source': 'data\\Master and Margarita Bulgakov.md', 'start_index': 9524}),
  0.7986563492482723),
 (Document(page_content='[Chapter 29: The Fate of the Master and Margarita is Decided]{.italic}\n\n[]{.calibre11}[]{#The_Master_and_Margarita_split_045.html#filepos1170033\n.calibre11}\n\n[[1]{.calibre8\nstyle="text-decoration:underline"}]{.calibre7}{.calibre6}\n[Resting his sharp chin on his fist]{.italic} ... [Woland stand\nfixedly:]{.italic} Woland seems almost consciously to adopt the pose of\nRodin\'s famous sculpture known as the [Thinker,]{.italic} actually the\ncentral figure over his [Gates of Hell.]{.italic}\n\n[]{.calibre11}[]{#The_Master_and_Margarita_split_045.html#filepos1170386\n.calibre11}\n\n[[2]{.calibre8\nstyle="text-decoration:underline"}]{.calibre7}{.calibre6}\n[to Timiriazev]{.italic}: That is, to the statue of the botanist and\nfounder of the Russian school of plant physiology, Kliment Arkadyevich\nTimiriazev (1843 - 1910), on Tverskoy Boulevard near the Nikitsky Gates.\n\n[Chapter 30: It\'s Time! It\'s Time!]{.italic}\n\n[]{.calibre11}[]{#The_Master_and_Margarita_split_045.html#filepos1170802\n.calibre11}', metadata={'source': 'data\\Master and Margarita Bulgakov.md', 'start_index': 982775}),
  0.7978212830562611)]

The output contains the top 3 results from the database search for the query "Who are the main characters of the book The Master and Margarita?". All the retrieved results have very similar scores, ranging from 0.798 to 0.809. This indicates that the system retrieved documents with nearly the same relevance score.

Despite having queried for the main characters, only the third document provides partial context that includes "the Master" and "Margarita" in the story. The other documents, though relevant to the book's publication history, do not provide the information needed to answer the query effectively. The scores of the documents are close in range, but the relevance varies greatly. While Document 1 has the highest score, it does not contribute much to answering the query directly.

I suppose for improvement, that metadata such as publication details, ISBNs, and copyright information should be ignored.

The next step ensures that only results with a high enough relevance score are considered, helping to maintain the quality of the retrieved information.

if len(results) == 0 or results[0][1] < 0.7:
    print(f'Unable to find matching results.')

In the next code, I create a simple but powerful tool called a prompt template. A prompt is a way to instruct the AI on how to answer a question using specific information.

Here's what this prompt template does:

Provides Context: It takes the relevant information from the database and includes it in the context section. Asks a Question: It then inserts the actual question that needs to be answered based on that context.

This setup helps the AI focus only on the provided context when answering, making sure the response is accurate and relevant. It’s a straightforward way to get precise answers from the AI, tailored to the information you have.

PROMPT_TEMPLATE = """
Answer the question based only on the following context:

{context}

---

Answer the question based on the above context: {question}
"""

In this code, I combine the context and question into a formatted prompt and send it to the AI model to generate a response. Here's a breakdown of what happens:

  • Create Context: The 'context_text' is built by joining the relevant text chunks (from the search results) with separators like "\n\n---\n\n" to maintain structure.
  • Format the Prompt: The prompt template is filled with the context_text and the query_text (the question being asked) using 'ChatPromptTemplate'. This creates the final prompt, which is what the AI will use to generate its response.
  • Invoke the Model: The 'ChatOpenAI' model is called with the formatted prompt, and it returns the answer based on the context provided.
  • Format the Response: Finally, the code formats the response, displaying both the AI's answer and the sources from which the information was pulled. This gives clear output showing the AI's reasoning and where the data came from.

This entire process ensures that the AI answers the question with accuracy.

context_text = "\n\n---\n\n".join([doc.page_content for doc, _score in results]) 
prompt_template = ChatPromptTemplate.from_template(PROMPT_TEMPLATE)
prompt = prompt_template.format(context=context_text, question=query_text)

print(prompt)

model = ChatOpenAI()
response_text = model.invoke(prompt)

sources = [doc.metadata.get("source", None) for doc, _score in results]
formatted_response = f"Response: {response_text}\nSources: {sources}"
print(formatted_response)

Here is the output:

Human: 
Answer the question based only on the following context:

At his death, Bulgakov left [The Master and Margarita]{.italic} in a
slightly unfinished state. It contains, for instance, certain
inconsistencies --- two versions of the 'departure' of the master and
Margarita, two versions of Yeshua's entry into Yershalaim, two names for
Yeshua's native town. His final revisions, undertaken in October of
1939, broke off near the start of Book Two. Later he dictated some
additions to his wife, Elena Sergeevna, notably the opening paragraph of
Chapter 32 ('Gods, my gods! How sad the evening earth!'). Shortly after
his death in 1940, Elena Sergeevna made a new typescript of the novel.
In 1963, she prepared another typescript for publication, which differs
slightly from her 1940 text. This 1963 text was published by
[Moskva]{.italic} in November 1966 and January 1967. However, the
editors of the magazine made cuts in it amounting to some sixty typed
pages. These cut portions immediately appeared in [samizdat]{.italic}

---

[Penguin Books Ltd, Registered Offices: 80 Strand, London WC2R oRL,
England]{.calibre15}

[First published as ]{.calibre15}[[Master i
Margarita]{.italic}]{.calibre15}[ in serial form in
]{.calibre15}[[Moskva,]{.italic}]{.calibre15}[ 1966-7 ]{.calibre15}\
[This translation published in Penguin Books 1997]{.calibre15}

\
\
\

[Text copyright © Mikhail Bulgakov, 1966, 1967]{.calibre15}

[Translation, Further Reading and Notes copyright © Richard Pevear and
]{.calibre15}\
[Larissa Volokhonsky, 1997 ]{.calibre15}\
[Introduction copyright © Richard Pevear, 1997 ]{.calibre15}\
[All rights reserved]{.calibre15}

[eISBN : 978-1-440-67408-2]{.calibre15}

[Set in 10/12pt Monotype Garamond ]{.calibre15}\

---

[Chapter 29: The Fate of the Master and Margarita is Decided]{.italic}

[]{.calibre11}[]{#The_Master_and_Margarita_split_045.html#filepos1170033
.calibre11}

[[1]{.calibre8
style="text-decoration:underline"}]{.calibre7}{.calibre6}
[Resting his sharp chin on his fist]{.italic} ... [Woland stand
fixedly:]{.italic} Woland seems almost consciously to adopt the pose of
Rodin's famous sculpture known as the [Thinker,]{.italic} actually the
central figure over his [Gates of Hell.]{.italic}

[]{.calibre11}[]{#The_Master_and_Margarita_split_045.html#filepos1170386
.calibre11}

[[2]{.calibre8
style="text-decoration:underline"}]{.calibre7}{.calibre6}
[to Timiriazev]{.italic}: That is, to the statue of the botanist and
founder of the Russian school of plant physiology, Kliment Arkadyevich
Timiriazev (1843 - 1910), on Tverskoy Boulevard near the Nikitsky Gates.

[Chapter 30: It's Time! It's Time!]{.italic}

[]{.calibre11}[]{#The_Master_and_Margarita_split_045.html#filepos1170802
.calibre11}

---

Answer the question based on the above context: Who are the main characters of the book The Master and Margarita?

Response: content='The main characters of the book "The Master and Margarita" are The Master and Margarita.' response_metadata={'token_usage': {'completion_tokens': 22, 'prompt_tokens': 859, 'total_tokens': 881}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None} id='run-871035cc-8ebd-4b00-bcdc-8a37f5f5aa75-0' usage_metadata={'input_tokens': 859, 'output_tokens': 22, 'total_tokens': 881}
Sources: ['data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md']

As I mentioned above, the context used to answer the question consists mainly of publishing and copyright information from the text rather than any meaningful content about the plot or characters of the novel.The AI correctly identifies the main characters of The Master and Margarita as "the Master and Margarita." While accurate, the response is somewhat minimal. The novel has many other significant characters, such as Woland, Azazello, Behemoth, and Pontius Pilate. A more complete answer could include these key figures to provide a fuller understanding of the novel's cast. The answer is technically correct but lacks depth.

The AI used 881 tokens in total. This suggests that a significant amount of text was processed as part of the context, but the generated answer was relatively short and concise.

The finish_reason being 'stop' indicates that the model completed the answer naturally, without being cut off.

Let's try more complicated questions.

query_text = 'How does Wolands presence affect the characters and events?'
results = db.similarity_search_with_relevance_scores(query_text, k=3)

context_text = "\n\n---\n\n".join([doc.page_content for doc, _score in results])
prompt_template = ChatPromptTemplate.from_template(PROMPT_TEMPLATE)
prompt = prompt_template.format(context=context_text, question=query_text)

print(prompt)

model = ChatOpenAI()
response_text = model.invoke(prompt)

sources = [doc.metadata.get("source", None) for doc, _score in results]
formatted_response = f"Response: {response_text}\nSources: {sources}"
print(formatted_response)

Here is the output:

Human: 
Answer the question based only on the following context:

Several years passed, and the citizens began to forget Woland, Koroviev
and the rest. Many changes took place in the lives of those who suffered
from Woland and his company, and however trifling and insignificant
those changes are, they still ought to be noted.

Georges Bengalsky, for instance, after spending three months in the
clinic, recovered and left it, but had to give up his work at the
Variety, and that at the hottest time, when the public was flocking
after tickets: the memory of black magic and its exposure proved very
tenacious. Bengalsky left the Variety, for he understood that to appear
every night before two thousand people, to be inevitably recognized and
endlessly subjected to jeering questions of how he liked it better, with
or without his head, was much too painful.

---

And, finally, Woland also flew in his true image. Margarita could not
have said what his horse's bridle was made of, but thought it might be
chains of moonlight, and the horse itself was a mass of darkness, and
the horse's mane a storm cloud, and the rider's spurs the white flecks
of stars.

Thus they flew in silence for a long time, until the place itself began
to change below them. The melancholy forests drowned in earthly darkness
and drew with them the dim blades of the rivers. Boulders appeared and
began to gleam below, with black gaps between them where the moonlight
did not penetrate.

---

Woland raised his sword. Straight away the flesh of the head turned dark
and shrivelled, then fell off in pieces, the eyes disappeared, and soon
Margarita saw on the platter a yellowish skull with emerald eyes, pearl
teeth and a golden foot. The lid opened on a hinge.

'Right this second, Messire,' said Koroviev, noticing Woland's
questioning look, 'hell appear before you. In this sepulchral silence I
can hear the creaking of his patent leather shoes and the clink of the
goblet he has just set down on the table, having drunk champagne for the
last time in his life. Here he is.'

---

Answer the question based on the above context: How does Wolands presence affect the characters and events?

Response: content="Woland's presence causes fear, discomfort, and changes in the lives of the characters. Georges Bengalsky had to give up his work at the Variety due to the memory of black magic and the exposure he experienced. Margarita is in awe of Woland's true image and the transformation of the landscape below them as they fly. Woland's actions, such as raising his sword and transforming a head into a skull, instill a sense of dread and anticipation in the characters. Overall, Woland's presence brings about significant emotional and physical impact on the characters and events in the story." response_metadata={'token_usage': {'completion_tokens': 119, 'prompt_tokens': 513, 'total_tokens': 632}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None} id='run-cc836402-cbdc-44ae-924e-447011ffa034-0' usage_metadata={'input_tokens': 513, 'output_tokens': 119, 'total_tokens': 632}
Sources: ['data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md']

Woland's presence in The Master and Margarita is a catalyst for dramatic changes, fear, and awe. His actions leave lasting impacts on the characters' lives and influence the events of the story in both subtle and overt ways. The retrieved context and AI-generated response effectively illustrate these effects, portraying Woland as a powerful and terrifying force. The AI's response reflects a clear understanding of the material provided.

query_text = 'What is the relationship between the Master and Margarita, and how does it drive the plot?'
results = db.similarity_search_with_relevance_scores(query_text, k=3)

context_text = "\n\n---\n\n".join([doc.page_content for doc, _score in results])
prompt_template = ChatPromptTemplate.from_template(PROMPT_TEMPLATE)
prompt = prompt_template.format(context=context_text, question=query_text)

print(prompt)

model = ChatOpenAI()
response_text = model.invoke(prompt)

sources = [doc.metadata.get("source", None) for doc, _score in results]
formatted_response = f"Response: {response_text}\nSources: {sources}"
print(formatted_response)

The output is below:

Human: 
Answer the question based only on the following context:

measure. In style and form it is a counterpoint to the rest of the book.
Finally, rather late in the process, the master and Margarita appear,
with Margarita coming to dominate the second part of the novel. Her
story is a romance in the old sense --- the celebration of a beautiful
woman, of a true love, and of personal courage.

---

The first typescript of [The Master]{.italic} and [Margarita,]{.italic}
dating to 1938, was dictated to the typist by Bulgakov from this last
revision, with many changes along the way. In 1939 he made further
alterations in the typescript, the most important of which concerns the
fate of the hero and heroine. In the last manuscript version, the fate
of the master and Margarita, announced to them by Woland, is to follow
Pilate up the path of moonlight to find Yeshua and peace. In the
typescript, the fate of the master, announced to Woland by Matthew Levi,
speaking for Yeshua, is not to follow Pilate but to go to his 'eternal
refuge' with Margarita, in a rather German-Romantic setting, with
Schubert's music and blossoming cherry trees. Asked by Woland, 'But why
don't you take him with you into the light?' Levi replies in a sorrowful
voice, 'He does not deserve the light, he deserves peace.' Bulgakov,
still pondering the problem of the master's guilt (and his own, for what

---

In the evolution of [The Master and Margárita]{.italic}, the Moscow
satire of Woland and his retinue versus the literary powers and the
imposed normality of Soviet life in general is there from the first, and
comes to involve the master when he appears, acquiring details from the
writer's own life and with them a more personal tone alongside the
bantering irreverence of the demonic retinue. The Pilate story, on the
other hand, the story of an act of cowardice and an interrupted
dialogue, gains in weight and independence as Bulgakov's work
progresses. From a single inset episode, it becomes the centrepiece of
the novel, setting off the contemporary events and serving as their
measure. In style and form it is a counterpoint to the rest of the book.
Finally, rather late in the process, the master and Margarita appear,
with Margarita coming to dominate the second part of the novel. Her
story is a romance in the old sense --- the celebration of a beautiful

---

Answer the question based on the above context: What is the relationship between the Master and Margarita, and how does it drive the plot?

Response: content='The relationship between the Master and Margarita is a central focus of the novel and drives the plot forward. Margarita dominates the second part of the novel, and her story is described as a romance in the old sense, celebrating a beautiful woman, true love, and personal courage. The fate of the Master and Margarita is intertwined, with different versions of their fate being explored in the manuscript. Ultimately, their relationship and their journey together, whether it be towards finding peace or following Pilate up the path of moonlight, shape the narrative and themes of the novel. Their love story adds depth and emotion to the overall plot of the book.' response_metadata={'token_usage': {'completion_tokens': 132, 'prompt_tokens': 609, 'total_tokens': 741}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None} id='run-786a8f89-39b2-4c1e-bac5-ebd83ae26ad6-0' usage_metadata={'input_tokens': 609, 'output_tokens': 132, 'total_tokens': 741}
Sources: ['data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md']

The AI's response is largely based on introductory for the novel, not directly from the narrative itself. But the AI's response succinctly captures these elements based on the retrieved context.

query_text = 'What literary devices does Bulgakov use in The Master and Margarita to enhance the story?'
results = db.similarity_search_with_relevance_scores(query_text, k=3)

context_text = "\n\n---\n\n".join([doc.page_content for doc, _score in results])
prompt_template = ChatPromptTemplate.from_template(PROMPT_TEMPLATE)
prompt = prompt_template.format(context=context_text, question=query_text)

print(prompt)

model = ChatOpenAI()
response_text = model.invoke(prompt)

sources = [doc.metadata.get("source", None) for doc, _score in results]
formatted_response = f"Response: {response_text}\nSources: {sources}"
print(formatted_response)

Here is the output:

Human: 
Answer the question based only on the following context:

several pages the sensation of flight on a broomstick or the gathering
of the infamous dead at Satan's annual spring ball, can combine the most
acute sense of the fragility of human life with confidence in its
indestructibility. Bulgakov underscores the continuity of this verbal
world by having certain phrases --- 'Oh, gods, my gods', 'Bring me
poison', 'Even by moonlight I have no peace' - migrate from one
character to another, or to the narrator. A more conspicuous case is the
Pilate story itself, successive parts of which are told by Woland,
dreamed by the poet Homeless, written by the master, and read by
Margarita, while the whole preserves its stylistic unity. Narrow notions
of the 'imitation of reality' break down here. But [The Master and
Margarita]{.italic} is true to the broader sense of the novel as a
freely developing form embodied in the works of Dostoevsky and Gogol, of
Swift and Sterne, of Cervantes, Rabelais and Apuleius. The mobile but

---

These three stories, in form as well as content, embrace virtually all
that was excluded from official Soviet ideology and its literature. But
if the confines of 'socialist realism' are utterly exploded, so are the
confines of more traditional novelistic realism. [The Master and
Margarita]{.italic} as a whole is a consistently free verbal
construction which, true to its own premises, can re-create ancient
Jerusalem in the smallest physical detail, but can also alter the
specifics of the New Testament and play variations on its principal
figures, can combine the realities of Moscow life with witchcraft,
vampirism, the tearing off and replacing of heads, can describe for
several pages the sensation of flight on a broomstick or the gathering
of the infamous dead at Satan's annual spring ball, can combine the most
acute sense of the fragility of human life with confidence in its
indestructibility. Bulgakov underscores the continuity of this verbal

---

Bulgakov was known well enough, then. But, outside a very small group,
the existence of [The Master and Margarita]{.italic} was completely
unsuspected. That certainly accounts for some of the amazement caused by
its publication. It was thought that virtually all of Bulgakov had found
its way into print. And here was not some minor literary remains but a
major novel, the author's crowning work. Then there were the qualities
of the novel itself --- its formal originality, its devastating satire
of Soviet life, and of Soviet literary life in particular, its
'theatrical' rendering of the Great Terror of the thirties, the audacity
of its portrayal of Jesus Christ and Pontius Pilate, not to mention
Satan. But, above all, the novel breathed an air of freedom, artistic
and spiritual, which had become rare indeed, not only in Soviet Russia.
We sense it in the special tone of Bulgakov's writing, a combination of
laughter (satire, caricature, buffoonery) and the most unguarded

---

Answer the question based on the above context: What literary devices does Bulgakov use in The Master and Margarita to enhance the story?

Response: content='Bulgakov uses various literary devices in The Master and Margarita to enhance the story, including the migration of certain phrases from character to character or to the narrator, the use of multiple perspectives to tell the Pilate story, stylistic unity, formal originality, satire, caricature, buffoonery, and a combination of laughter and unguarded freedom in his writing. These devices help to create a unique and richly textured narrative that defies traditional novelistic realism and explores themes of human fragility and indestructibility, freedom, and the boundaries of Soviet ideology and literature.' response_metadata={'token_usage': {'completion_tokens': 118, 'prompt_tokens': 725, 'total_tokens': 843}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None} id='run-e007d9de-576e-426b-8d09-948dd71b2251-0' usage_metadata={'input_tokens': 725, 'output_tokens': 118, 'total_tokens': 843}
Sources: ['data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md', 'data\\Master and Margarita Bulgakov.md']

Actually I am impressed that for a such complicated question the response accurately captures the essence of how Bulgakov employs literary devices to enhance the story. It identifies key devices and explains their impact on the narrative, reflecting the broader themes and stylistic choices described in the context.

Conclusion

Working on this project was a great experience, and I really enjoyed exploring how an AI can answer questions based on embedded documents. Using "The Master and Margarita" as my test text, I was impressed by how well the model handled both simple and more complex questions. It was amazing to see it tackle complicated literary themes and concepts. However, there were some mistakes in the vector contexts, which sometimes led to incomplete or less detailed answers. In the future, I would like to experiment with improving the quality of the context and try different approaches, such as fine-tuning the model or using larger and more varied datasets.

One thing that could be improved in the future is adjusting the chunk size and adding overlaps. Some of the context chunks were too large or disconnected, which might have caused the AI to miss important details. By refining the chunk size and overlaps, I believe the AI could better capture the nuances of the text, leading to more accurate and detailed answers.

This project showed me the potential of AI in literary analysis, and I’m excited to keep exploring new ways to enhance its accuracy and depth in answering challenging questions. You can find the full project on my GitHub here.

© 2025

Elena Medvedeva. Created by Elena Aseeva. Some assets are created by freepik.com