AI & Generative AI

Generative AI — models that can write text, generate images, and hold conversations — is one of the fastest-growing areas in software today. This file covers the core concepts and practical Python patterns for working with Large Language Models (LLMs).

1. AI Fundamentals Recap

As covered in the Machine Learning file, Generative AI is a specific application area within AI, heavily built on Deep Learning. What makes it distinctive is that instead of just predicting a number or category, a generative model produces new content — text, images, audio — resembling the data it was trained on.

Definition: Generative AI refers to models capable of producing new content — text, images, audio, or code — based on patterns learned from vast amounts of training data.

2. What is an LLM (Large Language Model)?

What is it?

An LLM is a deep learning model trained on enormous amounts of text, learning to predict the next most likely word (or piece of a word) in a sequence — and, through this simple mechanism trained at massive scale, developing the ability to write coherently, answer questions, summarize, translate, and reason through problems.

Definition: A Large Language Model is a deep neural network, typically built on a "Transformer" architecture, trained on massive text datasets to understand and generate human language.

How does it work? (Conceptual Overview)

1. Text is broken into small pieces called tokens (see below).
2. The model processes these tokens using a "Transformer" architecture (a design built around a mechanism called attention, which lets the model weigh how relevant every other word in the input is to each word it's currently processing).
3. Based on everything it has seen so far, the model predicts the most likely next token.
4. This process repeats, one token at a time, to generate an entire response.

Important Points

• LLMs don't "look up" answers from a database — they generate text based on learned statistical patterns, which means they can occasionally produce confident-sounding but incorrect information (often called "hallucination").
• Modern LLMs are accessed primarily through APIs (covered below) rather than run on a personal computer, due to their massive size.

3. Tokens

What is it?

A token is a small chunk of text — sometimes a whole word, sometimes just part of one — that an LLM actually processes. LLMs don't read text character-by-character or strictly word-by-word; they work in these token-sized chunks.

Simple Example (Conceptual)

"Understanding tokenization" might break down into tokens like:
["Understand", "ing", " token", "ization"]

Explanation: Common words are often a single token, while rarer or longer words get split into smaller sub-word pieces — this lets the model handle virtually any word, even ones it has never seen before, by combining familiar smaller pieces.

Why Tokens Matter Practically

• Cost — most LLM APIs charge based on the number of tokens processed (both the input you send and the output generated).
• Limits — every model has a maximum "context window" (the total number of tokens it can consider at once, for both the conversation history and its response).

Important Points

• As a rough estimate for English text, 1 token is roughly ¾ of a word (or roughly 4 characters).
• Understanding tokens helps explain both API costs and why very long conversations may eventually need to be trimmed or summarized.

4. Prompt Engineering

What is it?

Prompt engineering is the practice of carefully crafting the instructions (the "prompt") given to an LLM to get better, more reliable, more relevant responses.

Definition: Prompt engineering is the practice of designing input text to guide a language model toward producing the most useful and accurate output.

Key Techniques

1. Be clear and specific:

Weak prompt:   "Write about dogs"
Better prompt: "Write a 200-word beginner's guide to choosing a dog breed for a first-time owner living in an apartment"

2. Provide examples (few-shot prompting):

Classify the sentiment of each review as Positive or Negative.

Review: "This product exceeded my expectations!"
Sentiment: Positive

Review: "Completely broke after one use."
Sentiment: Negative

Review: "Works okay, nothing special."
Sentiment:

Explanation: Showing the model 2 examples of the exact input/output pattern you want ("few-shot") dramatically improves consistency compared to just describing the task in words alone ("zero-shot").

3. Ask for step-by-step reasoning (chain-of-thought):

Solve this step by step: If a train travels 60 km in 1.5 hours, what is its average speed?

Explanation: Explicitly asking the model to reason "step by step" often improves accuracy on problems requiring multiple logical steps, compared to asking for just the final answer directly.

4. Use a system prompt to set persistent behavior:

System: You are a helpful Python tutor. Always explain concepts using simple beginner-friendly language and include a code example.
User: What is a list comprehension?

Important Points

• Clear, specific instructions consistently outperform vague ones.
• Providing examples (few-shot) is one of the most reliable ways to improve output quality and consistency.
• Prompt engineering is an iterative process — testing and refining prompts based on the actual outputs received.

Practice

1. Write a zero-shot prompt and a few-shot prompt for the same task (e.g., extracting a person's name and city from a sentence), and compare which feels more reliable.

5. Embeddings

What is it?

An embedding converts text (a word, sentence, or document) into a list of numbers (a vector) that captures its meaning — texts with similar meanings end up with mathematically similar vectors, even if they use completely different words.

Definition: An embedding is a numerical vector representation of text that captures its semantic meaning, allowing similarity to be measured mathematically.

Simple Conceptual Example

python
# Conceptual illustration - actual embeddings have hundreds of dimensions
embedding_1 = get_embedding("The cat sat on the mat")
embedding_2 = get_embedding("A feline rested on the rug")
embedding_3 = get_embedding("Stock markets fell sharply today")

# embedding_1 and embedding_2 would be mathematically VERY similar
# (despite sharing almost no exact words), because their MEANING is similar

# embedding_3 would be mathematically very DIFFERENT from the other two

Measuring Similarity — Cosine Similarity

python
import numpy as np

def cosine_similarity(vec1, vec2):
    dot_product = np.dot(vec1, vec2)
    norm1 = np.linalg.norm(vec1)
    norm2 = np.linalg.norm(vec2)
    return dot_product / (norm1 * norm2)

similarity_score = cosine_similarity(embedding_1, embedding_2)
print(similarity_score)   # closer to 1 = more similar meaning

Explanation: Cosine similarity measures the angle between two vectors — a score close to 1 means very similar meaning, closer to 0 means unrelated, and negative values suggest opposite meaning.

Real-World Example

Embeddings power semantic search (finding documents by meaning, not just exact keyword matches), recommendation systems, and duplicate detection.

Important Points

• Embeddings capture meaning, not just literal word overlap.
• Similarity between embeddings is typically measured using cosine similarity.

6. Vector Databases

What is it?

A vector database is a specialized database designed to store embeddings and efficiently search for the most similar ones, even across millions of entries — far faster than comparing a query against every single stored embedding one by one.

Definition: A vector database stores embeddings and enables fast similarity search, finding the most semantically relevant items to a given query.

Conceptual Workflow

python
# 1. Convert your documents into embeddings and store them
documents = ["Python is a programming language", "Cats are popular pets", "Machine learning uses data"]
document_embeddings = [get_embedding(doc) for doc in documents]
vector_db.store(documents, document_embeddings)

# 2. When a user asks a question, embed their query too
query = "What is Python used for?"
query_embedding = get_embedding(query)

# 3. Find the most similar stored document(s)
results = vector_db.search(query_embedding, top_k=1)
print(results)   # returns: "Python is a programming language"

Real-World Example

Popular vector databases include Pinecone, Chroma, Weaviate, and FAISS (a library rather than a full database) — all used heavily in the RAG systems described next.

Important Points

• Vector databases are essential infrastructure for search and retrieval based on meaning rather than exact keyword matching.

7. RAG — Retrieval-Augmented Generation

What is it?

RAG combines a retrieval step (finding relevant information, usually via a vector database) with a generation step (an LLM writing a response) — letting an LLM answer questions using specific, up-to-date, or private information it was never originally trained on.

Definition: RAG (Retrieval-Augmented Generation) is a technique that retrieves relevant documents and provides them to an LLM as context, so it can generate accurate, grounded answers rather than relying solely on its training data.

Why We Need It

LLMs have a fixed training cutoff and no built-in knowledge of your company's private documents, today's news, or anything created after their training. RAG solves this by fetching relevant, current, or private information first, then handing it to the LLM alongside the user's question.

Conceptual RAG Pipeline

python
def answer_question(user_question):
    # Step 1: Retrieve relevant context
    query_embedding = get_embedding(user_question)
    relevant_docs = vector_db.search(query_embedding, top_k=3)

    # Step 2: Build a prompt combining the retrieved context and the question
    context = "\n".join(relevant_docs)
    prompt = f"""Answer the question using ONLY the context below.

Context:
{context}

Question: {user_question}
"""

    # Step 3: Send to the LLM for a grounded answer
    response = call_llm_api(prompt)
    return response

Explanation of the Code

• Instead of asking the LLM the question directly (relying only on what it learned during training), we first retrieve the most relevant real documents, then explicitly include them in the prompt.
• This grounds the LLM's answer in actual, verifiable source material, dramatically reducing the chance of it inventing an incorrect answer.

Real-World Example

A company chatbot answering questions about internal HR policies uses RAG: it retrieves the relevant HR policy document and includes it in the prompt, so the LLM answers based on the company's actual current policy, not generic guesses.

Important Points

• RAG is the standard technique for building LLM applications that need up-to-date or private/specific knowledge.
• RAG is generally cheaper and faster to set up than fine-tuning a model (retraining it on custom data), and is the more common first choice for most applications.

8. Calling an AI API from Python

What is it?

Most LLM providers offer an API (following the same request/response HTTP patterns covered in the APIs & HTTP file), letting you integrate AI capabilities directly into your own Python applications.

Simple Example (Generic Pattern)

python
import requests

API_URL = "https://api.example-ai-provider.com/v1/chat"
API_KEY = "YOUR_API_KEY_HERE"

def call_llm_api(prompt):
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    payload = {
        "model": "example-model-name",
        "messages": [{"role": "user", "content": prompt}]
    }

    response = requests.post(API_URL, headers=headers, json=payload, timeout=30)
    response.raise_for_status()
    data = response.json()
    return data["choices"][0]["message"]["content"]

reply = call_llm_api("Explain recursion in one sentence.")
print(reply)

Explanation of the Code

• This follows the exact same requests pattern covered in the APIs & HTTP file: headers for authentication, a JSON payload describing the request, and parsing the JSON response.
• The messages structure (a list of role/content pairs) is the standard way most chat-based AI APIs represent a conversation.
• Note: exact field names (choices, messages) vary between providers — always check your specific provider's documentation for the exact request/response format.

Common Mistakes

• Hardcoding API keys directly in code (same security concern covered in the APIs & HTTP and Security files) — use environment variables instead.
• Not setting a timeout, since LLM responses can sometimes take longer than typical API calls.
• Sending overly long conversation histories without managing the token limit, potentially causing the request to fail or become very expensive.

9. Building a Simple Chatbot

What is it?

A chatbot maintains a running conversation history, sending the full context back to the LLM on every turn — since the LLM itself has no memory between separate API calls.

Simple Example

python
conversation_history = [
    {"role": "system", "content": "You are a friendly, helpful assistant."}
]

def chat(user_message):
    conversation_history.append({"role": "user", "content": user_message})

    # (In a real implementation, this sends the full conversation_history to the API)
    reply = call_llm_api_with_history(conversation_history)

    conversation_history.append({"role": "assistant", "content": reply})
    return reply

print(chat("Hi, what's your name?"))
print(chat("What did I just ask you?"))   # works because history is preserved

Explanation of the Code

• Every message (from both the user and the assistant) is appended to conversation_history, and the entire history is sent with every new request.
• This is why the model can answer "What did I just ask you?" — it isn't remembering anything itself between calls; the full conversation is simply being re-sent each time.

Important Points

• LLMs are "stateless" between API calls — all conversational memory must be managed explicitly by your own application code.
• Long conversations eventually need strategies like summarizing older messages, since the context window has a maximum token limit.

10. AI Agents and Function/Tool Calling

What is it?

An AI agent is an LLM-powered system that can take actions — not just generate text, but decide to call specific functions/tools (like searching the web, running code, or querying a database) to accomplish a task.

Definition: An AI agent uses an LLM to decide which actions or tools to invoke, based on the user's request, in order to complete a task that requires more than just generating text.

Simple Conceptual Example — Function/Tool Calling

python
def get_weather(city):
    # (In reality, this would call a real weather API)
    return f"The weather in {city} is sunny, 28°C"

available_tools = {
    "get_weather": get_weather
}

def handle_user_request(user_message):
    # The LLM decides: "this question needs the get_weather tool"
    # and specifies which function to call and with what arguments
    tool_name = "get_weather"        # (in reality, determined by the LLM's response)
    tool_arguments = {"city": "Mumbai"}   # (also determined by the LLM)

    result = available_tools[tool_name](**tool_arguments)

    # The result is sent back to the LLM, which uses it to form a final natural-language reply
    return f"Based on the tool result: {result}"

print(handle_user_request("What's the weather like in Mumbai?"))

Explanation of the Code

• Instead of trying to answer a weather question purely from its training data (which would be outdated or simply invented), the LLM recognizes it needs real-time data and requests that a specific tool (get_weather) be called with specific arguments.
• Your code actually executes the real function, then hands the result back to the LLM, which incorporates it into a natural final response.

Real-World Example

A customer service AI agent might have tools for checking order status (querying a real database), processing a refund (calling a real payment API), and searching a knowledge base — deciding which tool(s) to use based on what the customer actually asks.

Important Points

• Function/tool calling is what turns an LLM from "just a text generator" into a system that can actually take real actions in the world.
• The LLM decides what to call and with what arguments; your own code is always responsible for actually executing the real function safely.

11. LangChain (Brief Overview)

What is it?

LangChain is a popular Python framework that provides pre-built components for common LLM application patterns — chaining prompts together, connecting to vector databases, and building agents — so you don't have to build every piece from scratch.

bash
pip install langchain

Why It Exists

Building RAG pipelines, agents, and multi-step LLM workflows involves a lot of repetitive "glue code" (formatting prompts, calling vector databases, parsing responses). LangChain provides standardized, reusable building blocks for all of this.

Important Points

• LangChain isn't strictly necessary for simple LLM applications (a direct API call, as shown earlier, may be simpler and more transparent) — it becomes more valuable as an application's complexity grows (multi-step chains, agents, RAG pipelines).
• The broader ecosystem also includes similar tools (like LlamaIndex, focused more specifically on RAG), each with slightly different strengths.

Comparison Table — Key Generative AI Concepts

Common Beginner Mistakes — Summary for This Section

• Assuming an LLM "remembers" previous conversations automatically — memory must be managed explicitly by resending history.
• Treating LLM output as always factually correct, without verification (hallucination is a real, known limitation).
• Hardcoding API keys directly in code.
• Skipping RAG when building a knowledge-specific application, leading to outdated or invented answers.
• Overcomplicating a simple use case with a heavy framework (like LangChain) when a direct API call would be simpler and clearer.

Cheat Sheet — AI & Generative AI

python
import requests

def call_llm_api(prompt):
    headers = {"Authorization": f"Bearer {API_KEY}"}
    payload = {"model": "model-name", "messages": [{"role": "user", "content": prompt}]}
    response = requests.post(API_URL, headers=headers, json=payload, timeout=30)
    return response.json()["choices"][0]["message"]["content"]

# Conversation history pattern
history = [{"role": "system", "content": "You are a helpful assistant."}]
history.append({"role": "user", "content": "Hello!"})

# Cosine similarity for embeddings
import numpy as np
similarity = np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))

Mini Project: AI Chatbot

Objective

Build a command-line chatbot that maintains conversation history and calls an LLM API to generate responses.

Requirements

• Maintain a running conversation history across multiple turns.
• Send the full history to the API on every turn.
• Handle API errors gracefully.

(Note: replace `API_URL`, `API_KEY`, and the response-parsing logic with your actual chosen provider's specific documentation — the structure below is a generic, representative pattern.)

Concepts Used

requests, exception handling, lists/dictionaries (conversation history), functions, loops.

Complete Code

python
import requests

API_URL = "https://api.example-ai-provider.com/v1/chat"
API_KEY = "YOUR_API_KEY_HERE"

conversation_history = [
    {"role": "system", "content": "You are a friendly, helpful assistant. Keep answers concise."}
]

def call_llm_api(messages):
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    payload = {
        "model": "example-model-name",
        "messages": messages
    }

    try:
        response = requests.post(API_URL, headers=headers, json=payload, timeout=30)
        response.raise_for_status()
        data = response.json()
        return data["choices"][0]["message"]["content"]
    except requests.exceptions.RequestException as e:
        return f"Sorry, something went wrong: {e}"

print("Chatbot ready! Type 'quit' to exit.\n")

while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        print("Goodbye!")
        break

    conversation_history.append({"role": "user", "content": user_input})
    reply = call_llm_api(conversation_history)
    conversation_history.append({"role": "assistant", "content": reply})

    print(f"Bot: {reply}\n")

Code Explanation

• conversation_history grows with every turn, and the entire list is sent with each API call, giving the illusion of memory even though the underlying LLM itself is stateless.
• The try/except block (from the Exception Handling file) ensures a network issue or API error doesn't crash the whole chatbot session.
• The while True loop keeps the conversation going until the user types "quit."

Sample Interaction

Chatbot ready! Type 'quit' to exit.

You: What's a good way to learn Python?
Bot: Practice daily with small projects, read others' code, and build things you're personally interested in.

You: Can you summarize what you just said in 5 words?
Bot: Practice, build projects, read code.

You: quit
Goodbye!

Possible Improvements

• Add a token/message limit, trimming or summarizing older messages in long conversations.
• Save conversation history to a file (using File Handling techniques) so it persists between sessions.
• Add function/tool calling so the chatbot can look up real information (like the weather) when asked.

Challenge Task

Extend the chatbot into a simple RAG system: load a small text file of FAQs, retrieve the most relevant one(s) based on the user's question (using basic keyword matching or embeddings), and include them as context in the prompt before calling the API.

Interview Questions

Q1. What is a token in the context of LLMs? Answer: A small chunk of text (sometimes a whole word, sometimes part of one) that an LLM processes as its basic unit — the amount of text an API call uses and costs is measured in tokens.

Q2. What is prompt engineering? Answer: The practice of carefully designing input text (instructions, examples, structure) to guide an LLM toward producing more accurate, relevant, and reliable output.

Q3. What is an embedding? Answer: A numeric vector representation of text that captures its semantic meaning, allowing similarity between different pieces of text to be measured mathematically, even if they use completely different words.

Q4. What problem does RAG solve? Answer: LLMs have no built-in knowledge of information created after their training or private/specific data. RAG retrieves relevant real documents first and includes them in the prompt, letting the LLM generate answers grounded in accurate, current, or private information.

Q5. Why do chatbots need to resend the entire conversation history with every API call? Answer: LLMs are stateless between separate API calls — they have no memory of previous turns on their own. The full conversation history must be explicitly included in every request for the model to have context on what was discussed earlier.

Q6. What is an AI agent, and how does function/tool calling relate to it? Answer: An AI agent is a system where an LLM decides which actions or external tools/functions to invoke to complete a task, rather than only generating text. Function/tool calling is the specific mechanism by which the LLM specifies which function to call and with what arguments, with the actual execution handled by the application's own code.

Q7. What is "hallucination" in the context of LLMs? Answer: When a model generates confident-sounding but factually incorrect or fabricated information, since it's generating text based on learned patterns rather than looking up verified facts.

Practice Questions

Beginner

1. Write a zero-shot prompt and a few-shot prompt for classifying whether a sentence is a question or a statement, and compare them.
2. Explain, in your own words, why an LLM might occasionally generate incorrect information.
3. Write a Python function call_llm_api(prompt) (using placeholder logic) demonstrating the standard request/response pattern used to call an AI API.
4. Explain the difference between a token and a word.
5. Write out a simple conversation history list (as Python dictionaries) representing 3 turns of a conversation.

Intermediate

1. Implement a cosine_similarity function from scratch and test it on two hardcoded example vectors.
2. Design (in comments/pseudocode) a simple RAG pipeline for a customer support chatbot that answers questions using a company's FAQ document.
3. Extend the AI Chatbot mini project to save conversation history to a JSON file after each session, and reload it when the program starts.
4. Write a function that estimates a rough token count for a given string (e.g., using a simple word-count-based approximation), and explain its limitations compared to real tokenization.
5. Describe, in your own words, the difference between fine-tuning a model and using RAG.

Challenge

1. Build a simple keyword-based (non-embedding) FAQ retrieval system: given a list of Q&A pairs and a user question, find and return the most relevant answer based on shared keywords.
2. Design (in pseudocode) a simple AI agent with two tools (get_weather and get_stock_price), and describe how the agent would decide which tool to use for a given user question.
3. Extend the AI Chatbot mini project to include a basic tool-calling system: if the user's message contains the word "weather," call a (placeholder) get_weather() function and include its result in the final response.