• ✦ Embeddings are a type of representation that bridges the human understanding of language to that of a machine.

  • In the context of Large Language Models (LLMs), to be specific, we are dealing with text embeddings.
  • There are other types of embeddings, such as image, audio, and video embeddings.
  • Embeddings are a powerful technique in machine learning that allows us to represent data in a lower-dimensional space while preserving its semantic meaning.
  • This approach has revolutionized various fields, including natural language processing (NLP), computer vision, and more.
• ✦ They are a distributed representation for text that is perhaps one of the key breakthroughs for the impressive performance of deep learning methods on challenging natural language processing problems.

  • Large language models like GPT-4, Gemini, or BERT use word embeddings as the first layer of the model. We know, BERT is not that "large" compared to the other two, but it's still considered a significant advancement in natural language processing.

  • These models convert each word into a dense vector and feed it into the model. The models then use these vectors to predict the next word in a sentence (in the case of GPT-4) or to understand the context of a word (in the case of BERT).

  • These models are trained on a large corpus of text, so they learn the semantic meaning of words. For example, the word “king” is closer in this space to “queen” than it is to “apple”.

  • They are representations of text in a N-dimensional space where words that have the same meaning have a similar representation.

    • The text is translated into numbers, specifically into vectors.
    • That's why we will often see some articles describe embeddings as vectors too.
    • Essential, text embeddings is a vector (i.e., a list) of floating point numbers.
    • In other words, it represents words in a coordinate system where related words, based on a corpus of relationships, are placed closer together.

  • The number of values in a text embedding — known as its “dimension” — depends on the embedding technique (the process of producing the vector), as well as how much information you want it to convey.

  • The embeddings below shows a vector with 8 dimensions.
    

    

  • Table below show the common models with the dimensions of their embeddings

• ✦ Let’s try to visualize the concept. Imagine that we have a collection of sentences that we’ve turned into vectors, using a dense embedding technique.
  • If we simplify these vectors with hundreds of dimensions to just two dimensions, which we can plot them on a similarly designed two-dimensional grid.
  • For example, consider these seven pieces of text:

in_1 = "Flamingo spotted at the bird park"

in_2 = "Sea otter seen playing at the marine park"

in_3 = "Baby panda born at the city zoo"

in_4 = "Python developers prefer snake_case for variable naming"

in_5 = "New JavaScript framework aims to simplify coding"

in_6 = "C++ developers appreciate the power of OOP"

in_7 = "Java is a popular choice for enterprise applications"


list_of_input_texts = [in_1, in_2, in_3, in_4, in_5, in_6, in_7]
Copy

• ✦ Each of the 7 texts will converted into a vector (again, you can understand vector as list for our purpose). The diagram below shows the first text is converted into a vector. Imagine each of the 7 texts has it own vector that has 1536 numerical values. Here we assume we are using OpenAI's text-embedding-3-small.

• ✦ The diagram below show graph after we simplified the 7 vectors down to 2 dimensions and plot them onto the x and y axes.
  • Observe the distances between the different texts
  • Although the text that starts with "Python developers prefer snake_case", contains two animals, the embedding is further away from the three data points that are truly talking about real animals
  • It is closer to the other two data points that are about programming/coding

• ✦ The straightforward reason is that they can reduce data dimensionality and address the primary issue: the necessity for speed.

  • As AI’s capabilities continue to grow, scaling automation can face speed and cost constraints. This is where the recent rise in interest in Embeddings becomes significant.
    
  • The main application of these technologies is the demand for speed, especially when processing large volumes of text data.
    
  • This is particularly pertinent for large language models like the GPT series, whether they are closed or open-sourced, where the efficient processing of enormous amounts of text is vital.
    
  • Embeddings serve as engineering tools to tackle the challenge of processing large-scale text swiftly and cost-effectively.

• ✦ The initial phase of any Large Language Model (LLM) training is the most crucial: the neural network is constructed from a vast amount of data with an extensive number of features (let’s refer to them as details).

  • Language, of which the text is representing, contains many dimensions that are hard to specify or structurally quantify, including sentiment, grammar, meaning, and objects, just to mention a few.
    
  • The more dimensions there are, the more challenging it is for computers to analyze and learn from the data. This is where embeddings come in.
    
  • Data scientists employ embeddings to depict high-dimensional data in a low-dimensional space.
    
  • Think of embeddings as summaries.
    • They take high-dimensional data and condense it into a smaller, more manageable form, like picking out the key points from a long text.
    • This makes it easier and faster for AI models to process and understand the information. Just like summarizing a book saves you time and effort, embeddings help AI models work more efficiently.
    • Reducing the number of features while still capturing important patterns and relationships is the job of the Embeddings.
    • They allow AI models to learn and make predictions faster and with less computing power.

Embedding models have been used for a long time, primarily for training other LLMs or ML models.

The introduction of Retrieval Augmented Generation (RAG) and subsequently of Vector Store Databases has shed new light on these models.

They have a few common issues:

1. They have a context length limit, just like Large Language Models.
2. They usually excel at only one language (English).
3. High-dimensional vectors are typically required for optimal results.
4. They are usually trained for a specific task (text, image, or audio).

As research progressed, new state-of-the-art (text) embedding models began producing embeddings with increasingly higher output dimensions, meaning each input text is represented using more values. While this improves performance, it comes at the cost of efficiency and speed. Researchers were therefore motivated to create embedding models whose embeddings could be reasonably reduced in size without significantly sacrificing performance.

• Embeddings
• Handling Embeddings
• Applying Embeddings
• Retrieval Augmented Generation (RAG)
• Hands-on Walkthrough and Tasks

This is our new helper function to get embeddings by passing in a list of text to the function.

def get_embedding(input, model='text-embedding-3-small', dimensions=None):
    response = client.embeddings.create(
        input=input,
        model=model,
        dimensions=dimensions
    )
    return [x.embedding for x in response.data]
Copy

• ✦ The function can take in two different model
  • text-embedding-3-smallthat produces embeddings with 1536 dimension
  • text-embedding-3-large that produces embeddings with 3072 dimensions

Usage is priced per input token. Below is an example of how many pages of text that can be processed per US dollar (assuming ~800 tokens per page):

Using larger embeddings, for example storing them in a vector store for retrieval, generally costs more and consumes more compute, memory and storage than using smaller embeddings.

With OpenAI's new embedding models, both text-embedding-3-large and text-embedding-3-small allows builders to trade-off performance and cost of using embeddings.

• ✦ Specifically, builders can shorten embeddings (i.e. remove some numbers from the end of the sequence) without the embedding losing its concept-representing properties by passing in the dimensions API parameter.

• ✦ For example, on the MTEB benchmark, a text-embedding-3-large embedding can be shortened to a size of 256 while still outperforming an unshortened text-embedding-ada-002 (One of OpenAI's older embedding models) embedding with a size of 1,536.

• ✦ In general, using the dimensions parameter when creating the embedding is the suggested approach. Code below shows how the helper function is called with the dimensions specified as 512.

# Helper Function for Getting Embeddings
def get_embedding(input, model='text-embedding-3-small', dimensions=None):
    response = client.embeddings.create(
        input=input,
        model=model,
        dimensions=dimensions
    )
    return [x.embedding for x in response.data]

# Calling the function
text = "Python developers prefer snake_case for variable naming"
embeddings = get_embedding(text, dimensions=512)
Copy

• ✦ Visualizing data beyond three dimensions is inherently difficult due to our limited spatial intuition.
  • When working with complex embeddings, such as Large Language Models (LLMs) or other high-dimensional representations, it becomes practically impossible to directly visualize them in their original form.
  • One effective approach to make these embeddings more interpretable for humans is dimensionality reduction.
  • Techniques like Principal Component Analysis (PCA) and Uniform Manifold Approximation and Projection (UMAP) allow us to compress the data into a lower-dimensional space, typically two dimensions, while preserving its intrinsic structure.
  • By doing so, we can create scatter plots or heatmaps that reveal patterns, clusters, and relationships, making it easier for us to grasp the underlying information

Since embeddings capture semantic information, they allow us to compare a pair of texts based on their vector representations.

• ✦ One very common way to compare the distance between a pair of embeddings.

  • The distance between two vectors measures their relatedness.
  • Small distances suggest high relatedness
  • Large distances suggest low relatedness.
• ✦ With the distance between a pair of embeddings, we can then apply the distance in many other use cases such as:

  • Identify texts that semantically close to a target text, by identifying the texts that have short distance (i.e., closer) to the target text.
  • identify outliers, by identifying the datapoints that furthest away from the rest of typical datapoints
  • identify clusters, by grouping those datapoints that are located close to each other into distinct groups.

While embeddings offer significant advantages in various applications, they also pose substantial risks to privacy and data security.

Embeddings are essentially numerical representations of text data, and despite their seemingly abstract nature, they can encode sensitive information about individuals or organizations.

• Embeddings - OpenAI API
• There and Back Again: An Embedding Attack Journey | IronCore Labs
• Understanding UMAP (pair-code.github.io)

• Embeddings
• Handling Embeddings
• Applying Embeddings
• Retrieval Augmented Generation (RAG)
• Hands-on Walkthrough and Tasks

• ✦ Embeddings are commonly used for (but not limited to):
  • Search (where results are ranked by relevance to a query string)
  • Clustering (where text strings are grouped by similarity)
  • Recommendations (where items with related text strings are recommended)
  • Anomaly detection (where outliers with little relatedness are identified)
  • Diversity measurement (where similarity distributions are analyzed)
  • Classification (where text strings are classified by their most similar label)

Here is the sample data used in the use cases below:

• ✦ After seeing some of these example use cases, you might think, “why should I care about these text embedding things? Can’t I just make use GPT-4 to analyze the text for me?

• ✦ Techniques like Retrieval Augmented Generated (RAG) or Fine-tuning allow tailoring the LLMs to specific problem domains. 

• ✦ However, it’s important to recognize that these systems are still in their early stages.  - Building a robust LLM system presents challenges such as high computational costs, security risks associated with large language models, unpredictable responses, and even hallucinations.

• ✦ On the other hand, text embeddings have a long history, are lightweight, and deterministic. 

  • Leveraging embeddings simplifies and reduces the cost of building LLM systems while retaining substantial value. By pre-computing text embeddings, you can significantly accelerate the training and inference process of LLMs. This leads to lower computational costs and faster development cycles. Additionally, embeddings capture semantic and syntactic information about text, providing a strong foundation for LLM performance.

  • It should be another tool in the NLP toolkit, allowing for efficient similarity search, clustering, and other tasks. Embeddings excel at capturing semantic and syntactic relationships between texts. This makes them invaluable for tasks like finding similar documents, grouping related content, and understanding the overall structure of a text corpus. By combining embeddings with LLMs, you can create more powerful and versatile applications.

• Embeddings
• Handling Embeddings
• Applying Embeddings
• Retrieval Augmented Generation (RAG)
• Hands-on Walkthrough and Tasks

Now that we understand how embeddings can be used to retrieve semantically related texts, it's time to explore probably the most popular and pragmatic application of embeddings: Retrieval Augmented Generation (RAG).

A Retrieval-Augmented Generation (RAG) system is a framework that enhances the accuracy and reliability of generative AI models by incorporating information from external sources.

LangChain provides a robust framework for building LLM applications. The framework includes many components to support common LLM operations such as prompt chaining, chat memory management, and, of course, RAG.

We recommend using LangChain or equivalent frameworks for implementing RAG, instead of writing your code from scratch. These frameworks often offer the following benefits:

Ready-to-use Components

• ✦ Components are various modules/functions that we can use to handle many of the common operations in RAG, without having to write the code from scratch.
  • For example, Langchain provides components for us to easily read PDF files or PowerPoint files, connect to databases, or get the transcript of a YouTube video.
• ✦ Many of these components are based on contributions from large communities and research works that have proven to work effectively.
  • For example, Langchain has a rich set of advanced techniques for retrieving relevant documents, such as Contextual Compression, Self Query, and Parent Document – techniques that otherwise someone would have to understand from research papers or code repositories and then translate into Python code.
• ✦ Using a framework like Langchain allows us to focus on the business and application logic, so we can efficiently build and evaluate our proof-of-concept prototypes.

Community Support:

• ✦ These popular frameworks like Langchain have active communities, providing tutorials, examples, and documentation.
• ✦ Whether you're a beginner or an experienced developer, you'll find resources to guide you.

However, packages like LangChain are not without their shortcomings:

• ✦ Expect a learning curve to get familiar with the framework

  • While Langchain provides powerful tools for RAG, it does require some initial learning. Developers need to understand the components, syntax, and best practices.
• ✦ They are still in active development and may break your code

  • Updates may introduce changes, deprecate features, or even cause backward compatibility issues.
  • There are also chances where the documentation lags behind updates, or the available tutorials are based on older versions of the framework.
  • The suggestion is to avoid changing the version of the installed package unless it's necessary and you're ready to fix any broken code.
• ✦ Less flexibility compared to writing your own code

  • While Langchain streamlines RAG pipelines, it imposes certain constraints.
  • Customization beyond the provided components may be limited or challenging.
  • However, unless we are building something very unique, the components still serve as very useful building blocks for many common operations in LLMs.

There are 5 main steps in a typical RAG pipeline:

• 1. Document Loading
  • In this initial step, relevant documents are ingested and prepared for further processing.
• 2. Splitting & Chunking
  • The text from the documents is split into smaller chunks or segments.
  • These chunks serve as the building blocks for subsequent stages.
• 3. Storage
  • The embeddings (vector representations) of these chunks are created and stored in a vector store.
  • These embeddings capture the semantic meaning of the text.
• 4. Retrieval
  • When an online query arrives, the system retrieves relevant chunks from the vector store based on the query.
  • This retrieval step ensures that the system identifies the most pertinent information.
• 5. Output
  • Finally, the retrieved chunks are used to generate a coherent response.
  • This output can be in the form of natural language text, summaries, or other relevant content.

• Embeddings
• Handling Embeddings
• Applying Embeddings
• Retrieval Augmented Generation (RAG)
• Hands-on Walkthrough and Tasks

• 👆🏽 Click on the "Open Notebook" button below to open the Jupyter Notebook 

• 👆🏽 Click on the "Open Notebook" button below to open the Jupyter Notebook 

  

  Open Notebook

• ✦ While there is no submission required, we encourage you to share your solutions with your peers by pasting your link into the Sharing Board.

  • Feedback: By sharing your solutions, you can get insights, suggestions, and constructive criticism from your peers. This feedback can help you improve your approach and learn from others’ perspectives.

  • Learning from Peers: Since everyone may have different ways of solving problems, participating in these sessions allows you to see various approaches. You can learn alternative methods, explore different techniques, and gain a deeper understanding of the challenges.

• ✦ URL: https://miro.com/app/board/uXjVKojBjec=/?share_link_id=989058465513

• ✦ Passcode: abc-2024

• Deep Dive into RAG
• Improving Pre-Retrieval Processe
• Improving Retrieval Processed
• Improving Post-Retrieval Processed
• RAG Evaluation
• Further Reading: WOG RAG Playbook

• ✦ As there are so many ways to tune our RAG pipelines, how would we know which of the changes actually lead to better performance?

• ✦ Ragas is one of the frameworks designed to assess RAG-based applications.

  • It is a framework that provides us with the necessary ingredients to help us evaluate our RAG pipeline on a component level.
  • Ragas provides you with the tools based on the latest research for evaluating LLM-generated text to give you insights about your RAG pipeline.

• ✦ What’s interesting about Ragas is that it started out as a framework for “reference-free” evaluation. That means, instead of having to rely on human-annotated ground truth labels in the evaluation dataset, Ragas leverages LLMs under the hood to conduct the evaluations.

  • ✦ To evaluate the RAG pipeline, Ragas expects the following information:

    • question: The user query that is the input of the RAG pipeline.
    • answer: The generated answer from the RAG pipeline.
    • contexts: The contexts retrieved from the external knowledge source used to answer the question.
    • ground_truths: The ground truth answer to the question. This is the only human-annotated information. This information is only required for some of the matrices.
  • ✦ Leveraging LLMs for reference-free evaluation is an active research topic.

    • While using as little human-annotated data as possible makes it a cheaper and faster evaluation method, there is still some discussion about its shortcomings, such as bias.
    • However, some papers have already shown promising results. If you are interested, you can read more on the “Related Work” section of this Ragas paper.

• ✦ Note that the framework has expanded to provide metrics and paradigms that require ground truth labels (e.g., context_recall and answer_correctness)

• ✦ Additionally, the framework provides you with tooling for automatic test data generation.

Ragas provides you with a few metrics to evaluate a RAG pipeline component-wise as well as from end-to-end.

On a component level, Ragas provides you with metrics to evaluate the retrieval component (context_relevancy and context_recall) and the generative component (faithfulness and answer_relevancy) separately.

Most (if not all of) metrics are scaled to the range between 0 and 1, with higher values indicating a better performance.

Ragas also provides you with metrics to evaluate the RAG pipeline end-to-end, such as answer semantic similarity and answer correctness.

pip install ragas
Copy

from datasets import Dataset 
import os
from ragas import evaluate
from ragas.metrics import faithfulness, answer_correctness

os.environ["OPENAI_API_KEY"] = "your-openai-key"

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}

dataset = Dataset.from_dict(data_samples)

score = evaluate(dataset,metrics=[faithfulness,answer_correctness])
score.to_pandas()
Copy

Visit the documentation here: Introduction | Ragas

• Deep Dive into RAG
• Improving Pre-Retrieval Processe
• Improving Retrieval Processed
• Improving Post-Retrieval Processed
• RAG Evaluation
• Further Reading: WOG RAG Playbook

The Retrieval-Augmented Generation (RAG) Playbook is a comprehensive guide designed to help developers, particularly in Whole of Government (WOG), navigate the complexities of building and deploying RAG systems.

This playbook offers practical advice on constructing RAG applications, from no-code/low-code solutions to custom pipelines using open-source frameworks. It also provides metrics for evaluating RAG systems and includes experiments on realistic government use cases to demonstrate how to iteratively improve RAG performance.

As RAG technology evolves, this playbook serves as a foundational resource for understanding and leveraging its capabilities effectively.

RAG is only as good as the retrieved documents’ relevance and quality. Fortunately, an emerging set of techniques can be employed to design and improve RAG systems. The guide has focused on grouping and summarizing many of these techniques (see below above) and will share some high-level strategic guidance in the guide. Developers or builders can and should experiment with using different pieces together.

• Deep Dive into RAG
• Improving Pre-Retrieval Processe
• Improving Retrieval Processed
• Improving Post-Retrieval Processed
• RAG Evaluation
• Further Reading: WOG RAG Playbook

• ✦ The “Retrieval” step is key since it directly improves the context that the LLM has when generating a response.

  • It is the process whereby we are retrieve relevant context for a given query from the vector store or other databases.
  • Instead of using normal document chunk index retrieval we can use some modified methods which can be more efficient and give better contextual retrieval.
• ✦ The methods we will cover below are:

  • **Parent-Child Index Retrieval
  • Hierarchical Summary Index Retrieval
  • Self-Query Retriever

• ✦ Consider that we've developed a RAG system designed to identify potential diseases based on the symptoms entered during a consultation. If we're working with a Naive RAG, it's possible that it might only identify diseases sharing one or two symptoms, which could somewhat show that our application is not useful or even unusable.

• ✦ This scenario is perfectly suited for employing the Parent-Child Index Retrieval method.

  • This approach involves dividing large segments (referred to as the parent chunk) into smaller segments (known as the child chunk).
  • The advantage of creating smaller segments is that the information within them becomes more concentrated, ensuring that its value is not lost across extensive text passages.
• ✦ However, there's a minor issue with this approach:

  • To accurately locate the most pertinent documents, it's necessary to segment our documents into smaller pieces.
  • Conversely, it's crucial to supply the Large Language Model (LLM) with adequate context, which is best achieved by using larger segments.

The above points are illustrated in the subsequent image:

• ✦ The dilemma seems inescapable:

  • Embedding a shorter context allows the RAG to focus on more specific meaning but forgoes the broader context in the surrounding text. Embedding longer text, such as the entire body of text focuses on the overall meaning but may dilute the significance of individual sentences or phrases.
• ✦ This is where the Parent-child index retrieval method comes into play, promising to improve our approach.

  • The core concept involves subdividing the larger segments (Parent chunks/documents) into smaller ones (Child Chunks/documents).
  • After this subdivision, the process entails searching for the most relevant top K documents using the child chunks, then retrieving the parent chunks associated with these top K child documents.
• ✦ To bring this concept into practical application, a step-by-step explanation is most effective:

  1. Collect the documents and segment them into larger chunks (Parent chunks).
  2. Divide each parent chunk to generate smaller, child chunks.
  3. Store the child chunks (in their Vector Representation) within the Vector Store.
  4. Keep the parent chunks stored in memory (Vector representation for these is not necessary).

The process described is visually represented in the following image:

• ✦ To better understand this method, consider the following image that illustrates how it operates:
  

  

• ✦ Implementing this might sound daunting due to the need to establish a new database for the smaller chunks, maintain the parent chunks in memory, and track the relationship between parent and child chunks. Fortunately, LangChain simplifies this process significantly, making it straightforward to set up.

from langchain.retrievers import ParentDocumentRetriever  
from langchain.storage import InMemoryStore  
from langchain_text_splitters import RecursiveCharacterTextSplitter  
from langchain_openai import OpenAIEmbeddings  
from langchain_chroma import Chroma
  
 
# Some code for loading the documents are obmitted
# ...

parent_docs = documents  
  
# Embedding Model  
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")  
  
  
# Splitters  
child_splitter = RecursiveCharacterTextSplitter(chunk_size=200)  
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=800)  
  
# Stores  
store = InMemoryStore()  
vectorstore = Chroma(embedding_function=embeddings, collection_name="fullDoc", persist_directory="./JohnWick_db_parentsRD")  
  
  
parent_document_retriever = ParentDocumentRetriever(  
	vectorstore=vectorstore,  
	docstore=store,  
	child_splitter=child_splitter,  
	parent_splitter =parent_splitter  
)

Copy

• ✦ Do note that the number of chunks in the vector store (number of child chunks) should be much higher than the number of documents stored in memory (parent chunks). With the following code we can if this is true:

print(f"Number of parent chunks is: {len(list(store.yield_keys()))}")  
  
print(f"Number of child chunks is: {len(parent_document_retriever.vectorstore.get()['ids'])}")  
  
'''  
Number of parent chunks is: 75  
Number of child chunks is: 3701  
'''
Copy

Once we have our Parent Document Retriever, we just need to create our RAG based on this retriever and that would be it.

setup_and_retrieval = RunnableParallel({"question": RunnablePassthrough(), "context": parent_document_retriever })  

output_parser = StrOutputParser()  
  
  
parent_retrieval_chain = setup_and_retrieval | rag_prompt | chat_model | output_parser
Copy

LangChain Documentation: Parent Document Retriever | 🦜️🔗 LangChain

• ✦ This approach can be understood as the reversal of Parent-Child Index Retrieval that we just discussed above. It is also a more intelligent method as it takes into consideration the "semantic meaning of the child chunks" and groups semantically-similar child chunks together.

• ✦ RAPTOR is one of the hierarchical approach introduced by Stanford researchers.

  • RAPTOR introduces a novel approach to retrieval-augmented language models by constructing a recursive tree structure from documents
  • This allows for more efficient and context-aware information retrieval across large texts, addressing common limitations in traditional language models

• ✦ Based on user query, the summary document is retrieved and then relevant chunks are retrieved from that document.

# Intallation
!git clone https://github.com/parthsarthi03/raptor.git
!cd raptor
!pip install -r requirements.txt

# Setting Up
import os
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"


from raptor import RetrievalAugmentation

RA = RetrievalAugmentation()

# Adding Documents
with open('sample.txt', 'r') as file:
    text = file.read()
RA.add_documents(text)


Copy

• ✦ For detailed methodologies and implementations, refer to the original paper or the GitHub repo:
  • [2401.18059] RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (arxiv.org)
  • parthsarthi03/raptor: The official implementation of RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (github.com)

• ✦ Its main feature is that it is capable of performing searches in the vector store, applying filters based on the metadata. This approach is allegedly one of the most optimal methods to improve the efficiency of the retriever.

• ✦ We know that when we apply a “Naive retrieval”, we are calculating the similarity of all the chunks of the vector database with the query.

  • The more chunks the vector store has, the more similarity calculations will have to be done.
  • Now, imagine being able to do a prior filter based on the metadata, and only after selecting the chunks that meet the conditions imposed in relation to the metadata, we calculate similarity scores based on the filtered chunks. 
  • This can drastically reduce computational and time cost.
• ✦ Let’s look at a use case to fully understand when to apply this type of retreival.

  • Let’s imagine that we have stored in our vector database a large number of experiences and leisure offers.
    • The description of the experience is what we have encoded, using our embedding model.
    • Each offer has 3 key values or metadata:
      • Date
      • price
      • place.
  • Let’s imagine that a user is looking for an experience below:
    • An experience close to nature that is safe and family-friendly.
    • Furthermore, the price must be less than $50 and the place must be in California.
  • Therefore, it does not make sense to calculate similarities with chunks/experiences that do not comply with the metadata filter (which is based on the requirements by the user).
• ✦ This case is ideal for applying Self Query Retriever.

  • What this type of retriever allows us to do is perform a first filter through the metadata
  • Only then do we perform the similarity calculation between the chunks that meet the metadata requirements and the user input.

This technique can be summarized in two very specific steps:

• Query Constructor
• Query Translater

• The Ultimate Guide on Retrieval Strategies - RAG (part-4) - ChatGen

• Advanced RAG 01: Small-to-Big Retrieval

• Advanced RAG 06: Exploring Query Rewriting

• Advanced RAG 04: Re-ranking. From Principles to Two Mainstream… | by Florian June | Towards AI

• langchain/cookbook/rewrite.ipynb at master · langchain-ai/langchain (github.com)

• Deep Dive into RAG
• Improving Pre-Retrieval Processe
• Improving Retrieval Processed
• Improving Post-Retrieval Processed
• RAG Evaluation
• Further Reading: WOG RAG Playbook

• Once we have efficiently retrieved the context for a given query, we can further refine and optimize It to improve its relevance for a more optimal generation of the output answer.

• ✦ Re-ranking is a process of ordering the retrieved context chunks in the final prompt based on its score and relevancy.

• ✦ This is important as researchers found better performance when the most relevant context is positioned at the start of the prompt.

• ✦ **The technique consists of two very different steps:

  • Step 1:
    • Get a good amount of relevant docs based on the input/question. Normally we set the most relevant K.
    • For the first step, it is nothing more than what we usually use to make a basic RAG.
    • Vectorize our documents. vectorize the query and calculate the similarity with any metric of our choice.
  • Step 2:
    • Recalculate which of these documents are really relevant.
    • Discarding the other documents that are not really useful.
    • Re-order the relevant documents
    • The second step is something different from what we are used to seeing. This recalculation/reranking is executed by the reranking model or cross-encoder.

• ✦ You will have realized that the two methods provide the same result in the end, which is a metric that reflects the similarity between two texts. But there is a very important difference.

• ✦ You may ask. If it works better, why don’t we just use cross encoder directly with all chunks, instead of just limiting it to the top-K chunks? This is because:
  • it would be terribly expensive and causes heavy computation which lead to slowness.
  • For this reason, we make a first filter of the chunks closest in similarity to the query, reducing the use of the reranking model to only K times.

• ✦ To better understand the architecture of this method, let’s look at a visual example.
  
  
  
  The image shows the steps:

1. We obtain the query, which we encode into its vector form with a transformer and we compare it into the vector base.
2. Collect the documents most similar to the query from our database. We can use any retriever method (e.g., cosine similarity).
3. Next we use the cross-encoder model.
   • In the example shown in the image, this model will be used a total of 4 times.
   • Remember that the input of this model will be the query and a document/chunk, to collect the similarity of these two texts.
4. After the 4 calls have been made to this model in the previous step, 4 new values (between 0 and 1) of the similarity between the query and each of the documents will be obtained.
   • As can be seen, the chunk number 1 obtained in the Step 1 has dropped out into 4th place after reranking in Step 4.
5. Then, we add the first 3 chunks most relevant to the context.

• ✦ Now, a good question would be where to find the Cross-Encoder models or how to use?
  • One of the most straightforward way to use a powerful cross encoder model is to use the model made available by the company Cohere.
  • While there are many open-source models that can be used for this purpose, it is beyond the scope of this training to cover them all.
  • Due to the LangChain and its integration with Cohere, we only have to import the module that will execute the call to the Cohere cross-encoder model:

from langchain_cohere import CohereRerank  
from langchain.retrievers.contextual_compression import ContextualCompressionRetriever

os.environ["COHERE_API_KEY"] = "YOUR API KEY FROM COHERE"  
  
compressor = CohereRerank(top_n=3)

  
compression_retriever = ContextualCompressionRetriever(  
base_compressor=compressor,  
base_retriever=naive_retriever  
)
Copy

Let’s see a comparison between a Naive Retriever (e.g., distance between embeddings) and a Reranking Retriever

• ✦ Observations:
  • As we see from the result above, Naive Retriever returns us the top 10 chunks/documents.
  • After performing the reranking and obtaining the 3 most relevant documents/chunks, there are noticeable changes.
  • Notice how document number 16, which is in third position in relation to its relevance in the first retriever, becomes first position when performing the reranking.

• ✦ This method focus on improving the quality of retrieved docs.
  • Information most relevant to a query may be buried in a document with a lot of irrelevant text.
  • Passing that full document through your application can lead to more expensive LLM calls and poorer responses.
• ✦ Contextual compression is meant to fix this.
  • The idea is simple: instead of immediately returning retrieved documents as-is, you can compress them using the context of the given query, so that only the relevant information is returned. “Compressing” here refers to both compressing the contents of an individual document and filtering out documents wholesale.
  • For this, we can use ContextualCompressionRetriever from LangChain library to improve the quality of retrieved documents by compressing them.

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor
from langchain_openai import OpenAI

llm = OpenAI(temperature=0)
compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "Why do LLMs hallucinate?"
)
pretty_print_docs(compressed_docs)
Copy

LangChain Documentation: Contextual compression | 🦜️🔗 LangChain

• ✦ Prompt Compression is a method of compressing or shrinking BOTH the retrieved context or final prompt by removing irrelevant information.
  • Its aim is to reduce length of the input prompt to reduce cost, improve latency and efficiency of output generation by allowing the LLM to focus on a more concise context.
  • The core idea is to use LLM to generate compressed version of input prompt.

• ✦ Based on information on the repository, it was claimed that these tools offer an efficient solution to compress prompts by up to 20x, enhancing the utility of LLMs.
  • 💰 Cost Savings: Reduces both prompt and generation lengths with minimal overhead.
  • 📝 Extended Context Support: Enhances support for longer contexts, mitigates the "lost in the middle" issue, and boosts overall performance.
  • ⚖️ Robustness: No additional training needed for LLMs.
  • 🕵️ Knowledge Retention: Maintains original prompt information like ICL and reasoning.
  • 📜 KV-Cache Compression: Accelerates inference process.
  • 🪃 Comprehensive Recovery: GPT-4 can recover all key information from compressed prompts.

• ✦ Here is a snippet of code to show how to use the package.

# Install the package
!pip install llmlingua

from llmlingua import PromptCompressor

llm_lingua = PromptCompressor()
compressed_prompt = llm_lingua.compress_prompt(prompt, instruction="", question="", target_token=200)
Copy

• Advanced RAG 09: Prompt Compression | by Florian June | Apr, 2024 | AI Advances (medium.com)

• Deep Dive into RAG
• Improving Pre-Retrieval Processe
• Improving Retrieval Processed
• Improving Post-Retrieval Processed
• RAG Evaluation
• Further Reading: WOG RAG Playbook

• ✦ Retrieval Augmented Generation (RAG) is emerging as a crucial framework for industries and GenAI practitioners or enthusiasts to develop applications powered by Large Language Models (LLMs).
  • It offers significant potential to utilize LLMs in an optimal and efficient manner for creating comprehensive GenAI applications, including chatbots, search engines, and more, which we have seen in the previous topic.
  • RAG enables the dynamic integration of external knowledge sources, enhancing the accuracy and relevance of responses generated by these applications.

The basic or "Vanilla" RAG, also known as Naive RAG, exhibits several limitations, particularly when applied to complex use cases or in the development of production-ready applications.

 
As we saw in the previous topic, building an RAG prototype is relatively easy – investing around 20% of the effort yields an application with 80% performance. However, achieving a further 20% performance improvement requires the remaining 80% of the effort.

 
Below are some key reasons why Naive RAG may not always deliver the most effective and optimized outcomes.

 RAG is only as good as the retrieved documents’ relevance and quality. Fortunately, an emerging set of techniques can be employed to design and improve RAG systems.

 
The improvement of RAG is not just a matter of incremental updates, by installing newer Python package or calling any functions out-of-the-box, but many of them involves a comprehensive rethinking of its architecture and processes.

We can group the various improvements under 3 major categories:

• ✦ Pre-Retrieval Processes
• ✦ Retrieval Process
• ✦ Post-Retrieval Process

• Evaluation of RAG systems is essential to benchmark the overall performance of RAG output.

• To evaluate RAG we can use metrics like:

  • answer relevancy,
  • faithfulness for generation and context recall,
  • precision for retrieval.
• These metrics provide a structured way to assess the quality of the generated answers and the relevance of the information retrieved by the system.

  • However, the complexity and variability of RAG systems necessitate a more comprehensive and nuanced approach to evaluation.
• Enter RAGAS, a framework specifically designed for this purpose.

  • RAGAS offers a suite of tools and metrics tailored to evaluate RAG-based applications at a component level, providing a clear pathway for developers or enthusiasts to assess and enhance their applications systematically, allowing developers to fine-tune their systems with more confidence.

We will go into the details of RAG evaluation in 5. RAG Evaluation

• RAG from scratch: Overview by LangChain - Google Slides

• A Practitioners Guide to Retrieval Augmented Generation (RAG)

• What is Semantic Chunking And How It Works + Full Python Code (learnwithhasan.com)

• Enhancing RAG Efficiency: The Power of Semantic Chunking | by Sthanikam Santhosh | Mar, 2024 | Medium

• Deep Dive into RAG
• Improving Pre-Retrieval Processe
• Improving Retrieval Processed
• Improving Post-Retrieval Processed
• RAG Evaluation
• Further Reading: WOG RAG Playbook

• ✦ As name suggest this contains optimizations which are done before retrieval process to enhance quality retrieval of context.
• ✦ It includes:
  • Better Splitting & Chunking of Document,
  • query transformation*,
  • query routing*.

• ✦ As we have already seen in Naive RAG that chunks are nothing but the small parts of whole document and indexing is vector representation of these chunks which we store in Vector DB.

  • How we do splitting and chunking, and eventually embedding makes an impact on accurate retrieval which then improves generation quality and contextual confidence.
  • The simplest way for splitting and chunking is fixed size chunking like simple character splitter or word splitter but it is not as effective as it may not hold full context of specific subject which also known as context fragmentation.

• ✦ We quote a paragraph from GovTech RAG Playbook that perfectly sums up the challenges of finding the right balance between the chunk size and the accuracy of the RAG pipeline. We included the RAG Playbook under the "Further Readings" for Topic 5.

  Chunk and Overlap Size

  While it is possible to obtain an embedding for a document as long as it fits into the embedding model’s context length, embedding an entire document is not always an optimal strategy. It is common to segment documents into chunks and to specify an overlap size between chunks.

  Both of these parameters can help to facilitate the flow of context from one chunk to another, and the optimal chunk and overlap size to use is corpus specific. Embedding a single sentence focuses on its specific meaning but forgoes the broader context in the surrounding text. Embedding an entire body of text focuses on the overall meaning but may dilute the significance of individual sentences or phrases.

  Generally, longer and more complex queries benefit from smaller chunk sizes while shorter and simpler queries may not require chunking.

  Source: GovTech RAG Playbook

• ✦ While fixed-size chunking offers a straightforward approach, it often leads to context fragmentation, hindering the retrieval of accurate information.

  • To expand the number of options you can consider when building your RAG pipeline, this note introduces more sophisticated chunking techniques.

• ✦ Query transformation is a method of improving quality of user query by restructuring it to improve retrieval quality.

• ✦ It includes techniques like:

  • Query rewriting
  • Decomposing main query into multiple sub queries

• ✦ When we are having multiple vector stores / databases or various actions to perform on user query based on its context, then routing the user query in the right direction is very important for relevant retrieval and further generation.

• ✦ Using specific prompt and output parsers, we can use an LLM call to decide which action to perform or where to route the user query.

  • In fact, we have implemented this when we identified the types of the customer query and then directed the query to the correct departments in our Topic 3 notebook Notebook for Reference - Part 2
• ✦ If you're keen to use any frameworks, you can use prompt chaining or custom Agents to implement query routing in LangChain or LlamaIndex.

  • Don't worry if you don't understand what is "Agents" at this stage. We may come to that later in this training.

• Advanced RAG: How MultiQuery Retriever Work? | by Bytefer | Apr, 2024 | Level Up Coding (medium.com)
• Semantic Chunking for RAG. What is Chunking ? | by Plaban Nayak | The AI Forum | Apr, 2024 | Medium

• Towards AI Agents
• More Secure way to Store Credentials
• Writing & Running Python Scripts
• Hands-on Walkthrough and Tasks
• Create Multi-Agent Systems with CrewAI

CrewAI is an open-source framework designed to orchestrate and coordinate teams of autonomous AI agents, similar to Autogen. Think of it as a way to assemble and manage a group of AI assistants that collaborate to achieve a shared objective, much like a crew on a ship or a project team.

Here are some essential aspects of CrewAI:

• ✦ Emphasis on Collaboration: Unlike many AI frameworks that prioritize individual agents, CrewAI is built for seamless teamwork. Agents share information and tasks, leveraging “collaborative intelligence” to tackle complex challenges that would be daunting for a single agent to handle.
  
• ✦ Role-Specific Agents: Each agent within a CrewAI team can assume a distinct role—be it a data engineer, marketer, or customer service representative. This role-based approach allows you to customize the team according to the specific demands of your project.
  
• ✦ User-Friendly and Adaptable: CrewAI is designed with simplicity in mind, making it accessible even for those without a deep understanding of AI. Its flexibility means you can easily tailor it to meet your unique requirements, including the ability for each agent to utilize different large language models (LLMs) suited to their specific roles and tasks.

1. Building a Smart Assistant Platform: CrewAI can be leveraged to develop a team of agents capable of managing various tasks, such as scheduling appointments, arranging travel, and responding to user inquiries. This creates a comprehensive smart assistant that streamlines everyday activities.

2. Creating an Automated Customer Service System: With CrewAI, you can assemble a team of agents dedicated to handling customer inquiries, resolving issues, and providing support. This automated system enhances customer experience by ensuring timely and efficient responses.

3. Developing a Multi-Agent Research Team: CrewAI can facilitate the formation of a collaborative research team composed of agents that work together on projects. They can analyze data, generate hypotheses, and test ideas, making the research process more efficient and effective.

The CrewAI workflow process typically involves the following steps:

1. Agents: In this initial phase, you define the capabilities of your CrewAI workflow by specifying the agents involved. This includes outlining their roles and the skills they should possess, effectively determining who does what within the team.

2. Tasks: Next, you establish the specific objectives you want your agents to achieve. This step is crucial for guiding the agents toward accomplishing the desired outcomes.

3. Process: Here, you outline how CrewAI will utilize the defined agents and tasks to meet the overarching goals of your project. This involves mapping out the interactions and workflows that will drive the collaboration.

4. Run: Finally, you initiate the execution of your agents and tasks. Once the run is underway, assuming everything goes smoothly, CrewAI will generate results aimed at solving the stated objectives. This step marks the transition from planning to action, bringing your workflow to life.

• ✦ Focus

  • When mixing too much information, too many tools, or too much context, the models can lose important information (and also open up opportunities for more hallucination)
  • Another great advantage of the agent is they are able to focus:
    • Tools available
    • Context
    • Goal to achieve
• ✦ Tools

  • Can overload agents with too many tools
  • Smaller models may not know which is the context, which are the tools
  • Provide the agents with the key tools to do the job that they need.
• ✦ Memory

  • Make huge and immense differences for the agents
  • Recollect what they did in the past, learn from it, and apply their knowledge into future execution
  • Some frameworks offer different types of memory and different types of implementations for it.
  • CrewAI Agents have three types of memory:
    • I) Long-term memory
      • Memory that is remain even after the crew finishes
      • The memory is stored in a database locally
      • Everything after an agent completed the tasks, it self critiques itself to learn what it should have done better; or what are the things that should be in there that are not
      • Leads to self-improving agents
    • II) Short-term memory
      • Use only during the crew execution
      • When crew kick-off, it starts from the bank
      • Agents store different things that they learn in this memory
        • Allow agents to share knowledge, activities, and learnings with other agents. Agent 1 can tap into learnings from Agent 3.
        • Allows agents to share intermediate information even before providing “task completion” output.
    • III) Entity memory
      • Only being store during the execution
      • It stores what are the subjects that are being discussed
        
        

A tool in CrewAI is a skill or function that agents can utilize to perform various actions.

Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively. When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem

• ✦ Tools are essential for LLM Agents as they significantly enhance their capabilities.
• ✦ They enable agents to perform a wide range of tasks, from web searching and data analysis to content generation and collaboration.
• ✦ Tools also provide customizability, allowing developers to create or integrate specific functionalities tailored to their needs.

• Towards AI Agents
• More Secure way to Store Credentials
• Writing & Running Python Scripts
• Hands-on Walkthrough and Tasks
• Create Multi-Agent Systems with CrewAI

A single AI agent’s architecture encompasses the essential components that empower it to think, plan, and act within its environment. This sophisticated design typically includes:

Tools

• ✦ The agent learns to call external APIs or tools for extra information/context or capability that might be missing in the model weights (often hard to change after pre-training).
• ✦ This includes things like current information, mathematical engines, code execution capability, access to proprietary information sources, and many more.

Memory

• ✦ Short-term memory: 
  • In-context learning (See Prompt Engineering) can be thought of as utilizing short-term memory of the model to operate on a given problem. The context length window can be thought of as Short-term memory.
• ✦ Long-term memory: 
  • Providing the agent with the capability to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval. The Retrieval part in RAG can be thought of as Long-term memory.

Planning

• ✦ Subgoal & task decomposition: 
  • The agent breaks down larger tasks into smaller, manageable subgoals, enabling efficient handling of complex tasks.
• ✦ Reflection and refinement: 
  • The agent can do self-criticism (though doubtful in certain ways) and self-reflection over past actions, learn from mistakes, and refine them for future steps, thus improving the final results.

Together, these elements create an intelligent system that can autonomously solve problems. An AI agent can analyze an issue, devise a step-by-step plan, and confidently execute it, making it a transformative force in the world of artificial intelligence. Below is one example of a more detailed architecture of an AI Agent system.

However, the development and implementation of multi-agent systems come with their own set of challenges and risks.

Here are three criteria to determine whether you might need an agent:

• ✦ Does your application follow an iterative flow based on incoming data?

  • If your application processes data in a cyclical manner, where each iteration builds upon the previous one, it may be a strong candidate for an agent-based approach.
  • Agents can effectively manage and respond to new information as it arrives, allowing for continuous improvement and refinement of outputs.
  • This is particularly useful in scenarios like data analysis, where insights evolve as more data is processed.
    
• ✦ Does your application need to adapt and follow different flows based on previously taken actions or feedback along the way?

  • Applications that require dynamic decision-making based on past interactions or user feedback can greatly benefit from agents.
  • An agent can track the history of actions and outcomes, enabling it to adjust its strategy in real-time.
  • This adaptability is crucial in environments where user preferences or external conditions change frequently
• ✦ Is there a state space of actions that can be taken?

  • If your application involves a complex set of possible actions that can be executed in various sequences, rather than a simple linear pathway, it may require an agent to navigate this state space effectively.
  • Agents can explore multiple pathways and make decisions based on the current state, optimizing for the best outcomes.
  • This is particularly relevant in scenarios like game development, robotics, or any system where multiple strategies can lead to different results.

McKinsey’s most recent “State of AI” survey found that more than 72 percent of companies surveyed are deploying AI solutions, with a growing interest in GenAI. Given that activity, it would not be surprising to see companies begin to incorporate frontier technologies such as agents into their planning processes and future AI road maps. Agent-driven automation remains an exciting proposition, with the potential to revolutionize whole industries, bringing a new speed of action to work. That said, the technology is still in its early stages, and there is much development required before its full capabilities can be realized.

• Why agents are the next frontier of generative AI | McKinsey
• How to choose between AutoGen vs crewAI for Creating AI Agents - Skim AI

• Towards AI Agents
• More Secure way to Store Credentials
• Writing & Running Python Scripts
• Hands-on Walkthrough and Tasks
• Create Multi-Agent Systems with CrewAI

After getting familiar with Jupyter Notebook, especially Google Colab that is hosted remotely on a server, you would realize that it is very dangerous to specify our API key in the notebook or script.

What we have been using until this point is to rely on the getpass() function to allow users (in fact us), to input the API Key and store the value into a variable, shown below.

from openai import OpenAI  
from getpass import getpass  
  
openai_key = getpass("Enter your API Key:")  
client = OpenAI(api_key=openai_key)
Copy

This method helps keep the key secure by not hardcoding it into the script, which could be accidentally shared or exposed.

While the getpass() method helps keep the key secure by not hardcoding it into the script which which could be accidentally shared or exposed, this method is not suitable for scenarios where the Python script or application needs to run autonomously, without human interaction, such as:

1. Automated scripts or applications: If your script is part of a larger application or a scheduled task that runs automatically, there won't be a user present to input the API key each time it runs. This makes getpass() impractical.
2. Web applications or services: For applications deployed on a server to provide services over the web, requiring manual input of an API key upon each restart or deployment is not feasible. These applications often need to start and operate without human intervention.
3. Containerized applications: Applications deployed using containers (e.g., Docker) in cloud environments are designed to be easily replicated and scaled. Requiring manual input for each container instance is not practical.
4. Development and testing environments: In environments where continuous integration/continuous deployment (CI/CD) practices are followed, the deployment process is automated, and manual steps like entering an API key each time the application is tested or deployed are not suitable.

When building an application, the app may require access to a variety of APIs and other services, such as Google Sheet, AWS account, or Telegram messages. All these access would require some forms of credentials (i.e., username and password pair, API key).

Think of an environment variable as a special, secure place on your computer or server where you can store these credentials Your Python scripts or applications can access the credentials, such as the OpenAI API key, when they need to access the services, but the credentials aren't visible to anyone just looking through the code.

One way to set the environment variable is through a configuration file (.env).

• Towards AI Agents
• More Secure way to Store Credentials
• Writing & Running Python Scripts
• Hands-on Walkthrough and Tasks
• Create Multi-Agent Systems with CrewAI

A Python script is a file containing Python code that is intended to be directly executed.

• ✦ Unlike Jupyter Notebooks, which allow for an interactive coding experience with immediate feedback for each cell, Python scripts are run from start to finish by the Python interpreter.
• ✦ Scripts are ideal for projects and tasks that require automation or when deploying applications.

• ✦ This is a quick way to check if your Python has been installed correctly

• ✦ Open your terminal (Command Prompt on Windows, Terminal on macOS and Linux) and type:

  • python --version

• ✦ This command should return the version of Python installed.

  

A well-structured Python script not only makes your code more readable and maintainable but also adheres to the conventions that Python developers expect. This section will guide you through the essential components and good practices for structuring your Python scripts.

• ✦ Use Comments:
  • Comments (#) are crucial for making your code understandable to others and your future self.
• ✦ Follow Naming Conventions:
  • Use meaningful variable and function names. Python convention is to use snake_case for variable and function names.
• ✦ Modularize Your Code:
  • Break your code into functions or modules for better readability and reusability.
• ✦ Error Handling:
  • Use try and except blocks to handle potential errors in your scripts.
  • See our guide on 5. A Gentle Intro to Exception Handling in Python

• This resource covers:
  • 11 Beginner Tips for Learning Python Programming
  • Writing Comments in Python (Guide)
  • How to Write Beautiful Python Code With PEP 8
  • Getting Started With Python IDLE
  • Interacting With Python
  • Your Guide to the Python print() Function
  • Understanding the Python Traceback
  • Invalid Syntax in Python: Common Reasons for SyntaxError
  • Python Exceptions: An Introduction
  • Strings and Character Data in Python
  • Variables in Python
  • Unicode & Character Encodings in Python: A Painless Guide
  • PEP 8 — the Style Guide for Python Code
  • Python Type Checking (Guide)
  • Writing Comments in Python (Guide)
  • Documenting Python Code: A Complete Guide

• Towards AI Agents
• More Secure way to Store Credentials
• Writing & Running Python Scripts
• Hands-on Walkthrough and Tasks
• Create Multi-Agent Systems with CrewAI

• 👆🏽 Click on the "Open Notebook" button below to open the Jupyter Notebook 

• ✦ While there is no submission required, we encourage you to share your solutions with your peers by pasting your link into the Sharing Board.

  • Feedback: By sharing your solutions, you can get insights, suggestions, and constructive criticism from your peers. This feedback can help you improve your approach and learn from others’ perspectives.

  • Learning from Peers: Since everyone may have different ways of solving problems, participating in these sessions allows you to see various approaches. You can learn alternative methods, explore different techniques, and gain a deeper understanding of the challenges.

• ✦ URL: https://miro.com/app/board/uXjVKojBjec=/?share_link_id=989058465513

• ✦ Passcode: abc-2024