Ollama Embedding: How to Feed Data to AI for Better Response?

Mervin Praison
23 Feb 202405:39

Summary

TLDRThis video introduces the concept of using 'Olama' embeddings for creating a Retrival-Augmented Generation (RAG) application with enhanced performance. The tutorial demonstrates how to ingest data from URLs, convert it into embeddings using the high-context 'Nomic Embed Text' model, and store them in a Vector database like ChromaDB. The process involves splitting data into chunks, utilizing a web-based loader, and implementing a RAG chain with a language model named 'Mistal'. The result is a locally executable AI application that provides relevant answers based on the context. The video also guides viewers on setting up the application using LangChain and Gradio for a user-friendly interface, showcasing the power of local AI model servers.

Takeaways

  • πŸ“š The video introduces 'Ollama', a local AI model server that allows users to run large language models on their own machine.
  • πŸ” The project involves creating a Retrival-Augmented Generation (RAG) application with improved performance using embeddings.
  • 🌐 Data will be ingested from URLs, converted to embeddings, and stored in a Vector database for efficient retrieval.
  • πŸ“ˆ The video highlights 'Nomic Embed Text' as the chosen embedding model due to its higher context length and superior performance compared to OpenAI's models.
  • πŸ› οΈ The tutorial demonstrates how to use 'Lang Chain', a Python library, to put all the pieces of the RAG application together.
  • πŸ’» The script provides a step-by-step guide on how to split data from URLs, convert it into embeddings, and store them in a Chroma DB.
  • πŸ”‘ The video mentions the use of a 'web-based loader' to extract data from URLs and a 'character text splitter' for dividing the data into chunks with an overlap.
  • πŸ€– The process includes using a 'retriever' to fetch relevant documents when a question is asked, which is part of the RAG application.
  • πŸ“ The tutorial also covers how to create a user interface using 'Gradio' to make the RAG application more accessible.
  • πŸŽ₯ The presenter encourages viewers to subscribe to their YouTube channel for more Artificial Intelligence-related content.
  • πŸŽ‰ The video concludes by showcasing the completed RAG application with a user interface that can answer questions based on the provided context.

Q & A

  • What is the main topic of the video?

    -The main topic of the video is about using 'ollama' embeddings to create a Retrival-Augmented Generation (RAG) application with better performance.

  • What is the purpose of using embeddings in the application?

    -The purpose of using embeddings is to ingest data from URLs, convert it to embeddings, and store it in a Vector database to retrieve relevant data when a question is asked.

  • Which embeddings model is being used in the video?

    -The video uses the 'nomic' embed text model for its higher context length and surpassing performance over OpenAI embedding models.

  • What is the role of the Vector database in this context?

    -The Vector database, specifically Chroma DB, is used to store the embeddings of the ingested data, allowing for efficient retrieval of relevant information.

  • What is the language model used in conjunction with the embeddings?

    -The language model used in conjunction with the embeddings is 'mistal', which is used to generate responses to questions based on the retrieved data.

  • What is the significance of the RAG process in the application?

    -The RAG process is significant as it allows the application to retrieve relevant documents and then use the context to generate more accurate and relevant answers to questions.

  • What is the user interface tool used to interact with the application?

    -The user interface tool used to interact with the application is Gradio, which provides an easy-to-use interface for users to input URLs and questions.

  • How does the video guide the viewers on setting up the application?

    -The video guides viewers step by step, starting from installing necessary libraries, creating an 'app.py' file, defining the model, splitting data, converting to embeddings, and finally setting up the user interface.

  • What is the performance of the embeddings process as mentioned in the video?

    -The video mentions that the embeddings process took approximately 219 milliseconds, indicating a fast performance.

  • How can viewers stay updated with similar content?

    -Viewers are encouraged to subscribe to the presenter's YouTube channel, click the Bell icon to stay tuned, and like the video to help others find it.

  • What is the final outcome of the application after implementing the RAG process?

    -The final outcome is a RAG application that runs completely locally on the user's machine with zero cost, providing answers based on the context provided through the user interface.

Outlines

00:00

πŸš€ Building a Local AI Application with OLAMA Embedding

The video script introduces a process for creating a locally runnable AI application using OLAMA embeddings. The presenter explains how to ingest data from URLs, convert it into embeddings, and store it in a Vector database for efficient retrieval. The goal is to enhance the performance of a language model by providing relevant data, which is achieved by using the nomic embed text model for its superior context length. The script guides viewers through installing necessary libraries, setting up the model, and creating a user interface with Gradio. The presenter also emphasizes the importance of subscribing to their YouTube channel for more AI-related content and provides a step-by-step tutorial on implementing the application.

05:01

πŸ’» Demonstrating the Local AI Application with a User Interface

In the second paragraph, the script focuses on demonstrating the functionality of the locally created AI application. The presenter shows how to run the application using a terminal command and provides a live example of how to use the user interface. They input URLs and ask a question about 'olama', which the application processes to provide a context-based response. The response clarifies that 'olama' is a platform for running large language models locally across different operating systems. The presenter expresses excitement about the project and invites viewers to stay tuned for more similar content, encouraging likes, shares, and subscriptions to their channel.

Mindmap

Keywords

πŸ’‘Ollama

Ollama refers to a local AI model server that enables users to run large language models on their own machines. In the context of the video, Ollama is used to create a Retrival-Augmented Generation (RAG) application that can operate completely locally with zero cost. The script mentions using Ollama for embedding, which is a process of converting data into a format that can be understood and manipulated by machine learning models.

πŸ’‘Embedding

Embedding in the video script refers to the process of converting data, such as text, into numerical vectors that can be used in machine learning models. The script discusses using 'nomic embed text' for creating embeddings with higher context length, surpassing other embedding models like OpenAI's. Embeddings are crucial for the RAG application as they allow for the efficient storage and retrieval of information in the Vector database.

πŸ’‘Vector Database

A Vector Database is a type of database designed to store and manage vectorized data, which are numerical representations of information. In the script, the Vector database is used to store the embeddings created from the data ingested from URLs. This allows for efficient retrieval of relevant data when a question is asked, which is a key component of the RAG application being demonstrated.

πŸ’‘RAG Application

RAG stands for Retrieval-Augmented Generation and refers to a type of AI application that combines the capabilities of retrieval systems with generative models. The video script details the creation of a RAG application using Ollama for embeddings and a language model for generating responses. The application is designed to provide more relevant answers by retrieving and using contextually relevant data.

πŸ’‘Chroma DB

Chroma DB is a tool mentioned in the script for managing embeddings and performing retrieval tasks. It is used to store the embeddings created from the ingested data and to retrieve relevant documents when a question is asked. Chroma DB is integral to the functioning of the RAG application by providing a means to organize and access the embedded data.

πŸ’‘Language Model

A language model in the context of the video is a type of machine learning model that is trained to understand and generate human-like text. The script refers to two language models: 'nomic embed text' for creating embeddings and 'mistal' for generating responses in the RAG application. Language models are central to the video's theme as they are used to process and produce the textual output.

πŸ’‘Gradio

Gradio is a Python library used for creating user interfaces for machine learning models. In the script, Gradio is used to add a user interface to the RAG application, allowing users to input URLs and questions and receive responses. Gradio simplifies the interaction with the application, making it more accessible and user-friendly.

πŸ’‘Retrieval

Retrieval in the context of the video refers to the process of searching for and retrieving relevant information from a database in response to a query. The script describes using a retriever in the RAG application to find relevant documents from the Vector database when a question is asked, which is a critical step in providing accurate and contextually relevant answers.

πŸ’‘LangChain

LangChain is a Python library mentioned in the script for assembling various components of a language model application. It is used to import necessary modules and functions for creating the RAG application, such as web-based loaders, embeddings, and chat prompt templates. LangChain serves as the framework that brings together the different elements of the application.

πŸ’‘Prompt Template

A prompt template in the video script is a predefined structure or set of instructions used to guide the input into a language model. The script discusses using a 'chat prompt template' to format the questions and prompts for the language model. Prompt templates are important for ensuring that the model receives clear and structured input, which can improve the quality of the generated responses.

πŸ’‘Local AI Model Server

A local AI model server, as mentioned in the script in relation to Ollama, is a server that runs on an individual's own machine rather than relying on cloud-based services. This allows for the execution of large language models locally, which can offer benefits such as reduced latency and increased privacy. The video demonstrates creating a RAG application that can function as a local AI model server.

Highlights

Introduction to OLLama embedding for creating RAG applications with better performance.

Data ingestion from URLs, conversion to embeddings, and storage in a Vector database for efficient retrieval.

Utilization of Chroma DB and Nomic embedding for enhanced context length surpassing OpenAI models.

Demonstration of a user interface created with Gradio for interacting with the language model.

Explanation of why Nomic Embed Text is preferred for its higher context length.

Step-by-step guide on creating a RAG application using Lang Chain.

Installation instructions for Lang Chain packages using pip.

Creation of an app.py file and importing necessary modules from Lang Chain.

Definition and setup of the Mistral model for the RAG application.

Process of splitting data into chunks with a specified overlap for better context.

Conversion of documents into embeddings and storage in the Vector database using Chroma DB.

Performance comparison before and after RAG to demonstrate the improvement.

Invoking the RAG chain with a prompt template and obtaining the output.

Efficiency of the embedding process, taking approximately 219 milliseconds.

Clarification that OLLama is a local AI model server for running large language models locally.

Introduction of a user interface for the RAG application using Gradio.

Final demonstration of the RAG application with a user interface, asking a question and receiving a response.

Encouragement to subscribe to the YouTube channel for more AI-related content.

Transcripts

00:00

this is amazing now we have o llama

00:03

embedding you can create rag application

00:06

with better performance using this

00:08

embedding in this we are going to ingest

00:10

data from URL convert those to

00:13

embeddings and then store in Vector

00:15

database so when we ask a question a

00:17

relevant data will be sent to the lodge

00:19

language model using Ola and finally we

00:22

get a more relevant answer we are going

00:25

to use chroma DB nomic embedding and

00:27

mless large language model finally going

00:30

to add that in user interface using

00:32

gradio that's exactly what we're going

00:34

to see today let's get

00:35

[Music]

00:37

started hi everyone I'm really excited

00:39

to show you about ol Lama embedding

00:42

especially we're going to use nomic

00:44

embed text model for embedding why nomic

00:47

nomic embed text has a higher context

00:50

length and it surpasses open AI

00:53

embedding models you can see noric

00:55

embedding model performance in this

00:57

chart finally we are going to create a

00:59

user interface like this this I'm going

01:00

to take you through step by step on how

01:02

to do this but before that I regularly

01:04

create videos in regards to Artificial

01:05

Intelligence on my YouTube channel so do

01:07

subscribe and click the Bell icon to

01:08

stay tuned make sure to click the like

01:10

button so this video can be helpful for

01:11

many others like you in this we are

01:13

going to use l chain put all the pieces

01:16

together so pip install Lang chain Lang

01:18

chain community and Lang chain core and

01:19

then click enter next create a file

01:21

called app.py and let's open it inside

01:23

the F from longchain Community import

01:26

web based loader chroma embeddings chat

01:29

ol

01:30

runable pass through string out passer

01:33

chat prom template character text

01:35

splitter now we going to define the

01:37

model which is mistal so now we're going

01:39

to see four steps one is to retrieve the

01:41

data from the URL split the data and the

01:44

second step is to convert that to

01:45

embedding and store in Vector DB and

01:47

third we are going to perform the rag so

01:50

first split data into chunks so I'm

01:52

passing the list of URLs now I'm going

01:54

to use web base loader to extract all

01:56

the data from those URLs next combining

01:59

those data now we are going to use

02:01

character textt splitter and here we are

02:02

going to divide the chunk and the chunk

02:05

overlap is 100 for example this is a

02:07

chunk size and the overlap between

02:10

different chunks is chunk overlap next

02:13

splitting those documents now we have

02:15

completed the first step of splitting

02:17

the chunk next we're going to convert

02:19

those document into embeddings and store

02:21

them in Vector database so we going to

02:23

initiate chroma DB so from documents

02:26

that's where you pause all the documents

02:29

give a name for the collection and here

02:31

is where we are defining the olama

02:33

embering so the model name we are giving

02:35

is nomic Ember text next we are using

02:39

the retriever this is used to retrieve

02:41

relevant documents when we ask a

02:43

question so the third step is rag so we

02:46

going to compare before Rag and after

02:49

rag so printing for our reference before

02:52

rag template and getting the promt

02:54

template using chat promt template next

02:57

creating the rag chain so first The

03:00

Prompt will be sent to the lar language

03:02

model so the prompt is what is the topic

03:04

name and then that will be sent to the

03:06

LOD language model mistal and finally we

03:09

get the output now I'm going to print

03:11

and invoking the chain and providing the

03:14

topic which is Ol so that is before rag

03:16

next after rag the same process as

03:19

before so we are defining the rag

03:21

template here the main difference is

03:23

that we are providing the context prom

03:26

template as before and then we defining

03:28

the rack chain here we providing the

03:30

context and also the question finally we

03:32

are invoking the chain and asking a

03:34

question that's it only few lines of

03:36

code so first we extracted the data from

03:39

the URL and split the document into

03:40

chunks next we converted those to

03:43

embeddings store them in chroma DB next

03:46

we are passing the promt template and

03:48

invoking the chain now I'm going to run

03:50

this code so make sure you've downloaded

03:51

AMA then AMA pull nomic embed text to

03:56

pull the model also AMA pull mistal to

03:59

download m model now type Python app.py

04:01

and then click enter so here I have

04:03

added the log on the right hand side so

04:05

you can see the performance so you can

04:07

see embedding took 219 milliseconds

04:11

approximately so that is really fast and

04:13

also you got the answer here so before

04:15

rag I'm sorry for the confusion but

04:18

olama doesn't seem to be a widely

04:20

recognized term so after rag olama is a

04:24

local AI model server that allows user

04:27

to run large language model on their own

04:29

machine

04:30

now we have created a rag application

04:32

which can run completely locally on your

04:34

machine with zero cost now we are going

04:36

to add user interface to this so I've

04:38

modified the code a little bit added

04:40

gradio at the top then I moved

04:42

everything to the function process input

04:45

with the list of URLs and the question

04:47

so I'm going to split the URLs then do

04:49

the same process again and finally

04:51

assigning the function name here having

04:54

two inputs one is for entering the list

04:55

of URLs and another one to ask a

04:58

question finally in interface. launch

05:00

now I'm going to run this code in your

05:02

terminal Python ui. piy and then click

05:04

enter now I got the URL here I'm going

05:06

to open it so here's the URL I'm going

05:09

to provide this URL next going to

05:11

provide this URL and going to ask what

05:13

is olama and click submit now it's

05:16

processing the request and here is the

05:18

response based on the context provided

05:20

olama is a platform or software that

05:23

enables user to run and create large

05:25

language models locally with support for

05:28

Mac OS Linux and Windows this is

05:30

exciting I'm really excited about this

05:33

I'm going to create more videos similar

05:34

to this so stay tuned I hope you like

05:36

this video do like share and subscribe

05:38

thanks for watching

Rate This
β˜…
β˜…
β˜…
β˜…
β˜…

5.0 / 5 (0 votes)

Related Tags
AI Language ModelsLocal HostingNomic EmbeddingsMistal ModelRAG ApplicationVector DatabaseChroma DBWeb ScrapingLarge Language ModelUser InterfaceGradio