Run llama 2 with Ollama on Windows locally
TLDRThe video script provides a step-by-step guide on downloading and installing a software on Windows, followed by the process of downloading and running a machine learning model locally. It explains the commands to be used in PowerShell and the difference in commands when compared to the common prompt. The video also demonstrates how to operate the software offline and highlights the performance difference when using a CPU versus a graphics card. The script concludes by encouraging users to explore the variety of available models for local computation.
Takeaways
- 📂 Start by downloading the software by navigating to the official website and selecting the appropriate download link for your operating system.
- 🖥️ After downloading, open the file to initiate the installation process and follow the on-screen instructions.
- 🔄 Once installed, locate the software either on your Start menu or in the specified user directory.
- 💡 Run the software by finding its executable file, which can be launched from the command line interface (CLI).
- 📱 To use the software, open the terminal within the directory where the executable is located and run necessary commands.
- 🔑 Use specific commands like 'do/o' to deselect options and 'pool' to download models.
- 🛠️ Choose the model you want to use from the available options, such as 'Llama 2' for this tutorial.
- 📚 Learn about the different commands available in the software, such as 'serve', 'list', 'run', 'pull', and 'push'.
- 📋 Understand the syntax required for commands in PowerShell, which may differ from the common prompt.
- 🌐 Note that some functionalities require an internet connection, while others can be executed offline.
- 🚀 Experience may vary depending on your hardware capabilities, such as the presence of a graphics card for faster responses.
Q & A
What is the first step to download AMA?
-The first step is to open your web browser and navigate to the AMA download page by typing '.com' in the address bar.
How does one download AMA if they are using Windows?
-For Windows users, they should click on the 'Download for Windows' link on the AMA download page.
What should you do after the download is complete?
-After the download is complete, you should click on 'Open' to execute the downloaded file.
Where can the installed software be found on a Windows system?
-The installed software can be found in the Start menu or at the location 'C:\\Users\\[username]\\AppData\\Roaming\\Microsoft\\Windows\\Start Menu\\Programs'.
How is the terminal opened in the AMA software?
-To open the terminal, deselect everything, right-click on the empty space, and select 'Open Terminal'.
What command is used to see the list of available models in AMA?
-The 'list' command is used to see the list of available models.
How can a model be downloaded using the 'pool' command?
-To download a model, go to the terminal and type '/Lama pool [model name]', for example, '/Lama pool llama 2' to download the 'Llama 2' model.
What is the difference between running a model with 'do/o' and 'do/M' in PowerShell?
-In PowerShell, 'do/M' should be used to run the model, whereas in the common prompt, 'do/o' is used. The difference is the inclusion of a dot slash in PowerShell.
How can you check the list of models on your system?
-To check the list of models, open the terminal and type 'do/o l' or 'do/M l' in PowerShell, depending on the command prompt used.
What happens when you run a model offline?
-When running a model offline, the response is generated without any connection to the internet, as demonstrated by running the 'Llama 2' model while in airplane mode.
How can you find more information about a specific model?
-To find more information about a model, you can click on 'F' next to the model's name to view its description, including details like the model's size, creator, and last update.
Outlines
📥 Downloading and Installing AMA
The first paragraph outlines the process of downloading and installing AMA software. It begins with navigating to the website and selecting the appropriate download link based on the user's operating system. The speaker, using Windows, clicks on the Windows download link and waits for the file to download. After downloading, the user is instructed to open the file, which leads to the installation process. Once installed, the software can be found on the Start menu or in the specific directory based on the user's system settings. The executable file is located in the 'C:/users/[username]/AppData/Local/Programs' directory. The speaker then proceeds to run the software on the terminal by deselecting all options and opening a terminal window. The correct command is entered to begin the download of a specific model, which is 'Llama 2' in this case, and the user is left to wait for the model to download.
📝 Understanding and Running AMA Commands
The second paragraph delves into the specifics of using AMA commands within PowerShell and the common command prompt. It explains the necessary commands to run the executable, such as 'do/o' for PowerShell and 'olm' for the common command prompt. The paragraph also covers various AMA commands, including 'serve' to create a model, 'list' to show the models on the PC, 'run' to execute a model, 'pull' to download a model from a registry, and 'push' to upload a model to a registry. The 'cp' and 'rm' commands are briefly mentioned for copying and removing models, respectively. The 'help' command is highlighted as a resource for understanding all available commands. The speaker demonstrates the 'list' and 'help' commands, showing how to view the available models and get information about the commands. It is noted that models can be run offline, as long as they have been previously downloaded.
🌐 Running Models Offline and Exploring Options
The final paragraph discusses the variety of models available for offline use and provides guidance on how to run them. It emphasizes that users can simply type the model's name to run it on their PC. Additional information about the models, such as descriptions and updates, can be accessed by clicking on the 'F' key. The paragraph mentions a specific model with a 2.7 million parameter count, developed by Microsoft Research, and last updated seven weeks prior. The speaker encourages users to explore the different models and try them out on their computers, highlighting that they can function without an internet connection. The paragraph concludes with a reminder that running models offline is possible and thanks the viewer for watching.
Mindmap
Keywords
💡download
💡installation
💡terminal
💡model
💡offline
💡PowerShell
💡executable
💡registry
💡comments
💡CPU
💡graphics card
Highlights
The process begins by downloading AMA through a browser on various operating systems such as MacOS, Linux, and Windows.
For Windows users, there is a specific link to click on for downloading the software.
After downloading, the file initiates an installation process which leads to the software being accessible from the Start menu or its installation directory.
Running the software involves opening the terminal and deselecting all options before utilizing the shift and right-click combination to open the terminal.
The tutorial demonstrates the use of PowerShell on Windows, which differs from the common prompt by requiring a dot slash in commands.
The 'serve' command in PowerShell is used to start a server for the model, while 'list model' shows the available models on the PC.
The 'run' command is utilized to execute the model, and 'pool' is used to download a model from the registry.
The 'push' command allows for the uploading of a personal model to the registry, and 'list common' displays all available models from the AMA.
The 'CP' and 'RM' commands are used for copying and removing models, respectively, and 'help' provides information about the commands.
The transcript showcases the ability to run a model offline by turning off the internet connection and using the 'do/olma run' command.
Even without an internet connection, the model can function and provide responses, albeit slower on a CPU without a dedicated graphics card.
The transcript mentions the capability to run various models locally on a PC, providing examples of different models and their functionalities.
The model 'Llama 2' is highlighted as an example, which is a 2.7 million parameter model created by Microsoft Research and was last updated 7 weeks prior.
The tutorial emphasizes the practical application of running models offline, which can be beneficial for users with limited or no internet access.