Perplexity AI: How We Built the World's Best LLM-Powered Search Engine in 6 Months, w/ Less Than $4M
TLDRThe speaker discusses the journey of building a search engine that aims to revolutionize the way we find information. Starting with a focus on enterprise search and SQL, they pivoted to creating a research assistant powered by AI. The product development was driven by their own needs and the desire to answer complex queries that traditional search engines like Google couldn't handle. The talk highlights the iterative process of improving the AI, incorporating user feedback, and the shift towards building their own models for better control and customization. The speaker emphasizes the importance of innovation and staying ahead in the competitive landscape of AI technology.
Takeaways
- π The speaker discusses the journey of building a search engine for the future, highlighting the transition from traditional search engines to more intuitive and personalized research assistants.
- π§ The initial focus was on text-to-SQL, which was a pivot from consumer search, and led to the development of a research assistant capable of answering complex queries.
- π‘ The company's inception began with a focus on enterprise search and the challenges faced in understanding and navigating complex data sets without expert guidance.
- π Key investors like Elon Musk, Nat Friedman, and Jeff Dean played a significant role in backing the company and its vision for enterprise search with LLMs and SQL.
- π The development process was not linear, with many iterations and learnings along the way, including the integration of real-time search and summarization to enhance the chatbot experience.
- π€ The realization that a search engine needed to be plugged into a real data source to provide trustworthy answers was a critical insight that shaped product development.
- πΌ The importance of building a product that the creators and their friends use themselves was emphasized, as it helps in achieving product-market fit and provides a long-lasting motivation.
- π₯ The launch of the web search feature for internal use led to the creation of a Discord bot, which received positive feedback and demonstrated the potential of the AI product.
- π The launch of Perplexity, a search engine built on AI, coincided with the launch of Chat GPT, and the company continued to innovate by adding features like focused searches on platforms like Wikipedia and Stack Overflow.
- π οΈ The company's commitment to rapid iteration and improvement, as well as its ability to adapt to market changes and user feedback, has been a key factor in its success and competition with tech giants.
Q & A
What is the main goal of the research assistant being developed in the transcript?
-The main goal is to create the world's best research assistant that can answer any question directly, providing a more intuitive and useful experience than traditional search engines like Google.
How did the team initially approach the development of their search product?
-The team initially worked on text-to-SQL, focusing on enterprise search with LLMs and SQL, before transitioning to web search for their own use and eventually launching it publicly.
What challenges did the team face when starting the company?
-The team faced challenges such as difficulty in securing funding for search-related work, a lack of expertise in company building, and the complexity of understanding and searching over various data sources like Salesforce and HubSpot.
How did the team use their own product to solve real-world problems?
-The team used their product to answer their own questions about company operations, such as how to start an Uber server with Ruby, understanding SQL query languages, and navigating insurance policy terms.
What was the significance of the launch of Chat GPT and how did it impact the team's direction?
-The launch of Chat GPT was considered an 'iPhone moment for AI' and it broke several myths about chatbots and platforms. This encouraged the team to continue developing their product, despite initial doubts, as they saw the potential for real-world application and user interest.
What are some key features that differentiate Perplexity from other search engines?
-Perplexity differentiates itself by providing direct answers to complex queries, integrating real-time search indices, offering a conversational and multiplayer interface, and allowing users to upload files for more personalized research assistance.
How did the team improve their product based on user feedback and interactions?
-The team improved their product by iterating rapidly, adding features like focused searches on specific platforms (e.g., Wikipedia, Stack Overflow), making the search fully end-to-end conversational, and introducing the Copilot feature for a more dynamic and interactive browsing experience.
What is the significance of the fine-tuning API from Open AI in the development of Perplexity?
-The fine-tuning API from Open AI allowed Perplexity to achieve better performance in terms of speed and user experience. It enabled them to offer a product that is almost instantaneous in response time, significantly improving the user experience compared to earlier versions.
What is the role of open-source models in the development strategy of Perplexity?
-Open-source models play a role in the development strategy of Perplexity by providing a base for customization and improvement. The team at Perplexity sees the value in having their own models to ensure better control over pricing and product customization, while also leveraging open-source models for their development.
How does Perplexity ensure the quality and authenticity of the citations provided in its search results?
-Perplexity maintains the quality and authenticity of its citations by using its own page ranks for the web and algorithms that are effective at selecting relevant links. This approach enhances the relevance ranking and ensures that the search results provided are of high quality.
What are some future directions for Perplexity in terms of product development and market positioning?
-Future directions for Perplexity include further refining their own models, improving the product based on user feedback, and potentially moving towards serving their own models. They also aim to differentiate themselves as an end-to-end platform for research and collaboration, with features like collections that allow users to save and share information persistently.
Outlines
π Introduction to the Future of Search and Perplexity's Journey
The speaker begins by setting the stage for a discussion on the future of search engines, emphasizing the need for an intuitive and essential product for consumers. They introduce the concept of Perplexity, a research assistant designed to answer complex questions directly, contrasting it with the traditional,εΉΏε-heavy search experience. The narrative then shifts to the company's inception, revealing that the initial focus was on text-to-SQL, which eventually led to the development of a powerful research assistant. The speaker highlights the organic growth and evolution of the product, from its humble beginnings to its current state, spurred by the team's own need for a better search solution.
π Building a Product for Personal and Team Use
The speaker discusses the famous advice by Paul Graham and Y Combinator on building a product that you and your friends use. They share their experience of learning from the product's usage, especially in the area of company building and health insurance. The speaker emphasizes the limitations of traditional search engines like Google in answering complex queries and the utility of their AI-powered research assistant in filling this gap. They also share anecdotes of using the assistant to understand real-time events and improve their internal processes, highlighting the product's practical applications and its evolution into a more interactive and multiplayer platform.
π‘ Breaking Myths and Enhancing the AI Experience
The speaker shares how their product challenges existing myths about chatbots and search engines, inspired by the launch of Chat GPT. They discuss the development of features that allow for more specific searches on platforms like Wikipedia and Stack Overflow, significantly improving the coding queries. The narrative continues with the decision to focus solely on web search after the high API prices made other options unfeasible. The speaker also touches on the rapid iteration and improvement of their product, emphasizing the importance of speed and user experience, and the introduction of innovative features like Copilot, which enhances the browsing experience.
π― Fine-Tuning Models and Balancing Speed with Quality
The speaker delves into the technical aspects of fine-tuning models, comparing the performance of GPD 3.5 and GPT-4. They highlight the challenges of balancing speed, reliability, and user experience, and how they aim to combine the best aspects of both models. The speaker also discusses the significance of cost-effectiveness in using fine-tuned models and the improvements in throughput and latency. They share insights on how their product stands out in terms of utility and citation scores, and the strategic decision to serve their own models to maintain control over pricing and product customization.
π οΈ Enhancing the Research Workflow with File Uploads and Collaboration
The speaker introduces the file upload feature as an essential component of the research workflow, allowing users to upload documents for more personalized queries. They discuss the collaboration with Anthropic to enhance this feature and the positive reception it received. The speaker then introduces 'collections', a feature that transitions the product from a simple tool to a comprehensive platform, enabling users to save, organize, and collaborate on information. They emphasize the importance of this transition in differentiating their product and establishing it as a daily tool for users.
π The Role of Open Source and Custom Models in the Future
The speaker discusses the role of open source models and the decision to train their own models to have more control over the product. They share insights on the competition between fine-tuning existing models and using open source models, and how they aim to balance cost-effectiveness with the need for customization. The speaker also talks about their efforts to improve their custom inference stack for faster performance and the importance of controlling pricing to maintain a competitive edge. They conclude with a discussion on the potential future developments and improvements for their product.
Mindmap
Keywords
π‘Search
π‘Perplexity
π‘Enterprise Search
π‘Chatbot
π‘Fine-tuning
π‘AI
π‘Product-Market Fit
π‘User Experience (UX)
π‘Open Source
π‘Competition
π‘Innovation
Highlights
The speaker discusses the future of search and the creation of the world's best research assistant.
The speaker shares the non-linear journey of building a search product, starting with text to SQL which was unrelated to consumer search.
The company was initially focused on enterprise search with LLMs and SQL, attracting investments from prominent figures like Elon Musk and Nat Friedman.
The speaker emphasizes the importance of building products that the creators themselves use, as seen with their internal use of a Slack bot for company-related queries.
The launch of web search for internal use led to the creation of Perplexity, which happened a week after the launch of ChatGPT.
The speaker highlights the limitations of Google in answering complex queries compared to the capabilities of AI.
The transition from working on SQL for enterprises to focusing on web search due to high API prices and the popularity of the web search product.
The introduction of Copilot as a browsing companion, which represents a shift from a simple chatbot UI to a more interactive and dynamic experience.
The speaker discusses the technical improvements and iterations, including the use of fine-tuned models and the integration of OpenAI's API.
The launch of Collections as a step towards becoming an end-to-end platform for research and collaboration.
The speaker talks about the company's strategy of not just being a 'rapper' but also serving their own models, with the launch of llama models in Perplexity Labs.
The speaker shares insights on the competition with other search engines and the importance of continuous iteration and improvement.
The speaker addresses the challenges of integrating various APIs and the focus on providing a reliable and high-quality user experience.
The speaker discusses the decision to fine-tune models for better utility and safety, moving away from the limitations of models like LLM that prioritize safety over utility.
The speaker talks about the company's roadmap, including the development of their own infrastructure for fine-tuning models and the potential of serving their own models in the product.
The speaker highlights the importance of controlling pricing and the benefits of having their own models for customization and cost control.
The speaker concludes with a discussion on the role of open source, the company's plans for future model development, and the balance between using cloud services and their own models.