Last Week in AI #162 - Udio Song AI, TPU v5, Mixtral 8x22, Mixture-of-Depths, Musicians sign...

The episode begins with hosts Andre and Jeremy introducing themselves and sharing some personal updates. Andre, a Stanford PhD graduate, now works at a generative AI startup, while Jeremy is the co-founder of Gladstone AI, an AI National Security Company. They discuss a recent media appearance and its aftermath, as well as a potential new podcast intro song generated by AI. Jeremy also shares his surprise encounter with John Cron from the Super Data Science podcast and Stie St Lawrence, founder of Women and Data, in New York City.

The hosts dive into the AI music generation space, discussing the recent developments and the excitement surrounding tools like Yudo and Sunno. They mention the impressive quality of AI-generated music, which is almost indistinguishable from real songs. The conversation also touches on the potential applications of music generation, such as educational aids and marketing, and the significant investments in this area, highlighting the potential for game-changing impact.

The discussion shifts to Anthropic's launch of an external tool use feature for CLAI, enabling stock taker integrations and more. The hosts explain how this feature allows Claude to use third-party features and APIs, significantly enhancing its capabilities. They also delve into the accuracy of tool selection and use, which is a key metric in tracking progress towards AGI, and the potential implications of these advancements.

The hosts cover the development of building LLMs for code repair from Replit, discussing how AI tools are being integrated to fix bugs in source code. They highlight Replit's unique data advantage, derived from its platform's history of code edits, and the potential for AGI development in this area. The conversation also touches on the open-source movement and its significance in the AI community.

The episode continues with a discussion on the early reviews of the Human AI pin, a wearable device that functions as a new type of AI-first hardware. The hosts share insights on the device's initial reception, noting that it may require further refinement due to issues like slow response times and bugs. They also ponder the future of AI-driven devices and their potential to redefine hardware categories.

The hosts discuss Microsoft's integration of GPT-4 into its 365 Copilot, enhancing image generation capabilities for business subscribers. They delve into the improvements brought by GPT-4 Turbo, such as updated knowledge cutoff and better reasoning capabilities. The conversation also touches on the competitive landscape in AI, with companies like Google and OpenAI striving to outperform each other in terms of technology advancements.

The hosts highlight the expansion of AI editing tools to all Google Photos users, including features like Magic Eraser, photo blur, and portrait light. They discuss the implications of this move in the context of smartphone competition and the increasing integration of AI-powered features in devices. The conversation also touches on Google's announcement of the Cloud TPU v5p, its most powerful AI accelerator yet, and its potential impact on large language model training.

The hosts discuss Meta's unveiling of a new version of its custom AI chip, the Meta Training and Inference Accelerator (MTI). They compare Meta's efforts to those of Google and Microsoft in the AI chip space, highlighting the importance of hardware advancements in the tech industry. The conversation also explores the potential for profit in the hardware level of the AI stack and the strategic moves by major tech companies to develop their own chip designs.

The hosts talk about Intel's new AI accelerator, the Vatti Freeship, which is set to enhance training and inference performance. They discuss Intel's claims about the accelerator's performance and power efficiency compared to Nvidia's H100, and the potential impact on the semiconductor market. The conversation also touches on the importance of production scaling and the challenges Intel faces in catching up with competitors like Nvidia.

The hosts discuss Adobe's efforts to build an AI video generation model by paying photographers and artists to submit videos of everyday actions and emotions. They highlight Adobe's strategy of using proprietary content for training to avoid copyright issues, and the potential for this approach to become more common in the future. The conversation also touches on OpenAI's reported transcription of YouTube videos to train its GT4 model, and the legal gray areas surrounding data usage in AI training.

The hosts mention Weo's launch of its paid robotaxi service in Los Angeles, following a year of offering free tours. They discuss the expansion of the service, which has seen significant interest from the public, and the potential for further growth in other cities. The conversation also includes brief mentions of other companies testing similar services in different locations.

The hosts discuss recent changes in the ownership structure of OpenAI's startup fund, which was previously owned by Sam Altman. They highlight the unusual nature of the fund's structure and the potential implications of the shift to a more traditional partnership model. The conversation also touches on the lack of clarity surrounding the fund's operations and the broader intrigue surrounding OpenAI's organizational structure.

The hosts highlight the launch of Mistral's new Mix Draw 8X 22B model, an open-source AI model with a large parameter set and no restrictions on commercial use. They discuss the model's capabilities, its significance in the open-source community, and the potential impact on AI development. The conversation also touches on the challenges of using large models that require significant computational resources.

The hosts discuss Google's announcement of new additions to its Gemma family of lightweight open-source AI models, including Code Gemma for coding and Recurrent Gemma with a recurrent structure. They highlight the models' efficiency, performance, and potential applications, as well as the focus on throughput and the importance of efficient hardware use in AI systems.

The hosts discuss the launch of Aurora-M, the first open-source multilingual language model developed in accordance with the US executive order. They highlight the model's rigorous evaluation across various tasks and languages, and its alignment with safety and ethical considerations. The conversation also touches on the model's performance and the significance of its development in the context of regulatory frameworks.

The hosts delve into recent research and advancements in AI, including DeepMind's work on dynamically allocating compute in Transformer-based language models and Google's approach to scaling Transformers with infinite context. They discuss the potential impact of these developments on AI capabilities and the ongoing efforts to optimize AI models for better performance and efficiency.

The hosts discuss policy and safety issues in AI development, including a new legislation proposal by Representative Adam Schiff requiring organizations to disclose the use of copyrighted data in AI training. They also cover the case of a Google software engineer charged with stealing AI secrets and the broader implications for intellectual property and national security in the AI sector.

The hosts wrap up the episode with a lightning round of quick AI news stories, including a policy paper on responsible reporting for Frontier AI development, the US government's efforts to address AI electricity demands, a legal case blocking the use of AI-enhanced video as evidence, and a significant investment in AI by the Canadian government, including the launch of an AI safety institute.