MLOps.community

Robert Caulk is responsible for directing software development, enabling research, coordinating company projects, quality control, proposing external collaborations, and securing funding. He believes firmly in open-source, having spent 12 years accruing over 1000 academic citations building open-source software in domains such as machine learning, image analysis, and coupled physical processes. He received his Ph.D. from Université Grenoble Alpes, France, in computational mechanics. Unleashing Unconstrained News Knowledge Graphs to Combat Misinformation // MLOps Podcast #279 with Robert Caulk, Founder of Emergent Methods. // Abstract Indexing hundreds of thousands of news articles per day into a knowledge graph (KG) was previously impossible due to the strict requirement that high-level reasoning, general world knowledge, and full-text context *must* be present for proper KG construction. The latest tools now enable such general world knowledge and reasoning to be applied cost effectively to high-volumes of news articles. Beyond the low cost of processing these news articles, these tools are also opening up a new, controversial, approach to KG building - unconstrained KGs. We discuss the construction and exploration of the largest news-knowledge-graph on the planet - hosted on an endpoint at AskNews.app. During talk we aim to highlight some of the sacrifices and benefits that go hand-in-hand with using the infamous unconstrained KG approach. We conclude the talk by explaining how knowledge graphs like these help to mitigate misinformation. We provide some examples of how our clients are using this graph, such as generating sports forecasts, generating better social media posts, generating regional security alerts, and combating human trafficking. // Bio Robert is the founder of Emergent Methods, where he directs research and software development for large-scale applications. He is currently overseeing the structuring of hundreds of thousands of news articles per day in order to build the best news retrieval API in the world: https://asknews.app. // MLOps Swag/Merch https://shop.mlops.community/ // Related Links Website: https://emergentmethods.ai News Retrieval API: https://asknews.app --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Rob on LinkedIn: https://www.linkedin.com/in/rcaulk/ Timestamps: [00:00] Rob's preferred coffee [00:05] Takeaways [00:55] Please like, share, leave a review, and subscribe to our MLOps channels! [01:00] Join our Local Organizer Carousel! [02:15] Knowledge Graphs and ontology [07:43] Ontology vs Noun Approach [12:46] Ephemeral tools for efficiency [17:26] Oracle to PostgreSQL migration [22:20] MEM Graph life cycle [29:14] Knowledge Graph Investigation Insights [33:37] Fine-tuning and distillation of LLMs [39:28] DAG workflow and quality control [46:23] Crawling nodes with Phi 3 Llama [50:05] AI pricing risks and strategies [56:14] Data labeling and poisoning [58:34] API costs vs News latency [1:02:10] Product focus and value [1:04:52] Ensuring reliable information [1:11:01] Podcast transcripts as News [1:13:08] Ontology trade-offs explained [1:15:00] Wrap up

Guanhua Wang is a Senior Researcher in DeepSpeed Team at Microsoft. Before Microsoft, Guanhua earned his Computer Science PhD from UC Berkeley. Domino: Communication-Free LLM Training Engine // MLOps Podcast #278 with Guanhua "Alex" Wang, Senior Researcher at Microsoft. // Abstract Given the popularity of generative AI, Large Language Models (LLMs) often consume hundreds or thousands of GPUs to parallelize and accelerate the training process. Communication overhead becomes more pronounced when training LLMs at scale. To eliminate communication overhead in distributed LLM training, we propose Domino, which provides a generic scheme to hide communication behind computation. By breaking the data dependency of a single batch training into smaller independent pieces, Domino pipelines these independent pieces of training and provides a generic strategy of fine-grained communication and computation overlapping. Extensive results show that compared with Megatron-LM, Domino achieves up to 1.3x speedup for LLM training on Nvidia DGX-H100 GPUs. // Bio Guanhua Wang is a Senior Researcher in the DeepSpeed team at Microsoft. His research focuses on large-scale LLM training and serving. Previously, he led the ZeRO++ project at Microsoft which helped reduce over half of model training time inside Microsoft and Linkedin. He also led and was a major contributor to Microsoft Phi-3 model training. He holds a CS PhD from UC Berkeley advised by Prof Ion Stoica. // MLOps Swag/Merch https://shop.mlops.community/ // Related Links Website: https://guanhuawang.github.io/ DeepSpeed hiring: https://www.microsoft.com/en-us/research/project/deepspeed/opportunities/ Large Model Training and Inference with DeepSpeed // Samyam Rajbhandari // LLMs in Prod Conference: https://youtu.be/cntxC3g22oU --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Guanhua on LinkedIn: https://www.linkedin.com/in/guanhua-wang/ Timestamps: [00:00] Guanhua's preferred coffee [00:17] Takeaways [01:36] Please like, share, leave a review, and subscribe to our MLOps channels! [01:47] Phi model explanation [06:29] Small Language Models optimization challenges [07:29] DeepSpeed overview and benefits [10:58] Crazy unimplemented crazy AI ideas [17:15] Post training vs QAT [19:44] Quantization over distillation [24:15] Using Lauras [27:04] LLM scaling sweet spot [28:28] Quantization techniques [32:38] Domino overview [38:02] Training performance benchmark [42:44] Data dependency-breaking strategies [49:14] Wrap up

Thanks to the High Signal Podcast by Delphina: https://go.mlops.community/HighSignalPodcast Aditya Naganath is an experienced investor currently working with Kleiner Perkins. He has a passion for connecting with people over coffee and discussing various topics related to tech, products, ideas, and markets. AI's Next Frontier // MLOps Podcast #277 with Aditya Naganath, Principal at Kleiner Perkins. // Abstract LLMs have ushered in an unmistakable supercycle in the world of technology. The low-hanging use cases have largely been picked off. The next frontier will be AI coworkers who sit alongside knowledge workers, doing work side by side. At the infrastructure level, one of the most important primitives invented by man - the data center, is being fundamentally rethought in this new wave. // Bio Aditya Naganath joined Kleiner Perkins’ investment team in 2022 with a focus on artificial intelligence, enterprise software applications, infrastructure and security. Prior to joining Kleiner Perkins, Aditya was a product manager at Google focusing on growth initiatives for the next billion users team. He previously was a technical lead at Palantir Technologies and formerly held software engineering roles at Twitter and Nextdoor, where he was a Kleiner Perkins fellow. Aditya earned a patent during his time at Twitter for a technical analytics product he co-created. Originally from Mumbai India, Aditya graduated magna cum laude from Columbia University with a bachelor’s degree in Computer Science, and an MBA from Stanford University. Outside of work, you can find him playing guitar with a hard rock band, competing in chess or on the squash courts, and fostering puppies. He is also an avid poker player. // MLOps Swag/Merch https://shop.mlops.community/ // Related Links Faith's Hymn by Beautiful Chorus: ⁠⁠https://open.spotify.com/track/1bDv6grQB5ohVFI8UDGvKK?si=4b00752eaa96413b⁠⁠ Substack: ⁠⁠https://adityanaganath.substack.com/?utm_source=substack&utm_medium=web&utm_campaign=substack_profile⁠⁠With thanks to the High Signal Podcast by Delphina: https://go.mlops.community/HighSignalPodcastBuilding the Future of AI in Software Development // Varun Mohan // MLOps Podcast #195 - ⁠⁠https://youtu.be/1DJKq8StuTo⁠⁠Do Re MI for Training Metrics: Start at the Beginning // Todd Underwood // AIQCON - ⁠⁠https://youtu.be/DxyOlRdCofo --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Aditya on LinkedIn: https://www.linkedin.com/in/aditya-naganath/

Dr Vincent Moens is an Applied Machine Learning Research Scientist at Meta and an author of TorchRL and TensorDict in Pytorch. PyTorch for Control Systems and Decision Making // MLOps Podcast #276 with Vincent Moens, Research Engineer at Meta. // Abstract PyTorch is widely adopted across the machine learning community for its flexibility and ease of use in applications such as computer vision and natural language processing. However, supporting reinforcement learning, decision-making, and control communities is equally crucial, as these fields drive innovation in areas like robotics, autonomous systems, and game-playing. This podcast explores the intersection of PyTorch and these fields, covering practical tips and tricks for working with PyTorch, an in-depth look at TorchRL, and discussions on debugging techniques, optimization strategies, and testing frameworks. By examining these topics, listeners will understand how to effectively use PyTorch for control systems and decision-making applications. // Bio Vincent Moens is a research engineer on the PyTorch core team at Meta, based in London. As the maintainer of TorchRL (https://github.com/pytorch/rl) and TensorDict (https://github.com/pytorch/tensordict), Vincent plays a key role in supporting the decision-making community within the PyTorch ecosystem. Alongside his technical role in the PyTorch community, Vincent also actively contributes to AI-related research projects. Before joining Meta, Vincent worked as an ML researcher at Huawei and AIG. Vincent holds a Medical Degree and a PhD in Computational Neuroscience. // MLOps Swag/Merch https://shop.mlops.community/ // Related Links Musical recommendation: https://open.spotify.com/artist/1Uff91EOsvd99rtAupatMP?si=jVkoFiq8Tmq0fqK_OIEglg Website: github.com/vmoens TorchRL: https://github.com/pytorch/rl TensorDict: https://github.com/pytorch/tensordict LinkedIn post: https://www.linkedin.com/posts/vincent-moens-9bb91972_join-the-tensordict-discord-server-activity-7189297643322253312-Wo9J?utm_source=share&utm_medium=member_desktop --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vincent on LinkedIn: https://www.linkedin.com/in/mvi/