Mar 1, 2024
Software Delivered AI w/ Brian Stevens of Neural Magic
AZ TRT S05 EP08 (223) 2-25-2024
What We Learned This Week
- Neural Magic Deepsparse software helps B2B Clients incorporate AI into their tech stack
- Large Language Learning Models of AI can be costly & require massive computing power
- Their clients now control their AI Model
- Opensource AI Foundation Models for training
- AI uses a Recommendation Model
Guest: Brian Stevens
Chief Executive Officer of Neural Magic
Brian Stevens is chief executive officer of Neural Magic. A tech veteran with more than 30 years of experience, Brian has a rich history of building/advising high-impact companies and driving disruptions that transform the industry.
In his role at Neural Magic, Brian aims to democratize Generative AI for enterprises and make it more accessible and affordable to all.
In his career, Brian has served in a variety of executive roles at world-renowned companies including VP and CTO of Google Cloud, and CTO and EVP of Worldwide Engineering at Red Hat.
Brian currently serves on the board of directors of Nutanix and Genpact, and is a former member of the board of directors of the American Red Cross, IEEE, OpenStack Foundation, Data Gravity, and Pentaho.
Brian holds a master’s degree in computer systems from Rensselaer Polytechnic Institute and a bachelor’s degree in computer science from the University of New Hampshire. In his personal life, Brian is an accomplished carpenter and woodworker with a passion for refurbishing old homes.
NEURAL MAGIC
About: Neural Magic is an AI company, born out of the Massachusetts Institute of Technology (MIT), on a mission to help customers innovate with machine learning, without added complexity or cost. While pursuing research at MIT, founders Nir Shavit and Alexander Matveev launched Neural Magic, a software-delivered AI solution, to address their frustration with the constraints of GPUs and existing hardware.
Using Neural Magic’s DeepSparse Inference Runtime, customers can easily deploy deep learning models on commodity CPUs with GPU-class performance.
For more information, including all of Neural Magic’s offerings, visit https://neuralmagic.com/ or follow @neuralmagic on Twitter, LinkedIn, and YouTube.
Open Source AI for Business-2024 Is the Year to On Ramp
Brian Stevens, CEO of Neural Magic is at the helm of this growing trillion-dollar industry (proper source)
As enterprises prepare for 2024, the growing demand for AI optimization is top of mind.
Neural Magic is fulfilling that need with software-delivered AI. Enterprises use Neural Magic's runtime and open-source sparsification tools for maximum CPU speedups of NLP (including LLMs) and computer vision models.
“It is my goal to democratize AI using optimized CPUs as the onramp to generative AI, making it faster, affordable and agile for enterprises.” – Brian Stevens, CEO, Neural Magic
Neural Magic has created a software architecture for the future of machine learning with an open-source LLM (Large Language Models) approach that enables enterprises to leverage existing commodity hardware (x86 and ARM). The net result demonstrates the power of software and model optimization across different computing platforms to enhance the scalability and efficiency of AI workloads.
Neural Magic was founded in 2017 by MIT professors and research scientists. The company has raised more than $55M from blue chip investors including a recent $35M Series A led by NEA, with participation from Andreessen Horowitz, VMware, Verizon, Amdocs, Comcast, Pillar, and Ridgeline Ventures.
Neural Magic has strategic partnerships with CPU manufacturers like AMD and Intel, cloud providers like AWS and Google Cloud, and software vendors like Red Hat and Ultralytics. These partnerships allow Neural Magic to provide value at all levels of the development lifecycle, from the models themselves down to the silicon.
Notes
Seg 2
AI you’ve been around since the 1950s and retail businesses have been using AI for years now. Wayfair out of Boston has incorporated AI. They understand the shopper experience.
In its simplest form, AI uses the recommendation model - presents back to you similar things you’ve been searching for. If you like this, you’ll like something similar. This is now a very large part of business revenues, while also helping to better the user experience.
There are large language, AI models, which involve more math in the program and need more computing power. Harder to run this larger AI model.
Businesses need an AI division in their tech stack now. Many large companies have an dedicated AI Lab. This is similar to how they had built out a cloud model in the past. Nowadays though, business understand their models need more integrated.
You train an AI model with company data. Lots of data. Seven data set to start using reference material like Internet sites or Wikipedia. It does cost a lot to change this model.
There are options to build on an existing model open source AI programs. What is used now is called a foundational model, and then you train it on your company product catalog.
Seg 3
Brian‘s background is a computer science degree and software developer. He worked in New England. He’s a technologist, solve the problem in use case for tech product manager.
Worked at Redhat through 2001, and the.com crash. Also worked in open source in Linux platforms. Then was at Google cloud, working onsite in Mountain View, California. This was no remote jobs back in 2014. He was there for 5 years and helped with the company going from $50 mil to $10 bil revenue.
Move back to New England. Connected with a professor from MIT, who had started a company on AI software and Brian joined as CEO.
Company is called Neural Magic, and the website is neuralmagic.com. Deepsparse is their software stack which runs deep AI learning model that you deploy on servers.
They fine-tune the model and adapted it to customer stack. This is for businesses to optimize AI for customers. It is similar to an interface with company software. AI language model that is large needs. Lots of infrastructure.
What neural magic does is? It makes the model faster and more efficient.
Seg 4
ChatGPT has changed things with AI. AI interface is similar with API codes with response and language size. Need a model that meets needs with a data set that makes the models run more efficiently on lots of hardware.
Neural magics deep sparse is an inference server and training to deploy in production and could be days to train.
There are 3 challenges with AI for businesses.
First is expensive and enterprise does not control things when they use a hosted model or a hosted service like open AI.
Second, they have to feed data to the host to train the AI model and there are privacy and security issues.
Third you have a lifecycle , which must qualify and test the text stack with each time you update.
Neural Magic allows enterprises to own the AI model. Now they have security, privacy and updates are all handled. This is AI on their own terms and gives them options. Full control and it looks like just another application.
Open source AI machine learning model in the cloud and can work on servers like Oracle, Amazon - AWS, the Google cloud, or Microsoft Azure. Client has liability with software, and they still need to protect it.
More info go to neural magic.com can learn about the product, the marketing as well as the community.
Seg 1 - Clips from:
Artificial Intelligence (AI) – how the Algorithm Connects Us All
- BRT S02 EP43 (90) 10-24-2021
5 Things We Learned This Week:
- AI is inter-connected with so many technologies & you use AI often on a daily basis
- AI is a part of almost all industries from Healthcare, Finance to Defense
- Human in the Loop - humans will always be needed to Interpret the Data, but AI will assist
- Software Teams must be managed so the product is integrated properly in the bigger picture
- Moore’s Law – Each year computing power grows 2x as fast, but cuts the cost in half
Naru and his team are working on document management, where their AI program will be able to read documents and determine what the info is. Rising Cloud is another project they are building that manages a company cloud usage to improve costs.
Moore’s Law – Each year computing power grows 2x as fast, but cuts the cost in half
Cloud Computing happens in the cloud and internet for your programming vs Edge Computing that happens right on your phone and does not need to go out to the cloud. Bigger the data request or process determines if Cloud or Edge is the best choice.
People interact with AI (Artificial Intelligence) daily on their phone, email, internet search and beyond. User Agreements in your phone or websites you use say they can take your search data and use it to enhance your experience.
AI Search uses past searches by you, vs what are the popular other searches by other people on the internet. It happens so fast and has the best / popular search options loading before you are even done typing. This is called a Recommendation Engine, just like Netflix or Amazon find shows or products you may like. These recs are similar to what you have watched or bought previously, or in similar genres.
The downside is you may not see different options, just more of the same. AI determines what you see daily on the internet, and can create a silo effect. Inventives uses a common solution, called Human in the Loop to review what the AI is doing. Then the searches or recommendations are reviewed to see how accurate they are.
Full Show: HERE
Best of Biotech
from AZ Bio & Life Sciences to
Jellatech: HERE
Biotech Shows: HERE
AZ Tech Council Shows:
https://brt-show.libsyn.com/size/5/?search=az+tech+council
*Includes Best of AZ Tech Council show from 2/12/2023
‘Best Of’ Topic: https://brt-show.libsyn.com/category/Best+of+BRT
Thanks for Listening.
Please Subscribe to the BRT Podcast.
AZ Tech Roundtable 2.0 with Matt Battaglia
The show where Entrepreneurs, Top Executives, Founders, and Investors come to share insights about the future of business.
AZ TRT 2.0 looks at the new trends in business, & how classic industries are evolving.
Common Topics Discussed: Startups, Founders, Funds & Venture Capital, Business, Entrepreneurship, Biotech, Blockchain / Crypto, Executive Comp, Investing, Stocks, Real Estate + Alternative Investments, and more…
AZ TRT Podcast Home Page: http://aztrtshow.com/
‘Best Of’ AZ TRT Podcast: Click Here
Podcast on Google: Click Here
Podcast on Spotify: Click Here
More Info: https://www.economicknight.com/azpodcast/
KFNX Info: https://1100kfnx.com/weekend-featured-shows/
Disclaimer: The views and opinions expressed in this program are those of the Hosts, Guests and Speakers, and do not necessarily reflect the views or positions of any entities they represent (or affiliates, members, managers, employees or partners), or any Station, Podcast Platform, Website or Social Media that this show may air on. All information provided is for educational and entertainment purposes. Nothing said on this program should be considered advice or recommendations in: business, legal, real estate, crypto, tax accounting, investment, etc. Always seek the advice of a professional in all business ventures, including but not limited to: investments, tax, loans, legal, accounting, real estate, crypto, contracts, sales, marketing, other business arrangements, etc.