Ryzen AI Software
Next-Gen AMD Ryzen
Ryzen AI AMD
With interoperability with leading industry frameworks, AMD Ryzen AI software continues to support the open-source community and give developers the freedom to implement AI inference applications on PCs with AMD Ryzen AI technology. The Ryzen AI Software 1.2 release showcases new features like easier-to-use developer tools, more hardware and model support, and more. Take a look at the latest features!
New Hardware Assistance
Improved NPU Support: The most recent update offers exceptional performance by adding support for Strix Point NPUs.
- Up to 50 TOPS for Ryzen AI 9 365 and Ryzen AI 9 HX 370
- Ryzen AI 9 HX 375: 55 TOPS Maximum
Better iGPU Integration: Without having to remove and reload the software, developers can now easily switch between integrated GPUs (iGPU) and NPUs based on the model needs.
Increased Model Assistance
LLM Flow Support: For general-purpose flows and performance-optimized applications, Ryzen AI Software 1.2 supports new models in both the PyTorch and ONNX frameworks.
Ryzen AI LLM
Here are general-purpose, experimentally-useful, unoptimized LLMs. It is not appropriate to use these models for benchmarking.
- Application: Early development and prototyping using a wide range of LLMs.
- Performance: Just functional assistance.
- AMD Ryzen 7040 Series, AMD Ryzen 8040 Series, and AMD Ryzen AI 300 Series (and subsequent Generations) are supported platforms.
- Frameworks supported: PyTorch.
- Models that are supported: Numerous.
Performance-Optimized LLMs
On the AMD secure performance-optimized LLMs for benchmarking and production are available upon request.
- Relevance: Comparing and implementing particular LLMs.
- Performance: Extremely well-optimized.
- Platforms supported: AMD Ryzen AI 300 Series (as well as others to come).
- Frameworks supported include ONNX and PyTorch.
- Models supported: Qwen1.5, Llama2, and Llama 3.
- On the AMD secure SDXL-T Support (limited performance enhancement) is available upon request.
Ryzen AI NPU
New and Enhanced Developer Tools for Ease of Use
AI Analyzer: A tool to help identify bottlenecks and improve model performance through the study and visualization of model compilation and inference profiling.
Platform/NPU Inspection and Management: A new xrt-smi tool that combines three functions in one easy-to-use package: it checks the AI PC and NPU’s condition, runs sanity tests to verify the NPU, and sets its performance settings for maximum effectiveness.
Dynamic Power Management (DPM) Integration: By adjusting the dynamic power management register’s status, developers can now modify the Power Slider in Windows to “Performance,” “Balanced,” or “Power saver” during NPU inference. Because of this integration, developers have the flexibility to perform inference in various power profiles according to the requirements of the final application. Using the xrt-smi tools, developers can also adjust the power level.
ONNX model benchmarking: A tool for ONNX model benchmarking that makes performance evaluation and ONNX-based workflow optimization easier.
Smart Installer: The Ryzen AI software setup procedure is made simpler with the new Smart Installer. The Smart Installer is now a Windows-based MSI procedure through a unified executable instead of a manual Python script installation, thanks to a “single-click installation” of the NPU driver.
AI Ryzen
They continue to deliver state-of-the-art tools and improvements intended to turbocharge your AI development on AI PCs through regular software releases. Updates to the Ryzen AI Software 1.2 include improved usability, streamlined processes for peak performance, support for open frameworks, and resources designed to help you develop your AI skills.
AMD’s release notes for a detailed rundown of all the new features and enhancements in the 1.2 software version. Make sure you have access to the resources and tools necessary to push the limits of what is possible on AI PCs by subscribing to the most recent Ryzen AI software upgrades. This will keep you at the forefront of AI innovation.
AMD Ryzen AI
Ryzen AI Processor
On AMD Ryzen AI-powered PCs, the AMD Ryzen AI software comes with the tools and runtime libraries needed to optimize and implement AI inference. The Ryzen AI software facilitates the execution of apps on the AMD XDNA architecture’s neural processing unit (NPU), which is the first specifically designed AI processing silicon on a Windows x86 processor. Additionally, it supports an integrated GPU (iGPU).
Create AI Applications Quickly: With a simple installation process, an expanding pre-trained model zoo, and efficient AI models with ONNX runtime, you can get started in a matter of minutes.
Strategically Distribute and Offload Models: Using the APU architecture, you may assign and delegate models to the NPU and iGPU according to the different levels of computing that your application requires.
Increase Battery Life and Improve Efficiency: Offloading AI Models to the NPU will boost battery life3 and free up CPU and GPU resources.
There are three simple steps to creating AI apps using Ryzen AI
Utilize a Pre-trained Model First
To get started, use a pre-trained model in TensorFlow or PyTorch. Next, to make your model operate with the Ryzen AI methodology, convert it to the ONNX format.
Quantization
To quantize your model, change the floating-point parameters to representations with less precision, such as 8- or 16-bit integers. For this reason, AI Quantizer for ONNX offers a simple Post Training Quantization (PTQ) procedure.
Implement the Model
Your model is prepared for hardware deployment after quantization. To deploy the AI model, use C++ or Python APIs with ONNX Runtime. The ONNX Runtime’s AI Execution Provider optimizes workloads to guarantee peak performance and minimal power usage.
0 Comments