MemryX Launches Floating-Point AI Chips, Leading a New Era of Precision Computing

An advanced AI processing chip for edge computing, positioned on a circuit board with futuristic blue lighting. The chip features visual elements representing floating-point computing with numerical symbols hovering above it, while subtle outlines of industrial equipment and vehicles appear in the background.](The image shows a high-tech AI chip with blue lighting effects, representing MemryX’s floating-point processing technology for edge computing.)

Overview: The Rise of Edge Computing in AI Applications

In recent years, advancements in semiconductor manufacturing processes have dramatically improved hardware computing capabilities and data volumes, ushering in a new development phase for Computer Vision. Traditionally, image processing relied on pixel-by-pixel calculations, but now, supported by Big Data and mature Deep Learning techniques, AI models can quickly create various applications through fixed learning patterns from massive datasets.

The Dual Rise of AI Technology and Edge Computing in Industrial and Vehicle Applications

In the rapidly spreading wave of Artificial Intelligence (AI) technology, edge computing is becoming the core driving force for both industrial and vehicle applications. While traditional cloud computing offers powerful centralized processing capabilities, it increasingly reveals limitations in scenarios requiring real-time processing, high data throughput, and sensitive data protection. The rise of edge computing, which distributes computing power to factory floors, equipment endpoints, and inside vehicles, provides solutions for these domains.

Value in Industrial Applications

• Reduced Latency: Processing data from sensors and equipment locally ensures real-time response capabilities for production lines and industrial robots.
• Reduced Network Load: Completing big data pre-processing at the edge reduces pressure on both internal and external factory networks, improving operational efficiency.
• Privacy Protection: Completing critical industrial data processing at the edge prevents sensitive production data leakage, ensuring corporate confidentiality and user privacy.

Value in Vehicle Applications

• Real-time Decision Making: On-board edge devices process camera, LIDAR, and radar data in real-time, ensuring quick decision-making for autonomous vehicles.
• Local Computing Optimization: Reduces vehicle dependency on external networks while ensuring reliable execution even during network interruptions.
• Data Privacy Protection: Localizing the processing of in-vehicle data (such as driving behavior and location information) reduces dependency on the cloud and protects driver privacy.

With the explosion of generative AI in 2024, creating diverse AI application demands, edge computing has become not just a supplementary technology but a fundamental cornerstone of modern AI applications.

MemryX: Innovator in Edge Computing and Floating-Point Calculation

In the edge computing wave, MemryX acceleration cards stand out with their powerful BF16 floating-point computing capabilities and comprehensive software support, making them ideal for edge AI applications. Unlike traditional solutions focused on integer computing, floating-point computing is crucial for AI inference scenarios requiring high precision. MemryX solutions deliver outstanding performance of 20 TFLOPS at low power consumption (1W / 5 TFLOPS), becoming a key driver for applications like object detection, image recognition, limb recognition, semantic segmentation, depth estimation, and natural language processing.

Comprehensive Software Support: Helping Developers Deploy Quickly

MemryX not only excels in hardware performance but also provides a rich software ecosystem, including module evaluation tools, API interfaces, drivers, and development tools to help developers quickly integrate and optimize AI model operations. Its software support includes:

• Chip simulation performance (Simulator)
• Weight precision adjustment tools
• Model cropping tools
• AI DEMO GitHub resources
• Model Zoo resources
• Model eXplorer resources

Outstanding AI Performance

Using the MemryX MX3 chip to run the currently popular YOLOv8-Nano object detection algorithm, it can easily achieve around 350 frames per second.

MemryX: Revolutionary Edge Computing and Floating-Point Calculation

MemryX was founded in 2019 by Dr. Wei Lu and Dr. Zhengya Zhang, currently professors of electrical engineering at the University of Michigan, and is now led by former Qualcomm Vice President Keith Kressin as CEO. Its design philosophy focuses on being lightweight, energy-efficient, and highly precise (floating-point computation), with iterative computing power that doesn’t occupy main platform resources. MemryX won the 2022 EE Awards Asia Gold Selection – Most Promising Product. Stay updated with their latest information through their official LinkedIn account.

The latest MX3 chip from MemryX features low power consumption, high computing capability (1W / 5 TFLOPS), and high cross-platform integration. It primarily uses 16-bit floating-point computation (BF16) to ensure model accuracy and provides rich module resources and integration packages for a more comprehensive AI resource integration experience. The AI chip also includes high-frequency memory configuration to eliminate memory bottlenecks during high computation. It’s important to consider the module size being used—one chip can process approximately 10M parameter data. For example, if Module A has 40M parameters, four MX3 chips would be needed.

Key Advantages

1. Uses floating-point (BF16) calculations to ensure module accuracy
2. Doesn’t occupy system memory
3. Scalability (up to 16 chips can be connected)
4. Optimal data flow optimization to minimize data movement
5. Optimal module operability for secondary optimization with other hardware accelerators

Software Framework

The MemryX Developer Hub includes a Compiler, Runtime, and Utility Tools to drive the MemryX AI SoCs series:

Compiler

Neural Compiler provides various functions such as Multi-Model integration, Model Cropping, Multiple Input Streams, Shared Input Stream, Mixed-Precision Weights, and Resources Utilization display. With simple Command Line instructions, it helps developers quickly convert models from PyTorch, Keras, TensorFlow, TensorFlow Lite, and ONNX to the MemryX DFP module format.

Runtime

Provides optimized user experience. Using Benchmark with Model Zoo helps developers quickly evaluate hardware performance and accuracy. It also provides several open-source example DEMOs (MemryX Examples) and concise yet powerful APIs to help developers quickly implement and deploy AI applications.

• Benchmark (Python, C/C++)
• Accelerator API (Python, C/C++)

Utility Tools

• Simulator: Software provided by MemryX to help developers without MX3 chips complete performance evaluations
• Viewer: GUI interface provided by MemryX, including the compiler, simulator, and accelerator mentioned above
• DFP Inspect: A tool provided by MemryX for inspecting DFP files

DEMO Examples

MemryX provides many AI examples teaching developers how to implement AI applications step by step:
https://github.com/memryx/MemryX_eXamples/tree/release

Note: It must be noted that the following examples are all open-source models and not for commercial use!

Conclusion

As mentioned in the manufacturer’s press release, MemryX is changing the customer experience of edge AI applications through AI chips in the following ways:

1. High Frame Rate (FPS)

MemryX’s data flow and memory computing architecture are suitable for pipeline operations. A low-power MemryX M.2 card can simultaneously process 10 camera streams running one or more AI models, making it particularly suitable for applications requiring high real-time performance, such as video management systems.

2. High Model Precision and Automated Compilation

MemryX provides tools that require just one click to complete high-precision AI model compilation. MX3 supports floating-point operations (BF16), ensuring model accuracy and integrity without the need to retrain models or make additional adjustments. Compared to mainstream integer modules (INT), MemryX allows customers to quickly deploy efficient and accurate AI applications.

3. Maintaining Original Model Integrity

Unlike other solutions that require modifying models to fit hardware, MemryX supports direct compilation and running of original models on MX3, and provides optional model pruning and compression functions to achieve design optimization.

4. Automatic Pre/Post-Processing

MemryX automatically identifies and packages preprocessing and post-processing code in AI models, helping developers integrate quickly, reducing the complexity of manual adjustments, and improving deployment efficiency.

5. Excellent Scalability

MX3 can be used as a single chip or combined with multiple chips as a logical unit, supporting applications from single smart cameras to 16-chip edge servers. All configurations share the same software and interfaces without requiring additional hardware such as PCIe switches.

6. Low Power Design

Each MX3 chip consumes only 0.5-2.0W, and the entire 4-chip M.2 module consumes less than one-tenth of a mainstream GPU while providing more efficient edge AI performance.

7. Extensive Software and Hardware Support

MemryX supports various operating systems and x86, ARM, and RISC-V platforms, adapting to a wide range of hardware environments and providing flexibility for developers.

Therefore, with its innovative MX3 solution, MemryX is redefining the application scope of edge artificial intelligence. From efficient floating-point computing capabilities to rich software support, scalability, and low-power design, MemryX is laying a solid foundation for the future of edge computing. Its core technology not only solves many pain points in the current market but also provides developers and enterprise users with flexible, fast, and reliable AI deployment solutions. In addition, MemryX offers hardware acceleration solutions for several core platforms, integrating MX3 chips with peripheral hardware to fully unleash platform performance greater than the sum of its parts.

With the widespread application of artificial intelligence in retail, automotive, industrial, agricultural, and robotics industries, MemryX stands at the forefront of edge computing technology, providing customers with outstanding performance and higher value. In the future, MemryX will continue to drive technological innovation, becoming an indispensable partner in the field of AI edge computing. With the tools and examples provided by the manufacturer mentioned above, AI is no longer an unreachable dream—one only needs to follow the example steps to quickly implement any intelligent application.