Practical machine vision algorithms have always been a focus of both research and industrial implementation. For algorithm engineers, training an excellent algorithm is not difficult, but arranging a stable software carrier and hardware carrier for the algorithm model requires a significant investment in R&D costs (such as hardware decoding, compatibility with multiple camera brands, etc.) and hardware selection. Through long-term project practice (focusing on safe production, smart manufacturing, intelligent logistics, mining vision, and overseas machine vision business), our team has achieved a high level of maturity in completing a mature set of software carriers and hardware carriers for algorithm models, greatly facilitating the industrial implementation of algorithm models. And our technical staff gave it a cliché name: Xiaozhi Spirit Engine.
We are targeting professional software technicians, not general practitioners. So, dear readers, please act within your capabilities!

Xiaozhi Spirit devices, through stable and solid edge computing hardware and supporting basic software, help algorithm developers solve the problem of isolated AI model operation. Developers can quickly utilize the toolchain and source code (also supporting their own algorithm source code) open-sourced by the community to rapidly deploy and implement their business algorithm models.

What does the software carrier include?

The software carrier includes a complete set of built-in supporting software adapted to the algorithm model: for example, if the algorithm model is a bullet, the software carrier includes the barrel, sight, stock, safety, magazine, and other components that can fire the bullet.

As shown in the figure below:

So, what exactly does it include?

1. Industrial-grade video stream hardware decoding module: Beginners might use OpenCV, but it can cause system crashes and cannot run stably 7*24. It's fine for lab work, but not for stable project operation.
2. Hardware color space conversion module: The decoded image frames are in YUV data format, but AI analysis requires RGB three-channel numerical matrices. Color space conversion is a huge pressure for any CPU, so we use a hardware module to accomplish this function.
3. Industrial-grade database module: As the amount of data increases, the space occupied increases linearly, and the time for adding, deleting, modifying, and querying increases logarithmically (logN).
4. Storage management: The software can run stably forever, and when the space is full, it will automatically delete old data.
5. Video stream adaptability: Whether it's video streams from various brand cameras, DVRs, or online video streams, they can all be connected.
6. Complete BS software UI: Camera configuration, video playback, real-time inference result frame display, network management, performance monitoring, alarm data display, and more are provided with a mature UI.
7. Market-oriented features: The algorithm supports setting effective times, dual-network penetration support, remote tunnel access, real-time OSD overlay display, static file upgrades, and security settings.

Function Name	Function Description	Partial UI Screenshot
Real-time View	Real-time multi-screen preview, can display inference results in real-time
Video Access	Supports video streams from various brand cameras, drones, NVRs, and online real-time streams
Import Any Third-party Custom Algorithm	Supports importing model files from any third-party algorithm package
Network Management	Supports dual-network isolation, remote tunneling, and penetration between production and office networks
Authorization Management	Users can manage device authorization through a formed tool, helping users quickly complete productization
Security Settings	Users can set screen locks, verification codes, and other security features, helping users quickly complete productization
Performance Optimization	Fully utilizes NPU performance, optimizes the performance of multi-model cascading, and can achieve up to 150FPS with a thousand-yuan chip	-

What does the hardware carrier include?

Facing this title, the first question we consider is: why do we need a hardware carrier?
The answer is actually quite simple. If all algorithm models are deployed and run through GPU computing servers, then the hardware cost is the biggest issue. To solve this problem, our hardware carrier mainly includes edge industrial control machines equipped with NPU computing chips (of course, they can also run on servers).

Currently, we support two mainstream NPU chips:

1. Rockchip RK series
2. Huawei Ascend series

The biggest advantages of edge industrial control machines are:

1. A single device is cheaper
2. More video streams can be analyzed with the same computing power
3. Lower dependence on the operating environment, no air conditioning required!

Some friends may ask: There are a lot of NPU computing industrial control machines on the market, you can buy them anywhere.
In fact, it's not like that. Our technical team spent four years, from channels, e-commerce, Taiwan, and other dimensions, selecting over 300 types of hardware devices (all ARM-Linux operating systems). Less than 2% passed the technical tests (for example, executing a sudo upgrade command would reveal their true nature).
Among the various substandard products, we list a few issues to share with you:

1. Uneven video stream hardware decoding

   Some manufacturers' devices, when decoding multiple video streams concurrently, struggle to achieve stable and smooth decoding, resulting in "chunky" decoding. The displayed effect is that the AI inference results are choppy.

2. NTP time synchronization issues can prevent the device from turning on

   Amazing, right! It's like if your computer time and Earth time are not synchronized, it prevents your computer from turning on. The industrial control machine market is just that mixed!

3. Poor heat dissipation

   This is very common. Running algorithms at full power, the device gets hot enough to burn your hand. How can you expect it to have a long lifespan? (We require that under full power operation, the external surface temperature of the device should be as cool as your girlfriend's face.)

4. Garbage operating system

   This is also very common. Although they are all Ubuntu, Kylin V10, Debian operating systems, many brands' devices, due to kernel trimming and backward NPU adaptation technology, have various third-party software that cannot be installed, such as MySQL.

What pain points does the Xiaozhi Spirit Engine solve for the industrial implementation of visual algorithms?

1. High cost of third-party custom algorithms The user side only needs one professional technician to complete the direct industrialization of laboratory algorithm models. There is no need to pay high costs and customize algorithms with third-party companies.
2. Confidentiality of owner data cannot be opened In our project practice, a large number of project user data cannot leave the premises, such as nuclear power, logistics, prisons, and manufacturing plants.
3. Lack of cheap and stable software and hardware algorithm carriers Both cheap and stable and efficient, where can you find such a good thing? Now it exists...
4. Users need intellectual property rights If it's just about completing the project delivery, it's hard for the user side to accumulate their own core technology. But if the algorithm is trained by themselves and they have the complete AI source code, the meaning is different.

What have we opened up?

1. Complete source code for AI vision algorithm tasks: Includes object detection, instance segmentation, image classification, semantic segmentation, etc. Of course, users can also use their own source code to train models;
2. Complete toolchain for migrating algorithm models to hardware devices: Algorithm models are all trained on GPU graphics cards, and migration to edge chips, especially domestic chips, requires model quantization migration;
3. Data annotation toolchain: Users can choose their own annotation toolchain or use the one we provide;
4. Algorithm package creation toolchain: For the productization application of algorithm models, it needs to support user parameter configuration. We have opened up algorithm package creation, allowing users to make visual parameter configuration adjustments through a backend configuration file.

Link to the entire toolchain on Github: https://github.com/AIDrive-Research/Custom-Algorithm

Why don't our users choose to use the AI training and inference integrated visualization platform?

Hello, dear readers, since you have read this far, the reason why you don't choose the integrated training and inference visualization platform, you all understand...