AXERA-TECH/RIFE.axera

RIFE

This version of RIFE has been converted to run on the Axera NPU using w8a16 quantization.

This model has been optimized with the following LoRA:

Compatible with Pulsar2 version: 4.2

Convert tools links:

For those who are interested in model conversion, you can try to export axmodel through

• The repo of AXera Platform, which you can get the detail of guide

Support Platform

• AX650
  • M4N-Dock(爱芯派Pro)
  • M.2 Accelerator card

Chips	model	cost
AX650	RIFE	200 ms

How to use

Download all files from this repository to the device

root@ax650:~/rife# tree
.
|-- model
|   `-- rife_x2_720p.axmodel
|-- video
|   `-- demo.mp4
|`-- run_axmodel.py
|`-- ms_ssim.py
|`-- build_config.json
|`-- requirements.txt

Inference

Input Data:
|-- video
|   `-- demo.mp4

Inference with M.2 Accelerator card with gradio demo

$python3 gradio_demo.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 4.2 77cdc0c2
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 4.2 08e76396
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 4.2 08e76396

==================================================
🌐 视频插帧工具 Web UI 已启动！
🔗 本地访问: http://127.0.0.1:7860
🔗 局域网访问: http://10.126.33.124:7860
==================================================

Then open the local access link in the browser.

Inference with AX650 Host, such as M4N-Dock(爱芯派Pro)

root@ax650 ~/rife #python3 run_axmodel.py --model ./model/rife_x2_720p.axmodel --video ./video/demo.mp4
[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 4.2 77cdc0c2
input name: onnx::Slice_0
demo.mp4, 128.0 frames in total, 25.0FPS to 50.0FPS
The audio will be merged after interpolation process
 99%|██████████████████████████████████████▋| 127/128.0 [01:38<00:00,  1.29it/s]

Output: [INFO]: