RISC-V Global Forum 2020

The inference phase with low power and memory limited environments is critical for deploying neural network models on edge devices. One possible approach is adopting the quantization technique to represent activations and weights in lower bits. In this work, we present the efficiently inference quantized models by supporting TVM QNN flow with RISC-V SIMD computations. As RISC-V supports both Superword SIMD and Subword SIMD, we compile models by TVM and replace the computation kernel with designated LLVM intrinsic functions for mapping with RISC-V favorable SIMD instructions. Experiments shows 1.79-7.58x reduction of instruction count compared quantized model with FP32 implementation. The accuracy loss is acceptable by evaluating on 1k images. The benchmark including MobileNet and Inception series. All experiments are executed on Spike with RISC-V SIMD supports.