Skip to content

Commit de2e44c

Browse files
Vinya567Vinya567
committed
test: Move initializer_as_input ONNX test into qa/ and drop cpdir
Move the initializer_as_input ONNX QA test from onnxruntime_backend/test/ into qa/L0_initializer_as_input/ so the test lives in the same repo where it's consumed and follows the existing L0_ tier naming convention under qa/. Drop the corresponding cpdir from cibase_build() in build.py since the files are now part of the qa/ checkout. This addresses the `[WIP] other way than wildcard?` comment from #5630 (which introduced the cross-repo cpdir). CI now picks up the test from qa/ directly instead of via a cross-repo wildcard copy of onnxruntime_backend/test/*. A follow-up PR in triton-inference-server/onnxruntime_backend will remove the now-duplicated test/initializer_as_input/ directory.
1 parent 172e290 commit de2e44c

7 files changed

Lines changed: 268 additions & 3 deletions

File tree

‎build.py‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2094,9 +2094,6 @@ def cibase_build(
20942094
os.path.join(ort_install_dir, "test", "custom_op_test.onnx"),
20952095
os.path.join(ci_dir, "qa", "L0_custom_ops"),
20962096
)
2097-
# [WIP] other way than wildcard?
2098-
backend_tests = os.path.join(build_dir, "onnxruntime", "test", "*")
2099-
cmake_script.cpdir(backend_tests, os.path.join(ci_dir, "qa"))
21002097

21012098
# Need the build area for some backends so that they can be
21022099
# rebuilt with specific options.

‎qa/L0_initializer_as_input/README.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
<!--
2+
# Copyright 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
#
4+
# Redistribution and use in source and binary forms, with or without
5+
# modification, are permitted provided that the following conditions
6+
# are met:
7+
# * Redistributions of source code must retain the above copyright
8+
# notice, this list of conditions and the following disclaimer.
9+
# * Redistributions in binary form must reproduce the above copyright
10+
# notice, this list of conditions and the following disclaimer in the
11+
# documentation and/or other materials provided with the distribution.
12+
# * Neither the name of NVIDIA CORPORATION nor the names of its
13+
# contributors may be used to endorse or promote products derived
14+
# from this software without specific prior written permission.
15+
#
16+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
17+
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
19+
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
20+
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
21+
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
22+
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
23+
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
24+
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27+
-->
28+
29+
This test is originated in "onnxruntime_backend" repository to better
30+
represent the scope of the test, however, this test utilizes Triton utilities
31+
and assumes that the test is located under "qa" directory in "server" repository
32+
for accessing those utilities. Please make sure the test environment is properly
33+
set before running the test.

‎qa/L0_initializer_as_input/generate_test_model.py‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Copyright 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
#
3+
# Redistribution and use in source and binary forms, with or without
4+
# modification, are permitted provided that the following conditions
5+
# are met:
6+
# * Redistributions of source code must retain the above copyright
7+
# notice, this list of conditions and the following disclaimer.
8+
# * Redistributions in binary form must reproduce the above copyright
9+
# notice, this list of conditions and the following disclaimer in the
10+
# documentation and/or other materials provided with the distribution.
11+
# * Neither the name of NVIDIA CORPORATION nor the names of its
12+
# contributors may be used to endorse or promote products derived
13+
# from this software without specific prior written permission.
14+
#
15+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
16+
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
17+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
18+
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
19+
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
20+
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
21+
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
22+
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
23+
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
24+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26+
27+
import numpy as np
28+
import onnx
29+
30+
# Reference script on how the model used in this test is created
31+
if __name__ == "__main__":
32+
values = np.ones((5, 5)).astype(np.float32)
33+
onnx_dtype = onnx.TensorProto.FLOAT
34+
initialized_input = onnx.helper.make_tensor(
35+
name="INITIALIZER",
36+
data_type=onnx_dtype,
37+
dims=values.shape,
38+
vals=values.flatten().astype(float),
39+
)
40+
add = onnx.helper.make_node("Add", ["INPUT", "INITIALIZER"], ["OUTPUT"])
41+
42+
input = onnx.helper.make_tensor_value_info("INPUT", onnx_dtype, values.shape)
43+
initializer = onnx.helper.make_tensor_value_info(
44+
"INITIALIZER", onnx_dtype, values.shape
45+
)
46+
output = onnx.helper.make_tensor_value_info("OUTPUT", onnx_dtype, values.shape)
47+
48+
graph_proto = onnx.helper.make_graph(
49+
[add],
50+
"init_input",
51+
[input, initializer],
52+
[output],
53+
initializer=[initialized_input],
54+
)
55+
model_def = onnx.helper.make_model(graph_proto, producer_name="triton")
56+
onnx.save(model_def, "model.onnx")

‎qa/L0_initializer_as_input/models/add_with_initializer/1/model.onnx‎

269 Bytes
Binary file not shown.

‎qa/L0_initializer_as_input/models/add_with_initializer/config.pbtxt‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Copyright 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
#
3+
# Redistribution and use in source and binary forms, with or without
4+
# modification, are permitted provided that the following conditions
5+
# are met:
6+
# * Redistributions of source code must retain the above copyright
7+
# notice, this list of conditions and the following disclaimer.
8+
# * Redistributions in binary form must reproduce the above copyright
9+
# notice, this list of conditions and the following disclaimer in the
10+
# documentation and/or other materials provided with the distribution.
11+
# * Neither the name of NVIDIA CORPORATION nor the names of its
12+
# contributors may be used to endorse or promote products derived
13+
# from this software without specific prior written permission.
14+
#
15+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
16+
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
17+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
18+
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
19+
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
20+
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
21+
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
22+
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
23+
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
24+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26+
27+
platform: "onnxruntime_onnx"
28+
max_batch_size: 0
29+
input [
30+
{
31+
name: "INPUT"
32+
data_type: TYPE_FP32
33+
dims: [5, 5]
34+
},
35+
{
36+
name: "INITIALIZER"
37+
data_type: TYPE_FP32
38+
dims: [5, 5]
39+
optional: true
40+
}
41+
]
42+
output [
43+
{
44+
name: "OUTPUT"
45+
data_type: TYPE_FP32
46+
dims: [ 5, 5]
47+
}
48+
]

‎qa/L0_initializer_as_input/test.py‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
#!/usr/bin/env python
2+
# Copyright 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
#
4+
# Redistribution and use in source and binary forms, with or without
5+
# modification, are permitted provided that the following conditions
6+
# are met:
7+
# * Redistributions of source code must retain the above copyright
8+
# notice, this list of conditions and the following disclaimer.
9+
# * Redistributions in binary form must reproduce the above copyright
10+
# notice, this list of conditions and the following disclaimer in the
11+
# documentation and/or other materials provided with the distribution.
12+
# * Neither the name of NVIDIA CORPORATION nor the names of its
13+
# contributors may be used to endorse or promote products derived
14+
# from this software without specific prior written permission.
15+
#
16+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
17+
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
19+
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
20+
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
21+
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
22+
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
23+
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
24+
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27+
28+
import unittest
29+
30+
import numpy as np
31+
import tritonclient.http as httpclient
32+
33+
34+
class OptionalInputTest(unittest.TestCase):
35+
def setUp(self):
36+
self.client_ = httpclient.InferenceServerClient("localhost:8000")
37+
self.model_name_ = "add_with_initializer"
38+
self.input_data_ = np.zeros((5, 5)).astype(np.float32)
39+
self.input_ = httpclient.InferInput("INPUT", self.input_data_.shape, "FP32")
40+
self.input_.set_data_from_numpy(self.input_data_, binary_data=False)
41+
self.optional_input_ = httpclient.InferInput(
42+
"INITIALIZER", self.input_data_.shape, "FP32"
43+
)
44+
self.optional_input_.set_data_from_numpy(self.input_data_, binary_data=False)
45+
46+
def test_without_optional(self):
47+
# Send request without providing optional input, the ONNX model
48+
# should use stored initializer value (tensor of all 1s)
49+
results = self.client_.infer(self.model_name_, [self.input_])
50+
np.testing.assert_allclose(results.as_numpy("OUTPUT"), (self.input_data_ + 1))
51+
52+
def test_with_optional(self):
53+
# Send request with optional input provided, the ONNX model
54+
# should use provided value for the initializer
55+
results = self.client_.infer(
56+
self.model_name_, [self.input_, self.optional_input_]
57+
)
58+
np.testing.assert_allclose(
59+
results.as_numpy("OUTPUT"), (self.input_data_ + self.input_data_)
60+
)
61+
62+
63+
if __name__ == "__main__":
64+
unittest.main()

‎qa/L0_initializer_as_input/test.sh‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#!/bin/bash
2+
# Copyright 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
#
4+
# Redistribution and use in source and binary forms, with or without
5+
# modification, are permitted provided that the following conditions
6+
# are met:
7+
# * Redistributions of source code must retain the above copyright
8+
# notice, this list of conditions and the following disclaimer.
9+
# * Redistributions in binary form must reproduce the above copyright
10+
# notice, this list of conditions and the following disclaimer in the
11+
# documentation and/or other materials provided with the distribution.
12+
# * Neither the name of NVIDIA CORPORATION nor the names of its
13+
# contributors may be used to endorse or promote products derived
14+
# from this software without specific prior written permission.
15+
#
16+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
17+
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
19+
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
20+
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
21+
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
22+
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
23+
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
24+
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27+
28+
export CUDA_VISIBLE_DEVICES=0
29+
30+
SERVER=/opt/tritonserver/bin/tritonserver
31+
SERVER_ARGS="--model-repository=`pwd`/models"
32+
SERVER_LOG="./server.log"
33+
CLIENT_LOG="./test.log"
34+
source ../common/util.sh
35+
36+
rm -f *.log
37+
38+
run_server
39+
if [ "$SERVER_PID" == "0" ]; then
40+
echo -e "\n***\n*** Failed to start $SERVER\n***"
41+
cat $SERVER_LOG
42+
exit 1
43+
fi
44+
45+
RET=0
46+
47+
set +e
48+
49+
python test.py >>$CLIENT_LOG 2>&1
50+
if [ $? -ne 0 ]; then
51+
cat $CLIENT_LOG
52+
echo -e "\n***\n*** Test Failed\n***"
53+
RET=1
54+
fi
55+
56+
set -e
57+
58+
kill $SERVER_PID
59+
wait $SERVER_PID
60+
61+
if [ $RET -eq 0 ]; then
62+
echo -e "\n***\n*** Test Passed\n***"
63+
else
64+
echo -e "\n***\n*** Test FAILED\n***"
65+
fi
66+
67+
exit $RET

0 commit comments

Comments
 (0)