Skip to content

Commit 6ac3e66

Browse files
authored
feat!: add integration test for safety_identifier with openai client (#5018)
# What does this PR do? <!-- Provide a short summary of what this PR does and why. Link to relevant issues if applicable. --> <!-- If resolving an issue, uncomment and update the line below --> Related to #4817 ## Test Plan <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* --> ```console (llama-stack) gualiu@gualiu-mac llama-stack % pytest -s -v tests/integration/responses/test_openai_response.py --stack-config=server:ci-tests --inference-mode record-if-missing -k "safety_identifier" --setup=gpt --suite=responses --color=yes --embedding-model=sentence-transformers/nomic-ai/nomic-embed-text-v1.5 --capture=tee-sys =================================================================================================== test session starts ==================================================================================================== platform darwin -- Python 3.12.7, pytest-8.4.2, pluggy-1.6.0 -- /Users/gualiu/go/src/github.com/llamastack/llama-stack/.venv/bin/python3 cachedir: .pytest_cache metadata: {'Python': '3.12.7', 'Platform': 'macOS-26.3-arm64-arm-64bit', 'Packages': {'pytest': '8.4.2', 'pluggy': '1.6.0'}, 'Plugins': {'anyio': '4.9.0', 'html': '4.1.1', 'socket': '0.7.0', 'asyncio': '1.1.0', 'json-report': '1.5.0', 'timeout': '2.4.0', 'metadata': '3.1.1', 'cov': '6.2.1', 'nbval': '0.11.0'}} rootdir: /Users/gualiu/go/src/github.com/llamastack/llama-stack configfile: pyproject.toml plugins: anyio-4.9.0, html-4.1.1, socket-0.7.0, asyncio-1.1.0, json-report-1.5.0, timeout-2.4.0, metadata-3.1.1, cov-6.2.1, nbval-0.11.0 asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 11 items / 8 deselected / 3 selected tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier[txt=openai/gpt-4o] instantiating llama_stack_client Starting llama stack server with config 'ci-tests' on port 8321... Waiting for server at http://localhost:8321... (0.0s elapsed) Waiting for server at http://localhost:8321... (0.5s elapsed) Server is ready at http://localhost:8321 llama_stack_client instantiated in 2.045s PASSED [ 33%] tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_streaming[txt=openai/gpt-4o] PASSED [ 66%] tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_and_previous_response[txt=openai/gpt-4o] PASSED [100%]Terminating llama stack server process... Terminating process 98817 and its group... Server process and children terminated gracefully =================================================================================================== slowest 10 durations =================================================================================================== 2.06s setup tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier[txt=openai/gpt-4o] 0.14s call tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier[txt=openai/gpt-4o] 0.13s teardown tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_and_previous_response[txt=openai/gpt-4o] 0.03s call tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_and_previous_response[txt=openai/gpt-4o] 0.01s call tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_streaming[txt=openai/gpt-4o] (4 durations < 0.005s hidden. Use -vv to show these durations.) ======================================================================================= 3 passed, 8 deselected, 2 warnings in 2.41s ======================================================================================== ```
1 parent 640c2a2 commit 6ac3e66

5 files changed

Lines changed: 1330 additions & 0 deletions

tests/integration/responses/recordings/6e6111504b1f683d81f9db5b92d69ba17b2413ea9348b77c00c2f0483334dc07.json

Lines changed: 301 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)