Commit 6ac3e66
authored
feat!: add integration test for safety_identifier with openai client (#5018)
# What does this PR do?
<!-- Provide a short summary of what this PR does and why. Link to
relevant issues if applicable. -->
<!-- If resolving an issue, uncomment and update the line below -->
Related to #4817
## Test Plan
<!-- Describe the tests you ran to verify your changes with result
summaries. *Provide clear instructions so the plan can be easily
re-executed.* -->
```console
(llama-stack) gualiu@gualiu-mac llama-stack % pytest -s -v tests/integration/responses/test_openai_response.py --stack-config=server:ci-tests --inference-mode record-if-missing -k "safety_identifier" --setup=gpt --suite=responses --color=yes --embedding-model=sentence-transformers/nomic-ai/nomic-embed-text-v1.5 --capture=tee-sys
=================================================================================================== test session starts ====================================================================================================
platform darwin -- Python 3.12.7, pytest-8.4.2, pluggy-1.6.0 -- /Users/gualiu/go/src/github.com/llamastack/llama-stack/.venv/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.12.7', 'Platform': 'macOS-26.3-arm64-arm-64bit', 'Packages': {'pytest': '8.4.2', 'pluggy': '1.6.0'}, 'Plugins': {'anyio': '4.9.0', 'html': '4.1.1', 'socket': '0.7.0', 'asyncio': '1.1.0', 'json-report': '1.5.0', 'timeout': '2.4.0', 'metadata': '3.1.1', 'cov': '6.2.1', 'nbval': '0.11.0'}}
rootdir: /Users/gualiu/go/src/github.com/llamastack/llama-stack
configfile: pyproject.toml
plugins: anyio-4.9.0, html-4.1.1, socket-0.7.0, asyncio-1.1.0, json-report-1.5.0, timeout-2.4.0, metadata-3.1.1, cov-6.2.1, nbval-0.11.0
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 11 items / 8 deselected / 3 selected
tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier[txt=openai/gpt-4o]
instantiating llama_stack_client
Starting llama stack server with config 'ci-tests' on port 8321...
Waiting for server at http://localhost:8321... (0.0s elapsed)
Waiting for server at http://localhost:8321... (0.5s elapsed)
Server is ready at http://localhost:8321
llama_stack_client instantiated in 2.045s
PASSED [ 33%]
tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_streaming[txt=openai/gpt-4o] PASSED [ 66%]
tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_and_previous_response[txt=openai/gpt-4o] PASSED [100%]Terminating llama stack server process...
Terminating process 98817 and its group...
Server process and children terminated gracefully
=================================================================================================== slowest 10 durations ===================================================================================================
2.06s setup tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier[txt=openai/gpt-4o]
0.14s call tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier[txt=openai/gpt-4o]
0.13s teardown tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_and_previous_response[txt=openai/gpt-4o]
0.03s call tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_and_previous_response[txt=openai/gpt-4o]
0.01s call tests/integration/responses/test_openai_response.py::TestOpenAIResponses::test_openai_response_with_safety_identifier_streaming[txt=openai/gpt-4o]
(4 durations < 0.005s hidden. Use -vv to show these durations.)
======================================================================================= 3 passed, 8 deselected, 2 warnings in 2.41s ========================================================================================
```1 parent 640c2a2 commit 6ac3e66
5 files changed
Lines changed: 1330 additions & 0 deletions
File tree
- tests/integration/responses
- recordings
Lines changed: 301 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments