Safety features

[Wunderwaffel]

• Improved prompts by adding safety/security instructions
• Improved separation of roles and marking of untrusted inputs using xml-tags
• Rate limiting using slowapi
• Preventing requests with exceedingly large input sizes
• Safety canary for detecting output of system prompt
• Filtering of offensive user inputs using OpenAI's moderation API

[Copilot]

Pull request overview

This PR implements comprehensive safety and security features for a scientific paper search and analysis API, addressing prompt injection vulnerabilities, toxic content filtering, and DoS prevention through multiple defensive layers.

Changes:

• Added rate limiting using slowapi with configurable limits per endpoint (3-10 requests/minute)
• Implemented input validation with maximum length constraints on all user inputs (5k-100k characters depending on field)
• Introduced OpenAI moderation API integration to filter offensive content and a safety canary system to detect prompt leakage
• Enhanced prompts with XML tags to clearly separate trusted system instructions from untrusted user/paper content

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
backend/requirements.txt	Added slowapi 0.1.9 dependency for rate limiting
backend/app/core/config.py	Added SAFETY_CANARY configuration field for prompt leakage detection
backend/app/core/limiter.py	Created centralized rate limiter instance with 5/minute default
backend/app/core/safety.py	Implemented SafetyService with moderation checks and output validation
backend/.env.example	Added example SAFETY_CANARY configuration
backend/app/main.py	Integrated SlowAPI middleware and exception handlers
backend/app/routes/search_routes.py	Added rate limiting decorators and max_length validation on form inputs
backend/app/routes/paper_routes.py	Applied rate limiting to summary and chat endpoints
backend/app/schemas/search_dto.py	Added max_length constraint to search query field
backend/app/schemas/paper_dto.py	Added max_length constraints to all user input fields
backend/app/services/search_service.py	Integrated moderation checks before processing search queries
backend/app/services/paper_service.py	Added moderation on user queries and canary validation on outputs
backend/app/llm/openai/provider.py	Wrapped untrusted inputs in XML tags and implemented moderation API integration
backend/app/llm/openai/prompts.py	Enhanced prompts with security instructions and XML tag documentation
backend/app/repositories/search_repository.py	Added threshold parameter to filter low-quality search results

Comments suppressed due to low confidence (2)

backend/app/repositories/search_repository.py:18

• A new threshold parameter has been added with a default value of 0.4, but this is not documented in the docstring. The docstring should explain what the threshold represents (appears to be a maximum cosine distance for filtering results) and why 0.4 was chosen as the default value. Additionally, none of the callers are using this parameter, so it's unclear whether this change is fully integrated or if it's intended for future use.

        db: AsyncSession, embeddings: List[List[float]], limit: int = 5, threshold: float = 0.4
    ) -> List[Tuple[PaperModel, float]]:
        """
        Perform a vector search for papers based on a list of embeddings.
        Returns a list of (PaperModel, avg_distance) tuples ordered by ascending distance.

backend/app/llm/openai/provider.py:149

• The variable user_message_content is constructed with proper XML tags for the paper text and optional user intent, but it is never used in the API call. Instead, line 149 directly passes paper_text without the XML tags, which defeats the purpose of the security improvements mentioned in the PR description. The user content should be passed as user_message_content instead of paper_text to maintain the proper separation of untrusted inputs using XML tags.

        user_message_content = f"<paper_text>\n{paper_text}\n</paper_text>"

        if has_query:
            user_message_content += f"\n\n<user_intent>\n{query}\n</user_intent>"

        response = await self.client.responses.create(
            model=self._model,
            reasoning={"effort": "medium"},
            input=[
                {
                    "role": "developer",
                    "content": prompt_content,
                },
                {
                    "role": "user",
                    "content": paper_text,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.