[ENHANCEMENT]: Optimize API response payload and introduce compression for improved performance

[piyush06singhal]

Feature and its Use Cases

What is the feature?

This feature focuses on improving API response efficiency by reducing unnecessary data in responses and introducing response compression.

From analysis of the current backend, endpoints such as /get_mcq, /get_shortq, /get_boolq, and /get_problems return large payloads that include fields not used by the frontend. These include:

• Full processed input text (statement / Text)
• Context data for each question
• Extra options beyond required choices
• Metadata such as time_taken and options_algorithm

In some cases, especially /get_problems, multiple full outputs are combined in a single response, leading to significantly large payload sizes.

---

How would users benefit from it?

• Reduced API response size (removal of redundant fields)
• Faster response times, especially on slower networks
• Improved frontend performance due to lighter payloads
• Reduced bandwidth usage for repeated or large requests
• Better scalability under concurrent usage

---

What scenarios would this feature address?

• Large quiz generation requests causing high latency
• Combined endpoints returning duplicated and unnecessary data
• Mobile or low-bandwidth users experiencing slow responses
• Increased server load due to large JSON serialization

---

Proposed Improvements

1. Response Filtering

   • Return only required fields used by frontend:
     • question
     • answer
     • options (limited set)
     • question_type
   • Remove unused fields such as:
     • statement, context, extra_options, time_taken, options_algorithm

2. Response Compression

   • Introduce gzip compression using middleware (e.g., Flask-Compress)
   • Automatically compress JSON responses for large payloads

3. Combined Endpoint Optimization

   • Refactor /get_problems to avoid redundant data duplication
   • Ensure shared data (like input text) is not repeated across outputs

4. Optional Enhancement

   • Introduce lightweight response mode (e.g., ?minimal=true) for optimized frontend consumption

This enhancement improves performance without altering core functionality or requiring major architectural changes.

Additional Context

Additional Context

Initial analysis shows that API responses may contain a large percentage of redundant data, especially in combined endpoints.

For typical quiz generation requests, this can significantly increase payload size and latency. Introducing response filtering and compression can reduce response size substantially and improve system performance.

This change is backwards-compatible and can be implemented incrementally.

Code of Conduct

• I have joined the Discord server and will post updates there
• I have searched existing issues to avoid duplicates