Production-ready data quality testing framework for public APIs — built with pytest, requests, and JSON Schema validation. Demonstrates end-to-end ETL testing skills that European fintech and data startups actively seek.
This suite performs 15+ comprehensive data quality tests across two real public APIs:
| API | Base URL | Auth | Tests |
|---|---|---|---|
| REST Countries | https://restcountries.com/v3.1 |
None (free) | Schema, null checks, uniqueness, type consistency |
| CoinGecko | https://api.coingecko.com/api/v3 |
Free tier API key | Schema, market data quality, pagination, freshness |
| Category | Count | Description |
|---|---|---|
| Schema Validation | 10 | JSON Schema compliance, mandatory fields, format validation |
| Data Quality | 12 | Null checks, uniqueness, completeness, range validation |
| Pagination | 10 | Page limits, sequence consistency, boundary testing |
| Error Handling | 14 | 404s, invalid inputs, timeouts, response headers |
| Integration | 8 | ETL pipeline simulation, cross-API consistency, batch processing |
data-api-qa-suite/
├── src/
│ ├── config.py # Test configuration & thresholds
│ ├── api_client.py # HTTP clients with retry, rate limiting, logging
│ ├── schema_validator.py # JSON Schema definitions & validation engine
│ └── data_assertions.py # Reusable data quality assertion framework
├── tests/
│ ├── conftest.py # Pytest fixtures & shared resources
│ ├── test_rest_countries_schema.py # 10 schema tests
│ ├── test_rest_countries_data_quality.py # 10 data quality tests
│ ├── test_coingecko_schema.py # 10 schema tests
│ ├── test_coingecko_data_quality.py # 12 data quality tests
│ ├── test_pagination.py # 10 pagination tests
│ ├── test_error_handling.py # 14 error handling tests
│ └── test_integration.py # 8 integration tests
├── reports/ # Test output & coverage reports
├── requirements.txt # Python dependencies
├── pytest.ini # Pytest configuration
├── Makefile # Convenient command shortcuts
└── README.md # This file
| Tool | Purpose |
|---|---|
| pytest | Test framework with fixtures, markers, and parametrization |
| requests | HTTP client with session management |
| jsonschema | JSON Schema validation for API contract testing |
| pytest-html | HTML test report generation |
| pytest-cov | Code coverage analysis |
| python-dotenv | Environment configuration management |
git clone https://github.com/YOUR_USERNAME/data-api-qa-suite.git
cd data-api-qa-suite
pip install -r requirements.txtcp .env.example .env
# Edit .env to add CoinGecko API key (free at coingecko.com)# Run all tests
pytest
# Run specific test categories
pytest -m schema # Schema validation only
pytest -m data_quality # Data quality only
pytest -m pagination # Pagination tests
pytest -m error_handling # Error handling tests
pytest -m integration # Integration tests
# Run with HTML report
pytest --html=reports/test_report.html --self-contained-html
# Run with coverage
pytest --cov=src --cov-report=html
# Use Makefile shortcuts
make test
make test-schema
make test-dq
make test-coverage============================= test session starts ==============================
platform linux -- Python 3.11.4
rootdir: /data-api-qa-suite
configfile: pytest.ini
src/api_client.py ..............
tests/test_rest_countries_schema.py ..........
tests/test_rest_countries_data_quality.py ..........
tests/test_coingecko_schema.py ..........
tests/test_coingecko_data_quality.py ...........
tests/test_pagination.py ..........
tests/test_error_handling.py ..............
tests/test_integration.py ........
======================== 74 passed in 45.32s =========================
============================================================
COMBINED QUALITY REPORT
============================================================
Total Assertions: 1,247
Passed: 1,245
Failed: 2
Pass Rate: 99.84%
============================================================
# Validate every response against a JSON Schema
errors = SchemaValidator.validate(country, "country")
assert len(errors) == 0# Ensure critical fields are never null
null_count = sum(1 for c in data if c.get("population") is None)
null_rate = (null_count / len(data)) * 100
assert null_rate <= 5.0 # Max 5% nulls acceptable# Verify primary key uniqueness
assertion.assert_unique(cca3_codes, "country.cca3")# Verify pages don't overlap and are sequential
page_1_ids = {c["id"] for c in page_1}
page_2_ids = {c["id"] for c in page_2}
assert len(page_1_ids & page_2_ids) == 0# Test 404s, timeouts, and connection errors
response = client.get_country_by_name("nonexistent")
assert response.status_code == 404# Verify market data is recent
age = now - datetime.fromisoformat(last_updated)
assert age <= timedelta(hours=1)| Skill | How It's Demonstrated |
|---|---|
| Schema Validation | JSON Schema definitions for 4+ API entities |
| Data Quality | Null checks, uniqueness, range validation, completeness |
| ETL Testing | Pipeline simulation, batch processing, transformation readiness |
| API Testing | Retry logic, rate limiting, timeout handling, status codes |
| SQL Skills | Data type assertions, primary key validation, null rate analysis |
| Automation | pytest fixtures, markers, parametrization, CI-ready structure |
| Reporting | HTML reports, coverage analysis, quality scorecards |
data-qa etl-testing api-testing data-quality pytest python rest-api json-schema qa-automation data-engineering
MIT License — feel free to use as a template for your own data QA projects.
Built for Data Science & QA Engineering roles. If you're hiring for data quality, ETL testing, or API validation — this is exactly what junior candidates rarely have on their GitHub.