Skip to content

Commit 736d531

Browse files
committed
Update tests/README.md
1 parent 3f65415 commit 736d531

1 file changed

Lines changed: 31 additions & 142 deletions

File tree

tests/README.md

Lines changed: 31 additions & 142 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,8 @@
11
# Parse Patrol Test Suite
22

3-
## Testing
3+
## Unit Tests
44

5-
Testing is done in multiple ways to ensure both functionality and MCP server behavior:
6-
7-
### Automated Unit Tests
8-
9-
Run the comprehensive test suite using pytest:
5+
Run the comprehensive test suite using `pytest`:
106

117
```bash
128
# Run all tests with verbose output
@@ -23,94 +19,44 @@ uv run pytest tests/ --cov=parse_patrol
2319
The test suite includes:
2420

2521
- **MCP Server Initialization**: Tests that the unified MCP server loads correctly
26-
- **Parser Configuration**: Parametrized tests ensuring each parser has proper configuration
2722
- **Runtime Import Tests**: Validates that parsers work with their dependencies (gracefully skips if dependencies missing)
28-
- **Minimal Dependency Tests**: Tests core MCP functionality without optional parser dependencies
29-
30-
(`test_modular_mcp.py`)
31-
32-
- **Purpose**: Test MCP server initialization and parser configuration
33-
- **Speed**: Fast (< 1 second)
34-
- **Dependencies**: None (tests graceful handling of missing dependencies)
35-
- **Markers**: `unit`
36-
37-
Tests:
38-
39-
- MCP server initialization
40-
- Parser configuration validation
41-
- Individual parser import testing
42-
- Graceful dependency handling
43-
44-
#### Expected Test Results
45-
46-
```text
47-
✅ test_mcp_server_initialization - MCP server loads correctly
48-
✅ test_parser_config_exists[cclib parser] - Configuration present
49-
✅ test_individual_parser_imports[cclib] - Parser imports successfully
50-
⚠️ test_individual_parser_imports[iodata] - Skipped (dependency missing)
51-
```
52-
53-
### Continuous Integration
54-
55-
GitHub Actions automatically runs tests on all pushes and pull requests:
56-
57-
- **Multi-Python Testing**: Tests run on Python 3.9, 3.10, 3.11, and 3.12
58-
- **Code Quality**: Linting with ruff and optional type checking with mypy
59-
- **Coverage Reporting**: Test coverage is tracked and reported via Codecov
23+
- NOMAD database search functionality
24+
- File download and extraction
25+
- NOMAD `pydantic` schema
6026

61-
The CI ensures that:
27+
Note that actual testing of the parsers and their semantics is relegated to development projects for each instance.
6228

63-
- All tests pass across supported Python versions
64-
- Code follows consistent formatting and style
65-
- Dependencies install correctly with `uv`
66-
- Both minimal and full installation scenarios work(`test_nomad_integration.py`)
29+
### Pytest Markers
6730

68-
- **Purpose**: End-to-end testing with real quantum chemistry files
69-
- **Speed**: Slow (1-5 minutes, downloads files)
70-
- **Dependencies**: `requests`, parser libraries (`cclib`, `iodata`, etc.)
71-
- **Markers**: `integration`, `slow`
31+
- `unit`: Fast unit tests
32+
- `integration`: Tests requiring network access
33+
- `slow`: Tests that download large files (1-5 minutes)
7234

73-
Tests:
35+
Many of these markers indicated expected performance:
7436

75-
- NOMAD database search functionality
76-
- File download and extraction
77-
- Parser testing with real QM software output files
78-
- Multi-software workflow validation
79-
- Automatic cleanup of downloaded files
80-
81-
#### Expected Test Results
82-
83-
```text
84-
✅ test_nomad_search_functionality - NOMAD search works
85-
✅ test_nomad_download_and_parse[Gaussian] - Downloaded and parsed Gaussian files
86-
✅ test_nomad_download_and_parse[ORCA] - Downloaded and parsed ORCA files
87-
⚠️ test_nomad_download_and_parse[VASP] - Skipped (no VASP files found)
88-
✅ test_nomad_multi_software_workflow - Multi-software workflow successful
89-
```
37+
- **Unit tests**: < 1 second total
38+
- **NOMAD search test**: 5-10 seconds
9039

91-
### Agent Pipeline Testing
40+
### File Management
9241

93-
Test end-to-end workflows:
42+
- Downloads to `.data/[entry_id]/` directory
43+
- Automatic cleanup after each test
9444

95-
- The agent attempts to generate pipeline scripts using the MCP tools
96-
- Successful cases are stored in `.pipelines/scripts/`
97-
- Input data for processing is found in `.pipelines/data/`
98-
- Schema definitions and documentation resources are available in `.resources/`
45+
### Error Handling
9946

100-
This directory contains comprehensive tests for the Parse Patrol chemistry parsing package, including integration tests that download real computational chemistry files from NOMAD and test parsing functionality.
47+
- Skip if software files not found in NOMAD
48+
- Network timeout handling
49+
- Comprehensive cleanup on failures
10150

102-
## Supported Quantum Chemistry Software
51+
### Continuous Integration
10352

104-
The integration tests cover files from:
53+
GitHub Actions automatically runs all tests on pushes to a pull request:
10554

106-
- **Gaussian** (`.log`, `.out`, `.fchk` files)
107-
- **ORCA** (`.out`, `.inp` files)
108-
- **VASP** (`OUTCAR`, `POSCAR`, `CONTCAR` files)
109-
- **Q-Chem** (`.out`, `.in` files)
110-
- **NWChem** (`.out`, `.nw` files)
111-
- **Psi4** (`.out` files)
55+
- Tests run on Python 3.12
56+
<!-- - **Code Quality**: Linting with `ruff` and optional type checking with `mypy` -->
57+
- Test coverage is tracked and reported via Codecov
11258

113-
## Running Tests
59+
## Manually Running Tests
11460

11561
### Quick Setup
11662

@@ -127,43 +73,6 @@ uv run pytest -m "not slow"
12773
uv run pytest tests/test_nomad_integration.py -vv -s --tb=long --log-cli-level=INFO
12874
```
12975

130-
Or use the test interfaces:
131-
132-
```bash
133-
# Test specific software
134-
uv run python tests/run_nomad_tests.py --software Gaussian
135-
```
136-
137-
### Pytest Markers
138-
139-
- `unit`: Fast unit tests
140-
- `integration`: Tests requiring network access
141-
- `slow`: Tests that download large files (1-5 minutes)
142-
143-
## Integration Test Workflow
144-
145-
### For Each Supported Software
146-
147-
1. **Search NOMAD** for computational files from specific QM software
148-
2. **Download** raw files to temporary directory
149-
3. **Parse** files with compatible parsers
150-
4. **Validate** parsed data structure and content
151-
5. **Cleanup** all downloaded files automatically
152-
153-
### File Management
154-
155-
- Downloads to `.data/[entry_id]/` directory
156-
- Automatic cleanup after each test
157-
- Temporary directories for test isolation
158-
- No permanent files left behind
159-
160-
### Error Handling
161-
162-
- Skip if software files not found in NOMAD
163-
- Parser-specific error handling
164-
- Network timeout handling
165-
- Comprehensive cleanup on failures
166-
16776
## Troubleshooting
16877

16978
**"NOMAD dependencies not available":**
@@ -180,10 +89,11 @@ uv sync --parsers
18089

18190
**"No [Software] files found in NOMAD"**
18291

183-
- Expected behavior
184-
- NOMAD content varies
185-
- Tests skip gracefully
186-
- Try different formulas or wait for database updates
92+
All queries and downloads have been verified.
93+
While the contents of NOMAD-lab is dynamic, the number of hits should only grow.
94+
This implies that tests with an expected empty return value could auto-deprecate after time.
95+
96+
Regardless, visit [the NOMAD-lab website](https://nomad-lab.eu/nomad-lab/) directly and try the same query there.
18797

18898
**Network timeouts:**
18999

@@ -222,24 +132,3 @@ When adding new tests:
222132
3. **Clean up resources**: Use context managers or proper teardown
223133
4. **Test realistic scenarios**: Use real data when possible
224134
5. **Document expected behavior**: Include docstrings explaining test purpose
225-
226-
### Adding New Software Support
227-
228-
1. Add configuration to `QM_SOFTWARE_CONFIGS` in `test_nomad_integration.py`
229-
2. Specify expected file extensions and compatible parsers
230-
3. Test with `uv run python tests/run_nomad_tests.py --software [NewSoftware]`
231-
232-
## Performance Notes
233-
234-
- **Unit tests**: < 1 second total
235-
- **NOMAD search test**: 5-10 seconds
236-
- **Single software integration test**: 30-120 seconds
237-
- **Full integration suite**: 3-8 minutes
238-
239-
Tests are designed to be robust and handle:
240-
241-
- Network variability
242-
- NOMAD database changes
243-
- Missing optional dependencies
244-
- File format variations
245-
- Parser errors and limitations

0 commit comments

Comments
 (0)