|
| 1 | +# Parse Patrol Test Suite |
| 2 | + |
| 3 | +This directory contains comprehensive tests for the Parse Patrol chemistry parsing package, including integration tests that download real computational chemistry files from NOMAD and test parsing functionality. |
| 4 | + |
| 5 | +## Test Categories |
| 6 | + |
| 7 | +### 1. Unit Tests (`test_modular_mcp.py`) |
| 8 | +- **Purpose**: Test MCP server initialization and parser configuration |
| 9 | +- **Speed**: Fast (< 1 second) |
| 10 | +- **Dependencies**: None (tests graceful handling of missing dependencies) |
| 11 | +- **Markers**: `unit` |
| 12 | + |
| 13 | +Tests: |
| 14 | +- MCP server initialization |
| 15 | +- Parser configuration validation |
| 16 | +- Individual parser import testing |
| 17 | +- Graceful dependency handling |
| 18 | + |
| 19 | +### 2. Integration Tests (`test_nomad_integration.py`) |
| 20 | +- **Purpose**: End-to-end testing with real quantum chemistry files |
| 21 | +- **Speed**: Slow (1-5 minutes, downloads files) |
| 22 | +- **Dependencies**: `requests`, parser libraries (`cclib`, `iodata`, etc.) |
| 23 | +- **Markers**: `integration`, `slow` |
| 24 | + |
| 25 | +Tests: |
| 26 | +- NOMAD database search functionality |
| 27 | +- File download and extraction |
| 28 | +- Parser testing with real QM software output files |
| 29 | +- Multi-software workflow validation |
| 30 | +- Automatic cleanup of downloaded files |
| 31 | + |
| 32 | +## Supported Quantum Chemistry Software |
| 33 | + |
| 34 | +The integration tests cover files from: |
| 35 | + |
| 36 | +- **Gaussian** (`.log`, `.out`, `.fchk` files) |
| 37 | +- **ORCA** (`.out`, `.inp` files) |
| 38 | +- **VASP** (`OUTCAR`, `POSCAR`, `CONTCAR` files) |
| 39 | +- **Q-Chem** (`.out`, `.in` files) |
| 40 | +- **NWChem** (`.out`, `.nw` files) |
| 41 | +- **Psi4** (`.out` files) |
| 42 | + |
| 43 | +## Running Tests |
| 44 | + |
| 45 | +### Quick Setup |
| 46 | +```bash |
| 47 | +# Install test dependencies |
| 48 | +uv sync |
| 49 | + |
| 50 | +# Run all tests |
| 51 | +uv run pytest |
| 52 | + |
| 53 | +# Run only unit tests (fast) |
| 54 | +uv run pytest -m unit |
| 55 | + |
| 56 | +# Run only integration tests |
| 57 | +uv run pytest -m integration |
| 58 | + |
| 59 | +# Run without slow download tests |
| 60 | +uv run pytest -m "not slow" |
| 61 | +``` |
| 62 | + |
| 63 | +### Using the Test Runner |
| 64 | +```bash |
| 65 | +# Quick test (just check dependencies and run a few tests) |
| 66 | +uv run python tests/run_nomad_tests.py --quick |
| 67 | + |
| 68 | +# Test specific software |
| 69 | +uv run python tests/run_nomad_tests.py --software Gaussian |
| 70 | + |
| 71 | +# Full test suite (all software, all parsers) |
| 72 | +uv run python tests/run_nomad_tests.py |
| 73 | +``` |
| 74 | + |
| 75 | +### Manual Control |
| 76 | +```bash |
| 77 | +# Run specific test file |
| 78 | +uv run pytest tests/test_nomad_integration.py -v |
| 79 | + |
| 80 | +# Run specific test function |
| 81 | +uv run pytest tests/test_nomad_integration.py::test_nomad_search_functionality -v |
| 82 | + |
| 83 | +# Run with detailed output |
| 84 | +uv run pytest tests/test_nomad_integration.py -v -s |
| 85 | +``` |
| 86 | + |
| 87 | +## Test Configuration |
| 88 | + |
| 89 | +### Pytest Markers |
| 90 | +- `unit`: Fast unit tests |
| 91 | +- `integration`: Tests requiring network access |
| 92 | +- `slow`: Tests that download large files (1-5 minutes) |
| 93 | + |
| 94 | +### Environment Variables |
| 95 | +None required - tests adapt to available dependencies. |
| 96 | + |
| 97 | +### Dependencies |
| 98 | +- **Required**: `pytest`, `requests` |
| 99 | +- **Optional**: `cclib`, `qc-iodata`, quantum chemistry parser libraries |
| 100 | +- Tests will skip gracefully if dependencies are missing |
| 101 | + |
| 102 | +## Integration Test Workflow |
| 103 | + |
| 104 | +### For Each Supported Software: |
| 105 | +1. **Search NOMAD** for computational files from specific QM software |
| 106 | +2. **Download** raw files to temporary directory |
| 107 | +3. **Parse** files with compatible parsers (cclib, iodata, gaussian) |
| 108 | +4. **Validate** parsed data structure and content |
| 109 | +5. **Cleanup** all downloaded files automatically |
| 110 | + |
| 111 | +### File Management: |
| 112 | +- Downloads to `.data/[entry_id]/` directory |
| 113 | +- Automatic cleanup after each test |
| 114 | +- Temporary directories for test isolation |
| 115 | +- No permanent files left behind |
| 116 | + |
| 117 | +### Error Handling: |
| 118 | +- Graceful skipping if software files not found in NOMAD |
| 119 | +- Parser-specific error handling |
| 120 | +- Network timeout handling |
| 121 | +- Comprehensive cleanup on failures |
| 122 | + |
| 123 | +## Expected Test Results |
| 124 | + |
| 125 | +### Unit Tests |
| 126 | +``` |
| 127 | +✅ test_mcp_server_initialization - MCP server loads correctly |
| 128 | +✅ test_parser_config_exists[cclib parser] - Configuration present |
| 129 | +✅ test_individual_parser_imports[cclib] - Parser imports successfully |
| 130 | +⚠️ test_individual_parser_imports[iodata] - Skipped (dependency missing) |
| 131 | +``` |
| 132 | + |
| 133 | +### Integration Tests |
| 134 | +``` |
| 135 | +✅ test_nomad_search_functionality - NOMAD search works |
| 136 | +✅ test_nomad_download_and_parse[Gaussian] - Downloaded and parsed Gaussian files |
| 137 | +✅ test_nomad_download_and_parse[ORCA] - Downloaded and parsed ORCA files |
| 138 | +⚠️ test_nomad_download_and_parse[VASP] - Skipped (no VASP files found) |
| 139 | +✅ test_nomad_multi_software_workflow - Multi-software workflow successful |
| 140 | +``` |
| 141 | + |
| 142 | +## Troubleshooting |
| 143 | + |
| 144 | +### Common Issues |
| 145 | + |
| 146 | +**"NOMAD dependencies not available"** |
| 147 | +```bash |
| 148 | +uv sync # This will install all dependencies including requests |
| 149 | +``` |
| 150 | + |
| 151 | +**"No [Software] files found in NOMAD"** |
| 152 | +- Expected behavior - NOMAD content varies |
| 153 | +- Tests skip gracefully |
| 154 | +- Try different formulas or wait for database updates |
| 155 | + |
| 156 | +**"Parser not available"** |
| 157 | +```bash |
| 158 | +# Install specific parser using optional dependencies |
| 159 | +uv sync --cclib # for cclib parser |
| 160 | +# Or install all parsers |
| 161 | +uv sync --parsers |
| 162 | +``` |
| 163 | + |
| 164 | +**Network timeouts** |
| 165 | +- Tests include reasonable timeouts (30-120 seconds) |
| 166 | +- Check internet connection |
| 167 | +- NOMAD servers may be temporarily unavailable |
| 168 | + |
| 169 | +**Permission errors on cleanup** |
| 170 | +- Tests create temporary directories |
| 171 | +- Ensure write permissions in project directory |
| 172 | +- `.data/` directory will be created and cleaned up |
| 173 | + |
| 174 | +### Debug Mode |
| 175 | +```bash |
| 176 | +# Run with maximum verbosity |
| 177 | +uv run pytest tests/test_nomad_integration.py -vv -s --tb=long |
| 178 | + |
| 179 | +# Run single test with debug output |
| 180 | +uv run pytest tests/test_nomad_integration.py::test_nomad_search_functionality -vv -s |
| 181 | +``` |
| 182 | + |
| 183 | +## Contributing |
| 184 | + |
| 185 | +When adding new tests: |
| 186 | + |
| 187 | +1. **Use appropriate markers**: `@pytest.mark.integration`, `@pytest.mark.slow` |
| 188 | +2. **Handle missing dependencies gracefully**: Use try/except with `pytest.skip()` |
| 189 | +3. **Clean up resources**: Use context managers or proper teardown |
| 190 | +4. **Test realistic scenarios**: Use real data when possible |
| 191 | +5. **Document expected behavior**: Include docstrings explaining test purpose |
| 192 | + |
| 193 | +### Adding New Software Support |
| 194 | +1. Add configuration to `QM_SOFTWARE_CONFIGS` in `test_nomad_integration.py` |
| 195 | +2. Specify expected file extensions and compatible parsers |
| 196 | +3. Test with `uv run python tests/run_nomad_tests.py --software [NewSoftware]` |
| 197 | + |
| 198 | +## Performance Notes |
| 199 | + |
| 200 | +- **Unit tests**: < 1 second total |
| 201 | +- **NOMAD search test**: 5-10 seconds |
| 202 | +- **Single software integration test**: 30-120 seconds |
| 203 | +- **Full integration suite**: 3-8 minutes |
| 204 | + |
| 205 | +Tests are designed to be robust and handle: |
| 206 | +- Network variability |
| 207 | +- NOMAD database changes |
| 208 | +- Missing optional dependencies |
| 209 | +- File format variations |
| 210 | +- Parser errors and limitations |
0 commit comments