Add debugger updates and improve development tooling

.github/workflows/test-publish.yaml

@@ -12,7 +12,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest]
-        python-version: ["3.11", "3.12", "3.13"]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
 
     steps:
     - uses: actions/checkout@v5
@@ -23,8 +23,8 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install pymadng[tfs]
+        python -m pip install pymadng[tfs] pytest
 
     - name: Test with python
       run: |
-        python -m unittest tests/*.py
+        python -m pytest tests

.github/workflows/test-pymadng.yml

@@ -22,14 +22,16 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest]
-        python-version: ["3.11", "3.12", "3.13"]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
 
     steps:
     - uses: actions/checkout@v5
     - name: Set up Python ${{ matrix.python-version }} on ${{ matrix.os }}
       uses: actions/setup-python@v6
       with:
         python-version: ${{ matrix.python-version }}
+    - name: Set up uv
+      uses: astral-sh/setup-uv@v6
     - name: Get MAD Binaries
       run: |
         mkdir ./src/pymadng/bin
@@ -38,8 +40,12 @@ jobs:
         chmod +x ./src/pymadng/bin/mad_Linux ./src/pymadng/bin/mad_Darwin
     - name: Install dependencies
       run: |
-        python -m pip install --upgrade pip
-        python -m pip install -e .[tfs]
+        uv sync --extra tfs --extra test
     - name: Test with python
       run: |
-        python -m unittest tests/*.py
+        uv run pytest tests --cov=src/pymadng --cov-report=xml --cov-report=term
+    - name: Upload coverage to Codecov
+      if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11' && github.event_name == 'push' && github.ref == 'refs/heads/main'
+      uses: codecov/codecov-action@v5
+      with:
+        files: ./coverage.xml

.pre-commit-config.yaml

@@ -0,0 +1,14 @@
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.13.0
+    hooks:
+      - id: ruff-check
+        args: [--fix]
+      - id: ruff-format
+
+  - repo: https://github.com/streetsidesoftware/cspell-cli
+    rev: v9.3.3
+    hooks:
+      - id: cspell
+        args: [--no-progress, --no-summary]
+        files: ^(README\.md|CHANGELOG\.md|docs/source/.*\.(md|rst))$

CHANGELOG.md

@@ -1,3 +1,9 @@
+0.10.0 (2026/03/11) \
+Major debug update: PyMAD-NG now exposes the MAD-NG debugger directly through `MAD.breakpoint()` and `MAD.pydbg()`, with MAD-side aliases (`breakpoint`, `pydbg`, and `python_breakpoint`) available inside executed code. \
+Debugger sessions now support scripted commands for tests and automation, plus improved interactive terminal handling with a stable Python-rendered prompt. \
+Quitting from the debugger now shuts down the current MAD session cleanly without leaving pipe state inconsistent. \
+Expanded debugger test coverage and documentation for Python-driven MAD debugging workflows.
+
 0.9.0 (2026/02/25) \
 Remove support for Python 3.10, now only supporting Python 3.11 and above. \
 Replaced incorrect Warning call with logging.warning for proper warning handling. \

README.md

@@ -4,6 +4,7 @@
 
 [![PyPI version](https://img.shields.io/pypi/v/pymadng.svg)](https://pypi.org/project/pymadng/)
 [![Documentation Status](https://readthedocs.org/projects/pymadng/badge/?version=latest)](https://pymadng.readthedocs.io/en/latest/)
+[![codecov](https://codecov.io/gh/MethodicalAcceleratorDesign/MAD-NG.py/branch/main/graph/badge.svg)](https://codecov.io/gh/MethodicalAcceleratorDesign/MAD-NG.py)
 [![License](https://img.shields.io/github/license/MethodicalAcceleratorDesign/MAD-NG.py)](https://github.com/MethodicalAcceleratorDesign/MAD-NG.py/blob/main/LICENSE)
 
 ---
@@ -28,9 +29,16 @@ Before diving into PyMAD-NG, we recommend you:
 ### Explore Key Examples
 
 - **[LHC Matching Example](https://pymadng.readthedocs.io/en/latest/ex-lhc-couplingLocal.html)** – Real-world optics matching with intermediate feedback.
-- **[Examples Page](https://pymadng.readthedocs.io/en/latest/examples.html)** - List of examples in an easy to read format. 
+- **[Examples Page](https://pymadng.readthedocs.io/en/latest/examples.html)** - List of examples in an easy to read format.
 - **[GitHub Examples Directory](https://github.com/MethodicalAcceleratorDesign/MAD-NG.py/blob/main/examples/)** – List of available examples on the repository
 
+### A Few Things To Know Early
+
+- `MAD()` launches a real MAD-NG subprocess immediately. Prefer `with MAD() as mad:` in normal use.
+- `mad["x"] = value` writes a variable inside MAD-NG. `mad.x = value` only changes the Python wrapper object.
+- Communication is explicit: tell MAD-NG to `py:send(...)` before calling `mad.recv()`, and tell MAD-NG to `py:recv()` before sending Python data.
+- Many high-level results are references to MAD-NG objects. Use `.eval()` or conversion helpers such as `.to_df()` when you want Python-side values.
+
 If anything seems unclear:
 - Refer to the [API Reference](https://pymadng.readthedocs.io/en/latest/pymadng.html#module-pymadng)
 - Check the [MAD-NG Docs](https://madx.web.cern.ch/releases/madng/html/)
@@ -59,7 +67,7 @@ Examples are stored in the `examples/` folder.
 Run any script with:
 
 ```bash
-python3 examples/ex-fodos.py
+python3 examples/ex-fodo/ex-fodos.py
 ```
 
 You can also batch-run everything using:
@@ -82,7 +90,7 @@ python3 runall.py
 
 ## 🤝 Contributing
 
-We welcome contributions! See [`CONTRIBUTING.md`](docs/source/contributing.md) or the [Contributing Guide](https://pymadng.readthedocs.io/en/latest/contributing.html) in the docs.
+We welcome contributions! See [the contributing guide](docs/source/contributing.md) or the [Contributing Guide](https://pymadng.readthedocs.io/en/latest/contributing.html) in the docs.
 
 Bug reports, feature requests, and pull requests are encouraged.
 

cspell.config.yaml

@@ -0,0 +1,68 @@
+version: "0.2"
+language: en-GB
+
+ignorePaths:
+  - .git
+  - .venv
+  - build
+  - dist
+
+words:
+  - addopts
+  - assertf
+  - automodule
+  - beamline
+  - cmatrix
+  - CTPSA
+  - currentmodule
+  - dataframe
+  - dataframes
+  - defexpr
+  - eval-rst
+  - fodo
+  - fodos
+  - gmath
+  - gphys
+  - imatrix
+  - initialise
+  - kwargs
+  - ipairs
+  - irange
+  - lhcb
+  - linenos
+  - linters
+  - literalinclude
+  - logrange
+  - MADX
+  - MADNG
+  - madp
+  - matplotlib
+  - mtable
+  - mtbl
+  - ndarray
+  - numpy
+  - knl
+  - pathlib
+  - pyplot
+  - pydbg
+  - pymad
+  - pymadng
+  - PyPI
+  - pyproject
+  - pytest
+  - runall
+  - rtrn
+  - seqfile
+  - stdout
+  - stderr
+  - sterr
+  - testpaths
+  - tonumber
+  - tostring
+  - TPSA
+  - tpsa
+  - twiss
+  - venv
+  - vars
+  - xlabel
+  - ylabel

docs/source/communication.md

@@ -16,6 +16,10 @@ Key points:
 - Data is retrieved via `{func}`MAD.recv`()` after explicit instruction to send it.
 - MAD-NG stdout is redirected to Python, but not intercepted.
 
+```{tip}
+Think of PyMAD-NG as controlling a persistent remote interpreter. Python and MAD-NG do not share memory; they exchange commands, references, and serialised data through pipes.
+```
+
 ```{important}
 You must always **send instructions before sending data**, and **send a request before receiving data**.
 ```
@@ -34,6 +38,25 @@ mad.recv()                  # Receive the value → 42
 Both {func}`MAD.send` and {func}`MAD.recv` are the core communication methods.
 See the {class}`pymadng.MAD` reference for more details.
 
+### Two Common Patterns
+
+#### Pattern A: MAD-NG asks Python for data
+
+```python
+mad.send("arr = py:recv()")
+mad.send(my_array)
+```
+
+#### Pattern B: Python asks MAD-NG for data
+
+```python
+mad.send("tbl = twiss {sequence=seq}")
+mad.send("py:send(tbl)")
+tbl = mad.recv("tbl") # Receives the table as a Python object that is interactive and can be converted to a DataFrame
+```
+
+If you keep these two patterns distinct, most send/receive bugs become much easier to reason about.
+
 ---
 
 ## Supported Data Types
@@ -63,14 +86,18 @@ The following types can be sent from Python to MAD-NG:
   - {func}`MAD.send`
 * - `start, stop, size` as float, int
   - `range`, `logrange`
-  - `mad.send_rng()`, `mad.send_lrng()`
+  - `mad.send_range()`, `mad.send_logrange()`
 * - Complex structures (e.g., TPSA, CTPSA)
   - `TPSA`, `CTPSA`
-  - `mad.send_tpsa()`, `mad.send_ctpsa()`
+  - `mad.send_tpsa()`, `mad.send_cpx_tpsa()`
 ```
 
 For full compatibility, see the {mod}`pymadng.MAD` documentation.
 
+```{note}
+High-level MAD objects are often returned as references rather than copied Python objects. This is expected behaviour and is part of how PyMAD-NG avoids unnecessary data transfer.
+```
+
 ---
 
 ## Converting TFS Tables to DataFrames
@@ -113,6 +140,15 @@ mad.send(arr2)                 # DEADLOCK if previous data not yet received
 Always ensure each {func}`MAD.send` has a matching {func}`MAD.recv` if data is expected back.
 ```
 
+### A Safer Way to Think About It
+
+Before each operation, ask one question:
+
+- "Is MAD-NG waiting for Python?"
+- or "Is Python waiting for MAD-NG?"
+
+If both sides are waiting to receive, the session will deadlock.
+
 ---
 
 ## Scope: Local vs Global
@@ -137,6 +173,65 @@ mad.send("print(a + (b or 5))")  # b is nil → 10 + 5 = 15
 Use `local` to avoid polluting the global MAD-NG namespace.
 ```
 
+### Python Assignment vs MAD Assignment
+
+This is worth stating explicitly because it is a common first-use mistake:
+
+```python
+mad["x"] = 5   # writes x inside MAD-NG
+mad.x = 5      # writes an attribute on the Python wrapper only
+```
+
+If you intend to create a MAD-NG variable, always use square-bracket assignment.
+
+---
+
+## References and Materialising Values
+
+Many objects returned by the high-level interface are references to values living inside MAD-NG.
+
+```python
+ref = mad.math.sin(1)
+```
+
+To materialise a Python value, if it exists (rather than a reference), use the `eval()` method:
+
+```python
+value = ref.eval()
+```
+
+To materialise a table:
+
+```python
+df = mad.tbl.to_df()
+```
+
+This distinction is especially important when inspecting objects interactively, writing assertions in tests, or passing results into ordinary Python libraries.
+
+---
+
+## Executing Python Returned by MAD-NG
+
+{func}`MAD.recv_and_exec` executes Python code sent back from MAD-NG:
+
+```python
+mad.send("py:send([[print('hello from MAD')]])")
+mad.recv_and_exec()
+```
+
+The execution context automatically includes:
+
+- `mad`: the current PyMAD-NG session
+- `breakpoint`: the debugger bridge
+- `pydbg`: alias for the debugger bridge
+
+That means MAD-NG can request an interactive debugger stop like this:
+
+```python
+mad.send("py:send([[breakpoint()]])")
+mad.recv_and_exec()
+```
+
 ---
 
 ## Customising the Environment
@@ -174,10 +269,9 @@ See {meth}`pymadng.MAD.__init__` for all configuration options.
 ## Summary
 
 - Always match {func}`MAD.send` with {func}`MAD.recv` when data is expected.
-- Use `mad.to_df()` for table conversion.
+- Use `.to_df()` on MAD tables when you want a DataFrame.
 - Avoid deadlocks by receiving before sending again.
 - Manage scope using `local` wisely.
 - Use configuration flags to tailor behaviour.
 
 For more, see the {doc}`advanced_features`, {doc}`debugging`, and {doc}`function_reference` sections.
-