docs: add a hosted documentation site (#81)

Author: PsiACE

.github/workflows/docs.yml

@@ -0,0 +1,59 @@
+name: Documentation
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    name: Build documentation
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install just
+        uses: extractions/setup-just@v4
+        with:
+          just-version: "1.58.0"
+
+      - name: Install documentation dependencies
+        run: uv sync --locked --group docs --group docs-examples
+
+      - name: Build documentation
+        run: just docs
+
+      - name: Run documentation examples
+        run: just docs-examples
+
+  publish:
+    name: Trigger Read the Docs
+    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
+    needs: build
+    runs-on: ubuntu-24.04
+    timeout-minutes: 5
+
+    steps:
+      - name: Trigger the latest documentation build
+        env:
+          RTD_WEBHOOK_TOKEN: ${{ secrets.RTD_WEBHOOK_TOKEN }}
+          RTD_WEBHOOK_URL: ${{ secrets.RTD_WEBHOOK_URL }}
+        run: >-
+          curl --fail-with-body --retry 3
+          --data-urlencode "token=${RTD_WEBHOOK_TOKEN}"
+          --data-urlencode "branches=latest"
+          --data-urlencode "default_branch=main"
+          "${RTD_WEBHOOK_URL}"

.gitignore

@@ -12,3 +12,4 @@ opendalfs.egg-info/
 dist/
 .coverage
 coverage.xml
+docs/_build/

.readthedocs.yaml

@@ -0,0 +1,17 @@
+version: 2
+
+build:
+  os: ubuntu-24.04
+  tools:
+    python: "3.12"
+
+sphinx:
+  configuration: docs/conf.py
+  fail_on_warning: true
+
+python:
+  install:
+    - method: uv
+      command: sync
+      groups:
+        - docs

README.md

@@ -6,6 +6,9 @@
 
 `opendalfs` is a Python-based interface for file systems that enables interaction with different storage services by [Apache OpenDAL](https://github.com/apache/opendal). Through `opendalfs`, users can utilize fsspec's standard API to operate on all [storage services supported by OpenDAL](https://docs.rs/opendal/latest/opendal/services/index.html).
 
+Read the [opendalfs documentation](https://opendalfs.readthedocs.io/en/latest/)
+for installation, storage configuration, API details, and tested integrations.
+
 ## URL Protocols
 
 `opendalfs` registers multiple fsspec protocols in the form of `opendal+<service>`, for example:
@@ -38,70 +41,10 @@ receive their OpenDAL configuration through fsspec storage options.
 
 ## Installation
 
-### Basic Installation
-
 ```bash
 pip install opendalfs
 ```
 
-### Development Installation
-
-Install [uv](https://docs.astral.sh/uv/) and
-[just](https://just.systems/man/en/packages.html), then run:
-
-```bash
-just install
-```
-
-## Development Setup
-
-This project uses:
-
-- Python 3.12+ for the Python interface
-- ruff for code formatting and linting
-- ty for type checking
-- pytest for testing
-
-Run the local checks and unit tests with:
-
-```bash
-just check
-just unit
-```
-
-For development setup and guidelines, see our [Contributing Guide](https://github.com/fsspec/opendalfs/blob/main/CONTRIBUTING.md).
-
-## Benchmarks
-
-The benchmark script compares Arrow direct, opendalfs (fsspec), and s3fs (fsspec) on MinIO.
-
-```bash
-just install
-just bench --sizes 16,32,64 --files 4 --workers 4
-```
-
-Configure MinIO access via `OPENDAL_S3_ENDPOINT`, `OPENDAL_S3_BUCKET`,
-`OPENDAL_S3_REGION`, `OPENDAL_S3_ACCESS_KEY_ID`, and
-`OPENDAL_S3_SECRET_ACCESS_KEY`. Compose, tests, and benchmarks read the same
-values.
-
-The benchmark target starts MinIO from the root `docker-compose.yml`. Stop it
-when you finish:
-
-```bash
-just bench-down
-```
-
-For profiling, you can install a tool with `uv` (for example `py-spy`) and run:
-
-```bash
-uv tool install py-spy
-uv tool run py-spy record -o bench.svg -- python bench/bench_read_write.py --sizes 16,32,64 --files 4 --workers 4
-```
-
-High write concurrency can stall on some systems. If runs time out, reduce
-`--fsspec-workers`.
-
 ## Status
 
 See [Tracking issues of 0.1.0 version for opendalfs](https://github.com/fsspec/opendalfs/issues/6)

docs/check_examples.py

@@ -0,0 +1,30 @@
+import re
+from pathlib import Path
+
+import pytest
+
+DOCS_ROOT = Path(__file__).parent
+PYTHON_BLOCK = re.compile(r"^```python\n(.*?)^```$", re.MULTILINE | re.DOTALL)
+
+
+def python_example_files():
+    return [
+        path
+        for path in sorted(DOCS_ROOT.rglob("*.md"))
+        if PYTHON_BLOCK.search(path.read_text())
+    ]
+
+
+@pytest.mark.parametrize(
+    "path",
+    python_example_files(),
+    ids=lambda path: str(path.relative_to(DOCS_ROOT)),
+)
+def test_python_examples(path, tmp_path, monkeypatch):
+    """Run a page's examples in order, sharing the page's Python namespace."""
+    monkeypatch.chdir(tmp_path)
+    namespace = {"__name__": "__docs_example__"}
+
+    for index, match in enumerate(PYTHON_BLOCK.finditer(path.read_text()), start=1):
+        code = compile(match.group(1), f"{path} example {index}", "exec")
+        exec(code, namespace)  # noqa: S102 - executing documentation is the test

docs/concepts/architecture.md

@@ -0,0 +1,57 @@
+# Architecture
+
+`opendalfs` connects two existing abstractions. OpenDAL supplies storage
+services and operations. fsspec supplies the filesystem interface used by
+Python data libraries.
+
+## The request path
+
+When pandas opens an `opendal+s3` URL, the request passes through these layers:
+
+```text
+pandas
+  -> fsspec URL resolution
+  -> opendalfs filesystem method
+  -> OpenDAL Python operator
+  -> storage service
+```
+
+The reverse path returns bytes, metadata, directory entries, or a file-like
+object in the shape expected by fsspec.
+
+## One filesystem, one service configuration
+
+An {class}`opendalfs.OpendalFileSystem` owns one OpenDAL operator. The operator
+is configured for one service and one root. Create another filesystem for a
+second bucket, container, or root.
+
+```python
+from opendalfs import OpendalFileSystem
+
+incoming = OpendalFileSystem("s3", bucket="incoming", region="us-east-1")
+archive = OpendalFileSystem("s3", bucket="archive", region="us-east-1")
+```
+
+fsspec may cache instances created with identical arguments. This is useful for
+connection reuse, but it does not merge different service configurations.
+
+## What the adapter owns
+
+`opendalfs` is responsible for:
+
+- fsspec method shapes and metadata dictionaries
+- sync wrappers around async storage operations
+- buffered file behavior
+- URL protocol registration
+- translating URL authorities and paths into operator-relative paths
+- adapter settings such as retries and concurrent writes
+
+OpenDAL remains responsible for service clients, credentials, backend-specific
+configuration, and the native capability of each service.
+
+## Why integrations work
+
+Most integrations do not know about OpenDAL. They call fsspec with a URL,
+filesystem, mapping, or file-like object. Compatibility depends on how closely
+the adapter follows those fsspec contracts. The repository therefore tests
+real downstream entry points in addition to its own filesystem methods.

docs/concepts/capabilities.md

@@ -0,0 +1,48 @@
+# Capabilities
+
+Compatibility has more than one layer. A method can exist in fsspec while the
+selected storage service lacks the native operation needed to implement it.
+
+```text
+OpenDAL service capability
+  -> OpenDAL Python binding
+  -> opendalfs method
+  -> downstream library usage
+```
+
+## fsspec methods
+
+fsspec defines common filesystem methods such as `open`, `ls`, `find`, `copy`,
+and `rm`. It also implements composite methods in terms of smaller operations.
+For example, a recursive operation may combine listing, directory creation,
+and individual file transfers.
+
+`opendalfs` implements the primitive operations needed by those fsspec flows
+and runs the upstream abstract filesystem contract tests against memory and S3.
+
+## Service operations
+
+OpenDAL services do not all support the same operations. Read, write, list,
+copy, rename, multipart upload, and metadata behavior can differ. OpenDAL
+reports unsupported native operations as errors.
+
+Consult the [OpenDAL service directory](https://opendal.apache.org/services/)
+when choosing a backend. Test optional operations against the configured
+service before using them in a production workflow.
+
+## Integration evidence
+
+A passing pandas test proves that the tested pandas entry point works with the
+locked dependency version and the backends covered by CI. It does not prove
+every pandas storage workflow on every OpenDAL service.
+
+The {doc}`../integrations/index` labels this evidence as "CI-tested" and names
+the entry form used by each library. This keeps the claim tied to a concrete
+test instead of treating compatibility as all or nothing.
+
+## Where to report a mismatch
+
+If a valid fsspec usage fails before reaching an unsupported OpenDAL operation,
+report it to `opendalfs`. Include the service, URL or filesystem construction,
+operation, and a small reproducer. Backend-specific failures may also require
+an OpenDAL issue.

docs/concepts/index.md

@@ -0,0 +1,10 @@
+# Concepts
+
+These pages explain where `opendalfs` sits between OpenDAL and fsspec, and why
+compatibility depends on more than the presence of a filesystem method.
+
+```{toctree}
+
+architecture
+capabilities
+```

docs/conf.py

@@ -0,0 +1,51 @@
+from importlib.metadata import version as package_version
+
+project = "opendalfs"
+author = "opendalfs contributors"
+copyright = "2026, opendalfs contributors"  # noqa: A001
+
+release = package_version("opendalfs")
+version = release
+
+extensions = [
+    "myst_parser",
+    "numpydoc",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.autosummary",
+    "sphinx.ext.intersphinx",
+    "sphinx.ext.viewcode",
+    "sphinx_copybutton",
+    "sphinx_design",
+]
+
+autosummary_generate = True
+autodoc_typehints = "description"
+numpydoc_show_class_members = False
+myst_heading_anchors = 3
+myst_enable_extensions = ["colon_fence"]
+
+exclude_patterns = ["_build"]
+
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/3", None),
+    "fsspec": ("https://filesystem-spec.readthedocs.io/en/latest/", None),
+    "opendal": ("https://opendal.apache.org/docs/python/", None),
+    "pandas": ("https://pandas.pydata.org/docs/", None),
+    "dask": ("https://docs.dask.org/en/stable/", None),
+    "xarray": ("https://docs.xarray.dev/en/stable/", None),
+}
+
+html_theme = "pydata_sphinx_theme"
+html_title = "opendalfs"
+html_theme_options = {
+    "github_url": "https://github.com/fsspec/opendalfs",
+    "navigation_with_keys": True,
+    "show_toc_level": 2,
+    "use_edit_page_button": True,
+}
+html_context = {
+    "github_user": "fsspec",
+    "github_repo": "opendalfs",
+    "github_version": "main",
+    "doc_path": "docs",
+}