Skip to content

Commit 69ac46c

Browse files
nirajkarknirajkark
committed
Add aws-open-data package for AWS Open Data Registry submission.Includes the dataset README, registry YAML, tutorial notebook, an
the heritagegraph ontology/KG snapshot intended for S3 publication.
1 parent 2c0af51 commit 69ac46c

17 files changed

Lines changed: 190142 additions & 0 deletions

‎aws-open-data/README.md‎

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
# Using HeritageGraph — AWS Open Data Tutorial
2+
3+
HeritageGraph is an RDF knowledge graph of Nepal's **tangible and intangible
4+
cultural heritage**: temples, stupas, monasteries, stone water spouts (*dhunge
5+
dhara*), UNESCO World Heritage sites, festivals, Guthi institutions, ritual
6+
roles, deities, and more. Every fact carries machine-readable **provenance**
7+
(source, confidence, epistemic stance) and links back to Wikidata or
8+
OpenStreetMap where available.
9+
10+
Once published, the dataset will appear on the
11+
[Registry of Open Data on AWS](https://registry.opendata.aws/heritagegraph).
12+
Full ontology documentation lives at
13+
[https://cairnepal.github.io/heritagegraphontology/](https://cairnepal.github.io/heritagegraphontology/).
14+
Source code and releases are in the
15+
[CAIRNepal/heritagegraphontology](https://github.com/CAIRNepal/heritagegraphontology)
16+
repository on GitHub.
17+
18+
---
19+
20+
## Start here: guided notebook
21+
22+
The fastest way to explore the data is the
23+
[**Get to Know the HeritageGraph KG**](get-to-know-the-heritagegraph-kg.ipynb)
24+
notebook (AWS Open Data template). It walks through bucket layout, loading Turtle
25+
files, entity statistics, a map of geolocated heritage, and provenance-aware
26+
SPARQL queries.
27+
28+
**Run locally** (uses the bundled `heritagegraph/` data — no S3 bucket required):
29+
30+
```bash
31+
git clone https://github.com/CAIRNepal/heritagegraphontology.git
32+
cd heritagegraphontology/aws-open-data
33+
pip install rdflib matplotlib boto3 jupyter
34+
jupyter lab get-to-know-the-heritagegraph-kg.ipynb
35+
```
36+
37+
**Run against the public S3 bucket** (once provisioned): open the notebook,
38+
set `BUCKET` to the registry bucket name in the second code cell, and run all
39+
cells. Public buckets use unsigned S3 requests (no AWS account needed to read).
40+
41+
---
42+
43+
## Access the data
44+
45+
### From Amazon S3
46+
47+
After publication, files are under the `heritagegraph/` prefix, for example:
48+
49+
```
50+
s3://<bucket>/heritagegraph/
51+
├── LICENSE.txt
52+
├── README.md
53+
├── ontology/ HeritageGraph OWL, SHACL shapes, LinkML source
54+
├── kg/ RDF Turtle, one file per source / license
55+
└── examples/ SPARQL competency questions and sample results
56+
```
57+
58+
Download a single file with the AWS CLI:
59+
60+
```bash
61+
aws s3 cp s3://<bucket>/heritagegraph/kg/wikidata.ttl ./wikidata.ttl --no-sign-request
62+
```
63+
64+
Sync the full dataset:
65+
66+
```bash
67+
aws s3 sync s3://<bucket>/heritagegraph/ ./heritagegraph/ --no-sign-request
68+
```
69+
70+
### From this repository
71+
72+
Clone the repo and use the copy under `aws-open-data/heritagegraph/` — it matches
73+
the S3 layout exactly.
74+
75+
---
76+
77+
## What's in `heritagegraph/kg/`
78+
79+
Each file is a **named graph** governed by its source license. You can load
80+
them individually or in combination.
81+
82+
| File | License | Contents |
83+
|------|---------|----------|
84+
| `wikidata.ttl` | [CC0](https://creativecommons.org/publicdomain/zero/1.0/) | ~253 richly-described entities from Wikidata |
85+
| `osm.ttl` | [ODbL](https://opendatacommons.org/licenses/odbl/1-0/) | ~7,589 geolocated physical features (© OpenStreetMap contributors) |
86+
| `unesco.ttl` | Attribution | 8 UNESCO World Heritage cultural components |
87+
| `intangible.ttl` | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Curated festivals, Guthi, Kumari, deities, castes, rituals |
88+
| `crosswalk.ttl` | Mixed | `owl:sameAs` and provenance links (tangible layer) |
89+
| `intangible_crosswalk.ttl` | Mixed | `owl:sameAs` / typing links (intangible layer) |
90+
91+
**Tip:** To avoid ODbL share-alike obligations, use only `wikidata.ttl` +
92+
`unesco.ttl` (and optionally `intangible.ttl`). See `heritagegraph/LICENSE.txt`
93+
for full terms.
94+
95+
**Scale:** ~7,850 tangible entities + a curated intangible layer (~138k triples
96+
in total). See `heritagegraph/examples/build_stats.json` for per-class counts.
97+
98+
---
99+
100+
## Quick start: Python + rdflib
101+
102+
```python
103+
from rdflib import Graph
104+
105+
g = Graph()
106+
for name in [
107+
"wikidata", "osm", "unesco", "intangible",
108+
"crosswalk", "intangible_crosswalk",
109+
]:
110+
g.parse(f"heritagegraph/kg/{name}.ttl", format="turtle")
111+
112+
print(f"{len(g):,} triples loaded")
113+
```
114+
115+
### Example SPARQL — geolocated temples and stupas
116+
117+
```python
118+
q = """
119+
PREFIX hg: <https://cair-nepal.org/heritagegraph/>
120+
PREFIX crm: <http://www.cidoc-crm.org/cidoc-crm/>
121+
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
122+
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
123+
SELECT ?label ?wkt WHERE {
124+
?s a ?cls ; rdfs:label ?label ; crm:P55_has_current_location ?p .
125+
?p geo:asWKT ?wkt .
126+
FILTER(?cls IN (hg:ArchitecturalStructure, hg:Stupa, hg:BuddhistMonument))
127+
} LIMIT 10
128+
"""
129+
for row in g.query(q):
130+
print(row.label, "→", row.wkt)
131+
```
132+
133+
More queries: `heritagegraph/examples/competency-questions.md` (32 competency
134+
questions with sample results in `competency-questions-data-results.txt`).
135+
136+
---
137+
138+
## Quick start: Amazon Neptune
139+
140+
For production SPARQL at scale, load the Turtle files into
141+
[Amazon Neptune](https://aws.amazon.com/neptune/):
142+
143+
1. Upload `heritagegraph/kg/*.ttl` to an S3 bucket Neptune can read.
144+
2. Use bulk load (`neptune-loader`) or the Neptune Workbench to import.
145+
3. Query with SPARQL 1.1 — the graph uses standard prefixes (`crm:`, `geo:`,
146+
`prov:`, `hg:`).
147+
148+
The ontology and SHACL shapes in `heritagegraph/ontology/` define the schema and
149+
validation rules for any data you add.
150+
151+
Alternatives: [Apache Jena Fuseki](https://jena.apache.org/),
152+
[GraphDB](https://graphdb.ontotext.com/), or any RDF 1.1 triplestore.
153+
154+
---
155+
156+
## Namespaces
157+
158+
| Purpose | IRI |
159+
|---------|-----|
160+
| Classes & properties | `https://cair-nepal.org/heritagegraph/` |
161+
| Instance identifiers | `https://data.cair-nepal.org/heritagegraph/` |
162+
| Source vocabulary | `https://data.cair-nepal.org/heritagegraph/source/` |
163+
164+
---
165+
166+
## Licensing
167+
168+
This dataset combines multiple open sources under **different licenses**, kept
169+
in separate files so you can choose what to include:
170+
171+
- **CC0** — `wikidata.ttl` (public domain)
172+
- **ODbL** — `osm.ttl` (share-alike; attribute OpenStreetMap)
173+
- **CC BY 4.0** — `intangible.ttl` and the ontology schema
174+
- **Attribution** — `unesco.ttl` (UNESCO World Heritage Centre)
175+
176+
Any combined database that includes the OSM layer must comply with ODbL. Full
177+
terms: [`heritagegraph/LICENSE.txt`](heritagegraph/LICENSE.txt).
178+
179+
---
180+
181+
## Learn more
182+
183+
| Resource | Link |
184+
|----------|------|
185+
| Registry entry | [registry.opendata.aws/heritagegraph](https://registry.opendata.aws/heritagegraph) |
186+
| Ontology docs | [cairnepal.github.io/heritagegraphontology](https://cairnepal.github.io/heritagegraphontology/) |
187+
| GitHub repository | [github.com/CAIRNepal/heritagegraphontology](https://github.com/CAIRNepal/heritagegraphontology) |
188+
| Guided notebook | [get-to-know-the-heritagegraph-kg.ipynb](get-to-know-the-heritagegraph-kg.ipynb) |
189+
| Bucket README | [heritagegraph/README.md](heritagegraph/README.md) |
190+
| Contact | info@cair-nepal.org · [CAIR-Nepal](https://cair-nepal.org) |

0 commit comments

Comments
 (0)