Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,15 +49,33 @@ jobs:
sleep 1 # Wait for Memgraph a bit.
- name: Install Neo4j
run: |
docker run -p 7474:7474 -p 7688:7687 -d -v $HOME/neo4j/data:/data -v $HOME/neo4j/logs:/logs -v $HOME/neo4j/import:/var/lib/neo4j/import -v $HOME/neo4j/plugins:/plugins --env NEO4J_AUTH=neo4j/test neo4j:4.4.7
# Using Neo4j 4.4.18 which has cgroupv2 compatibility fixes for Ubuntu 24.04
docker run --name neo4j-test -p 7474:7474 -p 7688:7687 -d -v $HOME/neo4j/data:/data -v $HOME/neo4j/logs:/logs -v $HOME/neo4j/import:/var/lib/neo4j/import -v $HOME/neo4j/plugins:/plugins --env NEO4J_AUTH=neo4j/test neo4j:4.4.18
- name: Wait for Neo4j to be ready
run: |
echo "Waiting for Neo4j to be ready..."
for i in {1..60}; do
# Use cypher-shell inside the container to verify Neo4j is accepting connections
if docker exec neo4j-test cypher-shell -u neo4j -p test "RETURN 1" 2>/dev/null; then
echo "Neo4j is ready and accepting queries!"
break
fi
echo "Waiting for Neo4j... ($i/60)"
sleep 2
done
# Show Neo4j logs for debugging
echo "=== Neo4j logs ==="
docker logs neo4j-test 2>&1 | tail -30 || true
- name: Install More Packages
run: |
ps aux | grep memgraph
poetry install --all-extras
poe install-pyg-cpu
poe install-dgl
poe install-tfgnn
- name: Run Tests
run: |
export TF_USE_LEGACY_KERAS="1"
poetry run pytest -vvv -m "not slow and not ubuntu and not docker"
- name: Use the Upload Artifact GitHub Action
uses: actions/upload-artifact@v4
Expand Down Expand Up @@ -132,6 +150,8 @@ jobs:
run: |
poetry install --all-extras
poe install-pyg-cpu
poe install-tfgnn
export TF_USE_LEGACY_KERAS="1"
poetry run pytest -vvv -m "not slow and not ubuntu and not docker"
- name: Save Memgraph Logs
uses: actions/upload-artifact@v4
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ poetry install # No extras
poetry install -E arrow # Support for the CSV, Parquet, ORC and IPC/Feather/Arrow formats
poetry install -E dgl # DGL support (also includes torch)
poetry install -E docker # Docker support
poetry install -E tfgnn # TFGNN support
```

To run the tests, make sure you have an [active Memgraph instance](https://memgraph.com/docs/getting-started), and execute one of the following commands:
Expand All @@ -100,6 +101,7 @@ If you’ve installed only certain extras, it’s also possible to run their ass
poetry run pytest . -k "arrow"
poetry run pytest . -k "dgl"
poetry run pytest . -k "docker"
poetry run pytest . -k "tfgnn"
```

## Development (how to build)
Expand Down
77 changes: 71 additions & 6 deletions docs/how-to-guides/translators/export-python-graphs.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How to export data from Memgraph into Python graphs

GQLAlchemy holds translators that can export Memgraph graphs into Python graphs ([NetworkX](https://networkx.org/), [PyG](https://pytorch-geometric.readthedocs.io/en/latest/) or [DGL](https://www.dgl.ai/) graphs). These translators create a Python graph instance from the graph stored in Memgraph.
GQLAlchemy holds translators that can export Memgraph graphs into Python graphs ([NetworkX](https://networkx.org/), [PyG](https://pytorch-geometric.readthedocs.io/en/latest/), [DGL](https://www.dgl.ai/) or [TF-GNN](https://github.com/tensorflow/gnn) graphs). These translators create a Python graph instance from the graph stored in Memgraph.

[![docs-source](https://img.shields.io/badge/source-examples-FB6E00?logo=github&style=for-the-badge)](https://github.com/memgraph/gqlalchemy/tree/main/tests/transformations/translators)
[![docs-source](https://img.shields.io/badge/source-translators-FB6E00?logo=github&style=for-the-badge)](https://github.com/memgraph/gqlalchemy/tree/main/gqlalchemy/transformations/translators)
Expand All @@ -9,17 +9,37 @@ hood](https://img.shields.io/static/v1?label=Related&message=Under%20the%20hood&

In this guide you will learn how to:

- [**Export data from Memgraph into NetworkX graph**](#export-data-from-memgraph-into-networkx-graph)
- [**Export data from Memgraph into PyG graph**](#import-pyg-graph-into-memgraph)
- [**Export data from Memgraph into DGL graph**](#import-dgl-graph-into-memgraph)
- [How to export data from Memgraph into Python graphs](#how-to-export-data-from-memgraph-into-python-graphs)
- [General prerequisites](#general-prerequisites)
- [Export data from Memgraph into NetworkX graph](#export-data-from-memgraph-into-networkx-graph)
- [Prerequisites](#prerequisites)
- [Create and run a Python script](#create-and-run-a-python-script)
- [Export data from Memgraph into PyG graph](#export-data-from-memgraph-into-pyg-graph)
- [Prerequisites](#prerequisites-1)
- [Create and run a Python script](#create-and-run-a-python-script-1)
- [Export data from Memgraph into DGL graph](#export-data-from-memgraph-into-dgl-graph)
- [Prerequisites](#prerequisites-2)
- [Create and run a Python script](#create-and-run-a-python-script-2)
- [Export data from Memgraph into TF-GNN graph](#export-data-from-memgraph-into-tf-gnn-graph)
- [Prerequisites](#prerequisites-3)
- [Create and run a Python script](#create-and-run-a-python-script-3)
- [Learn more](#learn-more)

## General prerequisites
You need a running **Memgraph Platform instance**, which includes both the MAGE library and Memgraph Lab, a visual interface. To run the image, open a command-line interpreter and run the following Docker command:
You need **Memgraph Platform** running, which includes both the MAGE library and Memgraph Lab, a visual interface. To run it on Linux/macOS, run the following in your terminal:

```
docker run -it -p 7687:7687 -p 7444:7444 -p 3000:3000 memgraph/memgraph-platform:latest
curl https://install.memgraph.com | sh
```

To run it on Windows, execute the following command in the console:

```
iwr https://windows.memgraph.com | iex
```

The above command runs a script that downloads a Docker Compose file to your system, builds and starts `memgraph-mage` and `memgraph-lab` Docker services in two separate containers.

<details>
<summary>To export data from Memgraph, you first have to <b>create a graph in Memgraph</b>. To do that, expand this section and run the given Python script.</summary>

Expand Down Expand Up @@ -188,6 +208,51 @@ You will get the following output:

This means that the DGL graph has the correct number of node and edge types, total number of nodes and edges, as well as node and edge features. You can explore it more to see if it has all the required features.

## Export data from Memgraph into TF-GNN graph

### Prerequisites

Except for the [**general prerequisites**](#general-prerequisites), you also need to install [**TensorFlow GNN**](https://github.com/tensorflow/gnn). You can install it with:

```bash
pip install tensorflow-gnn
```

Note: TF-GNN requires TensorFlow 2.x. For TensorFlow 2.16 and above, you may need to set `TF_USE_LEGACY_KERAS=1` environment variable.

### Create and run a Python script

Create a new Python script `memgraph-to-tfgnn.py`, in the code editor of your choice, with the following code:

```python
import os
os.environ["TF_USE_LEGACY_KERAS"] = "1" # Required for TensorFlow 2.16+

from gqlalchemy.transformations.translators.tfgnn_translator import TFGNNTranslator

translator = TFGNNTranslator()
graph_tensor = translator.get_instance()

print("Node sets:", list(graph_tensor.node_sets.keys()))
print("Edge sets:", list(graph_tensor.edge_sets.keys()))

for node_set_name, node_set in graph_tensor.node_sets.items():
print(f"Node set '{node_set_name}': {node_set.sizes[0]} nodes")
print(f" Features: {list(node_set.features.keys())}")

for edge_set_name, edge_set in graph_tensor.edge_sets.items():
print(f"Edge set '{edge_set_name}': {edge_set.sizes[0]} edges")
print(f" Features: {list(edge_set.features.keys())}")
```

To run it, open a command-line interpreter and run the following command:

```python
python3 memgraph-to-tfgnn.py
```

You will get output showing the node sets, edge sets, and their features. The TF-GNN GraphTensor can now be used directly with TensorFlow GNN models for graph neural network training.

## Learn more

Head over to the [**Under the hood**](../../under-the-hood/python-graph-translators.md) section to read about implementation details. If you want to learn more about using NetworkX with Memgraph with interesting resources and courses, head over to the [**Memgraph for NetworkX developers**](https://memgraph.com/memgraph-for-networkx?utm_source=docs&utm_medium=referral&utm_campaign=networkx_ppp&utm_term=docsgqla%2Bhowto&utm_content=textlink) website. If you have any questions or want to connect with the Memgraph community, [**join our Discord server**](https://www.discord.gg/memgraph).
102 changes: 99 additions & 3 deletions docs/how-to-guides/translators/import-python-graphs.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How to import Python graphs into Memgraph

GQLAlchemy holds translators that can import Python graphs ([NetworkX](https://networkx.org/), [PyG](https://pytorch-geometric.readthedocs.io/en/latest/) or [DGL](https://www.dgl.ai/) graphs) into Memgraph. These translators take the Python graph object and translate it to the appropriate Cypher queries. The Cypher queries are then executed to create a graph inside Memgraph.
GQLAlchemy holds translators that can import Python graphs ([NetworkX](https://networkx.org/), [PyG](https://pytorch-geometric.readthedocs.io/en/latest/), [DGL](https://www.dgl.ai/) or [TF-GNN](https://github.com/tensorflow/gnn) graphs) into Memgraph. These translators take the Python graph object and translate it to the appropriate Cypher queries. The Cypher queries are then executed to create a graph inside Memgraph.

[![docs-source](https://img.shields.io/badge/source-examples-FB6E00?logo=github&style=for-the-badge)](https://github.com/memgraph/gqlalchemy/tree/main/tests/transformations/translators)
[![docs-source](https://img.shields.io/badge/source-translators-FB6E00?logo=github&style=for-the-badge)](https://github.com/memgraph/gqlalchemy/tree/main/gqlalchemy/transformations/translators)
Expand All @@ -12,14 +12,23 @@ In this guide you will learn how to:
- [**Import NetworkX graph into Memgraph**](#import-networkx-graph-into-memgraph)
- [**Import PyG graph into Memgraph**](#import-pyg-graph-into-memgraph)
- [**Import DGL graph into Memgraph**](#import-dgl-graph-into-memgraph)
- [**Import TF-GNN graph into Memgraph**](#import-tf-gnn-graph-into-memgraph)

## General prerequisites
You need a running **Memgraph Platform instance**, which includes both the MAGE library and Memgraph Lab, a visual interface. To run the image, open a command-line interpreter and run the following Docker command:
You need **Memgraph Platform** running, which includes both the MAGE library and Memgraph Lab, a visual interface. To run it on Linux/macOS, run the following in your terminal:

```
docker run -it -p 7687:7687 -p 7444:7444 -p 3000:3000 memgraph/memgraph-platform:latest
curl https://install.memgraph.com | sh
```

To run it on Windows, execute the following command in the console:

```
iwr https://windows.memgraph.com | iex
```

The above command runs a script that downloads a Docker Compose file to your system, builds and starts `memgraph-mage` and `memgraph-lab` Docker services in two separate containers.

## Import NetworkX graph into Memgraph

### Prerequisites
Expand Down Expand Up @@ -207,6 +216,93 @@ Click **Run Query** button to see the results.

You can notice that we have nodes labeled with `user` and `movie` and relationships of type `PLUS` and `MINUS`. Besides that, nodes and relationships have randomized array properties ad well as `dgl_id` property.

## Import TF-GNN graph into Memgraph

### Prerequisites

Except for the [**general prerequisites**](#general-prerequisites), you also need to install [**TensorFlow GNN**](https://github.com/tensorflow/gnn). You can install it with:

```bash
pip install tensorflow-gnn
```

Note: TF-GNN requires TensorFlow 2.x. For TensorFlow 2.16 and above, you may need to set `TF_USE_LEGACY_KERAS=1` environment variable.

### Create and run a Python script

Create a new Python script `tfgnn-graph.py` in the code editor of your choice, with the following code:

```python
import os
os.environ["TF_USE_LEGACY_KERAS"] = "1" # Required for TensorFlow 2.16+

import tensorflow as tf
import tensorflow_gnn as tfgnn
from gqlalchemy import Memgraph
from gqlalchemy.transformations.translators.tfgnn_translator import TFGNNTranslator

memgraph = Memgraph()
memgraph.drop_database()

# Create a TF-GNN GraphTensor
graph_tensor = tfgnn.GraphTensor.from_pieces(
node_sets={
"user": tfgnn.NodeSet.from_fields(
sizes=[3],
features={
"name": tf.constant(["Alice", "Bob", "Charlie"]),
"age": tf.constant([25, 30, 35], dtype=tf.int64),
}
),
"movie": tfgnn.NodeSet.from_fields(
sizes=[2],
features={
"title": tf.constant(["Inception", "Matrix"]),
"rating": tf.constant([8.8, 8.7], dtype=tf.float32),
}
),
},
edge_sets={
"LIKES": tfgnn.EdgeSet.from_fields(
sizes=[3],
adjacency=tfgnn.Adjacency.from_indices(
source=("user", tf.constant([0, 0, 1])),
target=("movie", tf.constant([0, 1, 0])),
),
features={
"score": tf.constant([5, 4, 5], dtype=tf.int64),
}
),
},
)

translator = TFGNNTranslator()

for query in translator.to_cypher_queries(graph_tensor):
memgraph.execute(query)
```

First, connect to a running Memgraph instance. Next, drop the database to be sure that it's empty. After that, create a TF-GNN GraphTensor with `user` and `movie` node sets, and a `LIKES` edge set connecting users to movies. The GraphTensor includes node features (`name`, `age` for users; `title`, `rating` for movies) and edge features (`score`). In the end, call `to_cypher_queries` on `TFGNNTranslator` instance to transform the TF-GNN graph to Cypher queries which will be executed in Memgraph.

To run it, open a command-line interpreter and run the following command:

```python
python3 tfgnn-graph.py
```

### Explore the graph

[Connect to Memgraph](htps://memgraph.com/docs/data-visualization/install-and-connect) via Memgraph Lab which is running at `localhost:3000`. Open the **Query Execution** section and write the following query:

```cypher
MATCH (n)-[r]->(m)
RETURN n, r, m;
```

Click **Run Query** button to see the results.

You can notice that we have nodes labeled with `user` and `movie` and relationships of type `LIKES`. Besides that, nodes and relationships have their respective properties as well as `tfgnn_id` property which maps to the node/edge index in the original TF-GNN GraphTensor.

## Learn more

Head over to the [**Under the hood**](../../under-the-hood/python-graph-translators.md) section to read about implementation details. If you want to learn more about using NetworkX with Memgraph with interesting resources and courses, head over to the [**Memgraph for NetworkX developers**](https://memgraph.com/memgraph-for-networkx?utm_source=docs&utm_medium=referral&utm_campaign=networkx_ppp&utm_term=docsgqla%2Bhowto&utm_content=textlink) website. If you have any questions or want to connect with the Memgraph community, [**join our Discord server**](https://www.discord.gg/memgraph).
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
sidebar_label: tfgnn_translator
title: gqlalchemy.transformations.translators.tfgnn_translator
---

## TFGNNTranslator Objects

```python
class TFGNNTranslator(Translator)
```

Translator for converting between Memgraph and TensorFlow GNN (TF-GNN) graph representations.

TF-GNN represents graphs as GraphTensor objects, which are composite tensors that can be used directly in TensorFlow operations. This translator handles the conversion between Memgraph's graph model and TF-GNN's GraphTensor.

TF-GNN is particularly useful for building Graph Neural Networks (GNNs) using TensorFlow and Keras. It supports both homogeneous and heterogeneous graphs with node and edge features.

#### to\_cypher\_queries

```python
def to_cypher_queries(graph_tensor: tfgnn.GraphTensor) -> List[str]
```

Produce Cypher queries for data saved as part of the TF-GNN GraphTensor. The method handles both homogeneous and heterogeneous graphs. If the graph is homogeneous, default labels will be used (`_N` as a node label and `_E` as edge label). The method converts 1D as well as multidimensional features. If there are some isolated nodes inside the GraphTensor, they won't get transferred. Nodes and edges created in Memgraph DB will, for consistency reasons, have property `tfgnn_id` set to the id they have as part of the TF-GNN graph. Note that this method doesn't insert anything inside the database, it just creates Cypher queries. To insert queries the following code can be used:

```python
from gqlalchemy import Memgraph
from gqlalchemy.transformations.translators.tfgnn_translator import TFGNNTranslator

memgraph = Memgraph()
graph_tensor = ... # Your TF-GNN GraphTensor
for query in TFGNNTranslator().to_cypher_queries(graph_tensor):
memgraph.execute(query)
```

**Arguments**:

- `graph_tensor` - A reference to the TF-GNN GraphTensor.

**Returns**:

List of Cypher queries.

#### get\_instance

```python
def get_instance() -> tfgnn.GraphTensor
```

Create an instance of TF-GNN GraphTensor from all nodes and edges that are inside Memgraph.

The translator converts Memgraph node labels to TF-GNN node sets and edge types to edge sets. Node and edge properties are converted to TF-GNN features with the following rules:

- String, Integer, Float, Boolean properties are converted to corresponding TensorFlow dtypes
- List properties are converted to tensors (dense if same length, ragged if different lengths)
- Missing properties result in ragged tensors with empty values
- Map properties are not supported and will raise an error
- Memgraph-specific types (temporal types, Enum, Point) are converted to string representation

**Returns**:

A TF-GNN GraphTensor instance representing the graph stored in Memgraph.
1 change: 1 addition & 0 deletions gqlalchemy/transformations/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@
NUM_NODES = "num_nodes"
DEFAULT_NODE_LABEL = "NODE"
DEFAULT_EDGE_TYPE = "RELATIONSHIP"
TFGNN_ID = "tfgnn_id"
12 changes: 10 additions & 2 deletions gqlalchemy/transformations/export/graph_transporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@
except ModuleNotFoundError:
PyGTranslator = None

try:
from gqlalchemy.transformations.translators.tfgnn_translator import TFGNNTranslator
except ModuleNotFoundError:
TFGNNTranslator = None


class GraphTransporter(Transporter):
"""Here is a possible example for using this module:
Expand All @@ -52,7 +57,7 @@ def __init__(
>>> transporter = GraphTransporter("dgl")
graph = transporter.export()
Args:
graph_type: dgl, pyg or nx
graph_type: dgl, pyg, nx or tfgnn
"""
super().__init__()
self.graph_type = graph_type.upper()
Expand All @@ -64,8 +69,11 @@ def __init__(
self.translator = PyGTranslator(host, port, username, password, encrypted, client_name, lazy)
elif self.graph_type == GraphType.NX.name:
self.translator = NxTranslator(host, port, username, password, encrypted, client_name, lazy)
elif self.graph_type == GraphType.TFGNN.name:
raise_if_not_imported(dependency=TFGNNTranslator, dependency_name="tensorflow-gnn")
self.translator = TFGNNTranslator(host, port, username, password, encrypted, client_name, lazy)
else:
raise ValueError("Unknown export option. Currently supported are DGL, PyG and NetworkX.")
raise ValueError("Unknown export option. Currently supported are DGL, PyG, NetworkX and TFGNN.")

def export(self):
"""Creates graph instance for the wanted export option."""
Expand Down
2 changes: 1 addition & 1 deletion gqlalchemy/transformations/graph_type.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
from enum import Enum

GraphType = Enum("GraphType", ["DGL", "PYG", "NX"])
GraphType = Enum("GraphType", ["DGL", "PYG", "NX", "TFGNN"])
Loading