NebulaGraph AI Suite with 4 line code to run Graph Algo on NebulaGraph
Documentation: https://github.com/wey-gu/nebulagraph-ai#documentation
Source Code: https://github.com/wey-gu/nebulagraph-ai
NebulaGraph AI Suite for Python (ng_ai) is a powerful Python library that offers APIs for data scientists to effectively read, write, analyze, and compute data in NebulaGraph.
With the support of a single-machine engine(NetworkX), or distributed computing environment using Spark, we could perform Graph Analysis and Algorithms on top of NebulaGraph in less than 10 lines of code, in unified and intuitive API.
Option 1: Try on your Desktop. Go with NebulaGraph Docker Extension!
Option 2: On Linux Server? Go with Nebula-Up ππ»
- Set up env with Nebula-Up following this guide.
- Install ng_ai with pip from the Jupyter Notebook with http://localhost:8888 (password:
nebula
). - Open the demo notebook and run cells one by one.
- Check the API Reference
pip install ng_ai
See more details in the docs
RETURN ng_ai("pagerank", ["follow"], ["degree"], "spark",
{space: "basketballplayer", max_iter: 10}, {write_mode: "insert"})
See also: examples/spark_engine.ipynb
Run Algorithm on top of NebulaGraph:
Note, there is also a query mode, refer to examples or docs for more details.
from ng_ai import NebulaReader
# read data with spark engine, scan mode
reader = NebulaReader(engine="spark")
reader.scan(edge="follow", props="degree")
df = reader.read()
# run pagerank algorithm
pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10)
Write back to NebulaGraph:
from ng_ai import NebulaWriter
from ng_ai.config import NebulaGraphConfig
config = NebulaGraphConfig()
properties = {"louvain": "cluster_id"}
writer = NebulaWriter(
data=df_result, sink="nebulagraph_vertex", config=config, engine="spark")
writer.set_options(
tag="louvain", vid_field="_id", properties=properties,
batch_size=256, write_mode="insert",)
writer.write()
Then we could query the result in NebulaGraph:
MATCH (v:louvain)
RETURN id(v), v.louvain.cluster_id LIMIT 10;
Basically the same as Spark Engine, but with engine="nebula"
, refer to examples or docs for more details.
- reader = NebulaReader(engine="spark")
+ reader = NebulaReader(engine="nebula")
ng_ai is a unified abstraction layer for different engines, the current implementation is based on Spark, NetworkX, DGL, and NebulaGraph, but it's easy to extend to other engines like Flink, GraphScope, PyG, etc.
βββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Spark Cluster β
β .βββββ. .βββββ. .βββββ. .βββββ. β
β ; : ; : ; : ; : β
βββΆβ : ; : ; : ; : ; β
β β β² β± β² β± β² β± β² β± β
β β `βββ' `βββ' `βββ' `βββ' β
Algo Spark β
Engineβββββββββββββββββββββββββββββββββββββββββββββββββββββ
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββ¬βββββββββββ
ββββ€ β β
β NebulaGraph AI Suite(ngai) β ngai-api ββββ
β β β β
β ββββββββββββ€ β
β ββββββββββ ββββββββ ββββββββββ βββββββ β β
β β Reader β β Algo β β Writer β β GNN β β β
βββββββββΆβ ββββββββββ ββββββββ ββββββββββ βββββββ β β
β β β β β β β β
β β ββββββββββββββ΄ββββ¬βββββββββ΄ββββββ ββββββββ β β
β β βΌ βΌ βΌ βΌ β β
β β βββββββββββββββ ββββββββββββββββ ββββββββββββββββββββββββ β β
β ββββ€ β SparkEngine β β NebulaEngine β β NetworkX ββ DGLEngineβ β β
β β β βββββββββββββββ ββββββββββββββββ ββββββββββββββββββββββββ β β
β β ββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β Spark β
β β βββββββββReader βββββββββββββ β
β Spark Query Mode β β
β Reader β β
βScan Mode βΌ βββββββββββ
β β βββββββββββββββββββββββββββββββββββββββββββββββββββββ¬ββββββββββ€ ngai-udfββββββββββββββββ
β β β β βββββββββββ€ β
β β β NebulaGraph Graph Engine Nebula-GraphD β ngai-GraphD β β
β β ββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββΌββββββββββββββββββββ β
β β β β β β
β β β NebulaGraph Storage Engine β β β
β β β β β β
β βββΆβ Nebula-StorageD β Nebula-Metad β β
β β β β β
β ββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββ β
β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β RETURN ng_ai("pagerank", ["follow"], ["degree"], "spark", {space:"basketballplayer"}) ββββ
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β β from ng_ai import NebulaReader β
β β β
β β # read data with spark engine, scan mode β
β β reader = NebulaReader(engine="spark") β
β β reader.scan(edge="follow", props="degree") β
ββββ df = reader.read() β
β β
β # run pagerank algorithm β
β pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
- Spark 2.4, 3.0(not yet tested)
- NebulaGraph 3.4+
- NebulaGraph Spark Connector 3.4+
- NebulaGraph Algorithm 3.1+
See HACKING.md for details.
This project is licensed under the terms of the Apache License 2.0.