Skip to content

Releases: Pometry/Raphtory

v0.14.0

02 Dec 17:33
@github-actions github-actions
v0.14.0
0022974
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.14.0 Latest
Latest

Cached View

We have added a new function .cache_view which builds a lightweight index of the nodes and edges present in the current view (i.e. when you have applied a window/layer filter etc). If you are running any global algorithms or analytical pipelines over views, this will make your analysis drastically faster!
Example:

g = Graph()

#add some updates

for windowed_graph in g.rolling("1 day"):
    cached = windowed_graph.cache_view() #We are gonna run several algorithms, so build an index
    rp.weakly_connected_components(cached) 
    rp.pagerank(cached)

Node and edge filter view

We have added new views for the filtering of Nodes and Edges based upon property values. This includes checking:

  • if a property exists/doesn't exist
  • if the property value is less than/greater than/equal to a give argument
  • if the property value is in/not in a list of given arguments.

Note the edge filters are currently disabled for PersistentGraph whilst we confirm there are no missing corner cases.

Python example:

from raphtory import Graph
g = Graph
# add some updates
graph.filter_edges(Prop("test_int") > 2)
graph.filter_exploded_edges(Prop("test_str") != "first")
graph.filter_nodes(Prop("node_bool").is_some())
graph.filter_nodes(Prop("node_int") in [2,2,4])

Graphql example:

      graph(path: "g") {
        nodes {
          nodeFilter(
            property: "prop1", 
            condition: {
              operator: ANY, 
              value: [10, 30, 50, 70]
            }
          ) {
            list {
              name
            }
          }
        }
      }

Create Node

Added a create_node function which works exactly the same as add_node but will fail if the node is already in the graph. This is mostly useful in Graphql, where it is harder to first check if a node exists, but has been exposed in python as well.

Example:

from raphtory import Graph
g = Graph()
g.create_node(1,1) #Returns fine
g.create_node(1,1) #Throws an exception
g.add_node(1,1) #Returns fine

Import as

Added a set of import_as functions which allow renaming of nodes and edges when importing from one graph into another.
Example:

from raphtory import Graph
g1 = Graph()
a = g1.add_node(1, "A") #create node A in graph1
g2 = Graph()
g2.import_node_as(a, "X") # import A into graph2 as X - this brings all updates and properties as well

e = g1.add_edge(1,"A","B"") # add edge A->B to graph1
g2.import_edge_as(e,("X","Y")) #import edge A->B into graph2 as X->Y - this brings all updates and properties with it

Python

  • When using the Property APIs with any numerical properties Raphtory will now return numpy arrays instead of python lists. This is better for memory usage, faster to hand over from rust, and means aggregations etc are a lot more straight forward.
  • Exposed the secondary time index, allowing mangement of updates which occur at the same time.
  • Changes Graph.add_property to Graph.add_properties to bring it in line with other APIs.
  • Fixed a bug in the repr where we were print the wrong edge info (#1808)
  • Added wrappers for constructing vecs from any python iterable, meaning Nodes and Edges can be handed over to import functions directly without collecting.

Algorithms

  • Added FastRP based on "Fast and Accurate Network Embeddings via Very Sparse Random Projection" by Haochen Chen, et al.
  • Added maximum-weighted matching based on "Efficient Algorithms for Finding Maximum Matching in Graphs" by Zvi Galil, et al.
  • Changed the return of in-component and out-component to include the distance from the starting node.

UI updates

  • We have added a Saved graphs page which enables you to open whole graphs and get some top level statistics on each of the graphs on your server. An example of this can be seen below.
  • A whole heap of small bug fixes! We have noted several more (thank you everyone that is reporting them) and shall be blasting through them over the coming weeks before Christmas).

image

GraphQL

  • Added the edge ID function which returns the names of the source and destination as an array.
  • Added explode and eplode_layers onto the edges object.
  • Added all node property filters to graphql - examples of these can be found here.
  • Added the namespace function onto graph/graphql to allow easier grouping by path.
  • Removes the ability to create RemoteGraph directly, can now only be done through the client

Core-Raphtory

  • Made lazy node state support time ops and layer ops. This allows you to e.g. get a windowed degree for all nodes in the graph. This is a step towards out new NodeState APIs which should be complete soon.
  • Exposed several low level APIs to make writing raphtory extensions easier.
  • Subgraphs creation is now faster as we no longer need to build a hashset. Counting nodes should also be much faster now as well.
  • Made the inner rust value accessible on python NodeState and LazyNodeState wrappers.
  • Exposed parquet_loaders in rust.
  • updated our pyo3 version for python bindings to the new APIs.
  • Removed snmalloc as the build started to fail due to some unknown upstream dependency.

Python Documentation

  • Drastically improved the stub generation for hints within python IDEs
  • Fixed many missing types/doc strings, incorrect/confusing descriptions
  • Added warning for missing docs (still some to fix, but will mean in future we can fix a lot quicker)

Datasets

  • Added some properties to the LOTR data for the basic graphRAG example.

What's Changed

New Contributors

Full Changelog: v0.13.1...v0.14.0

Contributors

fabianmurariu, shivam-880, and 6 other contributors
Assets 2
Loading

v0.13.1

24 Oct 12:06
@github-actions github-actions
v0.13.1
85a9eab
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.13.1

What's Changed

Full Changelog: v0.13.0...v0.13.1

Contributors

fabianmurariu, ricopinazo, and ljeub-pometry
Assets 2
Loading

v0.13.0

15 Oct 10:07
@github-actions github-actions
v0.13.0
0e0008f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.13.0

UI Alpha

  • We have released the first version of the Raphtory UI. This should work for any graph that you host within your GraphServer and is available at / by default. The graphql playground has been moved to /playground.
  • We have many more plans for this UI, but in the meantime if you notice it isn't handling your data correctly, or you find a bug please report and issue and we shall get it fixed.
  • Below is an example of the UI with the Lord of the Rings graph loaded:

Small tweaks

  • The python doc stubs now error when the return type is incorrect - all current errors have been fixed. We will start to enable more warning and tidy these up fully over the coming releases.
  • PyDirection is no more and direction arguments now take strings as input directly (The only way to construct a PyDirection was via passing in a string anyway so this seemed entirely confusing and useless).
  • Added layers to the edge repr to show what layers an edge/exploded edge is present in, e.g.

Bug fixes

  • to_df in AlgorithmResult no longer returns internal ids
  • Graph.edges.explode().to_df() is now equivalent to Graph.edges.to_df(explode=True), in particular the history is no longer duplicated for each exploded edge.
  • The EmbeddingFunction was changed to return a Result to be able to bubble up errors instead of panicking. These changes were propagated all the way up.
  • Path inputs in python now use PathBuf instead of String, removing a host of annoying issues, especially in windows.

What's Changed

Full Changelog: v0.12.1...v0.13.0

Contributors

fabianmurariu, shivam-880, and 3 other contributors
Assets 2
Loading

v0.12.1

03 Oct 08:50
@github-actions github-actions
v0.12.1
41282d0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.12.1

Release v0.12.1

Assets 2
Loading

v0.12.0

02 Oct 14:06
@github-actions github-actions
v0.12.0
446e7f1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.12.0

Obvious breaking changes

In our efforts to better support indexes over properties and vector representations of the graph we have changed the on-disk representation of a Raphtory graph to a folder. Within this folder we can store the graph itself, any vectors, indexes, metadata, etc. required to simplfy the transfer of a graph between your machine and a GraphServer, or between yourself and colleages working on the same data.

As such the function save_to_file() will now produce a folder containing this new structure. If you would like to continue having a singular file (for purposes of transfer or ease) you can instead call save_to_zip(). This zip can be directly be read by Raphtory when you call load_from_file() so don't worry about having to unzip later.

New Vector APIs and integration with the GraphServer

  • We have updated vector query APIs as per #1713. The new vector context makes it much easier to query the nodes/edges/graphs by both similarity and structural elements such as neighbours. We have also tried to make it a lot clearer what each function is bringing into the context.
  • The embedding function is now set globally for the GraphServer and the conversion between graph/nodes/edges -> Document is now specified via jinja templates. This is to make it possible to store a vectorised Graph on disk.
  • Vectors are now updated when a node/edge are updated.

Algorithms

  • In_components and out_components have been optimised to do the minimal number of checks before returning a result.
  • In_component and out_component have been added for when you only want to get the component for an individual node.

Graphql

  • In_component and out_component have been made available on Node within Graphql - this returns a vec of Node objects allowing you to get metadata/properties of the nodes within this component.
  • A generated Schema is now available via Graphql to see what the type of all properties are for both nodes and edges.
  • We have drastically simplified the plugin APIs for the GraphServer, and now allow both custom mutations and queries. An example of this can be seen here: https://github.com/Pometry/Raphtory/tree/master/examples/custom-gql-apis.
  • Added open telemetry tracing to the GraphServer, allowing you to track the speed of all raphtory queries.
  • Added better logging throughout the GraphServer.

Edge filtering Alpha

  • We have released an alpha of edge property filtering - this allows you to filter both whole edges or updates within edges (exploded edges) in a variety of useful ways (see below).
  • This is currently limited to the EventGraph whilst we fix some semantics for the PersistentGraph. Please let us know if you notice anything odd, or unexpected with these if you give them a go.
image

Latest and is_active

  • We have added a latest() function onto the graph, node and edge. This is the equivalent of doing x.at(graph.latest_time). This isn't a massive issue in rust/python, but is very helpful in graphql, where you would have to do an initial query to get the latest_time.
  • We have exposed an is_active() function to nodes and edges, allowing you to check if they have any updates within the current window. This is very useful if you are calling rolling or expanding on a node/edge.

Bug fixes and performance improvements

  • Floats are now supported timestamps within python.
  • Fixed an issue in the motif algorithms where self loops were not being correctly handled.
  • Parallelised reading from saved graphs and for generation of new graphs with materialise.
  • Fixed the constant properties function in graphql as it was not set to async.
  • Fixed several 'off-by-one' errors in the boundary checks for node/edge window inclusion within the PersistentGraph.
  • Added the event_graph function to Graph and persistent_graph function to PersistentGraph - these are basically just NoOps, but make it so in python you can call them without knowing what type of graph you currently have.

Commits

Full Changelog: v0.11.3...v0.12.0

Contributors

louisch, fabianmurariu, and 5 other contributors
Assets 2
Loading

v0.11.3

06 Sep 16:52
@github-actions github-actions
v0.11.3
764e7ea
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.11.3

Parallel python loaders

Through some elegant dancing around locks, the pandas and parquet loaders now ingest into Raphtory’s underlying graph shards with minimal contention between threads. This has led to an order of magnitude improvement in ingestion speed in several of our use cases!

An example of this can be seen below where the 129 million edges of the Graph500 SF23 dataset are ingested in 25 seconds on a laptop!

image

Other minor bug fixes

Full Changelog: v0.11.2...v0.11.3

Contributors

narnolddd and ljeub-pometry
Assets 2
Loading

v0.11.2

03 Sep 23:35
@github-actions github-actions
v0.11.2
dab5563
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.11.2

Release v0.11.2

Assets 2
Loading

v0.11.1

02 Sep 14:57
@github-actions github-actions
v0.11.1
fcf885a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.11.1

Release 0.11.1

Bug Fixes

  • Exposed delete on edge in python
  • Fixed missing DTime parsing on Python properties
  • Fixed a bug in import_node(s)/import_edge(s) when the graph is indexed with ints

Graphql

  • Added new RemoteGraph, RemoteNode and RemoteEdge classes to wrap updates to graph on a server. These functions can be seen below
  • Made Properties optional in add_updates for Node and Edge in Graphql
  • fixed a bug in batch add nodes, where you couldn't just add the node_type.
client = graphql.RaphtoryClient(url="http://localhost:1736")
### RaphtoryClient functions
client.new_graph(path)

### RemoteGraph Functions
client.graph(path).add_node()
client.graph(path).add_nodes()
client.graph(path).add_edge()
client.graph(path).add_edges()
client.graph(path).delete_edge()
client.graph(path).add_properties()
client.graph(path).add_constant_properties()
client.graph(path).update_constant_properties()

### RemoteNode functions
client.graph(path).node(name).add_constant_properties()
client.graph(path).node(name).set_node_type()
client.graph(path).node(name).update_constant_properties()
client.graph(path).node(name).add_updates()

### RemoteEdge functions
client.graph(path).edge(src,dst).delete()
client.graph(path).edge(src,dst).add_constant_properties()
client.graph(path).edge(src,dst).update_constant_properties()
client.graph(path).edge(src,dst).add_updates()

##Misc

  • Tided up graphql pytests as having a single file was driving me mad
  • Broke the graphql python module into smaller files

What's Changed

Full Changelog: v0.11.0...v0.11.1

Contributors

fabianmurariu and miratepuffin
Assets 2
Loading

v0.11.0

23 Aug 16:20
@github-actions github-actions
v0.11.0
9bb4299
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.11.0

Release v0.11.0

Cached Graph

  • We have updated the on-disk format of the graph to now be stable across versions.
  • This new storage format is also updatable, allowing deltas to be inserted into an already existing file instead of having to resave the whole graph.
  • You can now 'cache' a graph, attaching a file to it, and periodically save all changes which have been inserted. Great for checkpointing! See the example below:
from raphtory import Graph
g= Graph() #Create new graph

g.add_edge(1,1,2)
g.add_edge(3,4,5)
g.cache("example_graph.raph") #add some updates and write these to a file - keeping the file attached to the graph

g.add_node(2,3)
g.add_edge(10,2,3)
g.write_updates() #Add some new updates and write these out

GraphQL

  • The GraphQL server has had a massive overhaul, allowing it to now create, update, and manage a working directory full of graphs.
  • Graphs will now persist across server runs based on the new cache-graph file format described above.
  • These new APIs also allow you to export subgraphs from one graph into another.

A basic example of this can be seen below:

from raphtory import graphql

server = graphql.GraphServer("graphs/")
server.run()
client = server.get_client() #run a local server and get the client 
#create a new graph in the 'example_graphs' namespace
client.query(""" 
        mutation {
            newGraph(path:"example_graphs/graph1",graphType:EVENT)
        }
""")
#add an edge into the new graph
client.query("""
query add_edge{ 
        updateGraph(path: "example_graphs/graph1") {
            addEdge(time:1,src:"node1",dst:"node2",properties: [
                {key: "new", value: "new props are  awesome"}
                {key: "number", value: 0.7}
                {key: "map", value: { a: "hi"}}
            ]){success}
        }
}""")

Pandas and Parquet loader changes:

  • We have placed time as the second parameter within the graph.load_x_from_y to match g.add_node and g.add_edge.
  • All parameters named props in the graph.load_x_from_y functions have been updated to properties.
  • The Layer_in_df flag has been replaced by layer and layer_col which will now throw an error if both are set.
  • The node_type_in_df flag has been replaced by node_type and node_type_col which will now throw an error if both are set.
  • You can now set the node_type via load_node_props_from_x functions.
  • We have removed the static load_from_pandas and load_from_parquet functions from the graph as these had WAY too many arguments and were all around very confusing. Instead you can now create a graph first and use load_edges_from_X etc.
  • The Datareader behind the Pandas and Parquet loaders now chunks the data and loads these in in part, drastically reducing the memory overhead for large datasets on ingestion.

Minor changes

  • Optimised the way node IDs are stored/referenced, greatly increasing both lookup and insertion speed. This does mean, however, that a graph can only have String XOR Integer ids now, not both at the same time.
  • We have updated to rust 1.80.0

What's Changed

Full Changelog: v0.10.0...v0.11.0

Contributors

fabianmurariu, shivam-880, and 2 other contributors
Assets 2
Loading

v0.10.0

15 Jul 13:23
@github-actions github-actions
v0.10.0
d8752c3
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.10.0

What's Changed

New Contributors

Full Changelog: v0.9.3...v0.10.0

Contributors

fabianmurariu, shivam-880, and 4 other contributors
Assets 2
Loading