ReactGraph Part 4 – First (Re)Active Queries

Investigation into bringing together graph and reactive computing. This fourth post in the series showcases LINQ-to-graph and Rx types queries for both “pull” and “push” type computation.

• ReactGraph Part 1: Graphs and Reactive Computing – Historical Context
• ReactGraph Part 2: Remote evaluation of continuous queries
• ReactGraph Part 3 – The Data Model

Disclaimer

Remember that the goal of the investigation is to think about the abstractions. We don’t deal with implementation issues such as scale, reliability, hot spots, partitioning, locality of access, performance, etc. Those are very very important issues and the most difficult part of any implementation. If you are looking to work with graphs, you should really look at Neo4j.

Adding data

I picked a domain familiar to everyone… the social graph. First, let’s add some people…

var savas = await graph.NewNode("savas", new Person { FirstName = "Savas", LastName = "Parastatidis" });
var jim = await graph.NewNode("jim", new Person { FirstName = "Jim", LastName = "Webber" });
var erik = await graph.NewNode("erik", new Person { FirstName = "Erik", LastName = "Meijer" });
var adam = await graph.NewNode("adam", new Person { FirstName = "Adam", LastName = "Wolff" });

Now let’s create some relationships between these nodes…

await graph.NewEdge(
   "savas-friend-jim",
   savas.Id,
   "friend",
   jim.Id,
   new Friend { Since = new DateTime(1995, 10, 1) },
   true);

// ... More person-friend-person edges

await graph.NewEdge(
   "savas-colleague-erik-1",
   savas.Id,
   "colleague",
   erik.Id,
   new Coleague
   {
      From = new DateTime(2010, 10, 1),
      Until = new DateTime(2011, 10, 1),
      Company = "Microsoft"
   },
   true);

// ... More person-colleague-person edges

Let’s add some posts…

var post = await graph.NewNode(
   "savas-status-update-1",
   new StatusUpdate
   {
      Text = "Status update 1",
      CreatedOn = DateTime.Now
   });
await graph.NewEdge(
   "savas-post-status-update-1",
   user.Id,
   "post",
   post.Id,
   new Post { PostedOn = DateTime.Now });
// ... More posts 

In a similar way, we can add comments, representations of photos, likes, pokes, etc.

First query

First, a note about the difference between nodes/edges and the values they store. You will notice that when we write queries against the ReactGraph store, we are dealing with nodes and edges, not values. We will see operators that help us write queries against those values, taking advantage of LINQ’s deep integration with the language (e.g. inline completion).

// Get the INode in the graph with identity "savas"
var savasNode = await graph.Get("savas");

// Project the value stored at the INode to an instance of the 
// type Person. We only need this when we want to deal with the 
// value inside a node or an edge.
var savasValue = savasNode.Value<Person>();

Now, let’s get one’s friends. A common theme across all the examples is the heavy use of LINQ and existing query operators wherever possible. This was actually a key goal of this investigation. More specialized/advanced query operators could (and will) be writen.

var friends = savas
  .OutgoingEdges
  .Where(e => e.Predicate == "friend")
  .Select(e => e.Graph.Get(e.DestinationId).Result));
  
foreach (var f in friends) {
  WriteLine($"Savas friends with {f.Value<Person>().FirstName});
}

// --- Output:
// Jim
// Eric
// Adam

Cool eh?

Note that the e.Graph.Get(e.DestinationId).Result part is really ugly. Unfortunately, the LINQ operators don’t compose great when async delegates are used. This is an open design issue that I will try to address.

The above pattern is common enough that I have provided helper operators (extension methods)…

var friends = savas
  .Outgoing("friend")
  .DestinationNode();

In both cases, friends is of type IQueryable<INode>. If we wanted to project to the Person type, we could have written the query above as…

var friends = savas
  .Outgoing("friend")
  .DestinationNode()
  .ValueType<Person>();

ValueType() is another helper extension method on IQueryable<INode>.

I hope you are with me so far and having fun 🙂

Now let’s get a user’s posts and start getting reactive

We can use a very similar query to the one above to get the user’s posts…

var posts = savas
  .Outgoing("post")
  .DestinationNode()
  .ValueType<StatusUpdate>();
  
posts.ToList().ForEach(p => WriteLine(p.Text));
// --- Output
// Status update 1
// Status update 2
// ...

In fact, we could even create domain-specific operators to make writing such queries even easier. Tooling that automatically generates code for us can take care of the mandane task of writing such helper operators.

public static class SocialGraphOperators {
  public static IQueryable<Person> Posts(this IQueryable<INode> nodes) {
    return nodes
      .Outgoing("post")
      .DestinationNode()
      .ValueType<StatusUpdate>();
  }
}

// Now we can get the posts like this...
var posts = savas.Posts();
  
posts.ToList().ForEach(p => WriteLine(p.Text));
// --- Output
// Status update 1
// Status update 2
// ...

Hopefully no surprise so far. But if we want to get the most recent list of a user’s posts, we need to keep submitting the same query. Well, that’s not great. How could we express our interest to receive any new posts that are made? Let’s enter the world of our reactive graph.

As before, here’s how the query expression would look like without any helper operators…

var posts = savas
  .EdgeChanges
  .Where(e => e.Node.Id == e.Edge.SourceId && e.Edge.Predicate == "post")
  .Select(e => e.Graph.GetNode(e.Edge.DestinationId));

// Using helper operators provided by the ReactGraph LINQ library
var posts = savas
  .EdgeChanges
  .Outgoing("post")
  .DestinationNode();
  
// Now subscribe
posts.Subscribe(p => WriteLine($"New post: {p.Text}");

// Of course, there is no output... We need to wait for 
// something to happen. Let's add some posts
await AddNewPost("savas", "foo");
await AddNewPost("savas", "bar");

// We need to enter an event loop
ReadLine();

// --- Output
// New post: foo
// New post: bar

And now, here’s the same Rx query looks like with the helper operators provided with the ReactGraph library…

var posts = savas
  .EdgeChanges
  .Outgoing("post")
  .DestinationNode();  

In both cases, the type of posts is IQbservable<INode> (note the “Q”). This means that the expression tree can be remoted and hosted at a remote store (which isn’t the case yet with the current implementation of course).

Please note how similar the pull and push queries look. That’s the beauty of LINQ.

As before, we could add domain-specific operators to make the query even easier to read.

In subsequent posts, we are going to dive into even more complex queries, both pull and push.

Next: A user’s feed via “pull” and “push” graph queries.