Building a real-time analytics dashboard on streaming data

Just over a year ago Kenny Bastani wrote a blog post showing how to analyse GitHub Events using Apache Pinot and Kafka. In this blog post, we will build on Kenny’s work, but instead of using Apache Superset to visualise the data, we’ll be using a Python library called Streamlit.

Analysing GitHub Events with Pinot and Streamlit

The code used in this blog post is available in the mneedham/pinot-github-events repository if you want to follow along or try it out afterwards:

Pinot

To recap, Apache Pinot is a data store that’s optimised for user-facing analytical workloads. It’s good at running real-time queries on event data.

We’re going to…


Tracking Cryptocurrency trading using the LUSID investment data platform

I recently started working at a London FinTech company called Finbourne who make a cloud-based data platform for investment data called LUSID.

LUSID — A data platform for investment data.

It’s mostly used by hedge funds, pension companies, and investment banks to store their trading data, but you can use it to store your own personal investment data as well.

My colleague Mike McGarry has done exactly this, importing data from his personal financial accounts and investments, and then using LUSID to compute his net worth over time.

Reading Mike’s blog posts got me wondering if we could use LUSID to build a portfolio of holdings in different…


How is the Covid vaccination rollout going in different parts of England?

In the world of Covid data, Thursday afternoons are my favourite because that’s when NHS England publish their weekly vaccinations spreadsheet.

In this blog post, we’re going to explore the data published in the spreadsheet to learn how the vaccine rollout is going in different parts of the country.

NHS England Covid Vaccination Data Website

The Data Preamble

The spreadsheet contains data on the number of vaccinations given by age group and by region, as well as population estimates for each of these areas.

The data is a few days behind the aggregate data shown on the official dashboard, so we need to keep that in mind when doing…


Learn about the latest updates to NEuler, the Graph Data Science Playground, including a new community detection layout and GDS Sandbox integration.

NEuler (Neo4j Euler) is a UI that simplifies the onboarding process for users of Neo4j’s Graph Data Science Library (GDSL). It was first released in early 2019 when it was used to onboard users of GDSL’s predecessor, the Graph Algorithms Library.

Leonhard Euler

Over the last few months, we’ve made changes, which will (hopefully!) make it easier to use and get you up to speed quicker with GDSL.

The NEuler Developer Guide contains instructions for installing the app or you can watch the video below.

Installing NEuler

Let’s have a look at what changes have been made.

Perspectives-light

One usability problem that many…


NEuler, the Graph Algorithms Playground, now supports the Graph Data Science Library

Just under a year ago we released NEuler, the Graph Algorithms Playground, which made it easy for users to learn how to use the Graph Algorithms Library.

Its successor, the Graph Data Science Library, was recently released, which meant that NEuler needed to be updated to use that instead.

I’m happy to announce that as of version 0.1.16, NEuler is Graph Data Science Library ready. It has also been renamed to be the Graph Data Science Playground!

This version is only supported by Neo4j Desktop versions 1.2.5 and higher, so you’ll need to update that as well.

If you had…


Import databases created in Neo4j Desktop to Neo4j Aura, the graph database as a service.

So we’ve been using Neo4j via Neo4j Desktop for a while now, building up a formidable collection of databases, and then a few weeks ago we learned about the launch of this thing called Neo4j Aura.

Neo4j Aura

Neo4j Aura is the simplest way to run Neo4j in the cloud. Completely automated and fully-managed, Neo4j Aura delivers the world’s most flexible, reliable and developer-friendly graph database as a service. With Neo4j Aura, you leave the day-to-day management of your database to the same engineers who built Neo4j, freeing you to focus on building rich graph-powered applications.

We’d like to give it a…


The latest release of APOC introduces streaming support for exporting to JSON and GraphML, as well as other features.

This week we released version 3.5.0.6 of APOC, Neo4j’s standard library.

APOC

tl;dr

This release contains the following features and bug fixes:

  • streaming support for export JSON and GraphML
  • string multiplication and fill functions
  • removing custom procedures and functions
  • null issues with apoc.coll.containsAll and Mongo procedure
  • apoc.periodic.repeat has improved reporting for bad queries
  • apoc.graph.fromDocument considers whitelist property mappings

You can install the latest release directly from the Neo4j Desktop in the ‘Plugins’ section of your project. Jennifer Reif also has a detailed post explaining how to install plugins if you haven’t used any yet.


The latest releases of the Graph Algorithms Library add new functionality to the in memory graph.

Over the last month or so we’ve released new functionality for the Neo4j Graph Algorithms Library, in versions 3.5.11.0, 3.5.12.0 and 3.5.12.1.

tl;dr

These releases see the following features and bug fixes:

  • support for loading multiple relationship types
  • support for loading multiple node and relationship properties
  • Cypher projections no longer write to the database
  • introduction of the Approximate Nearest Neighbors algorithm that I’ve previously written about.

Note that this release is only compatible with Neo4j 3.5.9 and above. You can read more about the release in the release notes.

You can install the latest release directly from the Neo4j Desktop in…


In version 3.5.11.0 of the Neo4j Graph Algorithms Library we added the Approximate Nearest Neighbors or ANN procedure.

ANN leverages similarity algorithms to efficiently find more alike items. In this post, we’ll look at our motivation for creating this algorithm, where it can be used, and will show how to use it with the help of a worked example.

You can download the Graph Algorithms Library from the Neo4j Download Center or install it into Neo4j Desktop via the Graph Algorithms Playground Graph App

Nearest Neighbors

It’s now been almost a year since we added similarity algorithms to the Neo4j Graph Algorithms…


Building a Neo4j and React App for Football Transfers using the GRANDstack

A month ago I wrote a blog post about a Neo4j graph I’d created containing all the football transfers in the Summer 2019 Window.

The GRANDstack — GraphQL, React, Apollo, Neo4j

While I was able to explore the data easily using Cypher queries, I wanted to make life a bit easier for myself by putting a front end around the data.

It seemed like a good opportunity to create my first GRANDstack application, which is exactly what I did.

The GRANDstack starter kit template

I started by creating a new repository based on the GRANDstack starter kit GitHub template. We can find that template at github.com/grand-stack/grand-stack-starter:

Mark Needham

Developer Relations at Finbourne

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store