Using a D3.js Force-Directed Node Graph to visualise Meta Data Relationships in News Articles produced by EMM

Page

fasttube screenshot off emm - ellipsed

Abstract

This paper describes a method for visualising meta information in data sets, using just co-occurance as an indicator for relationship strength. The meta entities are shown as nodes on a social-network style graph that provides a birds-eye view of how connected/disconnected the various entites are.

Introduction

For this article we will be using news feeds produced by the European Media Monitor, EMM. The EMM processing chain analyses news stories and adds meta-data to them, eg Entities (like people or organisations), GeoLocations, Categories, Sentiment etc

We will first process the data into 2 sub-sets, the entities and the co-occurances.

then we will create a relationship graph of 1 set of meta entites,

afterwhich we will add the other entites.

we will also discuss how to filter and control the graph to improve the visualisation.

and a possible method to auto-tune the simulation based on statistical calculations (ie using a normal distrubtion curve to set bottom (and top) filter threshold(s))

Methods

Preparing the data

The EMM data is retrieved via a http request to a specific rss feed. (in the web browser this is proxied using JSONP to circumvent cross-domain restrictions)

The Meta data items eg Entities are extracted and stored in a unique array, which we’ll call nodes.

the co-occurence between 2 entities is stored as a javascript object, which we’ll call a link, and contains a reference to the 2 entities (source, and target), and a count value for how many times the co-occurance occured in the article set.

finally we end up with a set of nodes and a set of links, with count values for both.

Preparing the Force Directed Graph

Load D3.js in your browser and create a placeholder div element with an id eg. ‘chart’

Instruct D3 to create an SVG element inside the placeholder.

Instruct D3 to create a Force layout.

Create an update function.

Manipulating the Visualisation by manipulating the Graph configurations

This is done by customising the charge and link distance functions etc

Linking the Data to the Graph

This is done with a data-join.

Manipulating the Visualisation by manipulating the data

This is done by filtering the data, and re-applying the data-join.

Incorporating other meta-data sets

when extracting the meta data items we can wrap them in a standardised meta objects super class,

effectively normaising our data set ie treating all meta-data items equally

we can now run them all into the same process.

However since each item has it’s own subclass properties eg type, we can effectively distinguish and filter them, in or our as we desire.

Distinguising and Filtering meta-data subsets

a large network can be created that shows categories and entities, geolocations etc.,

however it looks confusing, therefore we will apply filters to show the relationships between different subsets dynamically.

Results

<show screenshots here>

Discussion/Conclusions

This is great tool for analysing meta information and their relationships to each other,

all the source code for the meta visualiser is released under LGPL or Apache2.0 Licence (to be decided) see Appendix.

 

Appendix

Source Code

Requirements:

D3.js (v3)

JQuery.js (v?)

—————-

MetaVisualiser.html

MetaVisualiser.js

MetaVisualiser.css