Posted Tuesday April 29, 2014

Launching the Open Data Compass

Today saw the launch of the Arachnys Open Data Compass: a culmination of several months of additional research work building on top of our ongoing indexing of the world’s compliance-relevant data. The tool gives a powerful overview of where in the world has the best provision of the types of business-critical data necessary for due diligence research.

Releasing this index was partly motivated by demand from our customers: we are frequently asked where our global coverage is ‘best’. While this question is simple to answer in terms of number of sources or the presence or absence of high-value repositories, such as a central company registry, we wanted to give a more detailed account of the depth of information available to an investigator in as many countries and territories as possible. A second motivation was to have a more data-focused and comprehensive public outlet for our research work, to accompany this blog.

This is, in fact, the second year that we have published the Compass, though there have been several important updates:

  • We covered all 215 countries and territories included in the main Arachnys app, including countries in the developed world that usually fall outside our region of focus;
  • Our scores were calculated based on scoring of individual sources, rather than simply scoring at the country level;
  • We invited commentary from external experts to discuss our findings in context with their own regional insight;
  • We undertook a more detailed regional and thematic analysis, some of which is published on the Compass microsite, and some in our full report.

The research process has thrown up plenty of counterintuitive findings, interesting trends and instructive discussions on how to approach a project of this nature. Most fundamental of all, what features should contribute to a good score? The Compass values openness of data availability highly but, although there are very precise existing definitions of what constitutes Open Data, we took the view that using such strict criteria would heavily penalise many of the developing parts of the world that are of most interest to our users, and obscure the existence of useful business intelligence. As such, we decided to take a more qualitative approach, scoring more highly for the scope and depth of data sources, as well as the convenience in accessing and searching them.

Another key question was which data sources should contribute to the score. While, in general, a small but growing handful of nations is quickly outstripping the rest with the launch of data portals providing access to everything from child-related product recalls to buffalo censuses, the Compass focuses on a standardised collection of data sources that were widely comparable across all the markets we reviewed. We found that this approach enabled us to be more consistent in our methodology and to make robust comparisons between countries with widely different levels of development.

Finally, what do the results tell us? We think that the Compass scores give a valuable measure of an interplay of factors including anti-corruption, compliance, development, transparency and tech infrastructure. Given this, the top and bottom of the rankings may be no surprise, with New Zealand, the United Kingdom and Australia taking the podium, and the lower ranges dominated by sub-Saharan Africa, North Korea and small island states. There are some notable outliers, however, on several of the above factors: China and India, noted for their struggles with political corruption, score well, boosted by the high volumes of online public data to be found on provincial and state government e-services. Conversely, paywalls and other access restrictions prevent more developed and tech-friendly countries such as the USA, Singapore and South Korea from performing better in the index.

We’re delighted to be launching this free tool and hope you enjoy exploring our results and analysis on

Stay current with the latest from Arachnys

Subscribe today