Developers

This section of the DataHub documentation is for developers. Here you can learn about the design of the platform and how to get DataHub running locally or on your own servers, and the process for contributing enhancements and bug fixes to the code.

Gitter

We use following GitHub repositories for DataHub platform:

  • DEPLOY - Automated deployment
  • FRONTEND - Frontend application in node.js
  • ASSEMBLER - Data assembly line
  • AUTH - A generic OAuth2 authentication service and user permission manager.
  • SPECSTORE - API server for managing a Source Spec Registry
  • BITSTORE - A microservice for storing blobs i.e. files.
  • RESOLVER - A microservice for resolving datapackage URLs into more human readable ones

  • DOCS - Documentations

graph TD subgraph Repos frontend[Frontend] assembler[Assembler] auth[Auth] specstore[Specstore] bitstore[Bitstore] resolver[Resolver] docs[Docs] end subgraph Sites dhio[datahub.io] dhdocs[docs.datahub.io] docs --> dhdocs end deploy((DEPLOY)) deploy --> dhio frontend --> deploy assembler --> deploy auth --> deploy specstore --> deploy bitstore --> deploy resolver --> deploy

Install

We use several different services to run our platform, please follow the installation instructions here:

Deploy

For deployment of the application in a production environment, please see the deploy page.

DataHub CLI

The DataHub CLI is a Node JS lib and command line interface to interact with an DataHub instance.

CLI code