Skip to content

Latest commit

 

History

History
98 lines (65 loc) · 3.23 KB

README.md

File metadata and controls

98 lines (65 loc) · 3.23 KB
Raw

🔭 Grants Data cache & ETL Pipeline

For exploring grants data using SQL queries in Postgres

DB schema visualized

Table of Contents

🚪 Introduction

This is a NodeJS/typescript ETL framework, powered by the Allo indexer data API. This ETL pipeline queries the Allo indexer data API and populates the existing Postgres db in docker (see docker-compose.yml file). Database schema is managed using Prisma (a Typescript ORM).

🏁 Quick start

  • Install Docker. Visit https://www.docker.com/ to install.

  • After Docker installation, open Terminal to this repo location and run:

docker compose up

PGAdmin

  • Once all containers are running and healthy, open http://localhost:5555 to open PGAdmin.

  • Login using default login details:

Email: [email protected]
Password: admin
  • Load and configure your server:
Config Name Value
Host name/address host.docker.internal
Port 5432
Username postgres
Password postgres

PGAdmin add new server PGAdmin server name PGAdmin add server config

Metabase

  • Once all containers are running and healthy, open http://localhost:3000 to open Metabase UI.

  • Login using default login details:

Email: [email protected]
Password: password420

Metabase Dashboard

🪄 Run sample query

  • Click on the Query tool (image below) and copy any sample query from ./queries 😉

PGAdmin run new query Metabase run new query

💪🏾 Local Development

You can modify the schema file (located at ./prisma/schema.prisma) as needed. To kick off indexing for a particular chain, you can run:

yarn run etl --chain [chainId]

Currently available chains are:

Chain ChainId
Ethereum Mainnet 1
Optimism Mainnet 10
Fantom Opera 250
Public Goods Network (PGN) 424

For more hands-on use cases like creating a custom ETL pipeline like indexing data from the blockchain via RPC using tools like chainsauce.

😎 Contribute

You can be a cool anon and send in a PR. If you have an SQL query that has helped you and you feel will be helpful to other Anons 👻 and explorers 🔭, feel free to fork this, add it to ./queries, and send in a PR.