Indexer + API app

The project has 3 parts

• Postgres DB - schema defined in supabase hosted by supabase, uses some of their tools
• Indexer "ETL" go code in go-indexer - long-running server that
  • connects to an ethereum node
  • reads events for specific contracts (configured in config)
  • write events to DB
  • transforms events into data required for the app (accounts, positions, etc)
• Next JS API app in app - easy way to host the API. reads from the same schema. Deployed on vercel.

Database

Prerequisites

• npm
• docker
• supabase installed globally (npm install supabase --save-dev)

Start locally

start docker

npx supabase start
npx supabase status

Deploy schema change

npx supabase migration up

Generate types

# for indexer
npx supabase gen types --lang go --local > go-indexer/database/types.go

# for api server
npx supabase gen types --lang typescript --local > app/types/database.ts

Clear database and apply migrationx

npx supabase db reset
## remember to re-generate types

Restart system

npx supabase stop --no-backup ## drops DB

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker network prune

docker volume ls
docker volume rm <volume name>

Indexer

Prereqs

• go
• docker

Local development

1. install dependencies

go mod tidy

2. set config in go-indexer/config/config.yaml

• add contracts info + abis
• check supabase local studio (http://127.0.0.1:54323) for anon key and connection string

3. run server

# For development (default)
./start-indexer.sh

# For production
./start-indexer.sh prod

Run with nix

nix run .#indexer

Testing

go test ./...  

go test ./go-indexer/indexer -v

Health checks

Health checks should be deployed for every process. Currently there are only 2:

1. historicla loading + event ingestion (main routine)

curl http://localhost:8080/health

2. event processing (transform.go)

curl http://localhost:8081/health

Deploying

1. stop server + update
2. run database migration doc
3. if you don't want to refetch all historical events, update go-indexer/config/config.yaml to fetch events after a specific block
4. restart server

API server

Simple nextjs app to provide an API for the indexer database

Prereqs

• node.js
• npm

Start API server

1. install dependencies

npm install

2. copy .env.example

• SUPABASE_URL: http url for postgres db
• SUPABASE_ANON_KEY: read-only key
• API_KEY: random string to give access to API

3. start server

npm run dev

Visit the api at localhost:3000/v1

API docs and schema

Docs served at /docs. Generated automatically