Chathooks - A webhook formatter for chat

Overview

Chathooks is a webhook proxy service that converts generic outbound webhook messages to a canonical CommonChat message format which is then sent to any chat / team messaging platform supported by the CommonChat chat post abstraction library.

This is useful because:

• many services with outbound webhooks need to be formatted before they can be consumed by an inbound webhook. This proxy service does the conversion so you don't have to. The code is all open source so customizations can be done easily.
• the conversion can be done one time for all chat / team messaging solutions supported by CommonChat.
• one service can proxy an arbitrary number of webhook sources and event types so you don't have to configure multiple inbound webhooks going to the same group / channel.

See the following video to get started quickly:

• Getting Started YouTube Video

Supported HTTP Engines

Chathooks supports multiple HTTP engines including the following:

• net/http
• valyala/fasthttp
• aws/aws-lambda-go

Supported Webhook Formats

Multiple input webhook formats are supported via handlers. New ones can be easily created by using the handlers.Handler interface.

1. Aha!
2. AppSignal
3. Apteligent/Crittercism
4. Bugsnag
5. Circle CI
6. Codeship
7. Confluence
8. Datadog
9. Desk.com
10. Enchant
11. GoSquared
12. Heroku
13. Librato
14. Magnum CI
15. Marketo
16. Nixstats (Slack webhook proxy)
17. OpsGenie
18. Papertrail
19. Pingdom
20. Raygun
21. Runscope
22. Semaphore CI, Deploy
23. StatusPage
24. Travis CI
25. Userlike
26. VictorOps

Here is an exmaple Webhook Message from Travis CI formatted for Glip.

The above are "outbound" webhook formats from the perspective of the service providing the events.

A special webhook format supported is the Slack "inbound" webhook format. This format message isn't sent by Slack but is accepted by Slack. It is useful to allow modification of messages already formatted for Slack.

• Slack

Supported Chat Services

Chathooks can post messages to any service supported by CommonChat. New services can be added by creating an adapter using the commonchat.Adapter interface.

Note: The emoji to URL is designed to take a icon_emoji value and convert it to a URL. EmojiURLFormat is a fmt format string with one %s verb to represent the emoji string without :. You can use any emoji image service. The example shows the emoji set from github.com/wpeterson/emoji forked and hosted at grokify.github.io/emoji/.

Installation

Local

$ go get github.com/grokify/chathooks

Heroku

or

$ heroku create
$ git push heroku master
$ heroku open

AWS Lambda

The following will generate a main.zip file to upload to AWS Lambda.

$ go get github.com/grokify/chathooks
$ cd chathooks
$ sh aws-package.sh
$ ls main.zip

Configuration

Environment Variables

Chathooks uses two environment variables:

Variable Name	Value
CHATHOOKS_ENGINE	The engine to be used: awslambda for aws/aws-lambda-go, nethttp for net/http and fasthttp for valyala/fasthttp. Leave empty for eawsy/aws-lambda-go-shim as it does not require a server to be started.
CHATHOOKS_TOKENS	Comma-delimited list of verification tokens. No extra leading or trailing spaces.

Using the net/http and fasthttp Engines

1. To adjust supported handlers, edit server.go to add and remove handlers.
2. Set the CHATHOOKS_ENGINE environment variable to nethttp or fasthttp.
3. Start the service in main.go.

For testing purposes, use:

$ go run main.go

For production services, compile and run the binary:

$ go build main.go
$ ./main

Using the AWS Engine

To use the AWS Lambda engine, you need an AWS account. If you don't hae one, the free trial account includes 1 million free Lambda requests per month forever and 1 million free API Gateway requests per month for the first year.

Installing using the AWS Lambda UI

Installation via AWS Lambda

See the AWS docs for deployment:

https://docs.aws.amazon.com/lambda/latest/dg/lambda-go-how-to-create-deployment-package.html

Using the aws-cli you can use the following approach:

$ GOOS=linux go build main.go
$ zip main.zip ./main
# --handler is the path to the executable inside the .zip
$ aws lambda create-function \
  --region region \
  --function-name Chathook \
  --memory 128 \
  --role arn:aws:iam::account-id:role/execution_role \
  --runtime go1.x \
  --zip-file fileb://main.zip \
  --handler main

You can use the aws-package.sh shell script to package your handler.

Note: you can install the aws-cli using the appropriate command below:

$ pip install awscli
$ pip3 install awscli

After installation, you may need to add your Python bin directory to your path. The following are examples for OS X.

$ export PATH=$PATH:/Users/~/Library/Python/2.7/bin
$ export PATH=$PATH:/Library/Frameworks/Python.framework/Versions/3.6/bin

Update Lambda Code:

You can update the Lambda function code using the following:

https://docs.aws.amazon.com/cli/latest/reference/lambda/update-function-code.html

$ aws lambda update-function-code --function-name='MyFunction' --zip-file='fileb://main.zip' --publish --region='us-east-1'

The aws-update.sh file has this command with default settings.

Make sure to set your AWS credentials file.

Manual Setup

• Configure AWS Lambda Function
  1. Create AWS Lambda function
  2. Set CHATHOOKS_ENGINE to awslambda
  3. Set CHATHOOKS_TOKEN as desired
  4. Set Handler to main
• Configure API Gateway
  1. Create "REST" API
  2. Select "New API"
  3. Enter "API name"
  4. Click "Create API"
  5. Select "Actions" > "Create Resource" on "/"
  6. Click "Configure as proxy resource"
  7. Use Resource Path {proxy+}
  8. Click "Enable API Gateway CORS"
  9. Click "Create Resource"
  10. Leave "Integration Type" as "Lambda Function Proxy"
  11. In "Lambda Function", paste in your Lamda ARN
  12. Click "Save"
  13. Click "Actions" > "Deploy API" and create stage if necessary.
  14. Copy "Invoke URL" which should end in {stageName}

Troubleshooting

• 401 Unauthorized Error: If you can run a test on API Gateway and have deployed your API but are running into a HTTP Status 401 Unauthorized error, check to see if you have configured the CHATHOOKS_TOKEN environment variable but not entered a token query string parameter.

Usage

Creating the Glip Webhook

1. create a Glip webhook
2. use webhook URL's GUID to create the proxy URL as shown below
3. use the proxy URL in your outbound webhook service

Query Parameter	Required?	URL
inputType	required	An handler service like marketo
outputType	required	An adapter service like glip
outputURL	required	A webhook URL or UID, e.g. 11112222-3333-4444-5555-666677778888
outputFormat	optional	one of [nocard,card]
token	optional	Must be included if service is configured to use auth tokens

The webhook proxy URLs support both inbound and outbound formats. When available, these should be represented in the handler key.

To create the Glip webhook and receive a webhook URL do the following:

Add the Webhook Integration

At the top of any conversation page, click the Settings gear icon and then click Add Integration.

Select the Glip Webhooks integration.

Get the Webhook URL

Once you get the URL, the proxy URL is created by appending the GUID (e.g. 1112222-3333-4444-5555-666677778888) to the proxy URL base, hooks?inputType=slack&outputType=glip (e.g. https://glip-proxy.example.com/hooks?inputType=slack&outputType=glip&url=1112222-3333-4444-5555-666677778888). Use the proxy URL in the app that is posting the Slack webhook and the payload will be sent to Glip.

Development

Testing new handlers

It's easy to test handlers by sending messages using the example messages per service.

Code is provided to both send the message locally (without HTTP) and over HTTP.

• local_send.go without HTTP
• proxy_send.go over HTTP

cURL

The following is an example curl command that can be used with any host, local or remote.

curl -XPOST 'https://example.com/webhook?inputType=datadog&outputType=glip&url=https://hooks.glip.com/webhook/11111111-2222-3333-4444-555566667777' --data "@docs/handlers/datadog/event-example_formatted1.json" -H 'Content-Type: application/json' --verbose

Adding Handlers

The easiest way to add a handler is to inspect the code of an existing handler and build something similar. It needs satisfy the handlers.Handler interface.

Notes

Maintenance

Use godep to rebuild the vendor directory with:

$ rm -rf Godeps
$ export GO15VENDOREXPERIMENT=1
$ godep save ./...

More information on deploying Go on Heroku here:

• https://devcenter.heroku.com/articles/go-support

Heroku

Set up your own Go app on Heroku easily using goheroku.

Credits

Chathooks is built using:

• net/http

• valyala/fasthttp

• aws/aws-lambda-go

• buaazp/fasthttprouter

• rs/zerolog