Webhooks are what drive communication between a Saas service and your application. With Saas being a common component in today’s applications, we have created a cheat sheet to minimise setup time when working with Webhooks.
If you ever have to work with Slack, Telegram, MailChimp, Paypal and many other Saas — you would have come across something called a Webhook. A webhook is a way for an application or platform to provide other applications with real time data, and to do so, you create a callback API (webhook) in your application to listen to the data returned by the other application.
To work, your local server needs to be reachable from the outside — which is not always straightforward.
When we develop an application with a webhook, we have to let the other application know the URI of the webhook. This is so it can push data to our application in real-time. For example, if I am developing a chatbot application for Slack, I should have a public URI that I can key into my app’s webhook field inside Slack’s developer console. That way, Slack knows where to push real-time data to.
Usually, we develop applications in our local environment with a local server. However, for webhooks to work — we need to allow our local server to be reachable from the outside. We can configure our router which stands in between our own private network and the cloud to forward traffic via a certain port on our internet gateway to our application. But this is not feasible in most cases as you need to have access to your router configuration, your router gateway needs to have a static IP address, and this can compromise your private network security.
There is one solution we have found useful…
Employing a Tunneling Service
A tunneling service is a service that implements a tunneling protocol that enables communication between networks. It enables private network communication to be sent across a public network such as the internet through Encapsulation.
There are quite a few options we can use when it comes to tunneling services. We can either roll out our own, or use one of the free services available in the wild. We detail them below:
Option 1: Rolling out our own tunneling service
Understand the pros and cons of this option
The beauty of having your own personal remote server acting as a tunnel is that once it’s setup, it’s always available and you can customise the server to include things like logging, dashboard, and support for multiple users.
However, it does take some technical know-how and time to either set up or customise it. The running remote server will also cost you some money to run, so it may not be worth it if you are only working with Webhook on an ad-hoc basis.
To get started:
You can create your own personal tunneling service using OpenSSH with a domain name and a remote server. The remote server can be created on the fly in an inexpensive way using DigitalOcean Droplet, AWS EC2 or similar services.
Once you have created your remote server and launched it, you want to edit your DNS record to point your domain name to the newly created server.
With these set up, you can now setup remote port forwarding on your development machine.
ssh -R 8888:localhost:3000 -N username@your-server-ip-address
This initiates a SSH tunnel, and all traffic between your development machine and the remote server are encrypted with symmetric encryption.
It forwards traffic from your-domain-name:8888 to localhost:3000 on your development machine. The -R flag stands for remote port forwarding, meaning traffic are forwarded from the remote server to your local development machine, and the -N flag prevents the command from landing into the remote server’s shell.
TLS support
Sometimes, the Webhook might require secure connection over HTTPS, as in the case of Facebook Messenger. In such cases, you can acquire a free certificate with Let’s Encrypt and setup a reverse HTTPS proxy on the remote server using NGINX.
NGINX Documentations on Reverse Proxy and Securing HTTP Traffic to Upstream Servers provide information on how to set this up.
Option 2: Turnkey solutions
Understand the pros and cons of this solution
If rolling out your own tunnel is too much of a hassle, turnkey solutions are a great way at no cost. With your own tunnel service, you can set it up to fulfill any kind of requirements you may need, but might require considerable time and effort to set up. The Turnkey solutions require almost zero effort to get up and running, but may fall short of certain requirements that you may need.
Getting Started
We will introduce two products for use — Ngrok and Argo tunnel. Which one do you go for depends on a number of factors including:
- Setup time and effort
- Reserved domain
- Single or multiple developer setting
- Security
- Cost
Ngrok
Ngrok is the go to solution for many developers with it’s extensive documentation, and features such as a dashboard to track your tunnel traffics and inspect the requests coming in. You can also secure your tunnel with a password and replay requests.
It comes as a binary that you can install to your system path to have access to the ngrok command anywhere in your command line.
To start a tunnel with Ngrok:
ngrok http 3000
This creates a tunnel to port 3000 and the ngrok command also gives you the public address(in both http and https) that you can use as the webhook URI.
The freemium version outputs a random forwarding URL in each new instance. This means that you have to update your webhook URI every time you run it. The paid version features custom domains and IP whitelisting among others.
The freemium version should be sufficient for most people, but if you do require the paid version for features such as custom domain, and have the time and know-how. Deploying your own tunneling service may work out to be more cost effective and scalable.
Argo Tunnel
Argo Tunnel is a free service from Cloudflare that exposes a server to the internet without opening any ports.
To use Argo Tunnel, you need to install the Cloudflare daemon — Cloudflared from https://developers.cloudflare.com/argo-tunnel/downloads
To start a tunnel with Cloudflared:
cloudflared tunnel --url localhost:3000
Similar to Ngrok, this creates a tunnel to port 3000 with a random subdomain of trycloudflare.com.
Argo Tunnel works similarly to Ngrok without the bells and whistles that Ngrok provides. If you are a Cloudflare developer, and want to use something that’s integrated with the cloudflared daemon, this may be for you.
Conclusion
No option is a silver bullet, but knowing the different options available, gives you the power to quickly make the right decision on the setup you need. This in turn will help you and your teams become more efficient so that your time can be better spent on creating more impactful products.
