Connecting IOT Networks Behind a NAT with Grenache

Bitfinex is, like many things in the cryptocurrency space, a highly distributed system. Our backend architecture is distributed, as is the vast majority of our team. This puts us in a position where creativity and innovation within problem solving is a must.

In this tutorial we take a look at how you can connect to servers which are behind a NAT, e.g. a home router, by breaking down the concept of UDP Holepunching whilst creating two basic server applications which can connect through a NAT. This is a relevant case study for IOT applications and a represents common problem for anyone who has attempted to run a server at home. 

 

Routing with local networks

Most of the routers we use at home today still use NAT, consisting of a public IP and an internal one, for the local network. Computers on the internal network are not reachable from the outside without routing.

Screen Shot 2018-07-22 at 12.46.46

One solution to solve this problem is to create a manual routing table. The table maps a port on our public IP address to an internal IP and port. This way our server becomes reachable from the public internet:

Screen Shot 2018-07-25 at 13.16.38

In this diagram our ISP assigns the public IP 111.111.111.111 to us. If someone connects to this IP, the request will arrive at our router. In our small local network a server is running with the IP 192.168.1.3. The two arrows show a routing we set up, the port 3000 on our server is mapped to the port 1800. When a request arrives at our public IP 111.111.111.111on port 1800, the router routes it to 192.168.1.3, port 3000.

Sometimes we can’t manually create or maintain a routing table for technical or practical reasons. In those cases, we can use a practice called UDP Holepunching. With a UDP Holepunch, we let the router create the mapping. 

 

How UDP Holepunching works

With UDP Holepunching we let the router create the mentioned mapping for us. Many popular applications rely on it and it’s a feature used more often than we think. One well-known example is BitTorrent, but also many voice chat and video chat clients make use of it.

Imagine again our server at home with a service listening on 3000. Now we send a UDP packet to another server on the internet, from exact the same internal IP and port (192.168.1.3:3000). Let’s say the server on the internet has the IP 8.8.8.8 and accepts data on port 1337. Our router will notice the outgoing packet and remember its destination. Our router will also assign a new external port when it sends the packet to the external destination. Let’s pretend the router uses the port 24000. For the external server the UDP packet then comes from 111.111.111.111:24000, but on our internal network we sent the packet from 192.168.1.3:3000.

Lets say our external target server now sends back a UDP packet within a short amount of time. Our router still remembers that we just sent out data to 8.8.8.8:1337 from this port and that it was originating from 192.168.1.3:3000. So it will route the packet to our local server, as a service for us. Now we have created a routing entry and the servers can continue to send data to each other.

 

UDP Holepunching with Grenache

After taking a look at the problem and the theoretical solution, let’s see how we can make use of it in daily life. At Bitfinex we use Grenache, a tiny library for creating distributed networks. Grenache uses the BitTorrent protocol for peer discovery and we recently added support for Holepunching.

The UDP Holepunch transport for Grenache makes it easy to connect services between different local networks to share data. A good example are IOT applications. It’s common in the IOT world that a local server offers services that should be accessible from the public internet.

In our example we build a service inspired by the fibonacci number generating service we created some time ago in ‘Bitfinex Loves Microservices With Grenache. This time we will make it ready for NAT home routers. We will also extend the fibonacci calculation to return the sequence of numbers.

To start, we have to spin up our own local BitTorrent based network:

npm i -g grenache-grape

grape --dp 20001 --aph 30001 --bn '127.0.0.1:20002'
grape --dp 20002 --aph 40001 --bn '127.0.0.1:20001'

 

Easy, isn’t it? We are running our own and private BitTorrent network now.

 

The Fibonacci calculation service

Our project will have two parts. A calculation service, which will run in our NAT at home. In the second part of the article we create a consumer, which needs to be located on a non-firewalled server. Later we will connect both with a UDP Holepunch.

Connecting RPC services where all parties are behind a NAT is possible, too. In this case we need an additional party. The additional party helps with the exchange of the IPs and ports. In case you are interested in it, there is an example for a setup with broker in the Grenache UTP repository.

For this article, we will use grenache-nodejs-utp instead of the HTTP or WebSocket transport used in other articles. It will provide us with helper methods for Holepunching. For communication it uses UDP/UTP sockets.

Let’s start with coding. Our calculation service will be located in a file called rpc_server_behind_nat.js. The heart of our service will be the calculation of the fibonacci sequence by length. When we pass in the maximum length as parameter, the function will return the fibonacci number sequence of that length:

When we now call fibonacci(10), we receive [1, 1, 2, 3, 5, 8, 13, 21, 34, 55] as result, the first ten numbers in the sequence.

Now we have to make everything available on the network. Make sure you have all required dependencies installed:

npm i --save grenache-nodejs-link
npm i --save https://github.com/bitfinexcom/grenache-nodejs-utp

 

We require the RPC Server component and a component called Link:

The link will connect to our private BitTorrent network. It will help us with advertising our service to other parties. Later in this article we will use the link for looking up services.

The following code will connect to the small private network we set up with the two Grape command line tools:

Then we inject the link into our server component, so the server can use the connection:

For the server to work, it has to listen on a port. By calling ‘listen’ we let it listen on free, randomly assigned port:

As a next step, we have to put our request handler into place. When a new request arrives, we want to calculate the fibonacci sequence and answer the request:

We now have to announce our fibonacci service on the Grenache network. This happens by calling link.announce. We will also look for possible consumers of our service. By calling peer.punch we send consumers on the newtork a UDP packet to start a connection setup with them:

That’s it! A small server connected to a private BitTorrent network in 44 lines of code.

The code for this section is also part of the examples on GitHub: rpc_server_behind_nat.js.

 

The data consumer

In the last section we created a calculation service. It runs within our own private BitTorrent network. The calculation service also looks up possible consumers. It uses the punch helper method to send UDP packets to them. The UDP packets will help the consumers to establish a connection. This way we can offer a service through our home router even behind a firewalled NAT.

In this section we create a data consumer. For learning purposes the data consumer is not firewalled in this article. However, there is an example in the Grenache UTP repository which implements a solution where both P2P parties can be firewalled.

For the data consumer we require both the RPC client and server. We put the code into a file called rpc_server_public.js:

Like in the previous section, we set up the link, server and let the server listen for connections:

We are going to use the RPC client, too. We have to start it:

In the data calculation service we were looking up data consumers on the network, to send them a UDP packet. Here we announce our consumer to the network, so it can be found by the calculation service.

You probably remember that the other service calls .punch(‘fibonacci_consumer’) every second. This looks up an announced fibonacci consumer on the network, and sends a UDP packet to the IP and port of the consumer.

In the next code section we will listen for those packets. With the UDP packet that we receive as part of the punch event, we get the public IP and port of the sender.

When we now send network data to the received address and IP, the home router routes it to the service running on the local network. This way we can set up an ad-hoc, P2P data connection with a server behind a NAT.

That’s all we need for the consumer – time to connect the different parts. On a side note, the code for the consumer is on GitHub, too. You can find it as part of the examples on GitHub: rpc_server_public.js.

 

Testing

After we have created our two services, we can run them on our local machine to see if they connect. With the two local grape instances running, we first run the rpc_server_public.js:

$ node rpc_server_public.js

 

Then we boot the consumer:

$ node rpc_server_public.js

 

If everything worked well, we get this output:

$ node rpc_server_public.js

listening on 51563
punch from { address: '127.0.0.1', family: 'IPv4', port: 52217, size: 2 }
punching back...
got data reply, sequence is:
[ 1, 1, 2, 3, 5, 8, 13, 21, 34, 55 ]

$ node rpc_server_behind_nat.js

listening on 52217
punch from { address: '127.0.0.1', family: 'IPv4', port: 51563, size: 2 }
received request, calculating & replying...

 

If everything works, we can stop the both servers and local Grape instances. In the next section we will connect from an external server into a NAT.

 

Usage behind NAT

We can now start the calculation service behind the NAT and connect from the external consumer to it. For this lets assume your public server has the IP 157.81.109.241.

Start a Grape instance on the public server:

DEBUG=* grape --dp 20001 --aph 30001 --bn '127.0.0.1:20002'

 

Then start a Grape instance locally on your machine, behind a NAT, and connect it to the other Grape:

DEBUG=* grape --dp 20002 --aph 30001 --bn '157.81.109.241:20001'

 

On the public server machine, start the consumer now:

node rpc_server_public.js

 

Start rpc_server_behind_nat.js on your local machine behind the router now. It looks up possible consumers and sends a UDP packet to the consumer. This establishes a temporary ad-hoc routing for the service running in the home network.

When the packet arrives at the public server, it will emit a punch event. The public server handles the event and gets the IP/port from the ad-hoc routing, kicking off the request for calculation. This is then routed to the server inside the local network for us – when it gets an answer, it logs the received result. 

You are now able to run servers behind a NAT!

 


 

In this article we showed the basics of UDP Holepunching. We created a data producer and a small consumer. The example works without an extra registry as connection helper. There exist more complex use cases where both consumer and producer are behind a NAT. We hope we were able to inspire you, and this knowledge is helpful for your own projects.

 

We use Grenache in production at Bitfinex. If you are interested in distributed systems and passionate about JavaScript, we are hiring!

The Bitfinex APIs are designed to allow complete access to the features provided by Bitfinex. Learn more about our API documentation here.

Stay up to date with Bitfinex on TwitterLinkedInTelegram, Facebook & YouTube

Tags:
No Comments

Sorry, the comment form is closed at this time.