[//]: # (title: How to build your personal infrastructure for data collection and visualization)
[//]: # (description: Use Grafana, PostgreSQL, Mosquitto and Platypush to collect data points on your infrastructure and be the real owner of your own data.)
Moreover, many hubs can usually talk only to their smartphone app, adding an app hell to the bridge hell.
One of the most sought goals nowadays when it comes to home automation is (or it should be) to find consistent ways to
communicate with as many devices as possible using the same tool, the same box and the same interface.
I’ve recently worked on achieving this goal in platypush. Other platforms have tackled the issue too (from HASS.io to
Openhab) but, in my opinion, they all still require a certain degree of user configuration (like specifying what each
device is and what it’s supposed to do) that in my opinion shouldn’t be required. In this article, we’ll go through what
Zigbee and Z-Wave actually are, what you need to set up a universal bridge for them using just a Raspberry Pi and a USB
dongle, and how to permanently move your existing smart bridges to the storage room.
## Zigbee vs. Z-Wave
Zigbee and Z-Wave have arguably risen to become the dominant communication protocols in the smart home and IoT
industries. They share some common features, such as:
- They are better suited for low-powered devices. Wi-Fi is a relatively power-hungry protocol, needs to deal with packet
losses, relies on a centralised topology, and isn’t ideal for communication with devices that can be asleep for a long
time because of its high overhead on reconnection. Zigbee and Z-Wave have instead been designed to work well with
low-powered devices, be low-latency and optimized for sending less bytes on the network. Some Z-Wave or Zigbee devices
can theoretically run on the same battery for a couple of years; a similar Wi-Fi device won’t usually last more than a
couple of days.
- While Wi-Fi networks are usually configured using a star-based topology (one access point/router in the middle and all
the devices connect to it), Zigbee and Z-Wave have been designed to support more flexible topologies. Each device can
be both a client and a repeater on the network, each can directly connect to other devices and expand the network with
no need to introduce further routers or access points. This allows the creation of scalable *mesh* networks that can
include devices that would otherwise be out of range.
When it comes to the differences:
- **Number of devices**: Z-Wave networks are limited to 232 devices, while Zigbee supports up to 65k devices.
- **Maximum hops**: Z-Wave supports up to four hops between a device and the network controller, while Zigbee has
theoretically no such limitations.
- **Operating range**: Most Zigwave devices operate around the 2.4 GHz spectrum — although some specific devices also
use 784 MHz in China, 868 MHz in Europe and 915 MHz in the US and Australia. Z-Wave devices instead operate around the
850–900 MHz range (at 868.42 MHz in Europe, at 908.42 MHz in North America and other frequencies in other countries
depending on their regulations).
- **Signal range**: A thumb rule when it comes to electromagnetic waves is that, at fixed transmission power, the higher
the frequency the lower the range. It means that Z-Wave devices, which operate at lower frequencies, have usually a
longer range compared to Zigbee. An unobstructed Z-Wave signal can travel up to 100 meters outdoor, although a common
guideline is 30 meters indoor for unobstructed signals and 15 meters if there are walls in between. The indoor range
for Zigbee is instead usually limited around 12 meters. However, Zigbee networks theoretically support an unlimited
number of hops between the nodes and the controller, so the range can be easily extended by adding more nodes to the
network.
- **Signal reliability**: Z-Wave has a longer range than Zigbee. It means that, at a fixed distance, its signal is
usually more reliable. Moreover, Zigbee operates in the relatively “quieter” 800–900 MHz spectrum, therefore it
doesn’t have to share the crowded 2.4 GHz spectrum with Wi-Fi and Bluetooth devices. That overall leads to more
reliable communication.
- **Ownership and protocol**: Z-Wave is a proprietary technology developed and maintained by Sigma Designs. Sigma (
acquired by Silicon Labs in 2018) owns the protocol, licenses the compatible devices and runs the Z-Wave Alliance, and
grants certifications to the devices that comply with the standard. Its selling point is, in my opinion, the strong
enforcement of a shared protocol both on hardware and software side. Each sensor, configuration value or switch is
defined by a variable type, a range, a read-only vs. read-write policy and a structured representation that applies to
all the compliant devices. That makes it very easy to develop consistent interfaces that can comprehensively represent
and control any device as long as it speaks Z-Wave, even if the implementation doesn’t know exactly which device it
is. Zigbee, on the other hand, is an open standard maintained by the Zigbee Alliance. It also has a certification
process in place, but that comes in two parts — one part certifies the hardware, the other certifies the software. It
is possible to produce Zigbee certified hardware even if the software isn’t certified nor compatible with other
clients. While this makes the protocol much more open than Z-Wave, it also makes the task of writing an all-purpose
interfacing software much harder, as different devices may name their properties following different conventions.
That should cover most of the knowledge you need when it comes to the theory of Zigbee and Z-Wave. You’ll find many Zigbee and Z-Wave compatible smart devices around. Some examples of Zigbee devices are the Philips Hue and Ikea smart bulbs, Honeywell thermostats, Belkin smart plugs and bulbs, Bosch sensors and Osram products. Z-Wave includes around 65,000 compatible devices out there, including many garage doors, presence and temperature sensors, thermostats, dimmers, remote controls, smoke detectors and so on.
That should cover most of the knowledge you need when it comes to the theory of Zigbee and Z-Wave. You’ll find many
Zigbee and Z-Wave compatible smart devices around. Some examples of Zigbee devices are the Philips Hue and Ikea smart
bulbs, Honeywell thermostats, Belkin smart plugs and bulbs, Bosch sensors and Osram products. Z-Wave includes around
65,000 compatible devices out there, including many garage doors, presence and temperature sensors, thermostats,
dimmers, remote controls, smoke detectors and so on.
## Hardware and software
We’ll use a RaspberryPi in the following examples as a DIY bridge (any model and any distribution should work fine), and
Platypush as a home automation platform that also runs the Zigbee and Z-Wave integrations.
## Making your own Zigbee bridge
On the hardware side you’ll need:
- One or more Zigbee compatible devices, like Philips Hue, Ikea or Osram lights or Belkin switches.
- A Zigbee-to-USB adapter/sniffer.
The [CC2531](https://www.amazon.com/Gowoops-Protocol-Analyzer-Sniffer-802-15-4/dp/B07P5LY7Z6/ref=sr_1_6?keywords=zigbee+usb+adapter&link_code=qs&qid=1582640215&sr=8-6)
is one of the most popular options out there.
- A [Zigbee debugger+adapter cable](https://www.amazon.com/Debugger-Downloader-Programmed-Simulation-Programmer/dp/B07T13JX32/ref=pd_sbs_147_9?_encoding=UTF8&pd_rd_i=B07T13JX32&pd_rd_r=d44d5902-3147-4af9-ba8a-de7d31d4d1e0&pd_rd_w=j45M7&pd_rd_wg=TNdkl&pf_rd_p=7cd8f929-4345-4bf2-a554-7d7588b3dd5f&pf_rd_r=KRHVT3D830174F7FCB05&psc=1&refRID=KRHVT3D830174F7FCB05),
that you’ll need in order to flash the firmware on the dongle.
- A RaspberryPi or similar device (any model and distribution should work fine), or any computer that you want to use as
a bridge.
On the software side:
- Platypush uses [`zigbee2mqtt`](https://www.zigbee2mqtt.io/) as a backend to interact with the Zigbee dongle. Install
the zigbee2mqtt firmware on the dongle by following the instructions (for Windows, macOS and Linux)
on [their website](https://www.zigbee2mqtt.io/getting_started/flashing_the_cc2531.html).
You can check a list of the compatible devices [here](https://www.zigbee2mqtt.io/information/supported_devices.html).
- Install, start and enable an MQTT instance on the local machine, if you don’t have a server already running in your
network. If you’re running Debian/Ubuntu/Raspbian and want to use Mosquitto, for example:
```shell
[sudo] apt-get install mosquitto
[sudo] systemctl start mosquitto.service
[sudo] systemctl enable mosquitto.service
```
- Install zigbee2mqtt:
```shell
[sudo] apt-get install nodejs git make g++ gcc
git clone https://github.com/Koenkk/zigbee2mqtt
cd zigbee2mqtt
npm install
# Change mqtt.server and serial.port to respectively
# match your MQTT server and USB dongle device.
vi data/configuration
npm start
```
You can also make a systemd service out of it:
```yaml
[Unit]
Description=zigbee2mqtt
After=network.target
[Service]
ExecStart=/usr/bin/npm start
WorkingDirectory=/path/to/zigbee2mqtt
StandardOutput=inherit
StandardError=inherit
Restart=always
[Install]
WantedBy=multi-user.target
```
- Note that the zigbee2mqtt configuration file also includes a `permit_join` option. Set this to true while you’re
pairing your Zigbee devices for the first time, and set it to false afterwards to prevent other devices from
accidentally or malignantly join the network — you can always temporarily allow joins later.
- Once started and in `permit_join` mode, you can start pairing Zigbee devices to your new network. That is usually done
by doing a factory reset of the device. The procedure varies with the device: Philips Hue bulbs, for example, can be
reset either from the app (if they are paired to a bridge) or by pressing the ON/OFF buttons of a Hue dimmer
simultaneously for 10 seconds while keeping the dimmer within 10 cm from the lightbulb. Other Zigbee devices may
include a reset button instead.
- Once a Zigbee device joins the network, the zigbee2mqtt logs should show a trace like the following:
```
Successfully interviewed '0x00158d0001dc126a', device has successfully been paired
```
- Install Redis and Platypush with the Zigbee, HTTP and MQTT extensions:
```shell
[sudo] apt-get install redis-server
[sudo] systemctl start redis.service
[sudo] systemctl enable redis.service
pip install 'platypush[zigbee,http,mqtt]'
```
- Edit your `~/.config/platypush/config.yaml` file to enable the Zigbee and HTTP services:
```yaml
backend.http:
port: 8008
zigbee.mqtt:
host: localhost
backend.zigbee.mqtt:
enabled: true
```
- Start Platypush (either by running `platypush` or starting the `platypush.service` systemd service).
- Open `http://your-raspberry:8008/` in a browser. The web panel should include the Zigbee icon in the navigation bar,
you’ll be able to control your network from there.