Updated README

README.md

@@ -4,22 +4,16 @@ Platypush
 [![Build Status](https://ci.platypush.tech/status.svg)](https://ci.platypush.tech/latest.log)
 [![Documentation Status](https://ci.platypush.tech/docs/status.svg)](https://ci.platypush.tech/docs/latest.log)
 [![pip version](https://img.shields.io/pypi/v/platypush.svg?style=flat)](https://pypi.python.org/pypi/platypush/)
-[![License](https://img.shields.io/github/license/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/-/blob/master/LICENSE.txt)
+[![License](https://img.shields.io/github/license/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/src/branch/master/LICENSE.txt)
-[![Last Commit](https://img.shields.io/github/last-commit/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/-/commits/master/)
+[![Last Commit](https://img.shields.io/github/last-commit/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/commits/branch/master)
 [![Join chat on Matrix](https://img.shields.io/matrix/:platypush?server_fqdn=matrix.platypush.tech)](https://matrix.to/#/#platypush:matrix.platypush.tech)
-[![Contributions](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://git.platypush.tech/platypush/platypush/-/blob/master/CONTRIBUTING.md)
+[![Contributions](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://git.platypush.tech/platypush/platypush/src/branch/master/CONTRIBUTING.md)
 
 <!-- toc -->
 
-- [Architecture](#architecture)
+    + [Useful links](#useful-links)
-  * [Plugins](#plugins)
+- [Introduction](#introduction)
-  * [Actions](#actions)
+    + [What it can do](#what-it-can-do)
-  * [Backends](#backends)
-  * [Events](#events)
-  * [Hooks](#hooks)
-  * [Procedures](#procedures)
-  * [Cronjobs](#cronjobs)
-  * [The web interface](#the-web-interface)
 - [Installation](#installation)
   * [System installation](#system-installation)
     + [Install through `pip`](#install-through-pip)
@@ -31,17 +25,27 @@ Platypush
     + [Check the instructions reported in the documentation](#check-the-instructions-reported-in-the-documentation)
   * [Virtual environment installation](#virtual-environment-installation)
   * [Docker installation](#docker-installation)
+- [Architecture](#architecture)
+  * [Plugins](#plugins)
+  * [Actions](#actions)
+  * [Backends](#backends)
+  * [Events](#events)
+  * [Hooks](#hooks)
+  * [Procedures](#procedures)
+  * [Cronjobs](#cronjobs)
+  * [Entities](#entities)
+  * [The web interface](#the-web-interface)
 - [Mobile app](#mobile-app)
 - [Tests](#tests)
 - [Funding](#funding)
 
 <!-- tocstop -->
 
+#### Useful links
 - Recommended read: [**Getting started with Platypush**](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush).
 
-- The [blog](https://blog.platypush.tech) is in general a good place to get
+- The [blog](https://blog.platypush.tech) is a good place to get more insights
-  more insights on what you can build with it and inspiration about possible
+  and inspiration on what you can build.
-  usages.
 
 - The [wiki](https://git.platypush.tech/platypush/platypush/wiki) also
   contains many resources on getting started.
@@ -49,19 +53,19 @@ Platypush
 - Extensive documentation for all the available integrations and messages [is
   available](https://docs.platypush.tech/).
 
-- If you have issues/feature requests/enhancement ideas please [create an
+- If you have issues/feature requests/enhancements please [create an
-  issue](https://git.platypush.tech/platypush/platypush/-/issues).
+  issue](https://git.platypush.tech/platypush/platypush/issues).
-
-- A [Reddit channel](https://www.reddit.com/r/platypush) is also available for
-  more general questions.
 
 - A [Matrix instance](https://matrix.to/#/#platypush:matrix.platypush.tech) is
-  also available if you are looking for more interactive support.
+  available if you are looking for interactive support.
 
----
+- A [Reddit channel](https://www.reddit.com/r/platypush) is available for
+  general questions.
 
-Platypush is a general-purpose extensible platform for automation and
+## Introduction
-integration across multiple services and devices.
+
+Platypush is a general-purpose extensible platform for automation across
+multiple services and devices.
 
 It enables users to create their own self-hosted pieces of automation based on
 events (*if this happens then do that*)
@@ -70,16 +74,15 @@ everything you need to visualize and control under one roof.
 
 It takes some concepts from [IFTTT](https://ifttt.com),
 [Tasker](https://tasker.joaoapps.com/), [Microsoft
-Flow](https://flow.microsoft.com), [PushBullet](https://pushbullet.com) and
+Flow](https://flow.microsoft.com) and [Home
-[Home Assistant](https://www.home-assistant.io/) to provide an environment
+Assistant](https://www.home-assistant.io/) to provide an environment where the
-where the user can easily connect things together.
+user can easily connect things together.
 
-Its ideal home is a single-board computer like a RaspberryPi that you can
+It's built with compatibility and flexibility in mind, and it can easily run on
-configure to orchestrate any home automation and cloud automation in your own
+any device that can run a Python interpreter - from a Raspberry Pi zero, to an
-living room or garage, but it can easily run on any device that can run a
+old smartphone, to a beefy server.
-Python interpreter, and the bar for the hardware requirements is very low as
+
-well - I use it to run pieces of automation on devices as powerful as a
+#### What it can do
-RaspberryPi Zero or an old Nokia N900 with Linux.
 
 You can use Platypush to do things like:
 
@@ -110,13 +113,198 @@ You can use Platypush to do things like:
 - [Create a custom single hub for Zigbee and Z-Wave smart devices](https://blog.platypush.tech/article/Transform-a-RaspberryPi-into-a-universal-Zigbee-and-Z-Wave-bridge)
 - Build your own web dashboard with calendar, weather, news and music controls
   (basically, anything that has a Platypush web widget)
-- ...and much more (basically, anything that comes with a [Platypush plugin](https://docs.platypush.tech/en/latest/plugins.html))
+- ...and much more (basically, anything that comes with a [Platypush plugin](https://docs.platypush.tech))
+
+## Installation
+
+### System installation
+
+Platypush uses Redis to deliver and store requests and temporary messages:
+
+```yaml
+# Example for Debian-based distributions
+[sudo] apt-get install redis-server
+
+# Enable and start the service
+[sudo] systemctl enable redis
+[sudo] systemctl start redis
+```
+
+#### Install through `pip`
+
+```shell
+[sudo] pip3 install platypush
+```
+
+#### Install through a system package manager
+
+Note: currently only Arch Linux and derived distributions are supported.
+
+You can either install the
+[`platypush`](https://aur.archlinux.org/packages/platypush) package (for the
+latest stable version) or the
+[`platypush-git`](https://aur.archlinux.org/packages/platypush-git) package
+(for the latest git version) through your favourite AUR package manager. For
+example, using `yay`:
+
+```shell
+yay platypush
+# Or
+yay platypush-git
+```
+
+The Arch Linux packages on AUR are automatically updated upon new git commits
+or tags.
+
+#### Install from sources
+
+```shell
+git clone https://git.platypush.tech/platypush/platypush.git
+cd platypush
+[sudo] pip install .
+# Or
+[sudo] python3 setup.py install
+```
+
+### Installing the dependencies for your extensions
+
+After installing the base platform, you may want to check the dependencies and
+configuration required by the extensions that you wish to use. There are a few
+ways to check the dependencies required by an extension:
+
+#### Install via `extras` name
+
+All the extensions that require extra dependencies are listed in the
+[`extras_require` section under
+`setup.py`](https://git.platypush.tech/platypush/platypush/src/branch/master/setup.py#L84).
+
+#### Install via `manifest.yaml`
+
+All the plugins and backends have a `manifest.yaml` file in their source folder.
+Any extra dependencies are listed there
+
+If you followed the `extras` or `manifest.yaml` way to discover the
+dependencies, then you can install them in two ways:
+
+1. `pip` installation:
+
+```shell
+[sudo] pip3 install 'platypush[extra1,extra2,extra3]'
+```
+
+2. Sources installation:
+
+```shell
+cd $DIR_TO_PLATYPUSH
+[sudo] pip3 install '.[extra1,extra2,extra3]'
+```
+
+#### Check the instructions reported in the documentation
+
+If you follow this route then simply run the commands listed in the
+[plugin/backend documentation](https://docs.platypush.tech) to get the
+dependencies installed.
+
+After installing the dependencies, create a configuration file under
+`~/.config/platypush/config.yaml` (the application can load the configuration
+from another location through the `-c` option) containing the configuration of
+the backend and plugins that you want to use, and add any hooks and procedures
+for your use case.
+
+You can then start the service by simply running:
+
+```shell
+platypush
+```
+
+It's advised to run it as a systemd service though - simply copy the provided
+[`.service`
+file](https://git.platypush.tech/platypush/platypush/src/branch/master/examples/systemd/platypush.service)
+to `~/.config/systemd/user`, check if the path of `platypush` matches the path
+where it's installed on your system, and start the service via `systemctl`:
+
+```shell
+systemctl --user start platypush
+```
+
+### Virtual environment installation
+
+Platypush provides a script named `platyvenv` that can parse a `config.yaml`
+and automatically create a virtual environment (under
+`~/.local/share/platypush/venv/<device_id>`) with all the dependencies required
+by the configured integrations.
+
+1. Create the environment from a configuration file:
+
+    ```shell
+    platyvenv build -c /path/to/config.yaml
+    ```
+
+2. Start the service from the virtual environment:
+
+    ```shell
+        # device_id matches either the hostname or the device_id in config.yaml
+    platyvenv start device_id
+    ```
+
+3. Stop the instance:
+
+    ```shell
+    platyvenv stop device_id
+    ```
+
+4. Remove the instance:
+
+    ```shell
+    platyvenv rm device_id
+    ```
+
+[Wiki instructions](https://git.platypush.tech/platypush/platypush/wiki/Run-platypush-in-a-virtual-environment)
+
+### Docker installation
+
+You can also install Platypush in a container - the application provides a
+script named `platydock` that automatically creates a container instance from a
+`config.yaml`:
+
+1. Create the container from a configuration file:
+
+    ```shell
+    platydock build -c /path/to/config.yaml
+    ```
+
+2. Start the container:
+
+    ```shell
+        # device_id matches either the hostname or the device_id in config.yaml
+    platydock start device_id
+    ```
+
+3. Stop the instance:
+
+    ```shell
+    platydock stop device_id
+    ```
+
+4. Remove the instance:
+
+    ```shell
+    platydock rm device_id
+    ```
+
+Note that both the virtual environment and Docker container option offer the
+possibility to include extra YAML configuration files in the main `config.yaml`
+through the `include` directive (as long as they are in the same folder as the
+main `config.yaml`), as well as external Python scripts in a `scripts`
+directory in the same folder as the `config.yaml`.
+
+[Wiki instructions](https://git.platypush.tech/platypush/platypush/wiki/Run-platypush-in-a-container)
 
 ## Architecture
 
 The architecture of Platypush consists of a few simple pieces, orchestrated by
 a configuration file stored by default under
-[`~/.config/platypush/config.yaml`](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/config.yaml):
+[`~/.config/platypush/config.yaml`](https://git.platypush.tech/platypush/platypush/src/branch/master/examples/conf/config.yaml):
 
 ### Plugins
 
@@ -181,7 +369,7 @@ as simple JSON messages:
 
 [Full list](https://docs.platypush.tech/en/latest/backends.html)
 
-They are background services that either listen for messages on channels (like
+They are background services that listen for messages on channels (like
 an [HTTP
 backend](https://docs.platypush.tech/en/latest/platypush/backend/http.html), an
 [MQTT
@@ -189,22 +377,6 @@ instance](https://docs.platypush.tech/en/latest/platypush/backend/mqtt.html), a
 [Kafka
 instance](https://docs.platypush.tech/en/latest/platypush/backend/kafka.html),
 [Pushbullet](https://docs.platypush.tech/en/latest/platypush/backend/pushbullet.html)
-etc.) or monitor a device or a service for events (like a
-[sensor](https://docs.platypush.tech/en/latest/platypush/backend/sensor.html),
-a custom [voice
-assistant](https://docs.platypush.tech/en/latest/platypush/backend/assistant.google.html),
-a bridge running on a
-[Zigbee](https://docs.platypush.tech/en/latest/platypush/backend/zigbee.mqtt.html)
-or
-[Z-Wave](https://docs.platypush.tech/en/latest/platypush/backend/zwave.html),
-an [NFC card
-reader](https://docs.platypush.tech/en/latest/platypush/backend/nfc.html), a
-[MIDI
-device](https://docs.platypush.tech/en/latest/platypush/backend/midi.html), a
-[Telegram
-channel](https://docs.platypush.tech/en/latest/platypush/backend/chat.telegram.html),
-a [Bluetooth
-scanner](https://docs.platypush.tech/en/latest/platypush/backend/bluetooth.scanner.ble.html)
 etc.).
 
 If a backend supports the execution of requests (e.g. HTTP, MQTT, Kafka,
@@ -220,7 +392,6 @@ curl -XPOST -H 'Content-Type: application/json' -d '
 }' http://host:8008/auth
 
 # Execute a request
-
 curl -XPOST -H 'Content-Type: application/json' \
     -H "Authorization: Bearer $YOUR_TOKEN" -d '
 {
@@ -265,7 +436,7 @@ triggered. Hooks are the glue that connects events to actions, exposing a
 paradigm similar to IFTTT (_if a certain event happens then run these
 actions_). They can declared as:
 
-- Sections of the [`config.yaml`](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/config.yaml).
+- Sections of the [`config.yaml`](https://git.platypush.tech/platypush/platypush/src/branch/master/examples/conf/config.yaml).
   Example:
 
 ```yaml
@@ -289,7 +460,7 @@ event.hook.SearchSongVoiceCommand:
 
 - Stand-alone Python scripts stored under `~/.config/platypush/scripts` and
   will be dynamically imported at start time.
-  [Example](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/hook.py):
+  [Example](https://git.platypush.tech/platypush/platypush/src/branch/master/examples/conf/hook.py):
 
 ```python
 from platypush.event.hook import hook
@@ -413,6 +584,37 @@ def check_bt_device(**context):
     # on_device_off logic here
 ```
 
+### Entities
+
+Entities are a fundamental building block of Platypush. Most of the
+integrations will store their state or connected devices in the form of
+entities - e.g. the sensors detected by the Z-Wave/Zigbee/Bluetooth
+integration, or the lights connected to a Hue bridge, or your cloud nodes, or
+your custom Arduino/ESP machinery, and so on.
+
+Entities provide a consistent interface to interact with your integrations
+regardless of their type and the plugin that handles them. For instance, all
+temperature sensors will expose the same interface, regardless if they are
+Bluetooth or Zigbee sensors, and all the media plugins will expose the same
+interface, regardless if they manage Chromecasts, Kodi, Plex, Jellyfin or a
+local VLC player.
+
+Once you enable the HTTP backend and a few integrations that export entities
+and register a user, you can query the detected entities via:
+
+```shell
+curl -XPOST -H 'Content-Type: application/json' \
+    -H "Authorization: Bearer $YOUR_TOKEN" \
+    -d '{"type":"request", "action":"entities.get"}' \
+    http://localhost:8008/execute
+```
+
+All the entities expose the same interface and can be manipulated through the
+same API. Also, when an entity is updated it always emits an
+[`EntityUpdateEvent`](https://docs.platypush.tech/platypush/events/entities.html#platypush.message.event.entities.EntityUpdateEvent),
+so you can easily create hooks that react to these events and act on multiple
+types of entities.
+
 ### The web interface
 
 If
@@ -425,194 +627,9 @@ music controls and search, media library and torrent management, lights,
 Zigbee/Z-Wave devices and so on. The UI is responsive and mobile-friendly.
 
 The web service also provides means for the user to create [custom
-dashboards](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/dashboard.xml)
+dashboards](https://git.platypush.tech/platypush/platypush/src/branch/master/examples/conf/dashboard.xml)
 that can be used to show information from multiple sources on a large screen.
 
-## Installation
-
-### System installation
-
-Platypush uses Redis to deliver and store requests and temporary messages:
-
-```yaml
-    # Example for Debian-based distributions
-[sudo] apt-get install redis-server
-
-    # Enable and start the service
-[sudo] systemctl enable redis
-[sudo] systemctl start redis
-```
-
-#### Install through `pip`
-
-```shell
-[sudo] pip3 install platypush
-```
-
-#### Install through a system package manager
-
-Note: currently only Arch Linux and derived distributions are supported.
-
-You can either install the
-[`platypush`](https://aur.archlinux.org/packages/platypush) package (for the
-latest stable version) or the
-[`platypush-git`](https://aur.archlinux.org/packages/platypush-git) package
-(for the latest git version) through your favourite AUR package manager. For
-example, using `yay`:
-
-```shell
-yay platypush
-# Or
-yay platypush-git
-```
-
-The Arch Linux packages on AUR are automatically updated upon new git commits
-or tags.
-
-#### Install from sources
-
-```shell
-git clone https://git.platypush.tech/platypush/platypush.git
-cd platypush
-[sudo] pip install .
-    # Or
-[sudo] python3 setup.py install
-```
-
-### Installing the dependencies for your extensions
-
-After installing the base platform, you may want to check the dependencies and
-configuration required by the extensions that you wish to use. There are a few
-ways to check the dependencies required by an extension:
-
-#### Install via `extras` name
-
-All the extensions that require extra dependencies are listed in the
-[`extras_require` section under
-`setup.py`](https://git.platypush.tech/platypush/platypush/-/blob/master/setup.py#L72).
-
-#### Install via `manifest.yaml`
-
-All the plugins and backends have a `manifest.yaml` file in their source folder.
-Any extra dependencies are listed there
-
-If you followed the `extras` or `manifest.yaml` way to discover the
-dependencies, then you can install them in two ways:
-
-1. `pip` installation:
-
-```shell
-[sudo] pip3 install 'platypush[extra1,extra2,extra3]'
-```
-
-2. Sources installation:
-
-```shell
-cd $DIR_TO_PLATYPUSH
-[sudo] pip3 install '.[extra1,extra2,extra3]'
-```
-
-#### Check the instructions reported in the documentation
-
-If you follow this route then simply run the commands listed in the
-[plugin/backend documentation](https://docs.platypush.tech) to get the
-dependencies installed.
-
-After installing the dependencies, create a configuration file under
-`~/.config/platypush/config.yaml` (the application can load the configuration
-from another location through the `-c` option) containing the configuration of
-the backend and plugins that you want to use, and add any hooks and procedures
-for your use case.
-
-You can then start the service by simply running:
-
-```shell
-platypush
-```
-
-It's advised to run it as a systemd service though - simply copy the provided
-[`.service`
-file](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/systemd/platypush.service)
-to `~/.config/systemd/user`, check if the path of `platypush` matches the path
-where it's installed on your system, and start the service via `systemctl`:
-
-```shell
-systemctl --user start platypush
-```
-
-### Virtual environment installation
-
-Platypush provides a script named `platyvenv` that can parse a `config.yaml`
-and automatically create a virtual environment (under
-`~/.local/share/platypush/venv/<device_id>`) with all the dependencies required
-by the configured integrations.
-
-1. Create the environment from a configuration file:
-
-    ```shell
-    platyvenv build -c /path/to/config.yaml
-    ```
-
-2. Start the service from the virtual environment:
-
-    ```shell
-        # device_id matches either the hostname or the device_id in config.yaml
-    platyvenv start device_id
-    ```
-
-3. Stop the instance:
-
-    ```shell
-    platyvenv stop device_id
-    ```
-
-4. Remove the instance:
-
-    ```shell
-    platyvenv rm device_id
-    ```
-
-[Wiki instructions](https://git.platypush.tech/platypush/platypush/wiki/Run-platypush-in-a-virtual-environment)
-
-### Docker installation
-
-You can also install Platypush in a container - the application provides a
-script named `platydock` that automatically creates a container instance from a
-`config.yaml`:
-
-1. Create the container from a configuration file:
-
-    ```shell
-    platydock build -c /path/to/config.yaml
-    ```
-
-2. Start the container:
-
-    ```shell
-        # device_id matches either the hostname or the device_id in config.yaml
-    platydock start device_id
-    ```
-
-3. Stop the instance:
-
-    ```shell
-    platydock stop device_id
-    ```
-
-4. Remove the instance:
-
-    ```shell
-    platydock rm device_id
-    ```
-
-Note that both the virtual environment and Docker container option offer the
-possibility to include extra YAML configuration files in the main `config.yaml`
-through the `include` directive (as long as they are in the same folder as the
-main `config.yaml`), as well as external Python scripts in a `scripts`
-directory in the same folder as the `config.yaml`.
-
-[Wiki instructions](https://git.platypush.tech/platypush/platypush/wiki/Run-platypush-in-a-container)
-
 ## Mobile app
 
 An [official Android