A more robust cron start logic

If may happen (usually because of a race condition) that a cronjob has
already been started, but it hasn't yet changed its status from IDLE to
RUNNING when the scheduler checks it.

This fix guards the application against such events. If they occur, we
should just report them and move on, not terminate the whole scheduler.

.pre-commit-config.yaml

@@ -0,0 +1,32 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.1.0
+    hooks:
+    # -   id: trailing-whitespace
+    # -   id: end-of-file-fixer
+    -   id: check-yaml
+    -   id: check-json
+    -   id: check-xml
+    -   id: check-symlinks
+    -   id: check-added-large-files
+
+-   repo: https://github.com/Lucas-C/pre-commit-hooks-nodejs
+    rev: v1.1.2
+    hooks:
+    -   id: markdown-toc
+
+- repo: https://github.com/pycqa/flake8
+  rev: 4.0.1
+  hooks:
+    - id: flake8
+      additional_dependencies:
+        - flake8-bugbear
+        - flake8-comprehensions
+        - flake8-simplify
+
+-   repo: https://github.com/psf/black
+    rev: 22.3.0
+    hooks:
+    -   id: black

CHANGELOG.md

@@ -1,7 +1,105 @@
 # Changelog
 
 All notable changes to this project will be documented in this file.
-Given the high speed of development in the first phase, changes are being reported only starting from v0.20.2.
+Given the high speed of development in the first phase, changes are being
+reported only starting from v0.20.2.
+
+## [Unreleased]
+
+### Added
+
+- Added [Wallabag integration](https://git.platypush.tech/platypush/platypush/issues/224).
+- Added [Mimic3 TTS integration](https://git.platypush.tech/platypush/platypush/issues/226).
+
+## [0.23.6] - 2022-09-19
+
+### Fixed
+
+- Fixed album_id and list of tracks on `music.tidal.get_album`.
+
+## [0.23.5] - 2022-09-18
+
+### Added
+
+- Added support for web hooks returning their hook method responses back to the
+  HTTP client.
+
+- Added [Tidal integration](https://git.platypush.tech/platypush/platypush/pulls/223)
+
+- Added support for [OPML
+  subscriptions](https://git.platypush.tech/platypush/platypush/pulls/220) to
+  the `rss` plugin.
+
+- Better support for bulk database operations on the `db` plugin.
+
+### Fixed
+
+- Now supporting YAML sections with empty configurations.
+
+## [0.23.4] - 2022-08-28
+
+### Added
+
+- Added `matrix` integration
+  ([issue](https://git.platypush.tech/platypush/platypush/issues/2),
+  [PR](https://git.platypush.tech/platypush/platypush/pulls/217)).
+
+### Changed
+
+- Removed `clipboard` backend. Enabling the `clipboard` plugin will also enable
+  clipboard monitoring, with no need for an additional backend.
+
+## [0.23.3] - 2022-06-01
+
+### Added
+
+- Added `ntfy` integration (see #219).
+- Support for a default `config.yaml` if one isn't specified in the default
+  locations.
+
+### Changed
+
+- The HTTP server dependencies are now marked as required, since the default
+  `config.yaml` will have the HTTP backend enabled by default in order to allow
+  the creation of a first user.
+- Updated Vue.js frontend dependencies to the latest version.
+- Removed bulma from the frontend dependencies, making the frontend much
+  lighter and loading times much faster.
+- Other UI improvements.
+
+### Fixed
+
+- More reliable cronjobs in case of DST change or any clock changes in general
+  (see #217).
+- Fixed `--redis-queue` argument.
+
+## [0.23.2] - 2022-03-27
+
+### Added
+
+- Support for asynchronous events over GPIO PINs. It is now possible to specify
+  a list of `monitored_pins` in the [`gpio`
+  plugin](https://git.platypush.tech/platypush/platypush/-/blob/master/platypush/plugins/gpio/__init__.py)
+  configuration. A change in the value on those GPIO PINs (caused by e.g. a
+  button, a binary sensor or a probe) will trigger a
+  `platypush.message.event.gpio.GPIOEvent` that you can use in your automation
+  scripts.
+
+- Simplified script API to interact with platform variables
+  (closes [#206](https://git.platypush.tech/platypush/platypush/-/issues/206)).
+  You can now read and write stored variables in your Platypush scripts through
+  a much more intuitive interface compared to explicitly using the `variable`
+  plugin explicitly:
+
+```python
+from platypush.context import Variable
+
+# ...
+
+my_var = Variable.get('my_var')
+my_var = int(my_var) + 1
+Variable.set(my_var=my_var)
+```
 
 ## [0.23.0] - 2022-03-01
 
@@ -139,7 +237,7 @@ Given the high speed of development in the first phase, changes are being report
   operate instead of pydoc strings conventions or `config.yaml` conventions.
 
 - `platyvenv start`  now starts the environment process synchronously and it prints
-  stdout/stderr instead of redirecting it to the logs dir (previous behaviour: 
+  stdout/stderr instead of redirecting it to the logs dir (previous behaviour:
   `platyvenv start` used to start the process asynchronously and the logs were stored
   to `~/.local/share/platypush/venv/<env>/logs/<stdout|stderr>.log`).
 
@@ -179,17 +277,17 @@ Given the high speed of development in the first phase, changes are being report
 
 - Added `switchbot` plugin to interact with Switchbot devices over the cloud API instead of
   directly accessing the device's Bluetooth interface.
-  
+
 - Added `marshmallow` dependency - it will be used from now own to dump and document schemas
   and responses instead of the currently mixed approach with `Response` objects and plain
   dictionaries and lists.
-  
+
 - Support for custom MQTT timeout on all the `zwavejs2mqtt` calls.
 
 - Added generic joystick backend `backend.joystick.jstest` which uses `jstest` from the
   standard `joystick` system package to read the state of joysticks not compatible with
   `python-inputs`.
-  
+
 - Added PWM PCA9685 plugin.
 
 - Added Linux native joystick plugin, ``backend.joystick.linux``, for the cases where
@@ -246,7 +344,7 @@ Given the high speed of development in the first phase, changes are being report
 
 - Added `<Camera>` dashboard widget.
 
-- Added support for custom dashboard widgets with customized (see https://git.platypush.tech/platypush/platypush/-/wikis/Backends#creating-custom-widgets).
+- Added support for custom dashboard widgets with customized (see https://git.platypush.tech/platypush/platypush/wiki/Backends#creating-custom-widgets).
 
 - Added support for controls on `music.mpd` dashboard widget.
 
@@ -293,7 +391,7 @@ Given the high speed of development in the first phase, changes are being report
 
 - Added support for a static list of devices to actively scan to the `bluetooth.scanner` backend
   (see [#174](https://git.platypush.tech/platypush/platypush/-/issues/174)).
-  
+
 - Added `weather.openweathermap` plugin and backend, which replaces `weather.darksky`, since the
   Darksky API will be completely shut down by the end of 2021.
 
@@ -301,14 +399,14 @@ Given the high speed of development in the first phase, changes are being report
 
 - Cron expressions should adhere to the UNIX cronjob standard and use the machine local time,
   not UTC, as a reference (closes [#173](https://git.platypush.tech/platypush/platypush/-/issues/173)).
-  
+
 - Better management of Z-Wave values types from the UI.
 
 - Disable logging for `ZwaveValueEvent` events - they tend to be very verbose and
   can impact the performance on slower devices. They will still be published to the
   websocket clients though, so you can still debug Z-Wave values issues from the browser
   developer console (enable debug traces).
-  
+
 - Added suffix to the `zigbee.mqtt` backend default `client_id` to prevent clashes with
   the default `mqtt` backend `client_id`.
 

CONTRIBUTING.md

@@ -34,6 +34,6 @@ Guidelines:
     an `extras_require` entry.
   - In the plugin/backend class pydoc string.
   - In the `manifest.yaml` - refer to the Wiki (how to write
-    [plugins](https://git.platypush.tech/platypush/platypush/-/wikis/writing-your-own-plugins)
+    [plugins](https://git.platypush.tech/platypush/platypush/wiki/Writing-your-own-plugins)
-    and [backends](https://git.platypush.tech/platypush/platypush/-/wikis/writing-your-own-backends))
+    and [backends](https://git.platypush.tech/platypush/platypush/wiki/Writing-your-own-backends))
     for examples on how to write an extension manifest file.

MANIFEST.in

@@ -1,3 +1,4 @@
 recursive-include platypush/backend/http/webapp/dist *
 include platypush/plugins/http/webpage/mercury-parser.js
+include platypush/config/*.yaml
 global-include manifest.yaml

README.md

@@ -6,41 +6,82 @@ Platypush
 [![pip version](https://img.shields.io/pypi/v/platypush.svg?style=flat)](https://pypi.python.org/pypi/platypush/)
 [![License](https://img.shields.io/github/license/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/-/blob/master/LICENSE.txt)
 [![Last Commit](https://img.shields.io/github/last-commit/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/-/commits/master/)
+[![Join chat on Matrix](https://img.shields.io/matrix/:platypush?server_fqdn=matrix.platypush.tech)](https://matrix.to/#/#platypush:matrix.platypush.tech)
 [![Contributions](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://git.platypush.tech/platypush/platypush/-/blob/master/CONTRIBUTING.md)
 [![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/BlackLight/platypush.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/BlackLight/platypush/context:python)
 [![Language grade: JavaScript](https://img.shields.io/lgtm/grade/javascript/g/BlackLight/platypush.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/BlackLight/platypush/context:javascript)
 
-[[__TOC__]]
+<!-- toc -->
+
+- [Architecture](#architecture)
+  * [Plugins](#plugins)
+  * [Actions](#actions)
+  * [Backends](#backends)
+  * [Events](#events)
+  * [Hooks](#hooks)
+  * [Procedures](#procedures)
+  * [Cronjobs](#cronjobs)
+  * [The web interface](#the-web-interface)
+- [Installation](#installation)
+  * [System installation](#system-installation)
+    + [Install through `pip`](#install-through-pip)
+    + [Install through a system package manager](#install-through-a-system-package-manager)
+    + [Install from sources](#install-from-sources)
+  * [Installing the dependencies for your extensions](#installing-the-dependencies-for-your-extensions)
+    + [Install via `extras` name](#install-via-extras-name)
+    + [Install via `manifest.yaml`](#install-via-manifestyaml)
+    + [Check the instructions reported in the documentation](#check-the-instructions-reported-in-the-documentation)
+  * [Virtual environment installation](#virtual-environment-installation)
+  * [Docker installation](#docker-installation)
+- [Mobile app](#mobile-app)
+- [Tests](#tests)
+- [Funding](#funding)
+
+<!-- tocstop -->
 
 - Recommended read: [**Getting started with Platypush**](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush).
 
-- The [blog](https://blog.platypush.tech) is in general a good place to get more insights on what you can build with it and inspiration about possible usages.
+- The [blog](https://blog.platypush.tech) is in general a good place to get
+  more insights on what you can build with it and inspiration about possible
+  usages.
 
-- The [wiki](https://git.platypush.tech/platypush/platypush/-/wikis/home) also contains many resources on getting started.
+- The [wiki](https://git.platypush.tech/platypush/platypush/wiki) also
+  contains many resources on getting started.
 
-- Extensive documentation for all the available integrations and messages [is available](https://docs.platypush.tech/).
+- Extensive documentation for all the available integrations and messages [is
+  available](https://docs.platypush.tech/).
 
-- If you have issues/feature requests/enhancement ideas please [create an issue](https://git.platypush.tech/platypush/platypush/-/issues).
+- If you have issues/feature requests/enhancement ideas please [create an
+  issue](https://git.platypush.tech/platypush/platypush/-/issues).
 
-- A [Reddit channel](https://www.reddit.com/r/platypush) is also available for more general questions.
+- A [Reddit channel](https://www.reddit.com/r/platypush) is also available for
+  more general questions.
+
+- A [Matrix instance](https://matrix.to/#/#platypush:matrix.platypush.tech) is
+  also available if you are looking for more interactive support.
 
 ---
 
-Platypush is a general-purpose extensible platform for automation and integration across multiple services and devices.
+Platypush is a general-purpose extensible platform for automation and
+integration across multiple services and devices.
 
-It enables users to create their own self-hosted pieces of automation based on events (*if this happens then do that*)
+It enables users to create their own self-hosted pieces of automation based on
-and it provides a comprehensive and customizable user interface that collects everything you need to visualize and
+events (*if this happens then do that*)
-control under one roof.
+and it provides a comprehensive and customizable user interface that collects
+everything you need to visualize and control under one roof.
 
-It takes some concepts from [IFTTT](https://ifttt.com), [Tasker](https://tasker.joaoapps.com/),
+It takes some concepts from [IFTTT](https://ifttt.com),
-[Microsoft Flow](https://flow.microsoft.com), [PushBullet](https://pushbullet.com) and
+[Tasker](https://tasker.joaoapps.com/), [Microsoft
-[Home Assistant](https://www.home-assistant.io/) to provide an environment where the user can easily connect things
+Flow](https://flow.microsoft.com), [PushBullet](https://pushbullet.com) and
-together.
+[Home Assistant](https://www.home-assistant.io/) to provide an environment
+where the user can easily connect things together.
 
-Its ideal home is a single-board computer like a RaspberryPi that you can configure to orchestrate any home automation
+Its ideal home is a single-board computer like a RaspberryPi that you can
-and cloud automation in your own living room or garage, but it can easily run on any device that can run a Python
+configure to orchestrate any home automation and cloud automation in your own
-interpreter, and the bar for the hardware requirements is very low as well - I use it to run pieces of automation on
+living room or garage, but it can easily run on any device that can run a
-devices as powerful as a RaspberryPi Zero or an old Nokia N900 with Linux.
+Python interpreter, and the bar for the hardware requirements is very low as
+well - I use it to run pieces of automation on devices as powerful as a
+RaspberryPi Zero or an old Nokia N900 with Linux.
 
 You can use Platypush to do things like:
 
@@ -49,8 +90,10 @@ You can use Platypush to do things like:
 - [Create custom and privacy-secure voice assistants that run custom hooks on your phrases](https://blog.platypush.tech/article/Build-custom-voice-assistants)
 - Build integrations between [sensors](https://docs.platypush.tech/en/latest/platypush/backend/sensor.html),
   [cameras](https://docs.platypush.tech/en/latest/platypush/plugins/camera.pi.html),
-  [microphones](https://docs.platypush.tech/en/latest/platypush/plugins/sound.html) and
+  [microphones](https://docs.platypush.tech/en/latest/platypush/plugins/sound.html)
-  [machine learning models](https://docs.platypush.tech/en/latest/platypush/plugins/tensorflow.html) to create smart
+  and [machine learning
+  models](https://docs.platypush.tech/en/latest/platypush/plugins/tensorflow.html)
+  to create smart
   pieces of automation for e.g.
   [people detection](https://blog.platypush.tech/article/Detect-people-with-a-RaspberryPi-a-thermal-camera-Platypush-and-a-pinch-of-machine-learning)
   or [sound detection](https://blog.platypush.tech/article/Create-your-smart-baby-monitor-with-Platypush-and-Tensorflow)
@@ -62,35 +105,53 @@ You can use Platypush to do things like:
 - [Control your smart switches](https://docs.platypush.tech/en/latest/platypush/plugins/switch.html)
 - [Implement automated custom text-to-speech routines](https://docs.platypush.tech/en/latest/platypush/plugins/tts.html)
 - [Build any kind of interactions and automation routines with your Android device using Tasker](https://blog.platypush.tech/article/How-to-build-your-personal-infrastructure-for-data-collection-and-visualization)
-- Play [local videos](https://docs.platypush.tech/en/latest/platypush/plugins/media.mpv.html), YouTube videos and torrent media from any device and service, to any device
+- Play [local
+  videos](https://docs.platypush.tech/en/latest/platypush/plugins/media.mpv.html),
+  YouTube videos and torrent media from any device and service, to any device
 - [Get weather forecast events for your location and build automation routines on them](https://docs.platypush.tech/en/latest/platypush/plugins/weather.darksky.html)
 - [Create a custom single hub for Zigbee and Z-Wave smart devices](https://blog.platypush.tech/article/Transform-a-RaspberryPi-into-a-universal-Zigbee-and-Z-Wave-bridge)
-- Build your own web dashboard with calendar, weather, news and music controls (basically, anything that has a Platypush web widget)
+- Build your own web dashboard with calendar, weather, news and music controls
+  (basically, anything that has a Platypush web widget)
 - ...and much more (basically, anything that comes with a [Platypush plugin](https://docs.platypush.tech/en/latest/plugins.html))
 
 ## Architecture
 
-The architecture of Platypush consists of a few simple pieces, orchestrated by a configuration file stored by default
+The architecture of Platypush consists of a few simple pieces, orchestrated by
-under [`~/.config/platypush/config.yaml`](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/config.yaml):
+a configuration file stored by default under
+[`~/.config/platypush/config.yaml`](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/config.yaml):
 
-### [Plugins](https://docs.platypush.tech/en/latest/plugins.html)
+### Plugins
 
-They are integrations that do things - like
+[Full list](https://docs.platypush.tech/en/latest/plugins.html)
-[modify files](https://docs.platypush.tech/en/latest/platypush/plugins/file.html),
-[train and evaluate machine learning models](https://docs.platypush.tech/en/latest/platypush/plugins/tensorflow.html),
-[control cameras](https://docs.platypush.tech/en/latest/platypush/plugins/camera.pi.html),
-[read sensors](https://docs.platypush.tech/en/latest/platypush/plugins/gpio.sensor.dht.html),
-[parse a web page](https://docs.platypush.tech/en/latest/platypush/plugins/http.webpage.html),
-[control lights](https://docs.platypush.tech/en/latest/platypush/plugins/light.hue.html),
-[send emails](https://docs.platypush.tech/en/latest/platypush/plugins/mail.smtp.html),
-[control Chromecasts](https://docs.platypush.tech/en/latest/platypush/plugins/media.chromecast.html),
-[run voice queries](https://docs.platypush.tech/en/latest/platypush/plugins/assistant.google.html),
-[handle torrent transfers](https://docs.platypush.tech/en/latest/platypush/plugins/torrent.html) or
-control [Zigbee](https://docs.platypush.tech/en/latest/platypush/plugins/zigbee.mqtt.html) or
-[Z-Wave](https://docs.platypush.tech/en/latest/platypush/plugins/zwave.html) devices.
 
-The configuration of a plugin matches one-on-one that of its documented class constructor, so it's very straightforward
+Plugins are integrations that do things - like [modify
-to write a configuration for a plugin by reading its documentation:
+files](https://docs.platypush.tech/en/latest/platypush/plugins/file.html),
+[train and evaluate machine learning
+models](https://docs.platypush.tech/en/latest/platypush/plugins/tensorflow.html),
+[control
+cameras](https://docs.platypush.tech/en/latest/platypush/plugins/camera.pi.html),
+[read
+sensors](https://docs.platypush.tech/en/latest/platypush/plugins/gpio.sensor.dht.html),
+[parse a web
+page](https://docs.platypush.tech/en/latest/platypush/plugins/http.webpage.html),
+[control
+lights](https://docs.platypush.tech/en/latest/platypush/plugins/light.hue.html),
+[send
+emails](https://docs.platypush.tech/en/latest/platypush/plugins/mail.smtp.html),
+[control
+Chromecasts](https://docs.platypush.tech/en/latest/platypush/plugins/media.chromecast.html),
+[run voice
+queries](https://docs.platypush.tech/en/latest/platypush/plugins/assistant.google.html),
+[handle torrent
+transfers](https://docs.platypush.tech/en/latest/platypush/plugins/torrent.html)
+or control
+[Zigbee](https://docs.platypush.tech/en/latest/platypush/plugins/zigbee.mqtt.html)
+or [Z-Wave](https://docs.platypush.tech/en/latest/platypush/plugins/zwave.html)
+devices.
+
+The configuration of a plugin matches one-on-one that of its documented class
+constructor, so it's very straightforward to write a configuration for a plugin
+by reading its documentation:
 
 ```yaml
 light.hue:
@@ -102,9 +163,11 @@ light.hue:
 
 ### Actions
 
-Plugins expose *actions*, that match one-on-one the plugin class methods denoted by `@action`, so it's very
+Plugins expose *actions*, that match one-on-one the plugin class methods
-straightforward to invoke plugin actions by just reading the plugin documentation. They can be invoked directly from
+denoted by `@action`, so it's very straightforward to invoke plugin actions by
-your own scripts or they can be sent to the platform through any supported channel as simple JSON messages:
+just reading the plugin documentation. They can be invoked directly from your
+own scripts or they can be sent to the platform through any supported channel
+as simple JSON messages:
 
 ```json
 {
@@ -116,37 +179,54 @@ your own scripts or they can be sent to the platform through any supported chann
 }
 ```
 
-### [Backends](https://docs.platypush.tech/en/latest/backends.html)
+### Backends
 
-They are background services that either listen for messages on channels (like an
+[Full list](https://docs.platypush.tech/en/latest/backends.html)
-[HTTP backend](https://docs.platypush.tech/en/latest/platypush/backend/http.html), an
-[MQTT instance](https://docs.platypush.tech/en/latest/platypush/backend/mqtt.html), a
-[Kafka instance](https://docs.platypush.tech/en/latest/platypush/backend/kafka.html), a
-[Websocket service](https://docs.platypush.tech/en/latest/platypush/backend/websocket.html),
-[Pushbullet](https://docs.platypush.tech/en/latest/platypush/backend/pushbullet.html) etc.) or monitor a device or a
-service for events (like a [sensor](https://docs.platypush.tech/en/latest/platypush/backend/sensor.html), a custom
-[voice assistant](https://docs.platypush.tech/en/latest/platypush/backend/assistant.google.html), a bridge running on a
-[Zigbee](https://docs.platypush.tech/en/latest/platypush/backend/zigbee.mqtt.html) or
-[Z-Wave](https://docs.platypush.tech/en/latest/platypush/backend/zwave.html), an
-[NFC card reader](https://docs.platypush.tech/en/latest/platypush/backend/nfc.html), a
-[MIDI device](https://docs.platypush.tech/en/latest/platypush/backend/midi.html), a
-[Telegram channel](https://docs.platypush.tech/en/latest/platypush/backend/chat.telegram.html), a
-[Bluetooth scanner](https://docs.platypush.tech/en/latest/platypush/backend/bluetooth.scanner.ble.html) etc.).
 
-If a backend supports the execution of requests (e.g. HTTP, MQTT, Kafka, Websocket and TCP) then you can send requests
+They are background services that either listen for messages on channels (like
-to these services in JSON format. For example, in the case of the HTTP backend:
+an [HTTP
+backend](https://docs.platypush.tech/en/latest/platypush/backend/http.html), an
+[MQTT
+instance](https://docs.platypush.tech/en/latest/platypush/backend/mqtt.html), a
+[Kafka
+instance](https://docs.platypush.tech/en/latest/platypush/backend/kafka.html),
+a [Websocket
+service](https://docs.platypush.tech/en/latest/platypush/backend/websocket.html),
+[Pushbullet](https://docs.platypush.tech/en/latest/platypush/backend/pushbullet.html)
+etc.) or monitor a device or a service for events (like a
+[sensor](https://docs.platypush.tech/en/latest/platypush/backend/sensor.html),
+a custom [voice
+assistant](https://docs.platypush.tech/en/latest/platypush/backend/assistant.google.html),
+a bridge running on a
+[Zigbee](https://docs.platypush.tech/en/latest/platypush/backend/zigbee.mqtt.html)
+or
+[Z-Wave](https://docs.platypush.tech/en/latest/platypush/backend/zwave.html),
+an [NFC card
+reader](https://docs.platypush.tech/en/latest/platypush/backend/nfc.html), a
+[MIDI
+device](https://docs.platypush.tech/en/latest/platypush/backend/midi.html), a
+[Telegram
+channel](https://docs.platypush.tech/en/latest/platypush/backend/chat.telegram.html),
+a [Bluetooth
+scanner](https://docs.platypush.tech/en/latest/platypush/backend/bluetooth.scanner.ble.html)
+etc.).
+
+If a backend supports the execution of requests (e.g. HTTP, MQTT, Kafka,
+Websocket and TCP) then you can send requests to these services in JSON format.
+For example, in the case of the HTTP backend:
 
 ```shell
-# Get a token
+    # Get a token
 curl -XPOST -H 'Content-Type: application/json' -d '
   {
     "username": "$YOUR_USER",
     "password": "$YOUR_PASSWORD"
   }' http://host:8008/auth
 
-# Execute a request
+    # Execute a request
 
-curl -XPOST -H 'Content-Type: application/json' -H "Authorization: Bearer $YOUR_TOKEN" -d '
+curl -XPOST -H 'Content-Type: application/json' \
+    -H "Authorization: Bearer $YOUR_TOKEN" -d '
   {
     "type": "request",
     "action": "tts.say",
@@ -156,33 +236,38 @@ curl -XPOST -H 'Content-Type: application/json' -H "Authorization: Bearer $YOUR_
   }' http://host:8008/execute
 ```
 
-### [Events](https://docs.platypush.tech/en/latest/events.html)
+### Events
 
-When a certain event occurs (e.g. a JSON request is received, or a
+[Full list](https://docs.platypush.tech/en/latest/events.html)
-[Bluetooth device is connected](https://docs.platypush.tech/en/latest/platypush/events/bluetooth.html#platypush.message.event.bluetooth.BluetoothDeviceConnectedEvent),
+
-or a
+When a certain event occurs (e.g. a JSON request is received, or a [Bluetooth
-[Flic button is pressed](https://docs.platypush.tech/en/latest/platypush/events/button.flic.html#platypush.message.event.button.flic.FlicButtonEvent),
+device is
-or some
+connected](https://docs.platypush.tech/en/latest/platypush/events/bluetooth.html#platypush.message.event.bluetooth.BluetoothDeviceConnectedEvent),
-[speech is detected on the voice assistant service](https://docs.platypush.tech/en/latest/platypush/events/assistant.html#platypush.message.event.assistant.SpeechRecognizedEvent),
+or a [Flic button is
-or an
+pressed](https://docs.platypush.tech/en/latest/platypush/events/button.flic.html#platypush.message.event.button.flic.FlicButtonEvent),
-[RSS feed has new items](https://docs.platypush.tech/en/latest/platypush/events/http.rss.html#platypush.message.event.http.rss.NewFeedEvent),
+or some [speech is detected on the voice assistant
-or a
+service](https://docs.platypush.tech/en/latest/platypush/events/assistant.html#platypush.message.event.assistant.SpeechRecognizedEvent),
-[new email is received](https://docs.platypush.tech/en/latest/platypush/events/mail.html#platypush.message.event.mail.MailReceivedEvent),
+or an [RSS feed has new
-or a
+items](https://docs.platypush.tech/en/latest/platypush/events/http.rss.html#platypush.message.event.http.rss.NewFeedEvent),
-[new track is played](https://docs.platypush.tech/en/latest/platypush/events/music.html#platypush.message.event.music.NewPlayingTrackEvent),
+or a [new email is
-or an
+received](https://docs.platypush.tech/en/latest/platypush/events/mail.html#platypush.message.event.mail.MailReceivedEvent),
-[NFC tag is detected](https://docs.platypush.tech/en/latest/platypush/events/nfc.html#platypush.message.event.nfc.NFCTagDetectedEvent),
+or a [new track is
-or
+played](https://docs.platypush.tech/en/latest/platypush/events/music.html#platypush.message.event.music.NewPlayingTrackEvent),
-[new sensor data is available](https://docs.platypush.tech/en/latest/platypush/events/sensor.html#platypush.message.event.sensor.SensorDataChangeEvent),
+or an [NFC tag is
-or
+detected](https://docs.platypush.tech/en/latest/platypush/events/nfc.html#platypush.message.event.nfc.NFCTagDetectedEvent),
-[a value of a Zigbee device changes](https://docs.platypush.tech/en/latest/platypush/events/zigbee.mqtt.html#platypush.message.event.zigbee.mqtt.ZigbeeMqttDevicePropertySetEvent),
+or [new sensor data is
-etc.), the associated backend will trigger an [event](https://docs.platypush.tech/en/latest/events.html).
+available](https://docs.platypush.tech/en/latest/platypush/events/sensor.html#platypush.message.event.sensor.SensorDataChangeEvent),
+or [a value of a Zigbee device
+changes](https://docs.platypush.tech/en/latest/platypush/events/zigbee.mqtt.html#platypush.message.event.zigbee.mqtt.ZigbeeMqttDevicePropertySetEvent),
+etc.), the associated backend will trigger an
+[event](https://docs.platypush.tech/en/latest/events.html).
 
 ### Hooks
 
-Event hooks are custom pieces of logic that will be run when a certain event is triggered. Hooks are the glue that
+Event hooks are custom pieces of logic that will be run when a certain event is
-connects events to actions, exposing a paradigm similar to IFTTT (_if a certain event happens then run these actions_).
+triggered. Hooks are the glue that connects events to actions, exposing a
-They can declared as:
+paradigm similar to IFTTT (_if a certain event happens then run these
+actions_). They can declared as:
 
 - Sections of the [`config.yaml`](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/config.yaml).
   Example:
@@ -206,9 +291,10 @@ event.hook.SearchSongVoiceCommand:
           resource: ${output[0]['file']}
 ```
 
-- Stand-alone Python scripts stored under `~/.config/platypush/scripts` and will be dynamically imported at start time.
+- Stand-alone Python scripts stored under `~/.config/platypush/scripts` and
+  will be dynamically imported at start time.
   [Example](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/hook.py):
-  
+
 ```python
 from platypush.event.hook import hook
 from platypush.utils import run
@@ -227,13 +313,17 @@ def on_music_play_command(event, title=None, artist=None, **context):
 
 ### Procedures
 
-Procedures are pieces of custom logic that can be executed as atomic actions using `procedure.<name>` as an action name.
+Procedures are pieces of custom logic that can be executed as atomic actions
-They can be defined either in the `config.yaml` or as Python scripts stored under `~/.config/platypush/scripts` -
+using `procedure.<name>` as an action name.
-provided that the procedure is also imported in `~/.config/platypush/scripts/__init__.py` so it can be discovered by
-the service.
 
-YAML example for a procedure that can be executed when we arrive home and turns on the lights if the luminosity is lower
+They can be defined either in the `config.yaml` or as Python scripts stored
-that a certain thresholds, says a welcome home message using the TTS engine and starts playing the music:
+under `~/.config/platypush/scripts` - provided that the procedure is also
+imported in `~/.config/platypush/scripts/__init__.py` so it can be discovered
+by the service.
+
+YAML example for a procedure that can be executed when we arrive home and turns
+on the lights if the luminosity is lower that a certain thresholds, says a
+welcome home message using the TTS engine and starts playing the music:
 
 ```yaml
 procedure.at_home:
@@ -256,7 +346,7 @@ procedure.at_home:
 Python example:
 
 ```python
-# Content of ~/.config/platypush/scripts/home.py
+    # Content of ~/.config/platypush/scripts/home.py
 from platypush.procedure import procedure
 from platypush.utils import run
 
@@ -270,11 +360,12 @@ def at_home(**context):
   run('music.mpd.play')
 ```
 
-In either case, you can easily trigger the at-home procedure by sending an action request message to a backend - for
+In either case, you can easily trigger the at-home procedure by sending an
-example, over the HTTP backend:
+action request message to a backend - for example, over the HTTP backend:
 
 ```shell
-curl -XPOST -H 'Content-Type: application/json' -H "Authorization: Bearer $YOUR_TOKEN" -d '
+curl -XPOST -H 'Content-Type: application/json' \
+    -H "Authorization: Bearer $YOUR_TOKEN" -d '
   {
     "type": "request",
     "action": "procedure.at_home"
@@ -283,15 +374,18 @@ curl -XPOST -H 'Content-Type: application/json' -H "Authorization: Bearer $YOUR_
 
 ### Cronjobs
 
-Cronjobs are pieces of logic that will be run at regular intervals, expressed in crontab-compatible syntax.
+Cronjobs are pieces of logic that will be run at regular intervals, expressed
-They can be defined either in the `config.yaml` or as Python scripts stored under `~/.config/platypush/scripts` as
+in crontab-compatible syntax. They can be defined either in the `config.yaml`
-functions labelled by the `@cron` decorator.
+or as Python scripts stored under `~/.config/platypush/scripts` as functions
+labelled by the `@cron` decorator.
 
-Note that seconds are also supported (unlike the standard crontab definition), but, for back-compatibility with the
+Note that seconds are also supported (unlike the standard crontab definition),
-standard crontab format, they are at the end of the cron expression, so the expression is actually in the format
+but, for back-compatibility with the standard crontab format, they are at the
+end of the cron expression, so the expression is actually in the format
 `<minute> <hour> <day_of_month> <month> <day_of_week> <second>`.
 
-YAML example for a cronjob that is executed every 30 seconds and checks if a Bluetooth device is nearby:
+YAML example for a cronjob that is executed every 30 seconds and checks if a
+Bluetooth device is nearby:
 
 ```yaml
 cron.check_bt_device:
@@ -310,7 +404,7 @@ cron.check_bt_device:
 Python example:
 
 ```python
-# Content of ~/.config/platypush/scripts/bt_cron.py
+    # Content of ~/.config/platypush/scripts/bt_cron.py
 from platypush.cron import cron
 from platypush.utils import run
 
@@ -325,15 +419,18 @@ def check_bt_device(**context):
 
 ### The web interface
 
-If [`backend.http`](https://docs.platypush.tech/en/latest/platypush/backend/http.html) is enabled then a web interface
+If
-will be provided by default on `http://host:8008/`. Besides using the `/execute` endpoint for running requests, the
+[`backend.http`](https://docs.platypush.tech/en/latest/platypush/backend/http.html)
-built-in web server also provides a full-featured interface that groups together the controls for most of the plugins -
+is enabled then a web interface will be provided by default on
-e.g. sensors, switches, music controls and search, media library and torrent management, lights, Zigbee/Z-Wave devices
+`http://host:8008/`. Besides using the `/execute` endpoint for running
-and so on. The UI is responsive and mobile-friendly.
+requests, the built-in web server also provides a full-featured interface that
+groups together the controls for most of the plugins - e.g. sensors, switches,
+music controls and search, media library and torrent management, lights,
+Zigbee/Z-Wave devices and so on. The UI is responsive and mobile-friendly.
 
-The web service also provides means for the user to create
+The web service also provides means for the user to create [custom
-[custom dashboards](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/dashboard.xml) that can
+dashboards](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/conf/dashboard.xml)
-be used to show information from multiple sources on a large screen.
+that can be used to show information from multiple sources on a large screen.
 
 ## Installation
 
@@ -342,36 +439,69 @@ be used to show information from multiple sources on a large screen.
 Platypush uses Redis to deliver and store requests and temporary messages:
 
 ```yaml
-# Example for Debian-based distributions
+    # Example for Debian-based distributions
 [sudo] apt-get install redis-server
 
-# Enable and start the service
+    # Enable and start the service
 [sudo] systemctl enable redis
 [sudo] systemctl start redis
 ```
 
-To install the core platform:
+#### Install through `pip`
-
-* The `pip` way:
 
 ```shell
 [sudo] pip3 install platypush
 ```
 
-* The sources way:
+#### Install through a system package manager
+
+Note: currently only Arch Linux and derived distributions are supported.
+
+You can either install the
+[`platypush`](https://aur.archlinux.org/packages/platypush) package (for the
+latest stable version) or the
+[`platypush-git`](https://aur.archlinux.org/packages/platypush-git) package
+(for the latest git version) through your favourite AUR package manager. For
+example, using `yay`:
+
+```shell
+yay platypush
+# Or
+yay platypush-git
+```
+
+The Arch Linux packages on AUR are automatically updated upon new git commits
+or tags.
+
+#### Install from sources
 
 ```shell
 git clone https://git.platypush.tech/platypush/platypush.git
 cd platypush
+[sudo] pip install .
+    # Or
 [sudo] python3 setup.py install
 ```
 
-Then install the extensions that you wish to use. There are a few ways to check the dependencies required by an
+### Installing the dependencies for your extensions
-extension:
 
-#### Check their `extras` name in [`extras_require` under `setup.py`](https://git.platypush.tech/platypush/platypush/-/blob/master/setup.py#L72).
+After installing the base platform, you may want to check the dependencies and
+configuration required by the extensions that you wish to use. There are a few
+ways to check the dependencies required by an extension:
 
-If you follow this route then you can install the extra dependencies in one of the following ways:
+#### Install via `extras` name
+
+All the extensions that require extra dependencies are listed in the
+[`extras_require` section under
+`setup.py`](https://git.platypush.tech/platypush/platypush/-/blob/master/setup.py#L72).
+
+#### Install via `manifest.yaml`
+
+All the plugins and backends have a `manifest.yaml` file in their source folder.
+Any extra dependencies are listed there
+
+If you followed the `extras` or `manifest.yaml` way to discover the
+dependencies, then you can install them in two ways:
 
 1. `pip` installation:
 
@@ -385,15 +515,18 @@ If you follow this route then you can install the extra dependencies in one of t
 cd $DIR_TO_PLATYPUSH
 [sudo] pip3 install '.[extra1,extra2,extra3]'
 ```
-  
-#### Check the dependencies/installation instructions reported under the plugin/backend documentation.
 
-If you follow this route then simply run the commands listed in the plugin/backend documentation to get the dependencies
+#### Check the instructions reported in the documentation
-installed.
 
-After installing the dependencies, create a configuration file under `~/.config/platypush/config.yaml` (the application
+If you follow this route then simply run the commands listed in the
-can load the configuration from another location through the `-c` option) containing the configuration of the backend
+[plugin/backend documentation](https://docs.platypush.tech) to get the
-and plugins that you want to use, and add any hooks and procedures for your use case.
+dependencies installed.
+
+After installing the dependencies, create a configuration file under
+`~/.config/platypush/config.yaml` (the application can load the configuration
+from another location through the `-c` option) containing the configuration of
+the backend and plugins that you want to use, and add any hooks and procedures
+for your use case.
 
 You can then start the service by simply running:
 
@@ -402,89 +535,100 @@ platypush
 ```
 
 It's advised to run it as a systemd service though - simply copy the provided
-[`.service` file](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/systemd/platypush.service) to
+[`.service`
-`~/.config/systemd/user`, check if the path of `platypush` matches the path where it's installed on your system, and
+file](https://git.platypush.tech/platypush/platypush/-/blob/master/examples/systemd/platypush.service)
-start the service via `systemctl`:
+to `~/.config/systemd/user`, check if the path of `platypush` matches the path
+where it's installed on your system, and start the service via `systemctl`:
 
 ```shell
 systemctl --user start platypush
 ```
 
-### [Virtual environment installation](https://git.platypush.tech/platypush/platypush/-/wikis/Run-platypush-in-a-virtual-environment)
+### Virtual environment installation
 
-Platypush provides a script named `platyvenv` that can parse a `config.yaml` and automatically create a virtual
+Platypush provides a script named `platyvenv` that can parse a `config.yaml`
-environment (under `~/.local/share/platypush/venv/<device_id>`) with all the dependencies required by the configured
+and automatically create a virtual environment (under
-integrations.
+`~/.local/share/platypush/venv/<device_id>`) with all the dependencies required
+by the configured integrations.
 
 1. Create the environment from a configuration file:
 
-```shell
+    ```shell
-platyvenv build -c /path/to/config.yaml
+    platyvenv build -c /path/to/config.yaml
-```
+    ```
 
 2. Start the service from the virtual environment:
 
-```shell
+    ```shell
-# device_id matches either the hostname or the device_id in config.yaml
+        # device_id matches either the hostname or the device_id in config.yaml
-platyvenv start device_id
+    platyvenv start device_id
-```
+    ```
 
 3. Stop the instance:
 
-```shell
+    ```shell
-platyvenv stop device_id
+    platyvenv stop device_id
-```
+    ```
 
 4. Remove the instance:
 
-```shell
+    ```shell
-platyvenv rm device_id
+    platyvenv rm device_id
-```
+    ```
 
-### [Docker installation](https://git.platypush.tech/platypush/platypush/-/wikis/Run-platypush-in-a-container)
+[Wiki instructions](https://git.platypush.tech/platypush/platypush/wiki/Run-platypush-in-a-virtual-environment)
 
-You can also install Platypush in a container - the application provides a script named `platydock` that automatically
+### Docker installation
-creates a container instance from a `config.yaml`:
+
+You can also install Platypush in a container - the application provides a
+script named `platydock` that automatically creates a container instance from a
+`config.yaml`:
 
 1. Create the container from a configuration file:
 
-```shell
+    ```shell
-platydock build -c /path/to/config.yaml
+    platydock build -c /path/to/config.yaml
-```
+    ```
 
 2. Start the container:
 
-```shell
+    ```shell
-# device_id matches either the hostname or the device_id in config.yaml
+        # device_id matches either the hostname or the device_id in config.yaml
-platydock start device_id
+    platydock start device_id
-```
+    ```
 
 3. Stop the instance:
 
-```shell
+    ```shell
-platydock stop device_id
+    platydock stop device_id
-```
+    ```
 
 4. Remove the instance:
 
-```shell
+    ```shell
-platydock rm device_id
+    platydock rm device_id
-```
+    ```
 
-Note that both the virtual environment and Docker container option offer the possibility to include extra YAML configuration
+Note that both the virtual environment and Docker container option offer the
-files in the main `config.yaml` through the `include` directive (as long as they are in the same folder as the main
+possibility to include extra YAML configuration files in the main `config.yaml`
-`config.yaml`), as well as external Python scripts in a `scripts` directory in the same folder as the `config.yaml`.
+through the `include` directive (as long as they are in the same folder as the
+main `config.yaml`), as well as external Python scripts in a `scripts`
+directory in the same folder as the `config.yaml`.
+
+[Wiki instructions](https://git.platypush.tech/platypush/platypush/wiki/Run-platypush-in-a-container)
 
 ## Mobile app
 
-An [official Android app](https://f-droid.org/en/packages/tech.platypush.platypush/) is provided on the F-Droid store.
+An [official Android
-It allows to easily discover and manage multiple Platypush services on a network through the web interface, and it
+app](https://f-droid.org/en/packages/tech.platypush.platypush/) is provided on
-easily brings the power of Platypush to your fingertips.
+the F-Droid store. It allows to easily discover and manage multiple Platypush
+services on a network through the web interface, and it easily brings the power
+of Platypush to your fingertips.
 
 ## Tests
 
-To run the tests simply run `pytest` either from the project root folder or the `tests/` folder.
+To run the tests simply run `pytest` either from the project root folder or the
-Or run the following command from the project root folder:
+`tests/` folder. Or run the following command from the project root folder:
 
 ```shell
 python -m tests
@@ -496,10 +640,11 @@ python -m tests
 
 If you use and love Platypush, please consider [buying me a coffee/beer](https://paypal.me/fabiomanganiello).
 
-I've been working on Platypush all by myself in my spare time for the past few years, and I've made sure that it remains
+I've been working on Platypush all by myself in my spare time for the past few
-open and free.
+years, and I've made sure that it remains open and free.
 
-If you like this product, please consider supporting - I'm definitely not planning to get rich with this project, but
+If you like this product, please consider supporting - I'm definitely not
-I'd love to have at least the costs for the server covered by users.
+planning to get rich with this project, but I'd love to have at least the costs
+for the server covered by users.
 
 Issues and requests opened by donors will also be given priority over others.

docs/source/backends.rst

@@ -3,7 +3,7 @@ Backends
 ========
 
 .. toctree::
-    :maxdepth: 2
+    :maxdepth: 1
     :caption: Backends:
 
     platypush/backend/adafruit.io.rst
@@ -17,7 +17,6 @@ Backends
     platypush/backend/button.flic.rst
     platypush/backend/camera.pi.rst
     platypush/backend/chat.telegram.rst
-    platypush/backend/clipboard.rst
     platypush/backend/covid19.rst
     platypush/backend/file.monitor.rst
     platypush/backend/foursquare.rst

docs/source/conf.py

@@ -71,7 +71,7 @@ master_doc = 'index'
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = None
+language = 'en'
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
@@ -138,15 +138,12 @@ latex_elements = {
     # The paper size ('letterpaper' or 'a4paper').
     #
     # 'papersize': 'letterpaper',
-
     # The font size ('10pt', '11pt' or '12pt').
     #
     # 'pointsize': '10pt',
-
     # Additional stuff for the LaTeX preamble.
     #
     # 'preamble': '',
-
     # Latex figure (float) alignment
     #
     # 'figure_align': 'htbp',
@@ -156,8 +153,7 @@ latex_elements = {
 # (source start file, target name, title,
 #  author, documentclass [howto, manual, or own class]).
 latex_documents = [
-    (master_doc, 'platypush.tex', 'platypush Documentation',
+    (master_doc, 'platypush.tex', 'platypush Documentation', 'BlackLight', 'manual'),
-     'BlackLight', 'manual'),
 ]
 
 
@@ -165,10 +161,7 @@ latex_documents = [
 
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
-man_pages = [
+man_pages = [(master_doc, 'platypush', 'platypush Documentation', [author], 1)]
-    (master_doc, 'platypush', 'platypush Documentation',
-     [author], 1)
-]
 
 
 # -- Options for Texinfo output ----------------------------------------------
@@ -177,9 +170,15 @@ man_pages = [
 # (source start file, target name, title, author,
 #  dir menu entry, description, category)
 texinfo_documents = [
-    (master_doc, 'platypush', 'platypush Documentation',
+    (
-     author, 'platypush', 'One line description of project.',
+        master_doc,
-     'Miscellaneous'),
+        'platypush',
+        'platypush Documentation',
+        author,
+        'platypush',
+        'One line description of project.',
+        'Miscellaneous',
+    ),
 ]
 
 
@@ -196,102 +195,108 @@ intersphinx_mapping = {'https://docs.python.org/': None}
 todo_include_todos = True
 
 autodoc_default_options = {
-    'inherited-members': True,
+    'members': True,
+    'show-inheritance': True,
 }
 
-autodoc_mock_imports = ['googlesamples.assistant.grpc.audio_helpers',
+autodoc_mock_imports = [
-                        'google.assistant.embedded',
+    'googlesamples.assistant.grpc.audio_helpers',
-                        'google.assistant.library',
+    'google.assistant.embedded',
-                        'google.assistant.library.event',
+    'google.assistant.library',
-                        'google.assistant.library.file_helpers',
+    'google.assistant.library.event',
-                        'google.oauth2.credentials',
+    'google.assistant.library.file_helpers',
-                        'oauth2client',
+    'google.oauth2.credentials',
-                        'apiclient',
+    'oauth2client',
-                        'tenacity',
+    'apiclient',
-                        'smartcard',
+    'tenacity',
-                        'Leap',
+    'smartcard',
-                        'oauth2client',
+    'Leap',
-                        'rtmidi',
+    'oauth2client',
-                        'bluetooth',
+    'rtmidi',
-                        'gevent.wsgi',
+    'bluetooth',
-                        'Adafruit_IO',
+    'gevent.wsgi',
-                        'pyperclip',
+    'Adafruit_IO',
-                        'pydbus',
+    'pyperclip',
-                        'inputs',
+    'pydbus',
-                        'inotify',
+    'inputs',
-                        'omxplayer',
+    'inotify',
-                        'plexapi',
+    'omxplayer',
-                        'cwiid',
+    'plexapi',
-                        'sounddevice',
+    'cwiid',
-                        'soundfile',
+    'sounddevice',
-                        'numpy',
+    'soundfile',
-                        'cv2',
+    'numpy',
-                        'nfc',
+    'cv2',
-                        'ndef',
+    'nfc',
-                        'bcrypt',
+    'ndef',
-                        'google',
+    'bcrypt',
-                        'feedparser',
+    'google',
-                        'kafka',
+    'feedparser',
-                        'googlesamples',
+    'kafka',
-                        'icalendar',
+    'googlesamples',
-                        'httplib2',
+    'icalendar',
-                        'mpd',
+    'httplib2',
-                        'serial',
+    'mpd',
-                        'pyHS100',
+    'serial',
-                        'grpc',
+    'pyHS100',
-                        'envirophat',
+    'grpc',
-                        'gps',
+    'envirophat',
-                        'picamera',
+    'gps',
-                        'pmw3901',
+    'picamera',
-                        'PIL',
+    'pmw3901',
-                        'croniter',
+    'PIL',
-                        'pyaudio',
+    'croniter',
-                        'avs',
+    'pyaudio',
-                        'PyOBEX',
+    'avs',
-                        'todoist',
+    'PyOBEX',
-                        'trello',
+    'todoist',
-                        'telegram',
+    'trello',
-                        'telegram.ext',
+    'telegram',
-                        'pyfirmata2',
+    'telegram.ext',
-                        'cups',
+    'pyfirmata2',
-                        'graphyte',
+    'cups',
-                        'cpuinfo',
+    'graphyte',
-                        'psutil',
+    'cpuinfo',
-                        'openzwave',
+    'psutil',
-                        'deepspeech',
+    'openzwave',
-                        'wave',
+    'deepspeech',
-                        'pvporcupine ',
+    'wave',
-                        'pvcheetah',
+    'pvporcupine ',
-                        'pyotp',
+    'pvcheetah',
-                        'linode_api4',
+    'pyotp',
-                        'pyzbar',
+    'linode_api4',
-                        'tensorflow',
+    'pyzbar',
-                        'keras',
+    'tensorflow',
-                        'pandas',
+    'keras',
-                        'samsungtvws',
+    'pandas',
-                        'paramiko',
+    'samsungtvws',
-                        'luma',
+    'paramiko',
-                        'zeroconf',
+    'luma',
-                        'dbus',
+    'zeroconf',
-                        'gi',
+    'dbus',
-                        'gi.repository',
+    'gi',
-                        'twilio',
+    'gi.repository',
-                        'Adafruit_Python_DHT',
+    'twilio',
-                        'RPi.GPIO',
+    'Adafruit_Python_DHT',
-                        'RPLCD',
+    'RPi.GPIO',
-                        'imapclient',
+    'RPLCD',
-                        'pysmartthings',
+    'imapclient',
-                        'aiohttp',
+    'pysmartthings',
-                        'watchdog',
+    'aiohttp',
-                        'pyngrok',
+    'watchdog',
-                        'irc',
+    'pyngrok',
-                        'irc.bot',
+    'irc',
-                        'irc.strings',
+    'irc.bot',
-                        'irc.client',
+    'irc.strings',
-                        'irc.connection',
+    'irc.client',
-                        'irc.events',
+    'irc.connection',
-                        'defusedxml',
+    'irc.events',
-                        ]
+    'defusedxml',
+    'nio',
+    'aiofiles',
+    'aiofiles.os',
+    'async_lru',
+]
 
 sys.path.insert(0, os.path.abspath('../..'))
 

docs/source/events.rst

@@ -3,7 +3,7 @@ Events
 ======
 
 .. toctree::
-    :maxdepth: 2
+    :maxdepth: 1
     :caption: Events:
 
     platypush/events/adafruit.rst
@@ -28,6 +28,7 @@ Events
     platypush/events/google.fit.rst
     platypush/events/google.pubsub.rst
     platypush/events/gotify.rst
+    platypush/events/gpio.rst
     platypush/events/gps.rst
     platypush/events/http.rst
     platypush/events/http.hook.rst
@@ -40,14 +41,17 @@ Events
     platypush/events/linode.rst
     platypush/events/log.http.rst
     platypush/events/mail.rst
+    platypush/events/matrix.rst
     platypush/events/media.rst
     platypush/events/midi.rst
     platypush/events/mqtt.rst
     platypush/events/music.rst
     platypush/events/music.snapcast.rst
+    platypush/events/music.tidal.rst
     platypush/events/nextcloud.rst
     platypush/events/nfc.rst
     platypush/events/ngrok.rst
+    platypush/events/ntfy.rst
     platypush/events/ping.rst
     platypush/events/pushbullet.rst
     platypush/events/qrcode.rst
@@ -70,6 +74,7 @@ Events
     platypush/events/weather.rst
     platypush/events/web.rst
     platypush/events/web.widget.rst
+    platypush/events/websocket.rst
     platypush/events/wiimote.rst
     platypush/events/zeroborg.rst
     platypush/events/zeroconf.rst

docs/source/index.rst

@@ -6,17 +6,17 @@ Welcome to the Platypush reference of available plugins, backends and event type
 For more information on Platypush check out:
 
 * The `main page`_ of the project
-* The `Gitlab page`_ of the project
+* The `Gitea page`_ of the project
 * The `online wiki`_ for quickstart and examples
 * The `Blog articles`_ for inspiration on use-cases possible projects
 
 .. _main page: https://platypush.tech
-.. _Gitlab page: https://git.platypush.tech/platypush/platypush
+.. _Gitea page: https://git.platypush.tech/platypush/platypush
-.. _online wiki: https://git.platypush.tech/platypush/platypush/-/wikis/home
+.. _online wiki: https://git.platypush.tech/platypush/platypush/wiki
 .. _Blog articles: https://blog.platypush.tech
 
 .. toctree::
-    :maxdepth: 3
+    :maxdepth: 2
     :caption: Contents:
 
     backends

docs/source/platypush/backend/clipboard.rst

@@ -1,5 +0,0 @@
-``clipboard``
-===============================
-
-.. automodule:: platypush.backend.clipboard
-    :members:

docs/source/platypush/events/chat.slack.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.chat.slack``
+``chat.slack``
-======================================
+==============
 
 .. automodule:: platypush.message.event.chat.slack
     :members:

docs/source/platypush/events/dbus.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.dbus``
+``dbus``
-================================
+========
 
 .. automodule:: platypush.message.event.dbus
     :members:

docs/source/platypush/events/gotify.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.gotify``
+``gotify``
-==================================
+==========
 
 .. automodule:: platypush.message.event.gotify
     :members:

docs/source/platypush/events/gpio.rst

@@ -0,0 +1,5 @@
+``gpio``
+========
+
+.. automodule:: platypush.message.event.gpio
+    :members:

docs/source/platypush/events/irc.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.irc``
+``irc``
-===============================
+=======
 
 .. automodule:: platypush.message.event.irc
     :members:

docs/source/platypush/events/matrix.rst

@@ -0,0 +1,5 @@
+``matrix``
+==========
+
+.. automodule:: platypush.message.event.matrix
+    :members:

docs/source/platypush/events/music.tidal.rst

@@ -0,0 +1,5 @@
+``music.tidal``
+===============
+
+.. automodule:: platypush.message.event.music.tidal
+    :members:

docs/source/platypush/events/ngrok.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.ngrok``
+``ngrok``
-=================================
+=========
 
 .. automodule:: platypush.message.event.ngrok
     :members:

docs/source/platypush/events/ntfy.rst

@@ -0,0 +1,5 @@
+``ntfy``
+========
+
+.. automodule:: platypush.message.event.ntfy
+    :members:

docs/source/platypush/events/rss.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.rss``
+``rss``
-===============================
+=======
 
 .. automodule:: platypush.message.event.rss
     :members:

docs/source/platypush/events/sun.rst

@@ -1,5 +1,5 @@
-``platypush.message.event.sun``
+``sun``
-===============================
+=======
 
 .. automodule:: platypush.message.event.sun
     :members:

docs/source/platypush/events/websocket.rst

@@ -0,0 +1,5 @@
+``websocket``
+=============
+
+.. automodule:: platypush.message.event.websocket
+    :members:

docs/source/platypush/plugins/dbus.rst

@@ -2,4 +2,4 @@
 ==========================
 
 .. automodule:: platypush.plugins.dbus
-    :members:
+    :exclude-members: DBusService, BusType

docs/source/platypush/plugins/matrix.rst

@@ -0,0 +1,5 @@
+``matrix``
+==========
+
+.. automodule:: platypush.plugins.matrix
+    :members: MatrixPlugin

docs/source/platypush/plugins/media.jellyfin.rst

@@ -1,5 +1,5 @@
 ``media.jellyfin``
-================================
+==================
 
 .. automodule:: platypush.plugins.media.jellyfin
 	:members:

docs/source/platypush/plugins/music.tidal.rst

@@ -0,0 +1,5 @@
+``music.tidal``
+===============
+
+.. automodule:: platypush.plugins.music.tidal
+    :members:

docs/source/platypush/plugins/ntfy.rst

@@ -0,0 +1,5 @@
+``ntfy``
+========
+
+.. automodule:: platypush.plugins.ntfy
+    :members:

docs/source/platypush/plugins/tts.mimic3.rst

@@ -0,0 +1,5 @@
+``tts.mimic3``
+==============
+
+.. automodule:: platypush.plugins.tts.mimic3
+    :members:

docs/source/platypush/plugins/wallabag.rst

@@ -0,0 +1,5 @@
+``wallabag``
+============
+
+.. automodule:: platypush.plugins.wallabag
+    :members:

docs/source/plugins.rst

@@ -3,7 +3,7 @@ Plugins
 =======
 
 .. toctree::
-    :maxdepth: 2
+    :maxdepth: 1
     :caption: Plugins:
 
     platypush/plugins/adafruit.io.rst
@@ -75,6 +75,7 @@ Plugins
     platypush/plugins/mail.smtp.rst
     platypush/plugins/mailgun.rst
     platypush/plugins/mastodon.rst
+    platypush/plugins/matrix.rst
     platypush/plugins/media.chromecast.rst
     platypush/plugins/media.gstreamer.rst
     platypush/plugins/media.jellyfin.rst
@@ -93,9 +94,11 @@ Plugins
     platypush/plugins/music.mpd.rst
     platypush/plugins/music.snapcast.rst
     platypush/plugins/music.spotify.rst
+    platypush/plugins/music.tidal.rst
     platypush/plugins/nextcloud.rst
     platypush/plugins/ngrok.rst
     platypush/plugins/nmap.rst
+    platypush/plugins/ntfy.rst
     platypush/plugins/otp.rst
     platypush/plugins/pihole.rst
     platypush/plugins/ping.rst
@@ -129,12 +132,14 @@ Plugins
     platypush/plugins/trello.rst
     platypush/plugins/tts.rst
     platypush/plugins/tts.google.rst
+    platypush/plugins/tts.mimic3.rst
     platypush/plugins/tv.samsung.ws.rst
     platypush/plugins/twilio.rst
     platypush/plugins/udp.rst
     platypush/plugins/user.rst
     platypush/plugins/utils.rst
     platypush/plugins/variable.rst
+    platypush/plugins/wallabag.rst
     platypush/plugins/weather.buienradar.rst
     platypush/plugins/weather.darksky.rst
     platypush/plugins/weather.openweathermap.rst

docs/source/responses.rst

@@ -3,7 +3,7 @@ Responses
 =========
 
 .. toctree::
-    :maxdepth: 2
+    :maxdepth: 1
     :caption: Responses:
 
     platypush/responses/bluetooth.rst

generate_missing_docs.py

@@ -6,11 +6,17 @@ from platypush.plugins import Plugin
 from platypush.utils.manifest import get_manifests
 
 
+def _get_inspect_plugin():
+    p = get_plugin('inspect')
+    assert p, 'Could not load the `inspect` plugin'
+    return p
+
+
 def get_all_plugins():
     manifests = {mf.component_name for mf in get_manifests(Plugin)}
     return {
         plugin_name: plugin_info
-        for plugin_name, plugin_info in get_plugin('inspect').get_all_plugins().output.items()
+        for plugin_name, plugin_info in _get_inspect_plugin().get_all_plugins().output.items()
         if plugin_name in manifests
     }
 
@@ -19,17 +25,17 @@ def get_all_backends():
     manifests = {mf.component_name for mf in get_manifests(Backend)}
     return {
         backend_name: backend_info
-        for backend_name, backend_info in get_plugin('inspect').get_all_backends().output.items()
+        for backend_name, backend_info in _get_inspect_plugin().get_all_backends().output.items()
         if backend_name in manifests
     }
 
 
 def get_all_events():
-    return get_plugin('inspect').get_all_events().output
+    return _get_inspect_plugin().get_all_events().output
 
 
 def get_all_responses():
-    return get_plugin('inspect').get_all_responses().output
+    return _get_inspect_plugin().get_all_responses().output
 
 
 # noinspection DuplicatedCode
@@ -100,16 +106,17 @@ Backends
 
 # noinspection DuplicatedCode
 def generate_events_doc():
+    from platypush.message import event as event_module
     events_index = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'docs', 'source', 'events.rst')
     events_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'docs', 'source', 'platypush', 'events')
-    all_events = sorted(event for event in get_all_events().keys())
+    all_events = sorted(event for event in get_all_events().keys() if event)
 
     for event in all_events:
-        event_file = os.path.join(events_dir, event[len('platypush.message.event.'):] + '.rst')
+        event_file = os.path.join(events_dir, event + '.rst')
         if not os.path.exists(event_file):
             header = '``{}``'.format(event)
             divider = '=' * len(header)
-            body = '\n.. automodule:: {}\n    :members:\n'.format(event)
+            body = '\n.. automodule:: {}.{}\n    :members:\n'.format(event_module.__name__, event)
             out = '\n'.join([header, divider, body])
 
             with open(event_file, 'w') as f:
@@ -127,21 +134,22 @@ Events
 ''')
 
         for event in all_events:
-            f.write('    platypush/events/' + event[len('platypush.message.event.'):] + '.rst\n')
+            f.write('    platypush/events/' + event + '.rst\n')
 
 
 # noinspection DuplicatedCode
 def generate_responses_doc():
+    from platypush.message import response as response_module
     responses_index = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'docs', 'source', 'responses.rst')
     responses_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'docs', 'source', 'platypush', 'responses')
-    all_responses = sorted(response for response in get_all_responses().keys())
+    all_responses = sorted(response for response in get_all_responses().keys() if response)
 
     for response in all_responses:
-        response_file = os.path.join(responses_dir, response[len('platypush.message.response.'):] + '.rst')
+        response_file = os.path.join(responses_dir, response + '.rst')
         if not os.path.exists(response_file):
             header = '``{}``'.format(response)
             divider = '=' * len(header)
-            body = '\n.. automodule:: {}\n    :members:\n'.format(response)
+            body = '\n.. automodule:: {}.{}\n    :members:\n'.format(response_module.__name__, response)
             out = '\n'.join([header, divider, body])
 
             with open(response_file, 'w') as f:
@@ -159,7 +167,7 @@ Responses
 ''')
 
         for response in all_responses:
-            f.write('    platypush/responses/' + response[len('platypush.message.response.'):] + '.rst\n')
+            f.write('    platypush/responses/' + response + '.rst\n')
 
 
 generate_plugins_doc()

platypush/__init__.py

@@ -23,13 +23,13 @@ from .message.response import Response
 from .utils import set_thread_name, get_enabled_plugins
 
 __author__ = 'Fabio Manganiello <info@fabiomanganiello.com>'
-__version__ = '0.23.1'
+__version__ = '0.23.6'
 
 logger = logging.getLogger('platypush')
 
 
 class Daemon:
-    """ Main class for the Platypush daemon """
+    """Main class for the Platypush daemon"""
 
     # Configuration file (default: either ~/.config/platypush/config.yaml or
     # /etc/platypush/config.yaml
@@ -51,8 +51,15 @@ class Daemon:
     # number of executions retries before a request fails
     n_tries = 2
 
-    def __init__(self, config_file=None, pidfile=None, requests_to_process=None,
+    def __init__(
-                 no_capture_stdout=False, no_capture_stderr=False, redis_queue=None):
+        self,
+        config_file=None,
+        pidfile=None,
+        requests_to_process=None,
+        no_capture_stdout=False,
+        no_capture_stderr=False,
+        redis_queue=None,
+    ):
         """
         Constructor
         Params:
@@ -80,8 +87,11 @@ class Daemon:
         logging.basicConfig(**Config.get('logging'))
 
         redis_conf = Config.get('backend.redis') or {}
-        self.bus = RedisBus(redis_queue=self.redis_queue, on_message=self.on_message(),
+        self.bus = RedisBus(
-                            **redis_conf.get('redis_args', {}))
+            redis_queue=self.redis_queue,
+            on_message=self.on_message(),
+            **redis_conf.get('redis_args', {})
+        )
 
         self.no_capture_stdout = no_capture_stdout
         self.no_capture_stderr = no_capture_stderr
@@ -98,33 +108,59 @@ class Daemon:
             args -- Your sys.argv[1:] [List of strings]
         """
         parser = argparse.ArgumentParser()
-        parser.add_argument('--config', '-c', dest='config', required=False,
+        parser.add_argument(
-                            default=None, help=cls.config_file.__doc__)
+            '--config',
-        parser.add_argument('--pidfile', '-P', dest='pidfile', required=False,
+            '-c',
-                            default=None, help="File where platypush will " +
+            dest='config',
-                            "store its PID, useful if you're planning to " +
+            required=False,
-                            "integrate it in a service")
+            default=None,
-        parser.add_argument('--no-capture-stdout', dest='no_capture_stdout',
+            help=cls.config_file.__doc__,
-                            required=False, action='store_true',
+        )
-                            help="Set this flag if you have max stack depth " +
+        parser.add_argument(
-                            "exceeded errors so stdout won't be captured by " +
+            '--pidfile',
-                            "the logging system")
+            '-P',
-        parser.add_argument('--no-capture-stderr', dest='no_capture_stderr',
+            dest='pidfile',
-                            required=False, action='store_true',
+            required=False,
-                            help="Set this flag if you have max stack depth " +
+            default=None,
-                            "exceeded errors so stderr won't be captured by " +
+            help="File where platypush will "
-                            "the logging system")
+            + "store its PID, useful if you're planning to "
-        parser.add_argument('--redis-queue', dest='redis_queue',
+            + "integrate it in a service",
-                            required=False, action='store_true',
+        )
-                            default=cls._default_redis_queue,
+        parser.add_argument(
-                            help="Name of the Redis queue to be used to internally deliver messages "
+            '--no-capture-stdout',
-                                 "(default: platypush/bus)")
+            dest='no_capture_stdout',
+            required=False,
+            action='store_true',
+            help="Set this flag if you have max stack depth "
+            + "exceeded errors so stdout won't be captured by "
+            + "the logging system",
+        )
+        parser.add_argument(
+            '--no-capture-stderr',
+            dest='no_capture_stderr',
+            required=False,
+            action='store_true',
+            help="Set this flag if you have max stack depth "
+            + "exceeded errors so stderr won't be captured by "
+            + "the logging system",
+        )
+        parser.add_argument(
+            '--redis-queue',
+            dest='redis_queue',
+            required=False,
+            default=cls._default_redis_queue,
+            help="Name of the Redis queue to be used to internally deliver messages "
+            "(default: platypush/bus)",
+        )
 
         opts, args = parser.parse_known_args(args)
-        return cls(config_file=opts.config, pidfile=opts.pidfile,
+        return cls(
-                   no_capture_stdout=opts.no_capture_stdout,
+            config_file=opts.config,
-                   no_capture_stderr=opts.no_capture_stderr,
+            pidfile=opts.pidfile,
-                   redis_queue=opts.redis_queue)
+            no_capture_stdout=opts.no_capture_stdout,
+            no_capture_stderr=opts.no_capture_stderr,
+            redis_queue=opts.redis_queue,
+        )
 
     def on_message(self):
         """
@@ -145,8 +181,10 @@ class Daemon:
                     logger.info('Dropped unauthorized request: {}'.format(msg))
 
                 self.processed_requests += 1
-                if self.requests_to_process \
+                if (
-                        and self.processed_requests >= self.requests_to_process:
+                    self.requests_to_process
+                    and self.processed_requests >= self.requests_to_process
+                ):
                     self.stop_app()
             elif isinstance(msg, Response):
                 logger.info('Received response: {}'.format(msg))
@@ -158,7 +196,7 @@ class Daemon:
         return _f
 
     def stop_app(self):
-        """ Stops the backends and the bus """
+        """Stops the backends and the bus"""
         from .plugins import RunnablePlugin
 
         for backend in self.backends.values():
@@ -173,7 +211,7 @@ class Daemon:
             self.cron_scheduler.stop()
 
     def run(self):
-        """ Start the daemon """
+        """Start the daemon"""
         if not self.no_capture_stdout:
             sys.stdout = Logger(logger.info)
         if not self.no_capture_stderr:

platypush/backend/clipboard/__init__.py

@@ -1,40 +0,0 @@
-import time
-from typing import Optional
-
-import pyperclip
-
-from platypush.backend import Backend
-from platypush.message.event.clipboard import ClipboardEvent
-
-
-class ClipboardBackend(Backend):
-    """
-    This backend monitors for changes in the clipboard and generates even when the user copies a new text.
-
-    Requires:
-
-        - **pyperclip** (``pip install pyperclip``)
-
-    Triggers:
-
-        - :class:`platypush.message.event.clipboard.ClipboardEvent` on clipboard update.
-
-    """
-
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        self._last_text: Optional[str] = None
-
-    def run(self):
-        self.logger.info('Started clipboard monitor backend')
-        while not self.should_stop():
-            text = pyperclip.paste()
-            if text and text != self._last_text:
-                self.bus.post(ClipboardEvent(text=text))
-
-            self._last_text = text
-            time.sleep(0.1)
-
-        self.logger.info('Stopped clipboard monitor backend')
-
-# vim:sw=4:ts=4:et:

platypush/backend/clipboard/manifest.yaml

@@ -1,8 +0,0 @@
-manifest:
-  events:
-    platypush.message.event.clipboard.ClipboardEvent: on clipboard update.
-  install:
-    pip:
-    - pyperclip
-  package: platypush.backend.clipboard
-  type: backend

platypush/backend/http/__init__.py

@@ -91,14 +91,16 @@ class HttpBackend(Backend):
                                  other music plugin enabled. -->
                             <Music class="col-3" />
 
-                            <!-- Show current date, time and weather. It requires a `weather` plugin or backend enabled -->
+                            <!-- Show current date, time and weather.
+                                 It requires a `weather` plugin or backend enabled -->
                             <DateTimeWeather class="col-3" />
                         </Row>
 
                         <!-- Display the following widgets on a second row -->
                         <Row>
                             <!-- Show a carousel of images from a local folder. For security reasons, the folder must be
-                                 explicitly exposed as an HTTP resource through the backend `resource_dirs` attribute. -->
+                                 explicitly exposed as an HTTP resource through the backend
+                                 `resource_dirs` attribute. -->
                             <ImageCarousel class="col-6" img-dir="/mnt/hd/photos/carousel" />
 
                             <!-- Show the news headlines parsed from a list of RSS feed and stored locally through the
@@ -151,11 +153,7 @@ class HttpBackend(Backend):
 
     Requires:
 
-        * **flask** (``pip install flask``)
+        * **gunicorn** (``pip install gunicorn``) - optional, to run the Platypush webapp over uWSGI.
-        * **bcrypt** (``pip install bcrypt``)
-        * **magic** (``pip install python-magic``), optional, for MIME type
-            support if you want to enable media streaming
-        * **gunicorn** (``pip install gunicorn``) - optional but recommended.
 
     By default the Platypush web server will run in a
     process spawned on the fly by the HTTP backend. However, being a
@@ -174,12 +172,22 @@ class HttpBackend(Backend):
     _DEFAULT_HTTP_PORT = 8008
     _DEFAULT_WEBSOCKET_PORT = 8009
 
-    def __init__(self, port=_DEFAULT_HTTP_PORT,
+    def __init__(
-                 websocket_port=_DEFAULT_WEBSOCKET_PORT,
+        self,
-                 bind_address='0.0.0.0',
+        port=_DEFAULT_HTTP_PORT,
-                 disable_websocket=False, resource_dirs=None,
+        websocket_port=_DEFAULT_WEBSOCKET_PORT,
-                 ssl_cert=None, ssl_key=None, ssl_cafile=None, ssl_capath=None,
+        bind_address='0.0.0.0',
-                 maps=None, run_externally=False, uwsgi_args=None, **kwargs):
+        disable_websocket=False,
+        resource_dirs=None,
+        ssl_cert=None,
+        ssl_key=None,
+        ssl_cafile=None,
+        ssl_capath=None,
+        maps=None,
+        run_externally=False,
+        uwsgi_args=None,
+        **kwargs
+    ):
         """
         :param port: Listen port for the web server (default: 8008)
         :type port: int
@@ -246,26 +254,37 @@ class HttpBackend(Backend):
         self.bind_address = bind_address
 
         if resource_dirs:
-            self.resource_dirs = {name: os.path.abspath(
+            self.resource_dirs = {
-                os.path.expanduser(d)) for name, d in resource_dirs.items()}
+                name: os.path.abspath(os.path.expanduser(d))
+                for name, d in resource_dirs.items()
+            }
         else:
             self.resource_dirs = {}
 
         self.active_websockets = set()
         self.run_externally = run_externally
         self.uwsgi_args = uwsgi_args or []
-        self.ssl_context = get_ssl_server_context(ssl_cert=ssl_cert,
+        self.ssl_context = (
-                                                  ssl_key=ssl_key,
+            get_ssl_server_context(
-                                                  ssl_cafile=ssl_cafile,
+                ssl_cert=ssl_cert,
-                                                  ssl_capath=ssl_capath) \
+                ssl_key=ssl_key,
-            if ssl_cert else None
+                ssl_cafile=ssl_cafile,
+                ssl_capath=ssl_capath,
+            )
+            if ssl_cert
+            else None
+        )
 
         if self.uwsgi_args:
-            self.uwsgi_args = [str(_) for _ in self.uwsgi_args] + \
+            self.uwsgi_args = [str(_) for _ in self.uwsgi_args] + [
-                ['--module', 'platypush.backend.http.uwsgi', '--enable-threads']
+                '--module',
+                'platypush.backend.http.uwsgi',
+                '--enable-threads',
+            ]
 
-        self.local_base_url = '{proto}://localhost:{port}'.\
+        self.local_base_url = '{proto}://localhost:{port}'.format(
-            format(proto=('https' if ssl_cert else 'http'), port=self.port)
+            proto=('https' if ssl_cert else 'http'), port=self.port
+        )
 
         self._websocket_lock_timeout = 10
         self._websocket_lock = threading.RLock()
@@ -275,7 +294,7 @@ class HttpBackend(Backend):
         self.logger.warning('Use cURL or any HTTP client to query the HTTP backend')
 
     def on_stop(self):
-        """ On backend stop """
+        """On backend stop"""
         super().on_stop()
         self.logger.info('Received STOP event on HttpBackend')
 
@@ -284,7 +303,9 @@ class HttpBackend(Backend):
                 self.server_proc.kill()
                 self.server_proc.wait(timeout=10)
                 if self.server_proc.poll() is not None:
-                    self.logger.info('HTTP server process may be still alive at termination')
+                    self.logger.info(
+                        'HTTP server process may be still alive at termination'
+                    )
                 else:
                     self.logger.info('HTTP server process terminated')
             else:
@@ -293,17 +314,25 @@ class HttpBackend(Backend):
                 if self.server_proc.is_alive():
                     self.server_proc.kill()
                 if self.server_proc.is_alive():
-                    self.logger.info('HTTP server process may be still alive at termination')
+                    self.logger.info(
+                        'HTTP server process may be still alive at termination'
+                    )
                 else:
                     self.logger.info('HTTP server process terminated')
 
-        if self.websocket_thread and self.websocket_thread.is_alive() and self._websocket_loop:
+        if (
+            self.websocket_thread
+            and self.websocket_thread.is_alive()
+            and self._websocket_loop
+        ):
             self._websocket_loop.stop()
             self.logger.info('HTTP websocket service terminated')
 
     def _acquire_websocket_lock(self, ws):
         try:
-            acquire_ok = self._websocket_lock.acquire(timeout=self._websocket_lock_timeout)
+            acquire_ok = self._websocket_lock.acquire(
+                timeout=self._websocket_lock_timeout
+            )
             if not acquire_ok:
                 raise TimeoutError('Websocket lock acquire timeout')
 
@@ -313,13 +342,19 @@ class HttpBackend(Backend):
         finally:
             self._websocket_lock.release()
 
-        acquire_ok = self._websocket_locks[addr].acquire(timeout=self._websocket_lock_timeout)
+        acquire_ok = self._websocket_locks[addr].acquire(
+            timeout=self._websocket_lock_timeout
+        )
         if not acquire_ok:
-            raise TimeoutError('Websocket on address {} not ready to receive data'.format(addr))
+            raise TimeoutError(
+                'Websocket on address {} not ready to receive data'.format(addr)
+            )
 
     def _release_websocket_lock(self, ws):
         try:
-            acquire_ok = self._websocket_lock.acquire(timeout=self._websocket_lock_timeout)
+            acquire_ok = self._websocket_lock.acquire(
+                timeout=self._websocket_lock_timeout
+            )
             if not acquire_ok:
                 raise TimeoutError('Websocket lock acquire timeout')
 
@@ -327,12 +362,15 @@ class HttpBackend(Backend):
             if addr in self._websocket_locks:
                 self._websocket_locks[addr].release()
         except Exception as e:
-            self.logger.warning('Unhandled exception while releasing websocket lock: {}'.format(str(e)))
+            self.logger.warning(
+                'Unhandled exception while releasing websocket lock: {}'.format(str(e))
+            )
         finally:
             self._websocket_lock.release()
 
     def notify_web_clients(self, event):
-        """ Notify all the connected web clients (over websocket) of a new event """
+        """Notify all the connected web clients (over websocket) of a new event"""
+
         async def send_event(ws):
             try:
                 self._acquire_websocket_lock(ws)
@@ -349,26 +387,35 @@ class HttpBackend(Backend):
             try:
                 loop.run_until_complete(send_event(_ws))
             except ConnectionClosed:
-                self.logger.warning('Websocket client {} connection lost'.format(_ws.remote_address))
+                self.logger.warning(
+                    'Websocket client {} connection lost'.format(_ws.remote_address)
+                )
                 self.active_websockets.remove(_ws)
                 if _ws.remote_address in self._websocket_locks:
                     del self._websocket_locks[_ws.remote_address]
 
     def websocket(self):
-        """ Websocket main server """
+        """Websocket main server"""
         set_thread_name('WebsocketServer')
 
         async def register_websocket(websocket, path):
-            address = websocket.remote_address if websocket.remote_address \
+            address = (
+                websocket.remote_address
+                if websocket.remote_address
                 else '<unknown client>'
+            )
 
-            self.logger.info('New websocket connection from {} on path {}'.format(address, path))
+            self.logger.info(
+                'New websocket connection from {} on path {}'.format(address, path)
+            )
             self.active_websockets.add(websocket)
 
             try:
                 await websocket.recv()
             except ConnectionClosed:
-                self.logger.info('Websocket client {} closed connection'.format(address))
+                self.logger.info(
+                    'Websocket client {} closed connection'.format(address)
+                )
                 self.active_websockets.remove(websocket)
                 if address in self._websocket_locks:
                     del self._websocket_locks[address]
@@ -379,8 +426,13 @@ class HttpBackend(Backend):
 
         self._websocket_loop = get_or_create_event_loop()
         self._websocket_loop.run_until_complete(
-            websocket_serve(register_websocket, self.bind_address, self.websocket_port,
+            websocket_serve(
-                             **websocket_args))
+                register_websocket,
+                self.bind_address,
+                self.websocket_port,
+                **websocket_args
+            )
+        )
         self._websocket_loop.run_forever()
 
     def _start_web_server(self):
@@ -415,8 +467,9 @@ class HttpBackend(Backend):
             self.websocket_thread.start()
 
         if not self.run_externally:
-            self.server_proc = Process(target=self._start_web_server(),
+            self.server_proc = Process(
-                                       name='WebServer')
+                target=self._start_web_server(), name='WebServer'
+            )
             self.server_proc.start()
             self.server_proc.join()
         elif self.uwsgi_args:
@@ -424,9 +477,11 @@ class HttpBackend(Backend):
             self.logger.info('Starting uWSGI with arguments {}'.format(uwsgi_cmd))
             self.server_proc = subprocess.Popen(uwsgi_cmd)
         else:
-            self.logger.info('The web server is configured to be launched externally but ' +
+            self.logger.info(
-                             'no uwsgi_args were provided. Make sure that you run another external service' +
+                'The web server is configured to be launched externally but '
-                             'for the webserver (e.g. nginx)')
+                + 'no uwsgi_args were provided. Make sure that you run another external service'
+                + 'for the webserver (e.g. nginx)'
+            )
 
 
 # vim:sw=4:ts=4:et:

platypush/backend/http/app/routes/execute.py

@@ -16,12 +16,14 @@ __routes__ = [
 @execute.route('/execute', methods=['POST'])
 @authenticate()
 def execute():
-    """ Endpoint to execute commands """
+    """Endpoint to execute commands"""
     try:
         msg = json.loads(request.data.decode('utf-8'))
     except Exception as e:
-        logger().error('Unable to parse JSON from request {}: {}'.format(request.data, str(e)))
+        logger().error(
-        return abort(400, str(e))
+            'Unable to parse JSON from request {}: {}'.format(request.data, str(e))
+        )
+        abort(400, str(e))
 
     logger().info('Received message on the HTTP backend: {}'.format(msg))
 
@@ -29,8 +31,10 @@ def execute():
         response = send_message(msg)
         return Response(str(response or {}), mimetype='application/json')
     except Exception as e:
-        logger().error('Error while running HTTP action: {}. Request: {}'.format(str(e), msg))
+        logger().error(
-        return abort(500, str(e))
+            'Error while running HTTP action: {}. Request: {}'.format(str(e), msg)
+        )
+        abort(500, str(e))
 
 
 # vim:sw=4:ts=4:et:

platypush/backend/http/app/routes/hook.py

@@ -1,9 +1,11 @@
 import json
 
-from flask import Blueprint, abort, request, Response
+from flask import Blueprint, abort, request, make_response
 
 from platypush.backend.http.app import template_folder
 from platypush.backend.http.app.utils import logger, send_message
+from platypush.config import Config
+from platypush.event.hook import EventCondition
 from platypush.message.event.http.hook import WebhookEvent
 
 
@@ -15,9 +17,23 @@ __routes__ = [
 ]
 
 
-@hook.route('/hook/<hook_name>', methods=['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'OPTIONS'])
+def matches_condition(event: WebhookEvent, hook):
-def _hook(hook_name):
+    if isinstance(hook, dict):
-    """ Endpoint for custom webhooks """
+        if_ = hook['if'].copy()
+        if_['type'] = '.'.join([event.__module__, event.__class__.__qualname__])
+
+        condition = EventCondition.build(if_)
+    else:
+        condition = hook.condition
+
+    return event.matches_condition(condition)
+
+
+@hook.route(
+    '/hook/<hook_name>', methods=['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'OPTIONS']
+)
+def hook_route(hook_name):
+    """Endpoint for custom webhooks"""
 
     event_args = {
         'hook': hook_name,
@@ -28,20 +44,54 @@ def _hook(hook_name):
     }
 
     if event_args['data']:
-        # noinspection PyBroadException
         try:
             event_args['data'] = json.loads(event_args['data'])
         except Exception as e:
-            logger().warning('Not a valid JSON string: {}: {}'.format(event_args['data'], str(e)))
+            logger().warning(
+                'Not a valid JSON string: %s: %s', event_args['data'], str(e)
+            )
 
     event = WebhookEvent(**event_args)
+    matching_hooks = [
+        hook
+        for hook in Config.get_event_hooks().values()
+        if matches_condition(event, hook)
+    ]
 
     try:
         send_message(event)
-        return Response(json.dumps({'status': 'ok', **event_args}), mimetype='application/json')
+        rs = default_rs = make_response(json.dumps({'status': 'ok', **event_args}))
+        headers = {}
+        status_code = 200
+
+        # If there are matching hooks, wait for their completion before returning
+        if matching_hooks:
+            rs = event.wait_response(timeout=60)
+            try:
+                rs = json.loads(rs.decode())  # type: ignore
+            except Exception:
+                pass
+
+            if isinstance(rs, dict) and '___data___' in rs:
+                # data + http_code + custom_headers return format
+                headers = rs.get('___headers___', {})
+                status_code = rs.get('___code___', status_code)
+                rs = rs['___data___']
+
+            if rs is None:
+                rs = default_rs
+                headers = {'Content-Type': 'application/json'}
+
+            rs = make_response(rs)
+        else:
+            headers = {'Content-Type': 'application/json'}
+
+        rs.status_code = status_code
+        rs.headers.update(headers)
+        return rs
     except Exception as e:
         logger().exception(e)
-        logger().error('Error while dispatching webhook event {}: {}'.format(event, str(e)))
+        logger().error('Error while dispatching webhook event %s: %s', event, str(e))
         abort(500, str(e))
 
 

platypush/backend/http/app/routes/logout.py

@@ -13,17 +13,19 @@ __routes__ = [
 
 @logout.route('/logout', methods=['GET', 'POST'])
 def logout():
-    """ Logout page """
+    """Logout page"""
     user_manager = UserManager()
-    redirect_page = request.args.get('redirect', request.headers.get('Referer', '/login'))
+    redirect_page = request.args.get(
+        'redirect', request.headers.get('Referer', '/login')
+    )
     session_token = request.cookies.get('session_token')
 
     if not session_token:
-        return abort(417, 'Not logged in')
+        abort(417, 'Not logged in')
 
-    user, session = user_manager.authenticate_user_session(session_token)
+    user, _ = user_manager.authenticate_user_session(session_token)
     if not user:
-        return abort(403, 'Invalid session token')
+        abort(403, 'Invalid session token')
 
     redirect_target = redirect(redirect_page, 302)  # lgtm [py/url-redirection]
     response = make_response(redirect_target)

platypush/backend/http/app/routes/plugins/camera/__init__.py

@@ -1,7 +1,8 @@
 import json
 from typing import Optional
 
-from flask import Response, Blueprint, request
+from flask import Blueprint, request
+from flask.wrappers import Response
 
 from platypush.backend.http.app import template_folder
 from platypush.backend.http.app.utils import authenticate
@@ -17,19 +18,22 @@ __routes__ = [
 
 
 def get_camera(plugin: str) -> CameraPlugin:
-    return get_plugin('camera.' + plugin)
+    plugin_name = f'camera.{plugin}'
+    p = get_plugin(plugin_name)
+    assert p, f'No such plugin: {plugin_name}'
+    return p
 
 
-def get_frame(session: Camera, timeout: Optional[float] = None) -> bytes:
+def get_frame(session: Camera, timeout: Optional[float] = None) -> Optional[bytes]:
-    with session.stream.ready:
+    if session.stream:
-        session.stream.ready.wait(timeout=timeout)
+        with session.stream.ready:
-        return session.stream.frame
+            session.stream.ready.wait(timeout=timeout)
+            return session.stream.frame
 
 
-def feed(plugin: str, **kwargs):
+def feed(camera: CameraPlugin, **kwargs):
-    plugin = get_camera(plugin)
+    with camera.open(**kwargs) as session:
-    with plugin.open(stream=True, **kwargs) as session:
+        camera.start_camera(session)
-        plugin.start_camera(session)
         while True:
             frame = get_frame(session, timeout=5.0)
             if frame:
@@ -77,8 +81,12 @@ def get_photo(plugin, extension):
 @authenticate()
 def get_video(plugin, extension):
     stream_class = StreamWriter.get_class_by_name(extension)
-    return Response(feed(plugin, stream_format=extension, frames_dir=None, **get_args(request.args)),
+    camera = get_camera(plugin)
-                    mimetype=stream_class.mimetype)
+    return Response(
+        feed(camera, stream=True, stream_format=extension, frames_dir=None,
+            **get_args(request.args)
+        ), mimetype=stream_class.mimetype
+    )
 
 
 @camera.route('/camera/<plugin>/photo', methods=['GET'])

platypush/backend/http/app/routes/plugins/tts/mimic3.py

@@ -0,0 +1,46 @@
+import requests
+from urllib.parse import urljoin
+
+from flask import abort, request, Blueprint
+
+from platypush.backend.http.app import template_folder
+
+mimic3 = Blueprint('mimic3', __name__, template_folder=template_folder)
+
+# Declare routes list
+__routes__ = [
+    mimic3,
+]
+
+
+@mimic3.route('/tts/mimic3/say', methods=['GET'])
+def proxy_tts_request():
+    """
+    This route is used to proxy the POST request to the Mimic3 TTS server
+    through a GET, so it can be easily processed as a URL through a media
+    plugin.
+    """
+    required_args = {
+        'text',
+        'server_url',
+        'voice',
+    }
+
+    missing_args = required_args.difference(set(request.args.keys()))
+    if missing_args:
+        abort(400, f'Missing parameters: {missing_args}')
+
+    args = {arg: request.args[arg] for arg in required_args}
+
+    rs = requests.post(
+        urljoin(args['server_url'], '/api/tts'),
+        data=args['text'],
+        params={
+            'voice': args['voice'],
+        },
+    )
+
+    return rs.content
+
+
+# vim:sw=4:ts=4:et:

platypush/backend/http/app/utils.py

@@ -35,13 +35,15 @@ def logger():
             'format': '%(asctime)-15s|%(levelname)5s|%(name)s|%(message)s',
         }
 
-        level = (Config.get('backend.http') or {}).get('logging') or \
+        level = (Config.get('backend.http') or {}).get('logging') or (
-            (Config.get('logging') or {}).get('level')
+            Config.get('logging') or {}
+        ).get('level')
         filename = (Config.get('backend.http') or {}).get('filename')
 
         if level:
-            log_args['level'] = getattr(logging, level.upper()) \
+            log_args['level'] = (
-                if isinstance(level, str) else level
+                getattr(logging, level.upper()) if isinstance(level, str) else level
+            )
         if filename:
             log_args['filename'] = filename
 
@@ -65,6 +67,7 @@ def get_message_response(msg):
 # noinspection PyProtectedMember
 def get_http_port():
     from platypush.backend.http import HttpBackend
+
     http_conf = Config.get('backend.http')
     return http_conf.get('port', HttpBackend._DEFAULT_HTTP_PORT)
 
@@ -72,6 +75,7 @@ def get_http_port():
 # noinspection PyProtectedMember
 def get_websocket_port():
     from platypush.backend.http import HttpBackend
+
     http_conf = Config.get('backend.http')
     return http_conf.get('websocket_port', HttpBackend._DEFAULT_WEBSOCKET_PORT)
 
@@ -89,17 +93,13 @@ def send_message(msg, wait_for_response=True):
 
     if isinstance(msg, Request) and wait_for_response:
         response = get_message_response(msg)
-        logger().debug('Processing response on the HTTP backend: {}'.
+        logger().debug('Processing response on the HTTP backend: {}'.format(response))
-                       format(response))
 
         return response
 
 
 def send_request(action, wait_for_response=True, **kwargs):
-    msg = {
+    msg = {'type': 'request', 'action': action}
-        'type': 'request',
-        'action': action
-    }
 
     if kwargs:
         msg['args'] = kwargs
@@ -113,8 +113,10 @@ def _authenticate_token():
 
     if 'X-Token' in request.headers:
         user_token = request.headers['X-Token']
-    elif 'Authorization' in request.headers and request.headers['Authorization'].startswith('Bearer '):
+    elif 'Authorization' in request.headers and request.headers[
-        user_token = request.headers['Authorization'][len('Bearer '):]
+        'Authorization'
+    ].startswith('Bearer '):
+        user_token = request.headers['Authorization'][7:]
     elif 'token' in request.args:
         user_token = request.args.get('token')
     else:
@@ -176,7 +178,10 @@ def _authenticate_csrf_token():
     if user is None:
         return False
 
-    return session.csrf_token is None or request.form.get('csrf_token') == session.csrf_token
+    return (
+        session.csrf_token is None
+        or request.form.get('csrf_token') == session.csrf_token
+    )
 
 
 def authenticate(redirect_page='', skip_auth_methods=None, check_csrf_token=False):
@@ -208,7 +213,9 @@ def authenticate(redirect_page='', skip_auth_methods=None, check_csrf_token=Fals
                 if session_auth_ok:
                     return f(*args, **kwargs)
 
-                return redirect('/login?redirect=' + (redirect_page or request.url), 307)
+                return redirect(
+                    '/login?redirect=' + (redirect_page or request.url), 307
+                )
 
             # CSRF token check
             if check_csrf_token:
@@ -217,15 +224,22 @@ def authenticate(redirect_page='', skip_auth_methods=None, check_csrf_token=Fals
                     return abort(403, 'Invalid or missing csrf_token')
 
             if n_users == 0 and 'session' not in skip_methods:
-                return redirect('/register?redirect=' + (redirect_page or request.url), 307)
+                return redirect(
+                    '/register?redirect=' + (redirect_page or request.url), 307
+                )
 
-            if ('http' not in skip_methods and http_auth_ok) or \
+            if (
-                    ('token' not in skip_methods and token_auth_ok) or \
+                ('http' not in skip_methods and http_auth_ok)
-                    ('session' not in skip_methods and session_auth_ok):
+                or ('token' not in skip_methods and token_auth_ok)
+                or ('session' not in skip_methods and session_auth_ok)
+            ):
                 return f(*args, **kwargs)
 
-            return Response('Authentication required', 401,
+            return Response(
-                            {'WWW-Authenticate': 'Basic realm="Login required"'})
+                'Authentication required',
+                401,
+                {'WWW-Authenticate': 'Basic realm="Login required"'},
+            )
 
         return wrapper
 
@@ -233,42 +247,57 @@ def authenticate(redirect_page='', skip_auth_methods=None, check_csrf_token=Fals
 
 
 def get_routes():
-    routes_dir = os.path.join(
+    routes_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'routes')
-        os.path.dirname(os.path.abspath(__file__)), 'routes')
     routes = []
     base_module = '.'.join(__name__.split('.')[:-1])
 
-    for path, dirs, files in os.walk(routes_dir):
+    for path, _, files in os.walk(routes_dir):
         for f in files:
             if f.endswith('.py'):
                 mod_name = '.'.join(
-                    (base_module + '.' + os.path.join(path, f).replace(
+                    (
-                        os.path.dirname(__file__), '')[1:].replace(os.sep, '.')).split('.')
+                        base_module
-                    [:(-2 if f == '__init__.py' else -1)])
+                        + '.'
+                        + os.path.join(path, f)
+                        .replace(os.path.dirname(__file__), '')[1:]
+                        .replace(os.sep, '.')
+                    ).split('.')[: (-2 if f == '__init__.py' else -1)]
+                )
 
                 try:
                     mod = importlib.import_module(mod_name)
                     if hasattr(mod, '__routes__'):
                         routes.extend(mod.__routes__)
                 except Exception as e:
-                    logger().warning('Could not import routes from {}/{}: {}: {}'.
+                    logger().warning(
-                                     format(path, f, type(e), str(e)))
+                        'Could not import routes from {}/{}: {}: {}'.format(
+                            path, f, type(e), str(e)
+                        )
+                    )
 
     return routes
 
 
 def get_local_base_url():
     http_conf = Config.get('backend.http') or {}
-    return '{proto}://localhost:{port}'.format(
+    bind_address = http_conf.get('bind_address')
+    if not bind_address or bind_address == '0.0.0.0':
+        bind_address = 'localhost'
+
+    return '{proto}://{host}:{port}'.format(
         proto=('https' if http_conf.get('ssl_cert') else 'http'),
-        port=get_http_port())
+        host=bind_address,
+        port=get_http_port(),
+    )
 
 
 def get_remote_base_url():
     http_conf = Config.get('backend.http') or {}
     return '{proto}://{host}:{port}'.format(
         proto=('https' if http_conf.get('ssl_cert') else 'http'),
-        host=get_ip_or_hostname(), port=get_http_port())
+        host=get_ip_or_hostname(),
+        port=get_http_port(),
+    )
 
 
 # vim:sw=4:ts=4:et:

platypush/backend/http/manifest.yaml

@@ -2,9 +2,6 @@ manifest:
   events: {}
   install:
     pip:
-    - flask
-    - bcrypt
-    - python-magic
     - gunicorn
   package: platypush.backend.http
   type: backend

platypush/backend/http/webapp/.eslintrc.js

@@ -0,0 +1,15 @@
+module.exports = {
+  root: true,
+  env: {
+    browser: true,
+    node: true,
+  },
+  extends: [
+    'plugin:vue/vue3-essential',
+    'eslint:recommended',
+  ],
+  plugins: [],
+  rules: {
+    'vue/multi-word-component-names': 0,
+  },
+}