Removed deprecated setup.py web_build command from platydock [see #195]

- Support for custom PopcornTime API mirror/base URL.

- Full support for TV series search.

- Fixed torrent search (now using a different PopcornTime API mirror).

- Migrated SASS engine from `node-sass` (currently deprecated and broken on Node 16) to `sass`.

- Fixed alignment of Z-Wave UI header on Chrome/Webkit.

.gitignore

@@ -18,5 +18,4 @@ platypush/notebooks
 platypush/requests
 /http-client.env.json
 /platypush/backend/http/static/css/dist
-/tests/etc/scripts
 /tests/etc/dashboards

.readthedocs.yml

@@ -1,7 +0,0 @@
-build:
-  image: latest
-
-python:
-  version: 3.7
-  setup_py_install: true
-

.travis.requirements

@@ -1,12 +0,0 @@
-pyyaml
-requests
-flask
-redis
-python-dateutil
-websockets
-bcrypt
-sqlalchemy
-croniter
-zeroconf>=0.27.0
-pyjwt
-pytest

.travis.yml

@@ -1,22 +0,0 @@
-language: python
-dist: xenial
-python:
-  - "3.7"
-
-install: "pip install -r .travis.requirements"
-
-script: pytest
-
-notifications:
-    email:
-        recipients:
-            - blacklight86@gmail.com
-        on_success: change
-        on_failure: change
-
-services:
-    - redis-server
-
-git:
-    submodules: false
-

CHANGELOG.md

@@ -3,6 +3,149 @@
 All notable changes to this project will be documented in this file.
 Given the high speed of development in the first phase, changes are being reported only starting from v0.20.2.
 
+## [Unreleased]
+
+- Added `music.spotify.connect` backend to emulate a Spotify Connect receiver through Platypush.
+
+## [0.21.1] - 2021-06-22
+
+### Added
+
+- Added `switchbot` plugin to interact with Switchbot devices over the cloud API instead of
+  directly accessing the device's Bluetooth interface.
+  
+- Added `marshmallow` dependency - it will be used from now own to dump and document schemas
+  and responses instead of the currently mixed approach with `Response` objects and plain
+  dictionaries and lists.
+  
+- Support for custom MQTT timeout on all the `zwavejs2mqtt` calls.
+
+- Added generic joystick backend `backend.joystick.jstest` which uses `jstest` from the
+  standard `joystick` system package to read the state of joysticks not compatible with
+  `python-inputs`.
+  
+- Added PWM PCA9685 plugin.
+
+- Added Linux native joystick plugin, ``backend.joystick.linux``, for the cases where
+  ``python-inputs`` doesn't work and ``jstest`` is too slow.
+
+### Changed
+
+- `switch.switchbot` plugin renamed to `switchbot.bluetooth` plugin, while the new plugin
+  that uses the Switchbot API is simply named `switchbot`.
+
+### Fixed
+
+- More robust reconnection logic on the Pushbullet backend in case of websocket errors.
+
+## [0.21.0] - 2021-05-06
+
+### Added
+
+- Support for custom PopcornTime API mirror/base URL.
+
+- Full support for TV series search.
+
+### Fixed
+
+- Fixed torrent search (now using a different PopcornTime API mirror).
+
+- Migrated SASS engine from `node-sass` (currently deprecated and broken on Node 16) to `sass`.
+
+- Fixed alignment of Z-Wave UI header on Chrome/Webkit.
+
+## [0.20.10] - 2021-04-28
+
+### Fixed
+
+- Fixed zwave/zwavejs2mqtt interoperability.
+
+## [0.20.9] - 2021-04-12
+
+### Added
+
+- Added zwavejs2mqtt integration (see [#186](https://git.platypush.tech/platypush/platypush/-/issues/186).
+
+### Fixed
+
+- Major LINT fixes.
+
+### Removed
+
+- Removed unmaintained integrations: TorrentCast and Booking.com
+
+## [0.20.8] - 2021-04-04
+
+### Added
+
+- Added `<Camera>` dashboard widget.
+
+- Added support for custom dashboard widgets with customized (see https://git.platypush.tech/platypush/platypush/-/wikis/Backends#creating-custom-widgets).
+
+- Added support for controls on `music.mpd` dashboard widget.
+
+### Fixed
+
+- Fixed zigbee2mqtt backend error in case of messages with empty payload (see [#184](https://git.platypush.tech/platypush/platypush/-/issues/184)).
+
+- Fixed compatibility with all versions of websocket-client - versions >= 0.58.0 pass a `WebSocketApp` object as a first
+  argument to the callbacks, as well as versions < 0.54.0 do, but the versions in between don't pass this argument.
+
+## [0.20.7] - 2021-03-26
+
+### Fixed
+
+- Fixed race condition on `media.vlc.stop` when clearing the VLC instance.
+
+- Fixed dashboard widgets custom classes being propagated both to the container and to the widget content [see #179]
+
+- Fixed compatibility with SQLAlchemy >= 1.4.
+
+## [0.20.6] - 2021-03-16
+
+### Added
+
+- Added `log.http` backend to monitor changes to HTTP log files
+  (see [#167](https://git.platypush.tech/platypush/platypush/-/issues/167)).
+
+- Added `file.monitor` backend, which replaces the `inotify` backend
+  (see [#172](https://git.platypush.tech/platypush/platypush/-/issues/172)).
+
+### Removed
+
+- Removed legacy `pusher` script and `local` backend.
+
+### Fixed
+
+- Fixed support for Z-Wave switches.
+
+- Fixed possible race condition on VLC stop.
+
+## [0.20.5] - 2021-03-12
+
+### Added
+
+- Added support for a static list of devices to actively scan to the `bluetooth.scanner` backend
+  (see [#174](https://git.platypush.tech/platypush/platypush/-/issues/174)).
+  
+- Added `weather.openweathermap` plugin and backend, which replaces `weather.darksky`, since the
+  Darksky API will be completely shut down by the end of 2021.
+
+### Fixed
+
+- Cron expressions should adhere to the UNIX cronjob standard and use the machine local time,
+  not UTC, as a reference (closes [#173](https://git.platypush.tech/platypush/platypush/-/issues/173)).
+  
+- Better management of Z-Wave values types from the UI.
+
+- Disable logging for `ZwaveValueEvent` events - they tend to be very verbose and
+  can impact the performance on slower devices. They will still be published to the
+  websocket clients though, so you can still debug Z-Wave values issues from the browser
+  developer console (enable debug traces).
+  
+- Added suffix to the `zigbee.mqtt` backend default `client_id` to prevent clashes with
+  the default `mqtt` backend `client_id`.
+
 ## [0.20.4] - 2021-03-08
 
 ### Added

CONTRIBUTING.md

@@ -0,0 +1,37 @@
+Thanks for considering contributing your work to make Platypush a better product!
+
+Contributions are more than welcome, and the follow the standard Gitlab procedure:
+
+- [Fork the repo](https://git.platypush.tech/platypush/platypush).
+- Prepare your changes.
+- [Submit a merge request](https://git.platypush.tech/platypush/platypush/-/merge_requests).
+
+Guidelines:
+
+- The code should ideally have no LINT warnings/issues.
+
+- Project conventions:
+  - 4 spaces to indent.
+  - RST format for classes and methods documentation 
+  - Run `python generate_missing_docs.py` if you are adding new plugins/backends to automatically
+    generate the doc templates. Make sure that you don't accidentally remove lines elements from
+    the docs because of missing dependencies on the machine you use to generate the docs.
+  - Naming conventions: plugin classes are named `<Module>Plugin` and backend classes are
+    named `<Module>Backend`, with `<Module>` being the (camel-case) representation of the
+    Python module of the plugin without the prefix - for example, the plugin under
+    `platypush.plugins.light.hue` must be named `LightHuePlugin`.
+
+- If possible, [add a test](https://git.platypush.tech/platypush/platypush/-/tree/master/tests)
+  for the new functionality. If you have built a new functionality that works with some specific
+  device or service then it's not required to write a test that mocks the whole service, but if
+  you are changing some of the core entities (e.g. requests, events, procedures, hooks, crons
+  or the bus) then make sure to add tests and not to break the existing tests.
+
+- If the feature requires an optional dependency then make sure to document it:
+
+  - In the class docstring (see other plugins and backends for examples)
+  - In [`setup.py`](https://git.platypush.tech/platypush/platypush/-/blob/master/setup.py#L72) as
+    an `extras_require` entry
+  - In [`requirements.txt`](https://git.platypush.tech/platypush/platypush/-/blob/master/requirements.txt) -
+    if the feature is optional then leave it commented and add a one-line comment to explain which
+    plugin or backend requires it.

README.md

@@ -1,12 +1,14 @@
 Platypush
 =========
 
-[![Build Status](https://ci.platypush.tech/status.svg)](https://ci.platypush.tech)
+[![Build Status](https://ci.platypush.tech/status.svg)](https://ci.platypush.tech/latest.log)
-[![Documentation Status](https://readthedocs.org/projects/platypush/badge/?version=latest)](https://docs.platypush.tech/en/latest/)
+[![Documentation Status](https://ci.platypush.tech/docs/status.svg)](https://ci.platypush.tech/docs/latest.log)
 [![pip version](https://img.shields.io/pypi/v/platypush.svg?style=flat)](https://pypi.python.org/pypi/platypush/)
 [![License](https://img.shields.io/github/license/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/-/blob/master/LICENSE.txt)
 [![Last Commit](https://img.shields.io/github/last-commit/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/-/commits/master/)
-[![Contributions](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://git.platypush.tech/platypush/platypush/-/issues)
+[![Contributions](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://git.platypush.tech/platypush/platypush/-/blob/master/CONTRIBUTING.md)
+[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/BlackLight/platypush.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/BlackLight/platypush/context:python)
+[![Language grade: JavaScript](https://img.shields.io/lgtm/grade/javascript/g/BlackLight/platypush.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/BlackLight/platypush/context:javascript)
 
 - Recommended read: [**Getting started with Platypush**](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush).
 
@@ -14,7 +16,7 @@ Platypush
 
 - The [wiki](https://git.platypush.tech/platypush/platypush/-/wikis/home) also contains many resources on getting started.
 
-- Extensive documentation for all the available integrations and messages is available on [ReadTheDocs](https://docs.platypush.tech/en/latest/).
+- Extensive documentation for all the available integrations and messages [is available](https://docs.platypush.tech/).
 
 - If you have issues/feature requests/enhancement ideas please [create an issue](https://git.platypush.tech/platypush/platypush/-/issues).
 
@@ -277,6 +279,48 @@ curl -XPOST -H 'Content-Type: application/json' -H "Authorization: Bearer $YOUR_
   }' http://host:8008/execute
 ```
 
+### Cronjobs
+
+Cronjobs are pieces of logic that will be run at regular intervals, expressed in crontab-compatible syntax.
+They can be defined either in the `config.yaml` or as Python scripts stored under `~/.config/platypush/scripts` as
+functions labelled by the `@cron` decorator.
+
+Note that seconds are also supported (unlike the standard crontab definition), but, for back-compatibility with the
+standard crontab format, they are at the end of the cron expression, so the expression is actually in the format
+`<minute> <hour> <day_of_month> <month> <day_of_week> <second>`.
+
+YAML example for a cronjob that is executed every 30 seconds and checks if a Bluetooth device is nearby:
+
+```yaml
+cron.check_bt_device:
+  cron_expression: '* * * * * */30'
+  actions:
+    - action: bluetooth.lookup_name
+      args:
+        addr: XX:XX:XX:XX:XX:XX
+
+    - if ${name}:
+        - action: procedure.on_device_on
+    - else:
+        - action: procedure.on_device_off
+```
+
+Python example:
+
+```python
+# Content of ~/.config/platypush/scripts/bt_cron.py
+from platypush.cron import cron
+from platypush.utils import run
+
+@cron('* * * * * */30')
+def check_bt_device(**context):
+  name = run('bluetooth.lookup_name').get('name')
+  if name:
+    # on_device_on logic here
+  else:
+    # on_device_off logic here
+```
+
 ### The web interface
 
 If [`backend.http`](https://docs.platypush.tech/en/latest/platypush/backend/http.html) is enabled then a web interface
@@ -432,9 +476,15 @@ platydock stop device_id
 4. Remove the instance:
 
 ```shell
-platyvdock rm device_id
+platydock rm device_id
 ```
 
+## Mobile app
+
+An [official Android app](https://f-droid.org/en/packages/tech.platypush.platypush/) is provided on the F-Droid store.
+It allows to easily discover and manage multiple Platypush services on a network through the web interface, and it
+easily brings the power of Platypush to your fingertips.
+
 ## Tests
 
 To run the tests simply run `pytest` either from the project root folder or the `tests/` folder.

docs/README.md

@@ -0,0 +1,21 @@
+# Platypush self-generated reference
+====================================
+
+This directory contains the Sphinx self-generated documentation for Platypush.
+
+Dependencies required to generate the documentation:
+
+```shell
+$ [sudo] pip install sphinx 'git+https://github.com/bashtage/sphinx-material.git'
+```
+
+To generate the HTML documentation:
+
+```shell
+$ make html
+```
+
+The output will be generated under `build/html`.
+
+Type `make` with no additional arguments to get a full list of the supported output formats.
+

docs/source/_ext/sphinx_marshmallow.py

@@ -0,0 +1,59 @@
+import importlib
+import json
+import os
+import re
+import sys
+from typing import Union, List
+
+from docutils import nodes
+from docutils.parsers.rst import Directive
+
+
+class SchemaDirective(Directive):
+    """
+    Support for response/message schemas in the docs. Format: ``.. schema:: rel_path.SchemaClass(arg1=value1, ...)``,
+    where ``rel_path`` is the path of the schema relative to ``platypush/schemas``.
+    """
+    has_content = True
+    _schema_regex = re.compile(r'^\s*(.+?)\s*(\((.+?)\))?\s*$')
+    _schemas_path = os.path.abspath(
+        os.path.join(
+            os.path.dirname(os.path.relpath(__file__)), '..', '..', '..', 'platypush', 'schemas'))
+
+    sys.path.insert(0, _schemas_path)
+
+    @staticmethod
+    def _get_field_value(field) -> str:
+        metadata = getattr(field, 'metadata', {})
+        return metadata.get('example', metadata.get('description', str(field.__class__.__name__).lower()))
+
+    def _parse_schema(self) -> Union[dict, List[dict]]:
+        m = self._schema_regex.match('\n'.join(self.content))
+        schema_module_name = '.'.join(['platypush.schemas', *(m.group(1).split('.')[:-1])])
+        schema_module = importlib.import_module(schema_module_name)
+        schema_class = getattr(schema_module, m.group(1).split('.')[-1])
+        schema_args = eval(f'dict({m.group(3)})')
+        schema = schema_class(**schema_args)
+        output = {
+            name: self._get_field_value(field)
+            for name, field in schema.fields.items()
+            if not field.load_only
+        }
+
+        return [output] if schema.many else output
+
+    def run(self):
+        content = json.dumps(self._parse_schema(), sort_keys=True, indent=2)
+        block = nodes.literal_block(content, content)
+        block['language'] = 'json'
+        return [block]
+
+
+def setup(app):
+    app.add_directive('schema', SchemaDirective)
+
+    return {
+        'version': '0.1',
+        'parallel_read_safe': True,
+        'parallel_write_safe': True,
+    }

docs/source/backends.rst

@@ -22,6 +22,7 @@ Backends
     platypush/backend/clipboard.rst
     platypush/backend/covid19.rst
     platypush/backend/dbus.rst
+    platypush/backend/file.monitor.rst
     platypush/backend/foursquare.rst
     platypush/backend/github.rst
     platypush/backend/google.fit.rst
@@ -31,16 +32,19 @@ Backends
     platypush/backend/http.poll.rst
     platypush/backend/inotify.rst
     platypush/backend/joystick.rst
+    platypush/backend/joystick.jstest.rst
+    platypush/backend/joystick.linux.rst
     platypush/backend/kafka.rst
     platypush/backend/light.hue.rst
     platypush/backend/linode.rst
-    platypush/backend/local.rst
+    platypush/backend/log.http.rst
     platypush/backend/mail.rst
     platypush/backend/midi.rst
     platypush/backend/mqtt.rst
     platypush/backend/music.mopidy.rst
     platypush/backend/music.mpd.rst
     platypush/backend/music.snapcast.rst
+    platypush/backend/music.spotify.connect.rst
     platypush/backend/nextcloud.rst
     platypush/backend/nfc.rst
     platypush/backend/nodered.rst
@@ -71,9 +75,12 @@ Backends
     platypush/backend/todoist.rst
     platypush/backend/travisci.rst
     platypush/backend/trello.rst
+    platypush/backend/weather.rst
     platypush/backend/weather.buienradar.rst
     platypush/backend/weather.darksky.rst
+    platypush/backend/weather.openweathermap.rst
     platypush/backend/websocket.rst
     platypush/backend/wiimote.rst
     platypush/backend/zigbee.mqtt.rst
     platypush/backend/zwave.rst
+    platypush/backend/zwave.mqtt.rst

docs/source/conf.py

@@ -18,12 +18,13 @@ import sys
 # import os
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
+sys.path.insert(0, os.path.abspath("./_ext"))
 
 
 # -- Project information -----------------------------------------------------
 
-project = 'platypush'
+project = 'Platypush'
-copyright = '2017-2020, Fabio Manganiello'
+copyright = '2017-2021, Fabio Manganiello'
 author = 'Fabio Manganiello'
 
 # The short X.Y version
@@ -50,6 +51,7 @@ extensions = [
     'sphinx.ext.viewcode',
     'sphinx.ext.githubpages',
     'sphinx_rtd_theme',
+    'sphinx_marshmallow',
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -86,7 +88,8 @@ pygments_style = 'sphinx'
 # a list of builtin themes.
 #
 # html_theme = 'haiku'
-html_theme = 'sphinx_rtd_theme'
+# html_theme = 'sphinx_rtd_theme'
+html_theme = 'sphinx_material'
 
 html_domain_indices = True
 
@@ -94,7 +97,52 @@ html_domain_indices = True
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-# html_theme_options = {}
+html_theme_options = {
+    'nav_title': 'Platypush documentation',
+    'repo_url': 'https://git.platypush.tech/platypush/platypush',
+    'repo_name': 'Source code',
+    'repo_type': 'gitlab',
+    'color_primary': 'green',
+    'color_accent': 'light-green',
+    'logo_icon': '&#128366',
+    'nav_links': [
+        {
+            'href': 'https://platypush.tech/',
+            'title': 'Homepage',
+            'internal': False,
+        },
+        {
+            'href': 'https://blog.platypush.tech/',
+            'title': 'Blog',
+            'internal': False,
+        },
+        {
+            'href': 'https://git.platypush.tech/platypush/platypush',
+            'title': 'Repository',
+            'internal': False,
+        },
+        {
+            'href': 'https://git.platypush.tech/platypush/platypush/-/wikis/home',
+            'title': 'Wiki',
+            'internal': False,
+        },
+        {
+            'href': 'https://chrome.google.com/webstore/detail/platypush/aphldjclndofhflbbdnmpejbjgomkbie',
+            'title': 'Chrome Extension',
+            'internal': False,
+        },
+        {
+            'href': 'https://addons.mozilla.org/en-US/firefox/addon/platypush/',
+            'title': 'Firefox Extension',
+            'internal': False,
+        },
+        {
+            'href': 'https://f-droid.org/en/packages/tech.platypush.platypush/',
+            'title': 'Android App',
+            'internal': False,
+        },
+    ],
+}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
@@ -109,8 +157,9 @@ html_domain_indices = True
 # default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
 # 'searchbox.html']``.
 #
-# html_sidebars = {}
+html_sidebars = {
-
+    '**': ['logo-text.html', 'globaltoc.html', 'localtoc.html', 'searchbox.html']
+}
 
 # -- Options for HTMLHelp output ---------------------------------------------
 
@@ -265,6 +314,7 @@ autodoc_mock_imports = ['googlesamples.assistant.grpc.audio_helpers',
                         'imapclient',
                         'pysmartthings',
                         'aiohttp',
+                        'watchdog',
                         ]
 
 sys.path.insert(0, os.path.abspath('../..'))
@@ -277,3 +327,5 @@ def skip(app, what, name, obj, skip, options):
 def setup(app):
     app.connect("autodoc-skip-member", skip)
 
+
+# vim:sw=4:ts=4:et:

docs/source/events.rst

@@ -18,6 +18,7 @@ Events
     platypush/events/covid19.rst
     platypush/events/custom.rst
     platypush/events/distance.rst
+    platypush/events/file.rst
     platypush/events/foursquare.rst
     platypush/events/geo.rst
     platypush/events/github.rst
@@ -27,13 +28,13 @@ Events
     platypush/events/gps.rst
     platypush/events/http.rst
     platypush/events/http.hook.rst
-    platypush/events/http.ota.booking.rst
     platypush/events/http.rss.rst
     platypush/events/inotify.rst
     platypush/events/joystick.rst
     platypush/events/kafka.rst
     platypush/events/light.rst
     platypush/events/linode.rst
+    platypush/events/log.http.rst
     platypush/events/mail.rst
     platypush/events/media.rst
     platypush/events/midi.rst

docs/source/platypush/backend/file.monitor.rst

@@ -0,0 +1,5 @@
+``platypush.backend.file.monitor``
+==================================
+
+.. automodule:: platypush.backend.file.monitor
+    :members:

docs/source/platypush/backend/joystick.jstest.rst

@@ -0,0 +1,5 @@
+``platypush.backend.joystick.jstest``
+=====================================
+
+.. automodule:: platypush.backend.joystick.jstest
+    :members:

docs/source/platypush/backend/joystick.linux.rst

@@ -0,0 +1,5 @@
+``platypush.backend.joystick.linux``
+====================================
+
+.. automodule:: platypush.backend.joystick.linux
+    :members:

docs/source/platypush/backend/local.rst

@@ -1,5 +0,0 @@
-``platypush.backend.local``
-===========================
-
-.. automodule:: platypush.backend.local
-    :members:

docs/source/platypush/backend/log.http.rst

@@ -0,0 +1,5 @@
+``platypush.backend.log.http``
+==============================
+
+.. automodule:: platypush.backend.log.http
+    :members:

docs/source/platypush/backend/music.spotify.connect.rst

@@ -0,0 +1,5 @@
+``platypush.backend.music.spotify.connect``
+===========================================
+
+.. automodule:: platypush.backend.music.spotify.connect
+    :members:

docs/source/platypush/backend/weather.openweathermap.rst

@@ -0,0 +1,5 @@
+``platypush.backend.weather.openweathermap``
+============================================
+
+.. automodule:: platypush.backend.weather.openweathermap
+    :members:

docs/source/platypush/backend/weather.rst

@@ -0,0 +1,5 @@
+``platypush.backend.weather``
+=============================
+
+.. automodule:: platypush.backend.weather
+    :members:

docs/source/platypush/backend/zwave.mqtt.rst

@@ -0,0 +1,5 @@
+``platypush.backend.zwave.mqtt``
+================================
+
+.. automodule:: platypush.backend.zwave.mqtt
+    :members:

docs/source/platypush/events/file.rst

@@ -0,0 +1,5 @@
+``platypush.message.event.file``
+================================
+
+.. automodule:: platypush.message.event.file
+    :members:

docs/source/platypush/events/http.ota.booking.rst

@@ -1,5 +0,0 @@
-``platypush.message.event.http.ota.booking``
-============================================
-
-.. automodule:: platypush.message.event.http.ota.booking
-    :members:

docs/source/platypush/events/log.http.rst

@@ -0,0 +1,5 @@
+``platypush.message.event.log.http``
+====================================
+
+.. automodule:: platypush.message.event.log.http
+    :members:

docs/source/platypush/plugins/http.request.ota.booking.rst

@@ -1,5 +0,0 @@
-``platypush.plugins.http.request.ota.booking``
-==============================================
-
-.. automodule:: platypush.plugins.http.request.ota.booking
-    :members:

docs/source/platypush/plugins/pwm.pca9685.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.pwm.pca9685``
+=================================
+
+.. automodule:: platypush.plugins.pwm.pca9685
+    :members:

docs/source/platypush/plugins/switch.switchbot.rst

@@ -1,6 +0,0 @@
-``platypush.plugins.switch.switchbot``
-======================================
-
-.. automodule:: platypush.plugins.switch.switchbot
-    :members:
-

docs/source/platypush/plugins/switchbot.bluetooth.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.switchbot.bluetooth``
+=========================================
+
+.. automodule:: platypush.plugins.switchbot.bluetooth
+    :members:

docs/source/platypush/plugins/switchbot.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.switchbot``
+===============================
+
+.. automodule:: platypush.plugins.switchbot
+    :members:

docs/source/platypush/plugins/video.torrentcast.rst

@@ -1,5 +0,0 @@
-``platypush.plugins.video.torrentcast``
-=======================================
-
-.. automodule:: platypush.plugins.video.torrentcast
-    :members:

docs/source/platypush/plugins/weather.openweathermap.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.weather.openweathermap``
+============================================
+
+.. automodule:: platypush.plugins.weather.openweathermap
+    :members:

docs/source/platypush/plugins/weather.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.weather``
+=============================
+
+.. automodule:: platypush.plugins.weather
+    :members:

docs/source/platypush/plugins/zwave._base.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.zwave._base``
+=================================
+
+.. automodule:: platypush.plugins.zwave._base
+    :members:

docs/source/platypush/plugins/zwave.mqtt.rst

@@ -0,0 +1,5 @@
+``platypush.plugins.zwave.mqtt``
+================================
+
+.. automodule:: platypush.plugins.zwave.mqtt
+    :members:

docs/source/plugins.rst

@@ -61,7 +61,6 @@ Plugins
     platypush/plugins/graphite.rst
     platypush/plugins/homeseer.rst
     platypush/plugins/http.request.rst
-    platypush/plugins/http.request.ota.booking.rst
     platypush/plugins/http.request.rss.rst
     platypush/plugins/http.webpage.rst
     platypush/plugins/ifttt.rst
@@ -105,6 +104,7 @@ Plugins
     platypush/plugins/ping.rst
     platypush/plugins/printer.cups.rst
     platypush/plugins/pushbullet.rst
+    platypush/plugins/pwm.pca9685.rst
     platypush/plugins/qrcode.rst
     platypush/plugins/redis.rst
     platypush/plugins/rtorrent.rst
@@ -119,9 +119,10 @@ Plugins
     platypush/plugins/stt.picovoice.hotword.rst
     platypush/plugins/stt.picovoice.speech.rst
     platypush/plugins/switch.rst
-    platypush/plugins/switch.switchbot.rst
     platypush/plugins/switch.tplink.rst
     platypush/plugins/switch.wemo.rst
+    platypush/plugins/switchbot.rst
+    platypush/plugins/switchbot.bluetooth.rst
     platypush/plugins/system.rst
     platypush/plugins/tcp.rst
     platypush/plugins/tensorflow.rst
@@ -137,11 +138,14 @@ Plugins
     platypush/plugins/user.rst
     platypush/plugins/utils.rst
     platypush/plugins/variable.rst
-    platypush/plugins/video.torrentcast.rst
+    platypush/plugins/weather.rst
     platypush/plugins/weather.buienradar.rst
     platypush/plugins/weather.darksky.rst
+    platypush/plugins/weather.openweathermap.rst
     platypush/plugins/websocket.rst
     platypush/plugins/wiimote.rst
     platypush/plugins/zeroconf.rst
     platypush/plugins/zigbee.mqtt.rst
     platypush/plugins/zwave.rst
+    platypush/plugins/zwave._base.rst
+    platypush/plugins/zwave.mqtt.rst

platypush/__init__.py

@@ -24,7 +24,7 @@ from .utils import set_thread_name
 
 
 __author__ = 'Fabio Manganiello <info@fabiomanganiello.com>'
-__version__ = '0.20.4'
+__version__ = '0.21.1'
 
 logger = logging.getLogger('platypush')
 

platypush/backend/__init__.py

@@ -6,10 +6,9 @@
 import logging
 import re
 import socket
-import threading
 import time
 
-from threading import Thread
+from threading import Thread, Event as ThreadEvent, get_ident
 from typing import Optional, Dict
 
 from platypush.bus import Bus
@@ -62,7 +61,7 @@ class Backend(Thread, EventGenerator):
         self.poll_seconds = float(poll_seconds) if poll_seconds else None
         self.device_id = Config.get('device_id')
         self.thread_id = None
-        self._stop_event = threading.Event()
+        self._stop_event = ThreadEvent()
         self._kwargs = kwargs
         self.logger = logging.getLogger('platypush:backend:' + get_backend_name_by_class(self.__class__))
         self.zeroconf = None
@@ -220,7 +219,7 @@ class Backend(Thread, EventGenerator):
 
     def run(self):
         """ Starts the backend thread. To be implemented in the derived classes if the loop method isn't defined. """
-        self.thread_id = threading.get_ident()
+        self.thread_id = get_ident()
         set_thread_name(self._thread_name)
         if not callable(self.loop):
             return

platypush/backend/adafruit/io.py

@@ -65,11 +65,11 @@ class AdafruitIoBackend(Backend):
     def on_message(self, msg):
         # noinspection PyUnusedLocal
         def _handler(client, feed, data):
-            # noinspection PyBroadException
             try:
                 data = float(data)
-            except:
+            except Exception as e:
-                pass
+                self.logger.debug('Not a number: {}: {}'.format(data, e))
+
             self.bus.post(FeedUpdateEvent(feed=feed, data=data))
 
         return _handler

platypush/backend/alarm.py

@@ -55,16 +55,17 @@ class Alarm:
         self._runtime_snooze_interval = snooze_interval
 
     def get_next(self) -> float:
-        now = datetime.datetime.now().replace(tzinfo=gettz())
+        now = datetime.datetime.now().replace(tzinfo=gettz())  # lgtm [py/call-to-non-callable]
 
         try:
             cron = croniter.croniter(self.when, now)
             return cron.get_next()
         except (AttributeError, croniter.CroniterBadCronError):
             try:
-                timestamp = datetime.datetime.fromisoformat(self.when).replace(tzinfo=gettz())
+                timestamp = datetime.datetime.fromisoformat(self.when).replace(
+                    tzinfo=gettz())  # lgtm [py/call-to-non-callable]
             except (TypeError, ValueError):
-                timestamp = (datetime.datetime.now().replace(tzinfo=gettz()) +
+                timestamp = (datetime.datetime.now().replace(tzinfo=gettz()) +  # lgtm [py/call-to-non-callable]
                              datetime.timedelta(seconds=int(self.when)))
 
             return timestamp.timestamp() if timestamp >= now else None

platypush/backend/bluetooth/scanner/__init__.py

@@ -1,6 +1,9 @@
-from typing import Dict, Optional
+import time
+from threading import Thread, RLock
+from typing import Dict, Optional, List
 
 from platypush.backend.sensor import SensorBackend
+from platypush.context import get_plugin
 from platypush.message.event.bluetooth import BluetoothDeviceFoundEvent, BluetoothDeviceLostEvent
 
 
@@ -20,10 +23,12 @@ class BluetoothScannerBackend(SensorBackend):
 
     """
 
-    def __init__(self, device_id: Optional[int] = None, scan_duration: int = 10, **kwargs):
+    def __init__(self, device_id: Optional[int] = None, scan_duration: int = 10,
+                 track_devices: Optional[List[str]] = None, **kwargs):
         """
         :param device_id: Bluetooth adapter ID to use (default configured on the ``bluetooth`` plugin if None).
         :param scan_duration:  How long the scan should run (default: 10 seconds).
+        :param track_devices: List of addresses of devices to actively track, even if they aren't discoverable.
         """
         super().__init__(plugin='bluetooth', plugin_args={
             'device_id': device_id,
@@ -31,17 +36,72 @@ class BluetoothScannerBackend(SensorBackend):
         }, **kwargs)
 
         self._last_seen_devices = {}
+        self._tracking_thread: Optional[Thread] = None
+        self._bt_lock = RLock()
+        self.track_devices = set(track_devices or [])
+        self.scan_duration = scan_duration
+
+    def _add_last_seen_device(self, dev):
+        addr = dev.pop('addr')
+        if addr not in self._last_seen_devices:
+            self.bus.post(BluetoothDeviceFoundEvent(address=addr, **dev))
+        self._last_seen_devices[addr] = {'addr': addr, **dev}
+
+    def _remove_last_seen_device(self, addr: str):
+        dev = self._last_seen_devices.get(addr)
+        if not dev:
+            return
+
+        self.bus.post(BluetoothDeviceLostEvent(address=addr, **dev))
+        del self._last_seen_devices[addr]
+
+    def _addr_tracker(self, addr):
+        with self._bt_lock:
+            name = get_plugin('bluetooth').lookup_name(addr, timeout=self.scan_duration).name
+
+        if name is None:
+            self._remove_last_seen_device(addr)
+        else:
+            self._add_last_seen_device({'addr': addr, 'name': name})
+
+    def _bt_tracker(self):
+        self.logger.info('Starting Bluetooth tracker')
+        while not self.should_stop():
+            trackers = []
+            for addr in self.track_devices:
+                tracker = Thread(target=self._addr_tracker, args=(addr,))
+                tracker.start()
+                trackers.append(tracker)
+
+            for tracker in trackers:
+                tracker.join(timeout=self.scan_duration)
+
+            time.sleep(self.scan_duration)
+
+        self.logger.info('Bluetooth tracker stopped')
+
+    def get_measurement(self):
+        with self._bt_lock:
+            return super().get_measurement()
 
     def process_data(self, data: Dict[str, dict], new_data: Dict[str, dict]):
         for addr, dev in data.items():
-            if addr not in self._last_seen_devices:
+            self._add_last_seen_device(dev)
-                self.bus.post(BluetoothDeviceFoundEvent(address=dev.pop('addr'), **dev))
-            self._last_seen_devices[addr] = {'addr': addr, **dev}
 
         for addr, dev in self._last_seen_devices.copy().items():
-            if addr not in data:
+            if addr not in data and addr not in self.track_devices:
-                self.bus.post(BluetoothDeviceLostEvent(address=dev.pop('addr'), **dev))
+                self._remove_last_seen_device(addr)
-                del self._last_seen_devices[addr]
+
+    def run(self):
+        self._tracking_thread = Thread(target=self._bt_tracker)
+        self._tracking_thread.start()
+        super().run()
+
+    def on_stop(self):
+        super().on_stop()
+        if self._tracking_thread and self._tracking_thread.is_alive():
+            self.logger.info('Waiting for the Bluetooth tracking thread to stop')
+            self._tracking_thread.join(timeout=self.scan_duration)
 
 
 # vim:sw=4:ts=4:et:

platypush/backend/button/flic/fliclib/aioflic.py

@@ -13,25 +13,28 @@ Bd addr are represented as standard python strings, e.g. "aa:bb:cc:dd:ee:ff".
 import asyncio
 from enum import Enum
 from collections import namedtuple
-import time
 import struct
 import itertools
 
+
 class CreateConnectionChannelError(Enum):
     NoError = 0
     MaxPendingConnectionsReached = 1
 
+
 class ConnectionStatus(Enum):
     Disconnected = 0
     Connected = 1
     Ready = 2
 
+
 class DisconnectReason(Enum):
     Unspecified = 0
     ConnectionEstablishmentFailed = 1
     TimedOut = 2
     BondingKeysMismatch = 3
 
+
 class RemovedReason(Enum):
     RemovedByThisClient = 0
     ForceDisconnectedByThisClient = 1
@@ -44,6 +47,7 @@ class RemovedReason(Enum):
 
     CouldntLoadDevice = 7
 
+
 class ClickType(Enum):
     ButtonDown = 0
     ButtonUp = 1
@@ -52,20 +56,24 @@ class ClickType(Enum):
     ButtonDoubleClick = 4
     ButtonHold = 5
 
+
 class BdAddrType(Enum):
     PublicBdAddrType = 0
     RandomBdAddrType = 1
 
+
 class LatencyMode(Enum):
     NormalLatency = 0
     LowLatency = 1
     HighLatency = 2
 
+
 class BluetoothControllerState(Enum):
     Detached = 0
     Resetting = 1
     Attached = 2
 
+
 class ScanWizardResult(Enum):
     WizardSuccess = 0
     WizardCancelledByUser = 1
@@ -75,6 +83,7 @@ class ScanWizardResult(Enum):
     WizardInternetBackendError = 5
     WizardInvalidData = 6
 
+
 class ButtonScanner:
     """ButtonScanner class.
 
@@ -90,6 +99,7 @@ class ButtonScanner:
         self._scan_id = next(ButtonScanner._cnt)
         self.on_advertisement_packet = lambda scanner, bd_addr, name, rssi, is_private, already_verified: None
 
+
 class ScanWizard:
     """ScanWizard class
 
@@ -113,6 +123,7 @@ class ScanWizard:
         self.on_button_connected = lambda scan_wizard, bd_addr, name: None
         self.on_completed = lambda scan_wizard, result, bd_addr, name: None
 
+
 class ButtonConnectionChannel:
     """ButtonConnectionChannel class.
 
@@ -133,7 +144,7 @@ class ButtonConnectionChannel:
 
     _cnt = itertools.count()
 
-    def __init__(self, bd_addr, latency_mode = LatencyMode.NormalLatency, auto_disconnect_time = 511):
+    def __init__(self, bd_addr, latency_mode=LatencyMode.NormalLatency, auto_disconnect_time=511):
         self._conn_id = next(ButtonConnectionChannel._cnt)
         self._bd_addr = bd_addr
         self._latency_mode = latency_mode
@@ -164,7 +175,9 @@ class ButtonConnectionChannel:
 
         self._latency_mode = latency_mode
         if not self._client._closed:
-            self._client._send_command("CmdChangeModeParameters", {"conn_id": self._conn_id, "latency_mode": self._latency_mode, "auto_disconnect_time": self._auto_disconnect_time})
+            self._client._send_command("CmdChangeModeParameters",
+                                       {"conn_id": self._conn_id, "latency_mode": self._latency_mode,
+                                        "auto_disconnect_time": self._auto_disconnect_time})
 
     @property
     def auto_disconnect_time(self):
@@ -178,7 +191,10 @@ class ButtonConnectionChannel:
 
         self._auto_disconnect_time = auto_disconnect_time
         if not self._client._closed:
-            self._client._send_command("CmdChangeModeParameters", {"conn_id": self._conn_id, "latency_mode": self._latency_mode, "auto_disconnect_time": self._auto_disconnect_time})
+            self._client._send_command("CmdChangeModeParameters",
+                                       {"conn_id": self._conn_id, "latency_mode": self._latency_mode,
+                                        "auto_disconnect_time": self._auto_disconnect_time})
+
 
 class FlicClient(asyncio.Protocol):
     """FlicClient class.
@@ -212,7 +228,8 @@ class FlicClient(asyncio.Protocol):
         ("EvtButtonSingleOrDoubleClick", "<IBBI", "conn_id click_type was_queued time_diff"),
         ("EvtButtonSingleOrDoubleClickOrHold", "<IBBI", "conn_id click_type was_queued time_diff"),
         ("EvtNewVerifiedButton", "<6s", "bd_addr"),
-        ("EvtGetInfoResponse", "<B6sBBhBBH", "bluetooth_controller_state my_bd_addr my_bd_addr_type max_pending_connections max_concurrently_connected_buttons current_pending_connections currently_no_space_for_new_connection nb_verified_buttons"),
+        ("EvtGetInfoResponse", "<B6sBBhBBH",
+         "bluetooth_controller_state my_bd_addr my_bd_addr_type max_pending_connections max_concurrently_connected_buttons current_pending_connections currently_no_space_for_new_connection nb_verified_buttons"),
         ("EvtNoSpaceForNewConnection", "<B", "max_concurrently_connected_buttons"),
         ("EvtGotSpaceForNewConnection", "<B", "max_concurrently_connected_buttons"),
         ("EvtBluetoothControllerStateChange", "<B", "state"),
@@ -223,8 +240,8 @@ class FlicClient(asyncio.Protocol):
         ("EvtScanWizardButtonConnected", "<I", "scan_wizard_id"),
         ("EvtScanWizardCompleted", "<IB", "scan_wizard_id result")
     ]
-    _EVENT_STRUCTS = list(map(lambda x: None if x == None else struct.Struct(x[1]), _EVENTS))
+    _EVENT_STRUCTS = list(map(lambda x: None if x is None else struct.Struct(x[1]), _EVENTS))
-    _EVENT_NAMED_TUPLES = list(map(lambda x: None if x == None else namedtuple(x[0], x[2]), _EVENTS))
+    _EVENT_NAMED_TUPLES = list(map(lambda x: None if x is None else namedtuple(x[0], x[2]), _EVENTS))
 
     _COMMANDS = [
         ("CmdGetInfo", "", ""),
@@ -244,18 +261,19 @@ class FlicClient(asyncio.Protocol):
     _COMMAND_NAMED_TUPLES = list(map(lambda x: namedtuple(x[0], x[2]), _COMMANDS))
     _COMMAND_NAME_TO_OPCODE = dict((x[0], i) for i, x in enumerate(_COMMANDS))
 
-    
+    @staticmethod
     def _bdaddr_bytes_to_string(bdaddr_bytes):
         return ":".join(map(lambda x: "%02x" % x, reversed(bdaddr_bytes)))
 
+    @staticmethod
     def _bdaddr_string_to_bytes(bdaddr_string):
         return bytearray.fromhex("".join(reversed(bdaddr_string.split(":"))))
 
-    def __init__(self, loop,parent=None):
+    def __init__(self, loop, parent=None):
         self.loop = loop
-        self.buffer=b""
+        self.buffer = b""
-        self.transport=None
+        self.transport = None
-        self.parent=parent
+        self.parent = parent
         self._scanners = {}
         self._scan_wizards = {}
         self._connection_channels = {}
@@ -269,11 +287,10 @@ class FlicClient(asyncio.Protocol):
         self.on_get_button_uuid = lambda addr, uuid: None
 
     def connection_made(self, transport):
-        self.transport=transport
+        self.transport = transport
         if self.parent:
             self.parent.register_protocol(self)
 
-        
     def close(self):
         """Closes the client. The handle_events() method will return."""
         if self._closed:
@@ -342,7 +359,9 @@ class FlicClient(asyncio.Protocol):
         channel._client = self
 
         self._connection_channels[channel._conn_id] = channel
-        self._send_command("CmdCreateConnectionChannel", {"conn_id": channel._conn_id, "bd_addr": channel.bd_addr, "latency_mode": channel._latency_mode, "auto_disconnect_time": channel._auto_disconnect_time})
+        self._send_command("CmdCreateConnectionChannel", {"conn_id": channel._conn_id, "bd_addr": channel.bd_addr,
+                                                          "latency_mode": channel._latency_mode,
+                                                          "auto_disconnect_time": channel._auto_disconnect_time})
 
     def remove_connection_channel(self, channel):
         """Remove a connection channel.
@@ -384,7 +403,6 @@ class FlicClient(asyncio.Protocol):
         """
         self._send_command("CmdGetButtonUUID", {"bd_addr": bd_addr})
 
-    
     def run_on_handle_events_thread(self, callback):
         """Run a function on the thread that handles the events."""
         if threading.get_ident() == self._handle_event_thread_ident:
@@ -399,7 +417,7 @@ class FlicClient(asyncio.Protocol):
                 items[key] = value.value
 
         if "bd_addr" in items:
-            items["bd_addr"] = FlicClient._bdaddr_string_to_bytes(items["bd_addr"])
+            items["bd_addr"] = FlicClient._bdaddr_string_to_bytes()
 
         opcode = FlicClient._COMMAND_NAME_TO_OPCODE[name]
         data_bytes = FlicClient._COMMAND_STRUCTS[opcode].pack(*FlicClient._COMMAND_NAMED_TUPLES[opcode](**items))
@@ -415,16 +433,16 @@ class FlicClient(asyncio.Protocol):
             return
         opcode = data[0]
 
-        if opcode >= len(FlicClient._EVENTS) or FlicClient._EVENTS[opcode] == None:
+        if opcode >= len(FlicClient._EVENTS) or FlicClient._EVENTS[opcode] is None:
             return
 
         event_name = FlicClient._EVENTS[opcode][0]
-        data_tuple = FlicClient._EVENT_STRUCTS[opcode].unpack(data[1 : 1 + FlicClient._EVENT_STRUCTS[opcode].size])
+        data_tuple = FlicClient._EVENT_STRUCTS[opcode].unpack(data[1: 1 + FlicClient._EVENT_STRUCTS[opcode].size])
         items = FlicClient._EVENT_NAMED_TUPLES[opcode]._make(data_tuple)._asdict()
 
         # Process some kind of items whose data type is not supported by struct
         if "bd_addr" in items:
-            items["bd_addr"] = FlicClient._bdaddr_bytes_to_string(items["bd_addr"])
+            items["bd_addr"] = FlicClient._bdaddr_bytes_to_string()
 
         if "name" in items:
             items["name"] = items["name"].decode("utf-8")
@@ -445,13 +463,14 @@ class FlicClient(asyncio.Protocol):
 
         if event_name == "EvtGetInfoResponse":
             items["bluetooth_controller_state"] = BluetoothControllerState(items["bluetooth_controller_state"])
-            items["my_bd_addr"] = FlicClient._bdaddr_bytes_to_string(items["my_bd_addr"])
+            items["my_bd_addr"] = FlicClient._bdaddr_bytes_to_string()
             items["my_bd_addr_type"] = BdAddrType(items["my_bd_addr_type"])
             items["bd_addr_of_verified_buttons"] = []
 
             pos = FlicClient._EVENT_STRUCTS[opcode].size
             for i in range(items["nb_verified_buttons"]):
-                items["bd_addr_of_verified_buttons"].append(FlicClient._bdaddr_bytes_to_string(data[1 + pos : 1 + pos + 6]))
+                items["bd_addr_of_verified_buttons"].append(
+                    FlicClient._bdaddr_bytes_to_string())
                 pos += 6
 
         if event_name == "EvtBluetoothControllerStateChange":
@@ -469,7 +488,8 @@ class FlicClient(asyncio.Protocol):
         if event_name == "EvtAdvertisementPacket":
             scanner = self._scanners.get(items["scan_id"])
             if scanner is not None:
-                scanner.on_advertisement_packet(scanner, items["bd_addr"], items["name"], items["rssi"], items["is_private"], items["already_verified"])
+                scanner.on_advertisement_packet(scanner, items["bd_addr"], items["name"], items["rssi"],
+                                                items["is_private"], items["already_verified"])
 
         if event_name == "EvtCreateConnectionChannelResponse":
             channel = self._connection_channels[items["conn_id"]]
@@ -494,10 +514,12 @@ class FlicClient(asyncio.Protocol):
             channel.on_button_click_or_hold(channel, items["click_type"], items["was_queued"], items["time_diff"])
         if event_name == "EvtButtonSingleOrDoubleClick":
             channel = self._connection_channels[items["conn_id"]]
-            channel.on_button_single_or_double_click(channel, items["click_type"], items["was_queued"], items["time_diff"])
+            channel.on_button_single_or_double_click(channel, items["click_type"], items["was_queued"],
+                                                     items["time_diff"])
         if event_name == "EvtButtonSingleOrDoubleClickOrHold":
             channel = self._connection_channels[items["conn_id"]]
-            channel.on_button_single_or_double_click_or_hold(channel, items["click_type"], items["was_queued"], items["time_diff"])
+            channel.on_button_single_or_double_click_or_hold(channel, items["click_type"], items["was_queued"],
+                                                             items["time_diff"])
 
         if event_name == "EvtNewVerifiedButton":
             self.on_new_verified_button(items["bd_addr"])
@@ -536,19 +558,16 @@ class FlicClient(asyncio.Protocol):
             del self._scan_wizards[items["scan_wizard_id"]]
             scan_wizard.on_completed(scan_wizard, items["result"], scan_wizard._bd_addr, scan_wizard._name)
 
-        
+    def data_received(self, data):
-    def data_received(self,data):
+        cdata = self.buffer + data
-        cdata=self.buffer+data
+        self.buffer = b""
-        self.buffer=b""
         while len(cdata):
             packet_len = cdata[0] | (cdata[1] << 8)
             packet_len += 2
-            if len(cdata)>= packet_len:
+            if len(cdata) >= packet_len:
                 self._dispatch_event(cdata[2:packet_len])
-                cdata=cdata[packet_len:]
+                cdata = cdata[packet_len:]
             else:
                 if len(cdata):
-                    self.buffer=cdata #unlikely to happen but.....
+                    self.buffer = cdata  # unlikely to happen but.....
                 break
-            
-

platypush/backend/button/flic/fliclib/fliclib.py

@@ -229,8 +229,8 @@ class FlicClient:
 		("EvtScanWizardButtonConnected", "<I", "scan_wizard_id"),
 		("EvtScanWizardCompleted", "<IB", "scan_wizard_id result")
 	]
-	_EVENT_STRUCTS = list(map(lambda x: None if x == None else struct.Struct(x[1]), _EVENTS))
+	_EVENT_STRUCTS = list(map(lambda x: None if x is None else struct.Struct(x[1]), _EVENTS))
-	_EVENT_NAMED_TUPLES = list(map(lambda x: None if x == None else namedtuple(x[0], x[2]), _EVENTS))
+	_EVENT_NAMED_TUPLES = list(map(lambda x: None if x is None else namedtuple(x[0], x[2]), _EVENTS))
 
 	_COMMANDS = [
 		("CmdGetInfo", "", ""),
@@ -250,9 +250,11 @@ class FlicClient:
 	_COMMAND_NAMED_TUPLES = list(map(lambda x: namedtuple(x[0], x[2]), _COMMANDS))
 	_COMMAND_NAME_TO_OPCODE = dict((x[0], i) for i, x in enumerate(_COMMANDS))
 
+	@staticmethod
 	def _bdaddr_bytes_to_string(bdaddr_bytes):
 		return ":".join(map(lambda x: "%02x" % x, reversed(bdaddr_bytes)))
 
+	@staticmethod
 	def _bdaddr_string_to_bytes(bdaddr_string):
 		return bytearray.fromhex("".join(reversed(bdaddr_string.split(":"))))
 
@@ -438,7 +440,7 @@ class FlicClient:
 			return
 		opcode = data[0]
 
-		if opcode >= len(FlicClient._EVENTS) or FlicClient._EVENTS[opcode] == None:
+		if opcode >= len(FlicClient._EVENTS) or FlicClient._EVENTS[opcode] is None:
 			return
 
 		event_name = FlicClient._EVENTS[opcode][0]

platypush/backend/camera/pi.py

@@ -34,7 +34,7 @@ class CameraPiBackend(Backend):
             return self.value == other
 
     # noinspection PyUnresolvedReferences,PyPackageRequirements
-    def __init__(self, listen_port, x_resolution=640, y_resolution=480,
+    def __init__(self, listen_port, bind_address='0.0.0.0', x_resolution=640, y_resolution=480,
                  redis_queue='platypush/camera/pi',
                  start_recording_on_startup=True,
                  framerate=24, hflip=False, vflip=False,
@@ -49,13 +49,17 @@ class CameraPiBackend(Backend):
 
         :param listen_port: Port where the camera process will provide the video output while recording
         :type listen_port: int
+
+        :param bind_address: Bind address (default: 0.0.0.0).
+        :type bind_address: str
         """
 
         super().__init__(**kwargs)
 
+        self.bind_address = bind_address
         self.listen_port = listen_port
         self.server_socket = socket.socket()
-        self.server_socket.bind(('0.0.0.0', self.listen_port))
+        self.server_socket.bind((self.bind_address, self.listen_port))
         self.server_socket.listen(0)
 
         import picamera
@@ -134,13 +138,13 @@ class CameraPiBackend(Backend):
                         self.logger.info('Client closed connection')
                         try:
                             self.stop_recording()
-                        except:
+                        except Exception as e:
-                            pass
+                            self.logger.warning('Could not stop recording: {}'.format(str(e)))
 
                         try:
                             connection.close()
-                        except:
+                        except Exception as e:
-                            pass
+                            self.logger.warning('Could not close connection: {}'.format(str(e)))
 
                         self.send_camera_action(self.CameraAction.START_RECORDING)
 

platypush/backend/file/monitor/__init__.py

@@ -0,0 +1,128 @@
+from typing import Iterable, Dict, Union, Any
+
+from watchdog.observers import Observer
+
+from platypush.backend import Backend
+from .entities.handlers import EventHandler
+from .entities.resources import MonitoredResource, MonitoredPattern, MonitoredRegex
+
+
+class FileMonitorBackend(Backend):
+    """
+    This backend monitors changes to local files and directories using the Watchdog API.
+
+    Triggers:
+
+        * :class:`platypush.message.event.file.FileSystemCreateEvent` if a resource is created.
+        * :class:`platypush.message.event.file.FileSystemDeleteEvent` if a resource is removed.
+        * :class:`platypush.message.event.file.FileSystemModifyEvent` if a resource is modified.
+
+    Requires:
+
+        * **watchdog** (``pip install watchdog``)
+
+    """
+
+    class EventHandlerFactory:
+        """
+        Create a file system event handler from a string, dictionary or ``MonitoredResource`` resource.
+        """
+
+        @staticmethod
+        def from_resource(resource: Union[str, Dict[str, Any], MonitoredResource]) -> EventHandler:
+            if isinstance(resource, str):
+                resource = MonitoredResource(resource)
+            elif isinstance(resource, dict):
+                if 'patterns' in resource or 'ignore_patterns' in resource:
+                    resource = MonitoredPattern(**resource)
+                elif 'regexes' in resource or 'ignore_regexes' in resource:
+                    resource = MonitoredRegex(**resource)
+                else:
+                    resource = MonitoredResource(**resource)
+
+            return EventHandler.from_resource(resource)
+
+    def __init__(self, paths: Iterable[Union[str, Dict[str, Any], MonitoredResource]], **kwargs):
+        """
+        :param paths: List of paths to monitor. Paths can either be expressed in any of the following ways:
+
+            - Simple strings. In this case, paths will be interpreted as absolute references to a file or a directory
+              to monitor. Example:
+
+              .. code-block:: yaml
+
+                backend.file.monitor:
+                    paths:
+                        # Monitor changes on the /tmp folder
+                        - /tmp
+                        # Monitor changes on /etc/passwd
+                        - /etc/passwd
+
+            - Path with monitoring properties expressed as a key-value object. Example showing the supported attributes:
+
+                .. code-block:: yaml
+
+                    backend.file.monitor:
+                        paths:
+                            # Monitor changes on the /tmp folder and its subfolders
+                            - path: /tmp
+                              recursive: True
+
+            - Path with pattern-based search criteria for the files to monitor and exclude. Example:
+
+                .. code-block:: yaml
+
+                    backend.file.monitor:
+                        paths:
+                            # Recursively monitor changes on the ~/my-project folder that include all
+                            # *.py files, excluding those whose name starts with tmp_ and
+                            # all the files contained in the __pycache__ folders
+                            - path: ~/my-project
+                              recursive: True
+                              patterns:
+                                - "*.py"
+                              ignore_patterns:
+                                - "tmp_*"
+                              ignore_directories:
+                                - "__pycache__"
+
+            - Path with regex-based search criteria for the files to monitor and exclude. Example:
+
+                .. code-block:: yaml
+
+                    backend.file.monitor:
+                        paths:
+                            # Recursively monitor changes on the ~/my-images folder that include all
+                            # the files matching a JPEG extension in case-insensitive mode,
+                            # excluding those whose name starts with tmp_ and
+                            # all the files contained in the __MACOSX folders
+                            - path: ~/my-images
+                              recursive: True
+                              case_sensitive: False
+                              regexes:
+                                - '.*\\.jpe?g$'
+                              ignore_patterns:
+                                - '^tmp_.*'
+                              ignore_directories:
+                                - '__MACOSX'
+
+        """
+
+        super().__init__(**kwargs)
+        self._observer = Observer()
+
+        for path in paths:
+            handler = self.EventHandlerFactory.from_resource(path)
+            self._observer.schedule(handler, handler.resource.path, recursive=handler.resource.recursive)
+
+    def run(self):
+        super().run()
+        self.logger.info('Initializing file monitor backend')
+        self._observer.start()
+        self.wait_stop()
+
+    def on_stop(self):
+        self.logger.info('Stopping file monitor backend')
+        self._observer.stop()
+        self._observer.join()
+        self.logger.info('Stopped file monitor backend')

platypush/backend/file/monitor/entities/handlers.py

@@ -0,0 +1,61 @@
+import os
+
+from watchdog.events import FileSystemEventHandler, PatternMatchingEventHandler, RegexMatchingEventHandler
+
+from platypush.backend.file.monitor.entities.resources import MonitoredResource, MonitoredPattern, MonitoredRegex
+from platypush.context import get_bus
+from platypush.message.event.file import FileSystemModifyEvent, FileSystemCreateEvent, FileSystemDeleteEvent
+
+
+class EventHandler(FileSystemEventHandler):
+    """
+    Base class for Watchdog event handlers.
+    """
+    def __init__(self, resource: MonitoredResource, **kwargs):
+        super().__init__(**kwargs)
+        resource.path = os.path.expanduser(resource.path)
+        self.resource = resource
+
+    def on_created(self, event):
+        get_bus().post(FileSystemCreateEvent(path=event.src_path, is_directory=event.is_directory))
+
+    def on_deleted(self, event):
+        get_bus().post(FileSystemDeleteEvent(path=event.src_path, is_directory=event.is_directory))
+
+    def on_modified(self, event):
+        get_bus().post(FileSystemModifyEvent(path=event.src_path, is_directory=event.is_directory))
+
+    def on_moved(self, event):
+        pass
+
+    @classmethod
+    def from_resource(cls, resource: MonitoredResource):
+        if isinstance(resource, MonitoredPattern):
+            return PatternEventHandler(resource)
+        if isinstance(resource, MonitoredRegex):
+            return RegexEventHandler(resource)
+        return cls(resource)
+
+
+class PatternEventHandler(EventHandler, PatternMatchingEventHandler):
+    """
+    Event handler for file patterns.
+    """
+    def __init__(self, resource: MonitoredPattern):
+        super().__init__(resource=resource,
+                         patterns=resource.patterns,
+                         ignore_patterns=resource.ignore_patterns,
+                         ignore_directories=resource.ignore_directories,
+                         case_sensitive=resource.case_sensitive)
+
+
+class RegexEventHandler(EventHandler, RegexMatchingEventHandler):
+    """
+    Event handler for regex-based file patterns.
+    """
+    def __init__(self, resource: MonitoredRegex):
+        super().__init__(resource=resource,
+                         regexes=resource.regexes,
+                         ignore_regexes=resource.ignore_regexes,
+                         ignore_directories=resource.ignore_directories,
+                         case_sensitive=resource.case_sensitive)

platypush/backend/file/monitor/entities/resources.py

@@ -0,0 +1,24 @@
+from dataclasses import dataclass
+from typing import Optional, List
+
+
+@dataclass
+class MonitoredResource:
+    path: str
+    recursive: bool = False
+
+
+@dataclass
+class MonitoredPattern(MonitoredResource):
+    patterns: Optional[List[str]] = None
+    ignore_patterns: Optional[List[str]] = None
+    ignore_directories: Optional[List[str]] = None
+    case_sensitive: bool = True
+
+
+@dataclass
+class MonitoredRegex(MonitoredResource):
+    regexes: Optional[List[str]] = None
+    ignore_regexes: Optional[List[str]] = None
+    ignore_directories: Optional[List[str]] = None
+    case_sensitive: bool = True

platypush/backend/google/pubsub.py

@@ -51,11 +51,10 @@ class GooglePubsubBackend(Backend):
     def _message_callback(self, topic):
         def callback(msg):
             data = msg.data.decode()
-            # noinspection PyBroadException
             try:
                 data = json.loads(data)
-            except:
+            except Exception as e:
-                pass
+                self.logger.debug('Not a valid JSON: {}: {}'.format(data, str(e)))
 
             msg.ack()
             self.bus.post(GooglePubsubMessageEvent(topic=topic, msg=data))

platypush/backend/http/app/routes/hook.py

@@ -31,8 +31,8 @@ def _hook(hook_name):
         # noinspection PyBroadException
         try:
             event_args['data'] = json.loads(event_args['data'])
-        except:
+        except Exception as e:
-            pass
+            logger().warning('Not a valid JSON string: {}: {}'.format(event_args['data'], str(e)))
 
     event = WebhookEvent(**event_args)
 

platypush/backend/http/app/routes/login.py

@@ -31,7 +31,7 @@ def login():
     if session_token:
         user, session = user_manager.authenticate_user_session(session_token)
         if user:
-            return redirect(redirect_page, 302)
+            return redirect(redirect_page, 302)  # lgtm [py/url-redirection]
 
     if request.form:
         username = request.form.get('username')
@@ -44,7 +44,7 @@ def login():
                                                    expires_at=expires)
 
         if session:
-            redirect_target = redirect(redirect_page, 302)
+            redirect_target = redirect(redirect_page, 302)  # lgtm [py/url-redirection]
             response = make_response(redirect_target)
             response.set_cookie('session_token', session.session_token, expires=expires)
             return response

platypush/backend/http/app/routes/logout.py

@@ -25,7 +25,7 @@ def logout():
     if not user:
         return abort(403, 'Invalid session token')
 
-    redirect_target = redirect(redirect_page, 302)
+    redirect_target = redirect(redirect_page, 302)  # lgtm [py/url-redirection]
     response = make_response(redirect_target)
     response.set_cookie('session_token', '', expires=0)
     return response

platypush/backend/http/app/routes/plugins/camera/__init__.py

@@ -45,14 +45,12 @@ def get_args(kwargs):
         if k == 'resolution':
             v = json.loads('[{}]'.format(v))
         else:
-            # noinspection PyBroadException
             try:
                 v = int(v)
-            except:
+            except (ValueError, TypeError):
-                # noinspection PyBroadException
                 try:
                     v = float(v)
-                except:
+                except (ValueError, TypeError):
                     pass
 
         kwargs[k] = v

platypush/backend/http/app/routes/plugins/media/stream.py

@@ -32,8 +32,8 @@ def add_media():
     args = {}
     try:
         args = json.loads(request.data.decode('utf-8'))
-    except:
+    except Exception as e:
-        abort(400, 'Invalid JSON request')
+        abort(400, 'Invalid JSON request: {}'.format(str(e)))
 
     source = args.get('source')
     if not source:

platypush/backend/http/app/routes/plugins/sound/__init__.py

@@ -40,7 +40,7 @@ def audio_feed(device, fifo, sample_rate, blocksize, latency, channels):
                  channels=channels)
 
     try:
-        with open(fifo, 'rb') as f:
+        with open(fifo, 'rb') as f:  # lgtm [py/path-injection]
             send_header = True
 
             while True:

platypush/backend/http/app/routes/register.py

@@ -31,10 +31,10 @@ def register():
     if session_token:
         user, session = user_manager.authenticate_user_session(session_token)
         if user:
-            return redirect(redirect_page, 302)
+            return redirect(redirect_page, 302)  # lgtm [py/url-redirection]
 
     if user_manager.get_user_count() > 0:
-        return redirect('/login?redirect=' + redirect_page, 302)
+        return redirect('/login?redirect=' + redirect_page, 302)  # lgtm [py/url-redirection]
 
     if request.form:
         username = request.form.get('username')
@@ -49,7 +49,7 @@ def register():
                                                        if not remember else None)
 
             if session:
-                redirect_target = redirect(redirect_page, 302)
+                redirect_target = redirect(redirect_page, 302)  # lgtm [py/url-redirection]
                 response = make_response(redirect_target)
                 response.set_cookie('session_token', session.session_token)
                 return response

platypush/backend/http/app/utils.py

@@ -23,7 +23,7 @@ _logger = None
 def bus():
     global _bus
     if _bus is None:
-        _bus = RedisBus(redis_queue=current_app.config['redis_queue'])
+        _bus = RedisBus(redis_queue=current_app.config.get('redis_queue'))
     return _bus
 
 
@@ -123,7 +123,8 @@ def _authenticate_token():
     try:
         user_manager.validate_jwt_token(user_token)
         return True
-    except:
+    except Exception as e:
+        logger().debug(str(e))
         return token and user_token == token
 
 

platypush/backend/http/request/__init__.py

@@ -78,11 +78,11 @@ class HttpRequest(object):
 
     def get_new_items(self, response):
         """ Gets new items out of a response """
-        raise ("get_new_items must be implemented in a derived class")
+        raise NotImplementedError("get_new_items must be implemented in a derived class")
 
     def __iter__(self):
         for (key, value) in self.request_args.items():
-            yield (key, value)
+            yield key, value
 
 
 class JsonHttpRequest(HttpRequest):
@@ -96,7 +96,7 @@ class JsonHttpRequest(HttpRequest):
         new_entries = []
 
         if self.path:
-            m = re.match('\$\{\s*(.*)\s*\}', self.path)
+            m = re.match(r'\${\s*(.*)\s*}', self.path)
             response = eval(m.group(1))
 
         for entry in response:

platypush/backend/http/request/rss/__init__.py

@@ -238,15 +238,15 @@ class RssUpdates(HttpRequest):
                     with open(digest_filename, 'w', encoding='utf-8') as f:
                         f.write(content)
                 elif self.digest_format == 'pdf':
-                    import weasyprint
+                    from weasyprint import HTML, CSS
                     from weasyprint.fonts import FontConfiguration
 
-                    body_style = 'body { {body_style} }'.format(body_style=self.body_style)
+                    body_style = 'body { ' + self.body_style + ' }'
                     font_config = FontConfiguration()
-                    css = [weasyprint.CSS('https://fonts.googleapis.com/css?family=Merriweather'),
+                    css = [CSS('https://fonts.googleapis.com/css?family=Merriweather'),
-                           weasyprint.CSS(string=body_style, font_config=font_config)]
+                           CSS(string=body_style, font_config=font_config)]
 
-                    weasyprint.HTML(string=content).write_pdf(digest_filename, stylesheets=css)
+                    HTML(string=content).write_pdf(digest_filename, stylesheets=css)
                 else:
                     raise RuntimeError('Unsupported format: {}. Supported formats: ' +
                                        'html or pdf'.format(self.digest_format))

platypush/backend/http/utils.py

@@ -50,8 +50,6 @@ class HttpUtils(object):
         for name, resource_path in resource_dirs.items():
             resource_path = os.path.abspath(os.path.expanduser(resource_path))
             if directory.startswith(resource_path):
-                subdir = re.sub('^{}(.*)$'.format(resource_path),
-                                '\\1', directory)
                 uri = '/resources/' + name
                 break
 
@@ -92,10 +90,11 @@ class HttpUtils(object):
 
     @classmethod
     def plugin_name_to_tag(cls, module_name):
-        return module_name.replace('.','-')
+        return module_name.replace('.', '-')
 
     @classmethod
     def find_templates_in_dir(cls, directory):
+        # noinspection PyTypeChecker
         return [
             os.path.join(directory, file)
             for root, path, files in os.walk(os.path.abspath(os.path.join(template_folder, directory)))