- The `inspect` plugin and the Sphinx inspection extensions now use the
same underlying logic.
- Moved all the common inspection logic under
`platypush.common.reflection`.
- Faster scanning of the available integrations and components through a
pool of threads.
- Added `doc_url` parameters.
- Migrated events and responses metadata scanning logic.
- Now expanding some custom Sphinx tag instead of returning errors when
running outside of the Sphinx context - it includes `:class:`,
`:meth:` and `.. schema::`.
Added an `add_dependencies` plugin to the Sphinx build process that
parses the manifest files of the scanned backends and plugins and
automatically generates the documentation for the required dependencies
and triggered events.
This means that those dependencies are no longer required to be listed
in the docstring of the class itself.
Also in this commit:
- Black/LINT for some integrations that hadn't been touched in a long
time.
- Deleted some leftovers from previous refactors (deprecated
`backend.mqtt`, `backend.zwave.mqtt`, `backend.http.request.rss`).
- Deleted deprecated `inotify` backend - replaced by `file.monitor` (see
#289).
I've tried my best to keep it around, but the endpoints seem to be
broken, they no longer have a link to their API v3 documentation, and
the API Explorer that was supposed to be in the dashboard is gone.
- If a Python optional dependency is available as a system package on
the target system, try and install it that route rather than pip. It's
usually faster and it decreases the risk of breaking system packages.
- Added support for apk dependencies in manifest files. This brings the
number of distros officially supported by all the extensions to four:
- Alpine
- Arch
- Debian
- Ubuntu
The Tornado WSGI container won't guarantee the termination of the
spawned workers upon termination, so the code of the backend has to take
care of it and terminate all the children processes of the server
process when it terminates.
This also means that `psutil` is now a required base dependency, as we
need to expand the process subtree under the webserver launcher.
Also, catch `AttributeError` on `self._proc.terminate` in the
`HttpBackend`, since the process may already have been terminated and
set to null by another worker process.
- The following logging namespaces are now used, to make it easier to
filter only log lines related to the logged application message:
- `platypush:events`
- `platypush:requests`
- `platypush:responses`
- Those messages are always logged as JSON, with no prefixes nor
suffixes.
- Requests are always logged when executed - no more delegation to the
upstream backend.
- Responses are always logged when fully populated (including `id`,
`origin`, `target` etc.), instead of being logged when still partially
populated. This makes it particularly easy to link request/response
IDs directly from the logs.
Optional top-level imports in Tornado route declarations will trigger
`ImportError`. While this will just mean that those routes will be
skipped, it will also generate a lot of noise on the logs.
The `inspect` plugin can now detect references to plugins, backends,
events, responses and schemas in docstrings and replace them either with
links to the documentation or auto-generated examples.
Display a popup modal instead of a confirm box to prompt the user to
install the PWA app.
`confirm` blocks the JavaScript engine when run in `beforeMount` and
therefore the browser won't be able to proceed with `event.prompt()`.
There are situations where you may not want to run the HTTP server in a
full blown WSGI-over-Tornado container - unit/integration tests and
embedded single-core devices are among those cases.
In those scenarios, we should allow the user to be able to run the
backend using the built-in Werkzeug server provided by Flask.
The frontend now calls `utils.rst_to_html` to render the docstrings as
HTML instead of dumping them as raw text.
Also, actions and arguments are now cached to improve performance.
This reverts commit 71401a4936.
Temporarily reverted this commit because the `reuse_address` on the
application's `listen` method has only been implemented in Tornado 6.2 -
and Debian stable still shipts Tornado 6.1.
The WSGI container is a good option to wrap a multi-modal webapp
(Flask + websocket routes), but it's constrained to a single-process
approach and queued/pre-buffered requests. That makes performance poor
when handling requests that may take a few seconds to complete.
Defined a `platypush.backend.http.ws` package with all the routes, a
base `WSRoute` class that all the websocket routes can extend, and a
logic in the HTTP backend to automatically scan the package to register
exposed websocket routes.
It was just too painful to find a combination of versions of gunicorn,
gevent, eventlet, pyuwsgi etc. that could work on all of my systems.
On the other hand, Tornado works out of the box with no headaches.
Also in this commit:
- Updated a bunch of outdated/required integration dependencies.
- Black'd and LINTed a couple of old plugins.
The eventlet API has way too many dependency issues with gunicorn.
Still TODO: Fix or at least mitigate the WSGI workers timeout issue when
they handle websocket connections.
The websocket service is no longer provided by a different service,
controlled by a different thread running on another port.
Instead, it's now exposed directly over Flask routes, using
WSGI+eventlet+simple_websocket.
Also, the SSL context options have been removed from `backend.http`, for
sake of simplicity. If you want to enable SSL, you can serve Platypush
through a reverse proxy like nginx.
Instead of iterating over each of the entities in a grouping to find out
which groups should be displayed based on the selector's policy, the
selector can directly keep its `selectedGroups` attribute in sync with
the index.
Added `waitress` dependency. For performance and security reasons, it's
better to always run the Flask application inside of a uWSGI server.
`waitress` also makes things easier by avoiding to ask the user to
manually provide the external executable arguments, as it was the case
with `uwsgi` and `gunicorn`.
It was broken by the previous refactor of the entities panel, which no
longer triggers the `watch` callback on the upstream `entityGroups`.
The new approach listens for entity updates on the frontend bus and
dynamically creates the entity groupings in `selectedGroups` if they are
missing.
Unlike the other entity groupings, which are 4-layered (`grouping ->
group -> entity_id -> entity`), the grouping by ID only needs 3 layers
(`grouping -> entity_id -> entity`).
- Don't recalculate entity groups every time. Instead, keep them in sync
every time an entity is added or removed.
- Removed `computedChildren` from the entity component - no null nodes
are guaranteed to be passed now, so there's no need for another
iteration on the list of children.
- `childrenByParentId` now only looks in the scope of the entity's
children instead of searching all the entities.
The animation has a big impact on page loading performance when the
system includes a high number of entities that all need their loading
animation to be render.
Multiple style improvements for the entity components. Among these:
- A more consistent style for entity values and toggler buttons.
- Fixed overflowing/underflowing entities on smaller/larger screen
sizes.
- Simplified the stylesheets for many entities as many component classes
have now been moved to `common.scss`.
The entity name and value in the component header may be arbitrarily
long and rendered on small screens.
We therefore need to ensure that the text won't overflow the screen
width.
Plus, `platypush.schemas.system` has now been split into multiple
submodules to avoid a single-file mega-module with all the system
schemas definitions.
There are probably more optimal ways of achieving this other than
passing a reference to the full list of entities to each of the
entities, such as running a BFS to recursively expand all the entities
within the child hierarchy of an entity.
This is needed because the entity needs to know which entities aren't
direct children, but are two or more layers down in the hierarchy, so
they should be passed to their own child entities.
Remove `backend.sensor.distance` and `gpio.sensor.distance`. They are
now replaced by the `sensor.hcsr04` integration, which is compatible
with the new `SensorPlugin` API.
Removed `backend.sensor.dht` and `gpio.sensor.dht`. They have been
merged into the new `sensor.dht` integration, which supports the new
`SensorPlugin` API.
Removed `backend.sensor.accelerometer` and `gpio.sensor.accelerometer`.
The logic has now been merged in the new `sensor.lis3dh` integration,
which is compatible with the new `SensorPlugin` API.
Removed legacy `backend.sensor.motion.pmw3901` and
`gpio.sensor.motion.pmw3901`. They have been merged in the new
`sensor.pmw3901` integration, compatible with the new `SensorPlugin`
API.
Removed the old `backend.sensor.bme280` and the old `gpio.sensor.bme280`
plugin. They have now been merged into the new `sensor.bme280` runnable
plugin, which extends the `SensorPlugin` API and supports entities.
`backend.serial` has been removed and the polling logic merged into the
`serial` plugin.
The `serial` plugin now supports the new entity engine as well.
- Support for cloud instances as native entities.
- Using Marshmallow dataclasses+schemas instead of custom `Response`
objects.
- Merge `linode` backend into `linode` plugin.
No replacements have been made for the OBEX backends (push and file
services). PyOBEX is too broken and unmaintained, and there are too many
poorly documented steps required to get an unprivileged user to run an
SDP service.