- If a Python optional dependency is available as a system package on
the target system, try and install it that route rather than pip. It's
usually faster and it decreases the risk of breaking system packages.
- Added support for apk dependencies in manifest files. This brings the
number of distros officially supported by all the extensions to four:
- Alpine
- Arch
- Debian
- Ubuntu
The Tornado WSGI container won't guarantee the termination of the
spawned workers upon termination, so the code of the backend has to take
care of it and terminate all the children processes of the server
process when it terminates.
This also means that `psutil` is now a required base dependency, as we
need to expand the process subtree under the webserver launcher.
Also, catch `AttributeError` on `self._proc.terminate` in the
`HttpBackend`, since the process may already have been terminated and
set to null by another worker process.
- Support for distinct `type` field on constructor and method arguments.
- Added `has_varargs` field.
- Added `required` field.
- Better logic for parsing arguments `default` values.
1. Added documentation to the README on the possible options to run the
Redis service.
2. Show a relevant message to the user if the application is run with
`--start-redis` and Redis couldn't start.
3. Some LINT/black chores on some files that hadn't been touched in a
while.
Also, the application is now using `XDG_CONFIG_HOME` and
`XDG_DATA_HOME` if available to lookup the configuration file and
working directory.
Closes: #60
The main application class has been moved from __init__ to the app
module.
__init__ will contain instead the relevant global variables and the
modules and objects exposed to external integrations - such as
`get_plugin` and `get_backend`, or the `main` itself.
This will make future integrations much easier - the global __init__
doesn't contain any business logic now, it can import anything without
fearing circular dependencies, and it can limit its exposed objects to
those that we want to expose to 3rd-party integrations and scripts.
It will also make it easier to extend the main entry point with
additional logic - such as a supervisor or an embedded Redis server.
- The following logging namespaces are now used, to make it easier to
filter only log lines related to the logged application message:
- `platypush:events`
- `platypush:requests`
- `platypush:responses`
- Those messages are always logged as JSON, with no prefixes nor
suffixes.
- Requests are always logged when executed - no more delegation to the
upstream backend.
- Responses are always logged when fully populated (including `id`,
`origin`, `target` etc.), instead of being logged when still partially
populated. This makes it particularly easy to link request/response
IDs directly from the logs.
Most of TypeError are due to the user passing wrong data. It usually
doesn't mean that we have to fail hard and reload the plugin, nor retry
the call with the same parameters.
Optional top-level imports in Tornado route declarations will trigger
`ImportError`. While this will just mean that those routes will be
skipped, it will also generate a lot of noise on the logs.
This can happen for many reasons - not only if the cache file is not
accessible, but also if the structure/signature of some pickled objects
has changed. In that case, we should invalidate the current cache and
re-initialize it instead of failing.
- The readiness condition should be `multiprocessing.Condition`, not
`threading.Condition` - in most of the cases it will be checked in a
multiprocess environment.
- Fixed parameter name for `write`.
Too much of a pain in the ass to handle, too many format options to
think of, too many combinations of pipelines to support, and if I don't
think of those beforehand then I'll have to offload all of that
complexity on the user.
The `inspect` plugin can now detect references to plugins, backends,
events, responses and schemas in docstrings and replace them either with
links to the documentation or auto-generated examples.
Display a popup modal instead of a confirm box to prompt the user to
install the PWA app.
`confirm` blocks the JavaScript engine when run in `beforeMount` and
therefore the browser won't be able to proceed with `event.prompt()`.
There are situations where you may not want to run the HTTP server in a
full blown WSGI-over-Tornado container - unit/integration tests and
embedded single-core devices are among those cases.
In those scenarios, we should allow the user to be able to run the
backend using the built-in Werkzeug server provided by Flask.
The frontend now calls `utils.rst_to_html` to render the docstrings as
HTML instead of dumping them as raw text.
Also, actions and arguments are now cached to improve performance.
This reverts commit 71401a4936.
Temporarily reverted this commit because the `reuse_address` on the
application's `listen` method has only been implemented in Tornado 6.2 -
and Debian stable still shipts Tornado 6.1.
The WSGI container is a good option to wrap a multi-modal webapp
(Flask + websocket routes), but it's constrained to a single-process
approach and queued/pre-buffered requests. That makes performance poor
when handling requests that may take a few seconds to complete.
The Tensorflow module may take a few seconds to load the first time and
slow down the first scan of the plugins.
All the Tensorflow imports should therefore be placed close to where
they are used instead of being defined at the top of the module.
Defined a `platypush.backend.http.ws` package with all the routes, a
base `WSRoute` class that all the websocket routes can extend, and a
logic in the HTTP backend to automatically scan the package to register
exposed websocket routes.
It was just too painful to find a combination of versions of gunicorn,
gevent, eventlet, pyuwsgi etc. that could work on all of my systems.
On the other hand, Tornado works out of the box with no headaches.
Also in this commit:
- Updated a bunch of outdated/required integration dependencies.
- Black'd and LINTed a couple of old plugins.
The eventlet API has way too many dependency issues with gunicorn.
Still TODO: Fix or at least mitigate the WSGI workers timeout issue when
they handle websocket connections.
The websocket service is no longer provided by a different service,
controlled by a different thread running on another port.
Instead, it's now exposed directly over Flask routes, using
WSGI+eventlet+simple_websocket.
Also, the SSL context options have been removed from `backend.http`, for
sake of simplicity. If you want to enable SSL, you can serve Platypush
through a reverse proxy like nginx.
Instead of iterating over each of the entities in a grouping to find out
which groups should be displayed based on the selector's policy, the
selector can directly keep its `selectedGroups` attribute in sync with
the index.