If may happen (usually because of a race condition) that a cronjob has
already been started, but it hasn't yet changed its status from IDLE to
RUNNING when the scheduler checks it.
This fix guards the application against such events. If they occur, we
should just report them and move on, not terminate the whole scheduler.
- Using tidalapi's `UserPlaylist.add` and `UserPlaylist.delete` methods
instead of defining my own through `_api_request`, so we won't have to
deal with the logic to set the ETag header.
- Added `remove_from_playlist` method.
- Wrapped insert/update/delete operations in transactions
- Proper (and much more efficient) bulk logic
- Better upsert logic
- Return inserted/updated records if the engine supports it
A `WebhookEvent` hook can now return a tuple in the format `(data,
http_code, headers)` in order to customize the HTTP status code and the
headers of a response.
When a client triggers a `WebhookEvent` by calling a configured webhook
over `/hook/<hook_name>`, the server will now wait for the configured
`@hook` function to complete and it will return the returned response
back to the client.
This makes webhooks much more powerful, as they can be used to proxy
HTTP calls or other services, and in general return something to the
client instead of just executing actions.
This class handles runnable plugins that have their own asyncio event
loop, without the pain usually caused by the management of multiple
threads + asyncio loops.
This class handles runnable plugins that have their own asyncio event
loop, without the pain usually caused by the management of multiple
threads + asyncio loops.
- Added initial synchronization and users cache.
- Added loop to poll for new events (TODO: use websocket after the first sync)
- Added login, sync and join actions
This is useful for two reason:
1. Slightly faster variable initialization times.
2. The cached variable object won't fail on the next `.get()`/`.set()`
if the `db` or `redis` plugins have failed for some reason.
The relevant clipboard monitoring logic has been moved to the
`clipboard` plugin. Thus, enabling the plugin should provide all the
feature, with no need for an additional backend.
The cron scheduler has been made more robust against changes in the
system clock (caused by e.g. DST changes, NTP syncs or manual setting).
A more granular management for cronjob events has been introduced, now
supporting a `TIME_SYNC` event besides the usual `STOP`. When the cron
scheduler detects a system clock drift (i.e. the timestamp offset before
and after a blocking wait is >1 sec) then all the cronjobs are notified
and forced to refresh their state.
If Platypush is supposed to work also without a manually created
`config.yaml`, and the HTTP backend is enabled by default in that
configuration, then Flask and companions should be among the required
dependencies.
The UI is now much faster, the build process completes within one minute
(it used to take >15 minutes), and the size of the bundles has been
reduced by ~70%.
- Added ability to listen for signals
- Improved introspection output
- `dbus` plugin and backend have now been merged
- Migrated from `dbus` to `pydbus`
The logic Watchdog applies to filter events based
on `ignore_directories`, `ignore_patterns` and
`ignore_regexes` isn't really sophisticated, and
it doesn't check whether a partial directory/file
name is used in one of the `ignore_*` patterns.
The `file.monitor` backend should therefore implement
this logic on its side.
gunicorn makes it easier to set up a uWSGI wrapper
around the web application, and it's easier to
install and document than handling uwsgi as an
external system dependency.
pychromecast 10.0 introduced some [breaking changes](https://github.com/home-assistant-libs/pychromecast/pull/556/files)
in the declaration of the Chromecast object -
namely, the `device` attribute has been renamed to
`cast_info`. The code of ChromecastPlugin has been
updated to guarantee compatibility in both cases.
- Provide relevant examples for schema fields with no description/examples based on the field type
- Fixed RST warnings in Slack plugin
- Fixed list of events in ngrok plugin
Newer versions of python-vlc have apparently
removed the `event.u` union object from the
events dispatched by the player, resulting in
player callback failures.
The occurrences of `event.u` have therefore
been replaced with the player methods to
correctly retrieve the current state.
- Better synchronization with the player upon `play` request - wait until the play started event is received before returning the status
- DBus exceptions should be caught and handled in the status call
- Removed replication of logic between media.get_youtube_url (action) and media.get_youtube_video_url (internal method)
- Added differentiation between MediaPlayEvent and MediaResumeEvent
- Support for custom PopcornTime API mirror/base URL.
- Full support for TV series search.
- Fixed torrent search (now using a different PopcornTime API mirror).
- Migrated SASS engine from `node-sass` (currently deprecated and broken on Node 16) to `sass`.
- Fixed alignment of Z-Wave UI header on Chrome/Webkit.
The error should not be swallowed and it should be instead propagated up
to calendar.get_upcoming_events, if it's called from that context. And
calendar.get_upcoming_events should be in charge of handling the
exceptions and make sure that failing to retrieve the events for one
calendar doesn't make the whole method fail.
For some reason, vlc event handlers are not re-entrant (https://github.com/oaubert/python-vlc/issues/44#issuecomment-378520074).
This means that the vlc API can't be used from an event handler,
and that an event handler that reacts to stop/end-of-stream by
releasing the player and the vlc instance will likely get stuck
and the app may eventually die with SIGSEGV.
Because of this design limitation on the vlc side, the plugin has
to run another thread in the main app that monitors the stop event
set by the event handler and releases the resources appropriately.
- Refactored the logic for executing a request and wrapping the response common to procedures, crons and event hook decorators into platypush.common.exec_wrapper
- Added mock getvalue() method to Logger to prevent PyCharm failures in the tests when sys.stdout is redirected to the Logger object
If we open multiple connections from multiple MQTT-based backends to the
same host, with some of them having no topics subscribed, Paho-MQTT can
mess up which client is supposed to receive which message and we may end
up with lost messages on the platypush_bus_mqt/<host> topic.
If we open multiple connections from multiple MQTT-based backends to the
same host, with some of them having no topics subscribed, Paho-MQTT can
mess up which client is supposed to receive which message and we may end
up with lost messages on the platypush_bus_mqt/<host> topic.
# This workflows will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
- Support for custom YouTube video/audio formats on media plugins.
### Fixed
- Responses for requests received over an MQTT backend are now delivered to the right topic
(`<device_base_topic>/responses/<msg_id>`).
- Various fixes on media plugins.
## [0.22.3] - 2021-10-01
### Added
- `gotify` integration (see #198).
## [0.22.2] - 2021-09-25
### Added
- `ngrok` integration (see #196).
## [0.22.1] - 2021-09-22
### Fixed
- `zigbee.mqtt` backend now no longer requires the MQTT backend/plugin to be enabled.
- Fixed bug on empty popcorn API responses.
### Changed
- Created CI Gitlab pipeline to replace the Platypush event-based pre-existing pipeline.
### Removed
- Removed docs references to removed/abstract integrations.
## [0.22.0] - 2021-09-16
### Changed
- Platypush now uses manifest files to describe plugins and backends. Each extension is now
expected to provide a `manifest.yaml` file in its folder, reporting its package name, pip
dependencies, required system packages and optional extra installation commands.
- Refactored `platyvenv`, `platydock`, documentation generation and plugin management engine.
They are now both faster and more robust, since they can rely on the manifest definition to
operate instead of pydoc strings conventions or `config.yaml` conventions.
- `platyvenv start` now starts the environment process synchronously and it prints
stdout/stderr instead of redirecting it to the logs dir (previous behaviour:
`platyvenv start` used to start the process asynchronously and the logs were stored
to `~/.local/share/platypush/venv/<env>/logs/<stdout|stderr>.log`).
### Removed
- Removed `Homeseer` integration - it was based on a Python integration that has now been
pulled out of PyPI and GitHub. A new integration may come in the future if there is enough
demand for it.
## [0.21.4] - 2021-08-24
### Fixed
- Fixed JWT token generation, since python-jwt >= 2.0 always returns strings (not bytes) upon `jwt.encode`.
## [0.21.3] - 2021-07-28
### Added
- Added `sun` plugin for sunrise/sunset events.
- Added `slack` integration.
## [0.21.2] - 2021-07-20
### Added
- Added `music.spotify` backend to emulate a Spotify Connect receiver through Platypush.
- Added `music.spotify` plugin.
- Added `music.spotify` UI integration.
## [0.21.1] - 2021-06-22
### Added
- Added `switchbot` plugin to interact with Switchbot devices over the cloud API instead of
directly accessing the device's Bluetooth interface.
- Added `marshmallow` dependency - it will be used from now own to dump and document schemas
and responses instead of the currently mixed approach with `Response` objects and plain
dictionaries and lists.
- Support for custom MQTT timeout on all the `zwavejs2mqtt` calls.
- Added generic joystick backend `backend.joystick.jstest` which uses `jstest` from the
standard `joystick` system package to read the state of joysticks not compatible with
`python-inputs`.
- Added PWM PCA9685 plugin.
- Added Linux native joystick plugin, ``backend.joystick.linux``, for the cases where
``python-inputs`` doesn't work and ``jstest`` is too slow.
### Changed
- `switch.switchbot` plugin renamed to `switchbot.bluetooth` plugin, while the new plugin
that uses the Switchbot API is simply named `switchbot`.
### Fixed
- More robust reconnection logic on the Pushbullet backend in case of websocket errors.
## [0.21.0] - 2021-05-06
### Added
- Support for custom PopcornTime API mirror/base URL.
- Full support for TV series search.
### Fixed
- Fixed torrent search (now using a different PopcornTime API mirror).
- Migrated SASS engine from `node-sass` (currently deprecated and broken on Node 16) to `sass`.
- Fixed alignment of Z-Wave UI header on Chrome/Webkit.
## [0.20.10] - 2021-04-28
### Fixed
- Fixed zwave/zwavejs2mqtt interoperability.
## [0.20.9] - 2021-04-12
### Added
- Added zwavejs2mqtt integration (see [#186](https://git.platypush.tech/platypush/platypush/-/issues/186).
### Fixed
- Major LINT fixes.
### Removed
- Removed unmaintained integrations: TorrentCast and Booking.com
## [0.20.8] - 2021-04-04
### Added
- Added `<Camera>` dashboard widget.
- Added support for custom dashboard widgets with customized (see https://git.platypush.tech/platypush/platypush/wiki/Backends#creating-custom-widgets).
- Added support for controls on `music.mpd` dashboard widget.
### Fixed
- Fixed zigbee2mqtt backend error in case of messages with empty payload (see [#184](https://git.platypush.tech/platypush/platypush/-/issues/184)).
- Fixed compatibility with all versions of websocket-client - versions >= 0.58.0 pass a `WebSocketApp` object as a first
argument to the callbacks, as well as versions <0.54.0do,buttheversionsinbetweendon'tpassthisargument.
## [0.20.7] - 2021-03-26
### Fixed
- Fixed race condition on `media.vlc.stop` when clearing the VLC instance.
- Fixed dashboard widgets custom classes being propagated both to the container and to the widget content [see #179]
- Fixed compatibility with SQLAlchemy >= 1.4.
## [0.20.6] - 2021-03-16
### Added
- Added `log.http` backend to monitor changes to HTTP log files
(see [#167](https://git.platypush.tech/platypush/platypush/-/issues/167)).
- Added `file.monitor` backend, which replaces the `inotify` backend
(see [#172](https://git.platypush.tech/platypush/platypush/-/issues/172)).
### Removed
- Removed legacy `pusher` script and `local` backend.
### Fixed
- Fixed support for Z-Wave switches.
- Fixed possible race condition on VLC stop.
## [0.20.5] - 2021-03-12
### Added
- Added support for a static list of devices to actively scan to the `bluetooth.scanner` backend
(see [#174](https://git.platypush.tech/platypush/platypush/-/issues/174)).
- Added `weather.openweathermap` plugin and backend, which replaces `weather.darksky`, since the
Darksky API will be completely shut down by the end of 2021.
### Fixed
- Cron expressions should adhere to the UNIX cronjob standard and use the machine local time,
not UTC, as a reference (closes [#173](https://git.platypush.tech/platypush/platypush/-/issues/173)).
- Better management of Z-Wave values types from the UI.
- Disable logging for `ZwaveValueEvent` events - they tend to be very verbose and
can impact the performance on slower devices. They will still be published to the
websocket clients though, so you can still debug Z-Wave values issues from the browser
developer console (enable debug traces).
- Added suffix to the `zigbee.mqtt` backend default `client_id` to prevent clashes with
the default `mqtt` backend `client_id`.
## [0.20.4] - 2021-03-08
### Added
- Added SmartThings integration.
- Support for custom Redis message queue name over the `--redis-queue` argument.
### Fixed
- Refactored tests to use `pytest` instead of `unittest`.
- Some major bug fixes on procedures and hooks context evaluation.
## [0.20.3] - 2021-02-28
### Fixed
- Several bug fixes on the VLC plugin, including proper management of stop/end-of-stream, volume set and missing integration requirements in `requirements.txt` and `setup.py`.
## [0.20.2] - 2021-02-27
### Fixed
- More stable ZeroConf backends registration logic in case of partial or missing results.
- Improved and refactored integration tests.
### Added
- Support for passing context variables (${}) from YAML procedures/hooks/crons to Python procedure/hooks/crons.
[![Join chat on Matrix](https://img.shields.io/matrix/:platypush?server_fqdn=matrix.platypush.tech)](https://matrix.to/#/#platypush:matrix.platypush.tech)
- Advised read: [**Getting started with Platypush**](https://medium.com/@automationguru/automate-your-house-your-life-and-everything-else-around-with-platypush-dba1cd13e3f6) (Medium article).
<!-- toc -->
- The [wiki](https://github.com/BlackLight/platypush/wiki) also contains many resources on getting started.
- [Architecture](#architecture)
* [Plugins](#plugins)
* [Actions](#actions)
* [Backends](#backends)
* [Events](#events)
* [Hooks](#hooks)
* [Procedures](#procedures)
* [Cronjobs](#cronjobs)
* [The web interface](#the-web-interface)
- [Installation](#installation)
* [System installation](#system-installation)
+ [Install through `pip`](#install-through-pip)
+ [Install through a system package manager](#install-through-a-system-package-manager)
+ [Install from sources](#install-from-sources)
* [Installing the dependencies for your extensions](#installing-the-dependencies-for-your-extensions)
+ [Install via `extras` name](#install-via-extras-name)
+ [Install via `manifest.yaml`](#install-via-manifestyaml)
+ [Check the instructions reported in the documentation](#check-the-instructions-reported-in-the-documentation)
- Extensive documentation for all the available integrations and messages is available on [ReadTheDocs](https://platypush.readthedocs.io/en/latest/).
<!-- tocstop -->
- Also check other [Medium stories](https://medium.com/tag/platypush/archive) to get more insights on what you can build with it and inspiration about possible usages.
- Recommended read: [**Getting started with Platypush**](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush).
- If you have issues/feature requests/enhancement ideas please post a [GitHub issue](https://github.com/BlackLight/platypush/issues).
A [Reddit channel](https://www.reddit.com/r/platypush) is also available for more general questions.
- The [blog](https://blog.platypush.tech) is in general a good place to get
more insights on what you can build with it and inspiration about possible
usages.
- The [wiki](https://git.platypush.tech/platypush/platypush/wiki) also
contains many resources on getting started.
- Extensive documentation for all the available integrations and messages [is
available](https://docs.platypush.tech/).
- If you have issues/feature requests/enhancement ideas please [create an
- A [Reddit channel](https://www.reddit.com/r/platypush) is also available for
more general questions.
- A [Matrix instance](https://matrix.to/#/#platypush:matrix.platypush.tech) is
also available if you are looking for more interactive support.
---
Imagine Platypush as some kind of [IFTTT](https://ifttt.com) on steroids - or [Tasker](https://tasker.joaoapps.com/), or [Microsoft Flow](https://flow.microsoft.com), or [PushBullet](https://pushbullet.com) on steroids.
Platypush aims to turn any device in a smart hub that can control things, interact with cloud services and send messages to other devices. It's a general-purpose lightweight platform to process any request and run any logic triggered by custom events.
Platypush is a general-purpose extensible platform for automation and
integration across multiple services and devices.
Imagine the ability of running any task you like, or automate any routine you like, on any of your devices. And the flexibility of executing actions through a cloud service, with the power of running them from your laptop, Raspberry Pi, smart home device or smartphone.
It enables users to create their own self-hosted pieces of automation based on
events (*if this happens then do that*)
and it provides a comprehensive and customizable user interface that collects
everything you need to visualize and control under one roof.
It takes some concepts from [IFTTT](https://ifttt.com),
Flow](https://flow.microsoft.com), [PushBullet](https://pushbullet.com) and
[Home Assistant](https://www.home-assistant.io/) to provide an environment
where the user can easily connect things together.
Its ideal home is a single-board computer like a RaspberryPi that you can
configure to orchestrate any home automation and cloud automation in your own
living room or garage, but it can easily run on any device that can run a
Python interpreter, and the bar for the hardware requirements is very low as
well - I use it to run pieces of automation on devices as powerful as a
RaspberryPi Zero or an old Nokia N900 with Linux.
You can use Platypush to do things like:
- Control your smart home lights
- Control your favourite music player
- Interact with your voice assistant
- Get events from your Google or Facebook calendars
- Read data from your sensors and trigger custom events whenever they go above or below some custom thresholds
- Control the motors of your robot
- Send automated emails
- Synchronize the clipboards on your devices
- Control your smart switches
- Implement custom text-to-speech commands
- Build any kind of interaction with your Android device using Tasker
- Play local videos, YouTube videos and torrent links
- Get weather forecast for your location
- Build your own web dashboard with calendar, weather, news and music controls (basically, anything that has a Platypush web widget)
- ...and much more (basically, anything that comes with a [Platypush plugin](https://platypush.readthedocs.io/en/latest/plugins.html))
- [Control your smart home lights](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush)
- [Control your music and synchronize it to multiple devices](https://blog.platypush.tech/article/Build-your-open-source-multi-room-and-multi-provider-sound-server-with-Platypush-Mopidy-and-Snapcast)
- [Create custom and privacy-secure voice assistants that run custom hooks on your phrases](https://blog.platypush.tech/article/Build-custom-voice-assistants)
- Build integrations between [sensors](https://docs.platypush.tech/en/latest/platypush/backend/sensor.html),
or [sound detection](https://blog.platypush.tech/article/Create-your-smart-baby-monitor-with-Platypush-and-Tensorflow)
- [Get events from your Google or Facebook calendars](https://docs.platypush.tech/en/latest/platypush/plugins/calendar.html)
- [Read data from your sensors and trigger custom events whenever they go above or below some custom thresholds](https://blog.platypush.tech/article/How-to-build-your-personal-infrastructure-for-data-collection-and-visualization)
- [Control and automate a self-built robot](https://docs.platypush.tech/en/latest/platypush/plugins/gpio.zeroborg.html)
- [Deliver automated newsletters from custom RSS digests](https://blog.platypush.tech/article/Deliver-customized-newsletters-from-RSS-feeds-with-Platypush)
- [Synchronize the clipboards on your devices](https://docs.platypush.tech/en/latest/platypush/plugins/clipboard.html)
- [Control your smart switches](https://docs.platypush.tech/en/latest/platypush/plugins/switch.html)
- [Build any kind of interactions and automation routines with your Android device using Tasker](https://blog.platypush.tech/article/How-to-build-your-personal-infrastructure-for-data-collection-and-visualization)
YouTube videos and torrent media from any device and service, to any device
- [Get weather forecast events for your location and build automation routines on them](https://docs.platypush.tech/en/latest/platypush/plugins/weather.darksky.html)
- [Create a custom single hub for Zigbee and Z-Wave smart devices](https://blog.platypush.tech/article/Transform-a-RaspberryPi-into-a-universal-Zigbee-and-Z-Wave-bridge)
- Build your own web dashboard with calendar, weather, news and music controls
(basically, anything that has a Platypush web widget)
- ...and much more (basically, anything that comes with a [Platypush plugin](https://docs.platypush.tech/en/latest/plugins.html))
Imagine the ability of executing all the actions above through messages delivered through:
## Architecture
- A web interface
- A JSON-RPC API
- Raw TCP messages
- Web sockets
- [PushBullet](https://pushbullet.com)
- [Kafka](https://kafka.apache.org)
- [Redis](https://redis.io)
- [MQTT](https://mqtt.org)
- ...amd much more (basically, anything that comes with a [Platypush backend](https://platypush.readthedocs.io/en/latest/backends.html))
The architecture of Platypush consists of a few simple pieces, orchestrated by
Imagine the ability of running the application, with lots of those bundled features, on any device that can comes with Python (_only compatible with version 3.6 and higher_). Platypush has been designed with performance in mind, it's been heavily tested on slower devices like Raspberry Pis, and it can run the web server features, multiple backends and plugins quite well even on a Raspberry Pi Zero - it's even been tested with some quite impressive performance on an older [Nokia N900](https://en.wikipedia.org/wiki/Nokia_N900), and of course you can run it on any laptop, desktop, server environment. It's been developed mainly with IoT in mind (and some of its features overlap with IoT frameworks like [Mozilla IoT](https://iot.mozilla.com) and [Android Things](https://developer.android.com/things/)), but nothing prevents you from automating any task on any device and environment.
Plugins are integrations that do things - like [modify