revil-O/platypush
1
0
Fork 0
forked from platypush/platypush
Code Pull requests Activity
3787 commits 7 branches 83 tags 136 MiB
Commit graph

1466 commits

Author SHA1 Message Date
Fabio Manganiello
52e353dc14
Expose the wrapped function in @action. 
Added a `wrapped` "hidden" parameter to the function returned by the
`@action` decorator.

We need this to access the underlying decorated function when e.g. we
need to access its specs or decorators.
 2023-10-09 22:35:08 +02:00
Fabio Manganiello
1e93af86f4
Fixed some broken docstring references. 2023-10-09 01:33:45 +02:00
Fabio Manganiello
53bdcb9604
A major rewrite of the inspect plugin. 
- The `inspect` plugin and the Sphinx inspection extensions now use the
  same underlying logic.

- Moved all the common inspection logic under
  `platypush.common.reflection`.

- Faster scanning of the available integrations and components through a
  pool of threads.

- Added `doc_url` parameters.

- Migrated events and responses metadata scanning logic.

- Now expanding some custom Sphinx tag instead of returning errors when
  running outside of the Sphinx context - it includes `:class:`,
  `:meth:` and `.. schema::`.
 2023-10-09 01:33:45 +02:00
Fabio Manganiello
4d52fd35b9
Skip None responses in the RSS plugin. 2023-10-04 22:08:11 +02:00
Fabio Manganiello
40d3ad1150
Removed <type> | None type hints. 
They break on Python < 3.10.
 2023-10-03 01:15:13 +02:00
Fabio Manganiello
3086dd86fc
LINT+Black+stability fixes for some plugins that hadn't been touched in a while. 
- media.mplayer
- media.omxplayer
- media.vlc
- music.mpd
- music.snapcast
 2023-10-01 22:55:06 +02:00
Fabio Manganiello
2aefc4e5c8
Several improvements for the Google integrations. 
1. Improved documentation. Every plugin now reports the exact steps to
   get the integration up and running with the right API scopes.

2. All Google plugins now have a standard process to get (and reuse) the
   client secret. Except for PubSub, Translate and Maps (which have
   their own flows), all the Google plugins now read the client secrets
   from `<WORKDIR>/credentials/google/client_secret.json` by default.

3. Black/LINT for some of those plugins, which hadn't been touched in a
   while.

4. The interface to pass API scopes is now leaner. It's now possible to
   pass a scope directly as e.g. `calendar.readonly` rather than
   `https://www.googleapis.com/auth/calendar.readonly`.

5. Improved the logic to retrieve the right scope tokens file. If e.g.
   an integration requires the role `A`, and a credentials file exists
   for the roles `A` and `B`, then this file will be used rather than
   prompting the user to authenticate again.
 2023-10-01 15:37:20 +02:00
Fabio Manganiello
5ca3757834
A more readable configuration for the calendar plugin. 
The old type configuration
(`platypush.plugins.calendar.name.CalendarNamePlugin`) is a bit clunky.

Instead, since the type will always be a plugin, we should encourage
the use of `calendar.name` directly to identify the type.
 2023-10-01 01:09:15 +02:00
Fabio Manganiello
966a6ce29e
httplib2 should be an explicit dependency for Google integrations. 
Plus, some misc LINT/Black chores.
 2023-10-01 00:52:59 +02:00
Fabio Manganiello
532f5479b3
Added full YAML example for chat.irc configuration. 2023-09-30 14:34:54 +02:00
Fabio Manganiello
4f5ccda353
Better documentation for the calendar plugin. 2023-09-30 13:31:41 +02:00
Fabio Manganiello
15d06fa5c2
Improved docstring parser logic. 2023-09-30 12:35:31 +02:00
Fabio Manganiello
905d6632e0
Misc documentation improvements. 2023-09-29 18:09:15 +02:00
Fabio Manganiello
0a3ec4b9f1
Merge branch 'master' into 311/auto-generate-deps-docs 2023-09-28 01:25:29 +02:00
Fabio Manganiello
b76f141b61
Catch response write errors in the MQTT callback. 
If the client that forwarded the request is no longer available (either
because an exception or a timeout was raised) then its I/O buffer and
event loop may be closed.

In this case, the response callback should handle and report the
exception, and still set the event, so that any other threads waiting
for the response can move on.
 2023-09-27 11:23:55 +02:00
Fabio Manganiello
ca7f042ccc
We shouldn't call dateutil.parser if t has already been deserialized to a datetime. 2023-09-27 11:20:10 +02:00
Fabio Manganiello
c311987741
Removed typing.Final from some of the most commonly used modules. 
`typing.Final` is not defined on Python < 3.8.
 2023-09-26 23:50:10 +02:00
Fabio Manganiello
9298f52443
Moved BluetoothPlugin to __init__.py. 
This is for consistency with other plugins, that all have their main
plugin class definition inside of `__init__.py`.
 2023-09-24 19:21:53 +02:00
Fabio Manganiello
c3337ccc6c
[#311] Docs deps autogen sphinx plugin. 
Added an `add_dependencies` plugin to the Sphinx build process that
parses the manifest files of the scanned backends and plugins and
automatically generates the documentation for the required dependencies
and triggered events.

This means that those dependencies are no longer required to be listed
in the docstring of the class itself.

Also in this commit:

- Black/LINT for some integrations that hadn't been touched in a long
  time.

- Deleted some leftovers from previous refactors (deprecated
  `backend.mqtt`, `backend.zwave.mqtt`, `backend.http.request.rss`).

- Deleted deprecated `inotify` backend - replaced by `file.monitor` (see
  #289).
 2023-09-24 17:00:08 +02:00
Fabio Manganiello
059f7a4b11 Merge branch 'master' into 316/rpm-support 2023-09-23 16:21:48 +02:00
Fabio Manganiello
b797add90d
Modify shebang header for Mercury JS script. 
The RPM build process wants an absolute path.
 2023-09-23 16:16:13 +02:00
Fabio Manganiello
79e24461cb
Coalesce to empty list if zigbee.mqtt.devices returns null. 2023-09-22 17:00:53 +02:00
Fabio Manganiello
55965e962c
Store the Philips Hue bridge configuration under our workdir. 
By default, the `phue` library will store the file containing the token
and the bridge configuration under `~/.python_hue`.

That's outside of our application folder, and it can't easily be copied
around or added to Docker volumes.

We should instead have it under `<WORKDIR>/light.hue/config.json`, in
line with what the other plugins do, and if `~/.python_hue` is available
but `<WORKDIR>/light.hue/config.json` isn't then we should copy the
legacy file to the new one.
 2023-09-22 16:58:44 +02:00
Fabio Manganiello
761f2768cb
[#316] Added RPM dependencies. 2023-09-21 23:20:06 +02:00
Fabio Manganiello
a6efaad26d
[#306] Removed Travis CI integration. 
I've tried my best to keep it around, but the endpoints seem to be
broken, they no longer have a link to their API v3 documentation, and
the API Explorer that was supposed to be in the dashboard is gone.
 2023-09-20 23:31:58 +02:00
Fabio Manganiello
3e8d6bb01e Removed typing-extensions dependency. 
The package isn't present on older versions of Debian.
 2023-09-18 14:05:39 +02:00
Fabio Manganiello
07c88c9530
Added logic to prevent socket leakage from paho-mqtt upon client stop. 2023-09-17 23:43:56 +02:00
Fabio Manganiello
3f4168eb69
Support for new Quality enumeration constants on tidalapi. 2023-09-17 22:40:26 +02:00
Fabio Manganiello
c6cda86b1c
LINT/regex fix for ESP plugin. 2023-09-17 17:10:40 +02:00
Fabio Manganiello
2e004c1a1e
get_all_plugins should use Message.Encoder for encoding the response. 
Some plugin information may include enums, tuples and other types that
aren't serializable by the default JSON serializer.
 2023-09-17 02:11:52 +02:00
Fabio Manganiello
2fcd623c51
Migrated zwave.mqtt integration. 
Merged the zwave.mqtt plugin with the listener and removed the
leftovers of the legacy zwave plugin.
 2023-09-16 03:58:19 +02:00
Fabio Manganiello
2c93049ee5
Catch all the exceptions in a plugin action wrapper. 
The @action decorator should capture all the exceptions,
log them and return them on `Response.errors`.

This ensures that uncaught exceptions from plugin
actions won't unwind out of control, and also that they
are logged and treated consistently across all the
integrations.
 2023-09-14 23:08:23 +02:00
Fabio Manganiello
ac72b2f7a8
Fixed management of state on zigbee.mqtt. 
Before the merge of the plugin and the listener those components
used to have their own separate state, which led to inconsistencies.
 2023-09-14 23:05:27 +02:00
Fabio Manganiello
5a514fdcce
Only support the run_topic logic on the MQTT plugin. 
Plugins that extend `MqttPlugin` shouldn't run messages as
requests, even if the parent MQTT plugin is configured to
do so.
 2023-09-14 01:09:03 +02:00
Fabio Manganiello
4cb5aa7acb
Prepend the class name to the string used to generate the MQTT client_id hash. 
If we include the class name by default then we won't have to
explicitly modify the client_id in the implementation classes
in order to prevent clashes.
 2023-09-14 01:06:53 +02:00
Fabio Manganiello
ddd8f1afdc
base_topic param in zigbee.mqtt renamed to topic_prefix. 
This is for sake of consistency with other integrations (like
`zwave.mqtt`) that also use the same parameter name for the MQTT topic
prefix.
 2023-09-07 21:32:56 +02:00
Fabio Manganiello
3de510da68
Migrated zigbee.mqtt integration. 
The plugin has been migrated to the new `mqtt` API and the legacy
listener that extended `MqttBackend` has been removed and merged into
the plugin.
 2023-09-06 02:44:56 +02:00
Fabio Manganiello
33a1ef39e4
Refactored and merged backend.mqtt logic into mqtt plugin. 2023-09-06 02:43:45 +02:00
Fabio Manganiello
b6c0ff799b
Rewritten the http.webpage plugin. 2023-09-03 17:33:25 +02:00
Fabio Manganiello
669f2eb2d2
LINT/black for tts.mimic3 plugin. 2023-09-02 12:40:34 +02:00
Fabio Manganiello
35416f3ee3
Some LINT on the old http.request.rss plugin. 2023-09-02 01:01:16 +02:00
Fabio Manganiello
2c46b6fe14
Added git a required manifest dependency when needed. 
It is needed for packages that install pip packages via git.
 2023-08-20 21:19:15 +02:00
Fabio Manganiello
a99ffea37c
Fixed apt dependencies for mpd plugin. 2023-08-19 21:46:08 +02:00
Fabio Manganiello
69706eaabe
s/logger/_logger/ in the plugins module. 
The `logger` name may clash with the context of an action, where
`logger` may have been set to something else.
 2023-08-19 13:32:19 +02:00
Fabio Manganiello
1cb686bdab
Updated the inspect plugin to the new manifest utils interface. 2023-08-19 13:31:48 +02:00
Fabio Manganiello
1825b492b3
Replaced Config.workdir with Config.get_workdir(). 
Again, Python < 3.9 doesn't like class properties.
 2023-08-19 13:21:24 +02:00
Fabio Manganiello
c2b3ec8ce3
Fixed manifest files with outdated formats. 2023-08-19 12:54:33 +02:00
Fabio Manganiello
2cab836bdf
Entity.columns class property replaced by Entity.get_columns method. 
Again, Python < 3.9 doesn't like the combination of `@property` +
`@classmethod`.
 2023-08-18 17:20:53 +02:00
Fabio Manganiello
a9cdff900e
_variable should be an external global function rather than a class property. 
The combination of `@property` + `@classmethod` isn't supported on
Python < 3.9.
 2023-08-18 16:16:47 +02:00
Fabio Manganiello
ca95490412
Added timeout parameter to requests.get in the rss plugin. 2023-08-18 15:53:30 +02:00
Fabio Manganiello
5dd7345c0b
Sync the latest parse timestamps in main instead of __init__ in rss. 
We should load the latest timestamps from the db when the thread starts
instead of doing it in the constructor.

The constructor may be invoked when the entities engine hasn't been
initialized yet, and result in deadlocks.
 2023-08-18 15:51:11 +02:00
Fabio Manganiello
bf7d060b81
Added @ensure_initialized decorator to actions in variable. 
The `variable` plugin may break in the constructor the first time the
application is started.

That's because it tries to initialize the cache of stored variables, but
the local database hasn't yet been initialized.

That's because plugins are registered _before_ the entities engine is
initialized, as the entities engine assumes that it already has plugins
to scan for entities.

Therefore, the initialization of the `variable` plugin's cache should be
lazy (only done upon the first call to `get`/`set` etc.), in order to
prevent deadlock situations where the plugin waits for the engine to
start, but the engine will be initialized only after the plugin is
ready.

And the lazy initialization logic should also ensure that the entities
engine has been properly started (and emit a `TimeoutError` if that's
not the case), in order to prevent race conditions.
 2023-08-17 02:47:30 +02:00
Fabio Manganiello
98e9abde18
Extended manifest files with Python system packages (if available). 
- If a Python optional dependency is available as a system package on
  the target system, try and install it that route rather than pip. It's
  usually faster and it decreases the risk of breaking system packages.

- Added support for apk dependencies in manifest files. This brings the
  number of distros officially supported by all the extensions to four:

  - Alpine
  - Arch
  - Debian
  - Ubuntu
 2023-08-16 22:43:51 +02:00
Fabio Manganiello
00863a176e
Added application plugin. 2023-08-15 11:20:24 +02:00
Fabio Manganiello
f51beb271e
Large refactor + stability fixes for the external process control logic. 2023-08-15 11:12:21 +02:00
Fabio Manganiello
4062ddbcf0
More improvements to the inspect plugin. 
- Support for distinct `type` field on constructor and method arguments.

- Added `has_varargs` field.

- Added `required` field.

- Better logic for parsing arguments `default` values.
 2023-08-08 20:47:27 +02:00
Fabio Manganiello
9beb0a7af3
Skip :type: annotations from docstring args documentation. 2023-08-08 20:47:27 +02:00
Fabio Manganiello
2a30f060b4
Parse the arguments list from obj_type.__init__.__doc__ too. 2023-08-08 20:47:27 +02:00
Fabio Manganiello
e9a568fdd2
Unified interface to retrieve the Redis bus configuration. 
A common `utils.get_redis_conf` has been created to handle the cascade
fallback logic used to retrive the default Redis configuration.
 2023-07-24 01:04:13 +02:00
Fabio Manganiello
0dc380fa94
Removed dependency from prctl. 
Also, black'd and LINT-fixed some files that hadn't been touched in a
while.
 2023-07-23 19:17:30 +02:00
Fabio Manganiello
66981bd00b
Updated email addresses and black'd some old source files. 2023-07-22 23:02:44 +02:00
Fabio Manganiello
3eda0c6f17
[#268] Implemented XMPP integration. 2023-07-22 22:36:36 +02:00
Fabio Manganiello
cb04af0bbd
Catch TypeError when execution an action. 
Most of TypeError are due to the user passing wrong data. It usually
doesn't mean that we have to fail hard and reload the plugin, nor retry
the call with the same parameters.
 2023-07-14 22:20:27 +02:00
Fabio Manganiello
27cf1bec52
Removed some optional top-level imports. 
Optional top-level imports in Tornado route declarations will trigger
`ImportError`. While this will just mean that those routes will be
skipped, it will also generate a lot of noise on the logs.
 2023-07-01 03:13:38 +02:00
Fabio Manganiello
5e5403287e
The inspect plugin should not fail hard if the cache couldn't be loaded. 
This can happen for many reasons - not only if the cache file is not
accessible, but also if the structure/signature of some pickled objects
has changed. In that case, we should invalidate the current cache and
re-initialize it instead of failing.
 2023-06-28 01:33:12 +02:00
Fabio Manganiello
ba827b0248
Added sound plugin documentation. 
- Added example of how to stream audio over HTTP.

- Added Portaudio to the apt/pacman dependencies.
 2023-06-28 00:10:36 +02:00
Fabio Manganiello
a103ea49f1
Fixed ffmpeg/audio consumer synchronization upon timeout. 2023-06-27 15:12:15 +02:00
Fabio Manganiello
77f7cd8b90
Don't use typing.Self (yet). 
typing.Self has only been introduced in Python 3.10.
 2023-06-27 14:17:04 +02:00
Fabio Manganiello
f2540437b7
Sound plugin rewritten almost from scratch. 2023-06-27 13:31:38 +02:00
Fabio Manganiello
2f4229d7b1
pylint fixes for the camera plugin. 2023-06-16 15:40:05 +02:00
Fabio Manganiello
9aa8e4538a
Better termination logic for the ffmpeg audio converter. 2023-06-16 11:47:37 +02:00
Fabio Manganiello
a6351dddd4
Extracted AudioRecorder out of SoundPlugin. 2023-06-16 03:12:55 +02:00
Fabio Manganiello
da93f1b3b0
[Chore] pylint 2023-06-14 01:44:36 +02:00
Fabio Manganiello
c8786b75de
sound.recordplay merged into sound.record. 2023-06-12 22:15:02 +02:00
Fabio Manganiello
be794316a8
Merged sound.stream_recording and sound.record. 2023-06-12 13:06:02 +02:00
Fabio Manganiello
a415c5b231
Merged outfile/fifo logic in sound.stream_recording. 2023-06-12 12:33:14 +02:00
Fabio Manganiello
e238fcb6e4
Refactoring the sound plugin to use ffmpeg as a stream converter. 2023-06-11 12:48:49 +02:00
Fabio Manganiello
4587b262b0
Stream camera frames over HTTP using a Redis pub/sub mechanism. 2023-06-05 20:40:12 +02:00
Fabio Manganiello
d7208c6bbc
Refactored Tornado routes for native pub/sub support. 
The Redis pub/sub mechanism is now a native feature for Tornado routes
through the `PubSubMixin`.

(Plus, lint/black chore for the sound plugin)
 2023-05-30 21:58:27 +02:00
Fabio Manganiello
8b5eb82497
Camera stream writer fixes. 
- The readiness condition should be `multiprocessing.Condition`, not
  `threading.Condition` - in most of the cases it will be checked in a
  multiprocess environment.

- Fixed parameter name for `write`.
 2023-05-30 11:06:48 +02:00
Fabio Manganiello
4fffabd82a
Revert "Removed camera.gstreamer." 
This reverts commit b4d714df8a.
 2023-05-29 22:13:24 +02:00
Fabio Manganiello
4bf9c01ac9
Moved camera routes. 
Camera routes migrated from Flask blueprints to Tornado handlers.
 2023-05-27 22:24:45 +02:00
Fabio Manganiello
b4d714df8a
Removed camera.gstreamer. 
Too much of a pain in the ass to handle, too many format options to
think of, too many combinations of pipelines to support, and if I don't
think of those beforehand then I'll have to offload all of that
complexity on the user.
 2023-05-27 22:19:50 +02:00
Fabio Manganiello
1c40e5e843
Black'd the camera plugin and writer. 
Also, proper fix for the multi-inheritance problem of
the ffmpeg writers.
 2023-05-23 20:42:59 +02:00
Fabio Manganiello
a7aabd7c52
Fixed handling of :meth: docstring annotations with relative paths. 2023-05-22 16:32:30 +02:00
Fabio Manganiello
7eca1c27c9
Blackened the qrcode and pushbullet plugins 2023-05-22 02:33:54 +02:00
Fabio Manganiello
d7405ad05d
Added multiple parsers for the entities referenced in docstrings. 
The `inspect` plugin can now detect references to plugins, backends,
events, responses and schemas in docstrings and replace them either with
links to the documentation or auto-generated examples.
 2023-05-22 02:20:58 +02:00
Fabio Manganiello
b91c1eba6d
Parse :return: definitions from action docstrings too. 2023-05-21 03:05:19 +02:00
Fabio Manganiello
27d4a20418
Use reflection to infer the arguments of a Python user procedure 2023-05-17 17:17:59 +02:00
Fabio Manganiello
61ea3d79e4
Large refactor for the inspect plugin. 
More common logic has been extracted and all the methods and classes
have been documented and black'd.
 2023-05-17 00:05:22 +02:00
Fabio Manganiello
8447f9a854
Improved rendering of actions/arguments documentation. 
The frontend now calls `utils.rst_to_html` to render the docstrings as
HTML instead of dumping them as raw text.

Also, actions and arguments are now cached to improve performance.
 2023-05-14 15:06:34 +02:00
Fabio Manganiello
5f2d6dfeb5
Added utils.rst_to_html action. 2023-05-14 15:05:24 +02:00
Fabio Manganiello
3c83e7f412
A faster implementation for the inspect.get_* methods. 
Plugin/backend lookup is now done by inspecting the manifest files
instead of searching all the subpackages.
 2023-05-13 13:44:46 +02:00
Fabio Manganiello
cfedcd701e
Performance improvements when loading the Tensorflow plugin. 
The Tensorflow module may take a few seconds to load the first time and
slow down the first scan of the plugins.

All the Tensorflow imports should therefore be placed close to where
they are used instead of being defined at the top of the module.
 2023-05-11 19:48:22 +02:00
Fabio Manganiello
41233138ff
Blackened inspect module and extracted model defs to adjacent module. 2023-05-09 21:58:02 +02:00
Fabio Manganiello
56dc8d0972
Migrated the webapp to Tornado. 
It was just too painful to find a combination of versions of  gunicorn,
gevent, eventlet, pyuwsgi etc. that could work on all of my systems.

On the other hand, Tornado works out of the box with no headaches.

Also in this commit:

- Updated a bunch of outdated/required integration dependencies.
- Black'd and LINTed a couple of old plugins.
 2023-05-08 02:06:45 +02:00
Fabio Manganiello
d1f0e1976c
Exclude squashfs/loopback mounts from system.disk_info. 2023-05-06 18:53:16 +02:00
Fabio Manganiello
373788377b
Created two separate actions under variable to delete/unset. 
`delete` will actually remove the record from the database (same as
`unset`'s new behaviour), while `unset` will set it to null without
deleting it (same as the `unset`'s previous behaviour).
 2023-05-05 02:21:18 +02:00
Fabio Manganiello
2c254e8eb9
numpy and PIL should be required dependencies for all camera plugins. 2023-05-04 23:44:42 +02:00
Fabio Manganiello
3febfabdd7
Bluetooth LE blacklisted device notices moved info -> debug. 2023-05-01 22:10:06 +02:00