1656 commits

Author	SHA1	Message	Date
Fabio Manganiello	27da2becd7	[media] Added media.get_info action. ... It combines the `-j` and `-g` options of ytdl* to get both the stream URL and the track metadata.	2023-11-06 02:25:35 +01:00
Fabio Manganiello	6411688e65	[youtube] Don't fail if some items in the response have missing attrs.	2023-11-05 22:12:21 +01:00
Fabio Manganiello	71bca662a1	media.play should support anything supported by a yt_dlp extractor.	2023-11-05 02:40:38 +01:00
Fabio Manganiello	11c3b7820d	[torrent] Create download_dir if it doesn't exist.	2023-11-04 15:48:05 +01:00
Fabio Manganiello	2b12984c81	[youtube] Full plugin rewrite. ... Instead of relying on the official Google YouTube API (limited, subject to breaking changes with short/no notice depending on Google's strategy against scrapers, and with an initial setup that has a high cost), we'll just stick to Piped from now on. It's free, it doesn't require API keys, it's unlikely to change, it's not subject to Google's hostile practices against developers, and anybody can run an instance.	2023-11-04 12:11:44 +01:00
Fabio Manganiello	44d4ae2a96	[media] Made the youtube-dl executable configurable. ... `youtube-dl` is mostly dead and there are several forks available, thus we need to give the user the ability to pick which `youtube-dl` executable fork they want to use. Among these, `yt-dlp` is probably the most maintained today and it's also included in many default repos, so it's been added as an extra requirement for all the media plugins. Closes: #268	2023-11-04 11:57:09 +01:00
Fabio Manganiello	b4bf30945a	[media] Black/LINT for MediaPlugin.	2023-11-04 00:50:08 +01:00
Fabio Manganiello	efdb63443d	Renamed the youtube plugin. ... From `google.youtube` to `youtube`.	2023-11-04 00:07:29 +01:00
Fabio Manganiello	4c5366849d	[torrent] Updated plugin. ... - The default PopcornTime API host has changed, as popcorn-time.ga is no longer available. - The iMDb API now requires a paid tier even for a basic query. The official iMDb API layer (and the API key requirement) has thus been replaced with a dear ol' scraping of the frontend endpoint. - Pass of Black/LINT.	2023-11-03 20:54:46 +00:00
Fabio Manganiello	ba5d505c9b	[youtube] Black/LINT pass.	2023-11-03 20:54:46 +00:00
Fabio Manganiello	d9c4634ce8	[tts.mimic3] Plugin rewrite. ... The plugin now leverages the `sound` plugin for playback, like all other `tts` plugins now do, instead of an external `media` plugin. This also removes the need for the `/tts/mimic3/say` endpoint.	2023-10-31 00:44:05 +00:00
Fabio Manganiello	f960ec4bf4	[sound] Plugin refactor. ... - Added `input_format`/`output_format` options to both input and output audio streams. - Replaced the previous (confusing) occurrences of `ffmpeg_format` and `format`. - Added custom `dtype` option for `sound.play`. - Added `join` flag (default: false) to `sound.play` to wait for the playback to finish.	2023-10-31 00:44:05 +00:00
Fabio Manganiello	ef3b1d20f2	Updated MediaPlugin documentation snippet.	2023-10-31 00:44:05 +00:00
Fabio Manganiello	0cbd0a94d6	tts plugins should leverage sound`, not an external media player.	2023-10-31 00:44:05 +00:00
Fabio Manganiello	72b2625425	Updated tts plugin dependencies, as they now leverage the sound plugin.	2023-10-31 00:44:05 +00:00
Fabio Manganiello	42912581f9	[Fix] Always initialize AssistantPlugin._conversation_start_sound.	2023-10-23 21:30:49 +00:00
Fabio Manganiello	23e53f13f0	Removed/commented references to numpy.typing. ... That module isn't available yet on the numpy version installed on Debian `oldstable`.	2023-10-23 20:23:57 +00:00
Fabio Manganiello	fc869bf5db	[Fix] sounddevice arguments fix. ... Different versions of the `sounddevice` dependency may or may not return the `index` parameter when querying the available sound devices. Thus, the code should be ready for both cases.	2023-10-23 20:11:14 +00:00
Fabio Manganiello	cea3055369	Indent pre- and post-install shell scripts in docs.	2023-10-23 15:48:09 +00:00
Fabio Manganiello	a005958317	[Assistant] Added assistant entity type.	2023-10-23 16:02:05 +02:00
Fabio Manganiello	4c0e3a16b6	Removed Alexa/AVS integration. ... The integration was based on my old fork of the AVS service, which is no longer functional given the changes the the Amazon's backend side. A new `avs-device-sdk` is now available, but it seems that it requires lengthy compilation processes which are RaspberryPi-specific. Further investigation is needed for a new Alexa plugin - see #334.	2023-10-23 00:23:10 +02:00
Fabio Manganiello	fb017a8b35	[assistant.google] Added doc on how to configure audio devices.	2023-10-22 22:49:41 +02:00
Fabio Manganiello	645e8c8f77	Added updated configuration snippet for assistant.google plugin.	2023-10-22 21:53:15 +02:00
Fabio Manganiello	b46c00f015	Removed deprecated google.assistant.pushtotalk plugin. ... It only existed as a back-compatibility layer with armv6, since there was no build of the assistant library that worked on Raspberry Pi Zero. But that API layer has been discontinued by Google and it's no longer functional, so only the `assistant.google` integration (on x86_64 and armv7) is currently supported.	2023-10-22 19:57:55 +02:00
Fabio Manganiello	cce6c4c5ad	[#284] Merged assistant.google plugin and backend. ... This removes the deprecated `assistant.google` backend and also adds a new `conversation_start_sound` parameter.	2023-10-22 19:55:11 +02:00
Fabio Manganiello	e24438980a	[sound] Better logic for default sound device auto-detection.	2023-10-22 18:07:25 +02:00
Fabio Manganiello	e65eef34d3	[application] application.install should redirect stderr->stdout.	2023-10-22 01:44:39 +02:00
Fabio Manganiello	cd87cc3e43	Minor docstring fixes.	2023-10-21 18:08:29 +02:00
Fabio Manganiello	8634572c34	Small docstring fix.	2023-10-20 01:41:30 +02:00
Fabio Manganiello	0eae6fd48d	Misc dependencies and mock fixes. ... - `iputils` should be an explicit system dependency for `ping`. Some minimal systems (like some Docker images) may not have the `ping` command installed out of the box. - `hid` and `marshmallow_dataclass` should be among the auto-mocked modules.	2023-10-20 01:02:35 +02:00
Fabio Manganiello	4c6d966038	[Doc fix] Fixed indent in the documentation of the application.install action.	2023-10-19 12:13:08 +02:00
Fabio Manganiello	f0255549c8	[utils] Added utils.to_yaml action.	2023-10-18 22:50:52 +02:00
Fabio Manganiello	c7acc03c8f	[inspect] Added get_enabled_plugins and get_enabled_backends actions.	2023-10-18 22:10:32 +02:00
Fabio Manganiello	c05d887551	[application] Added application.install action. ... It can be used to programmatically install extensions.	2023-10-18 19:33:00 +02:00
Fabio Manganiello	81e99a0e22	[shell] Better buffering for the output sent to websockets.	2023-10-18 03:00:52 +02:00
Fabio Manganiello	66cba233e5	[config] Added config.get_config_file action.	2023-10-18 03:00:51 +02:00
Fabio Manganiello	20a2203e7e	[Shell plugin] Added support for async output over websockets.	2023-10-18 03:00:51 +02:00
Fabio Manganiello	891e05a219	[Execute UI] Added syntax highlight to JSON and shell snippets.	2023-10-13 23:12:04 +02:00
Fabio Manganiello	e382ad7650	Invert @ensure_initialized and @action annotations. ... Otherwise, we won't be able to access the wrapped function from the inspection logic.	2023-10-12 00:57:31 +00:00
Fabio Manganiello	1b6e9b4df9	Fixed a broken docstring reference.	2023-10-10 22:18:36 +02:00
Fabio Manganiello	52e353dc14	Expose the wrapped function in @action. ... Added a `wrapped` "hidden" parameter to the function returned by the `@action` decorator. We need this to access the underlying decorated function when e.g. we need to access its specs or decorators.	2023-10-09 22:35:08 +02:00
Fabio Manganiello	1e93af86f4	Fixed some broken docstring references.	2023-10-09 01:33:45 +02:00
Fabio Manganiello	53bdcb9604	A major rewrite of the inspect plugin. ... - The `inspect` plugin and the Sphinx inspection extensions now use the same underlying logic. - Moved all the common inspection logic under `platypush.common.reflection`. - Faster scanning of the available integrations and components through a pool of threads. - Added `doc_url` parameters. - Migrated events and responses metadata scanning logic. - Now expanding some custom Sphinx tag instead of returning errors when running outside of the Sphinx context - it includes `:class:`, `:meth:` and `.. schema::`.	2023-10-09 01:33:45 +02:00
Fabio Manganiello	4d52fd35b9	Skip None responses in the RSS plugin.	2023-10-04 22:08:11 +02:00
Fabio Manganiello	40d3ad1150	Removed <type> | None type hints. ... They break on Python < 3.10.	2023-10-03 01:15:13 +02:00
Fabio Manganiello	3086dd86fc	LINT+Black+stability fixes for some plugins that hadn't been touched in a while. ... - media.mplayer - media.omxplayer - media.vlc - music.mpd - music.snapcast	2023-10-01 22:55:06 +02:00
Fabio Manganiello	2aefc4e5c8	Several improvements for the Google integrations. ... 1. Improved documentation. Every plugin now reports the exact steps to get the integration up and running with the right API scopes. 2. All Google plugins now have a standard process to get (and reuse) the client secret. Except for PubSub, Translate and Maps (which have their own flows), all the Google plugins now read the client secrets from `<WORKDIR>/credentials/google/client_secret.json` by default. 3. Black/LINT for some of those plugins, which hadn't been touched in a while. 4. The interface to pass API scopes is now leaner. It's now possible to pass a scope directly as e.g. `calendar.readonly` rather than `https://www.googleapis.com/auth/calendar.readonly`. 5. Improved the logic to retrieve the right scope tokens file. If e.g. an integration requires the role `A`, and a credentials file exists for the roles `A` and `B`, then this file will be used rather than prompting the user to authenticate again.	2023-10-01 15:37:20 +02:00
Fabio Manganiello	5ca3757834	A more readable configuration for the calendar plugin. ... The old type configuration (`platypush.plugins.calendar.name.CalendarNamePlugin`) is a bit clunky. Instead, since the type will always be a plugin, we should encourage the use of `calendar.name` directly to identify the type.	2023-10-01 01:09:15 +02:00
Fabio Manganiello	966a6ce29e	httplib2 should be an explicit dependency for Google integrations. ... Plus, some misc LINT/Black chores.	2023-10-01 00:52:59 +02:00
Fabio Manganiello	532f5479b3	Added full YAML example for chat.irc configuration.	2023-09-30 14:34:54 +02:00
Fabio Manganiello	4f5ccda353	Better documentation for the calendar plugin.	2023-09-30 13:31:41 +02:00
Fabio Manganiello	15d06fa5c2	Improved docstring parser logic.	2023-09-30 12:35:31 +02:00
Fabio Manganiello	905d6632e0	Misc documentation improvements.	2023-09-29 18:09:15 +02:00
Fabio Manganiello	0a3ec4b9f1	Merge branch 'master' into 311/auto-generate-deps-docs	2023-09-28 01:25:29 +02:00
Fabio Manganiello	b76f141b61	Catch response write errors in the MQTT callback. ... If the client that forwarded the request is no longer available (either because an exception or a timeout was raised) then its I/O buffer and event loop may be closed. In this case, the response callback should handle and report the exception, and still set the event, so that any other threads waiting for the response can move on.	2023-09-27 11:23:55 +02:00
Fabio Manganiello	ca7f042ccc	We shouldn't call dateutil.parser if t has already been deserialized to a datetime.	2023-09-27 11:20:10 +02:00
Fabio Manganiello	c311987741	Removed typing.Final from some of the most commonly used modules. ... `typing.Final` is not defined on Python < 3.8.	2023-09-26 23:50:10 +02:00
Fabio Manganiello	9298f52443	Moved BluetoothPlugin to __init__.py. ... This is for consistency with other plugins, that all have their main plugin class definition inside of `__init__.py`.	2023-09-24 19:21:53 +02:00
Fabio Manganiello	c3337ccc6c	[#311] Docs deps autogen sphinx plugin. ... Added an `add_dependencies` plugin to the Sphinx build process that parses the manifest files of the scanned backends and plugins and automatically generates the documentation for the required dependencies and triggered events. This means that those dependencies are no longer required to be listed in the docstring of the class itself. Also in this commit: - Black/LINT for some integrations that hadn't been touched in a long time. - Deleted some leftovers from previous refactors (deprecated `backend.mqtt`, `backend.zwave.mqtt`, `backend.http.request.rss`). - Deleted deprecated `inotify` backend - replaced by `file.monitor` (see #289).	2023-09-24 17:00:08 +02:00
Fabio Manganiello	059f7a4b11	Merge branch 'master' into 316/rpm-support	2023-09-23 16:21:48 +02:00
Fabio Manganiello	b797add90d	Modify shebang header for Mercury JS script. ... The RPM build process wants an absolute path.	2023-09-23 16:16:13 +02:00
Fabio Manganiello	79e24461cb	Coalesce to empty list if zigbee.mqtt.devices returns null.	2023-09-22 17:00:53 +02:00
Fabio Manganiello	55965e962c	Store the Philips Hue bridge configuration under our workdir. ... By default, the `phue` library will store the file containing the token and the bridge configuration under `~/.python_hue`. That's outside of our application folder, and it can't easily be copied around or added to Docker volumes. We should instead have it under `<WORKDIR>/light.hue/config.json`, in line with what the other plugins do, and if `~/.python_hue` is available but `<WORKDIR>/light.hue/config.json` isn't then we should copy the legacy file to the new one.	2023-09-22 16:58:44 +02:00
Fabio Manganiello	761f2768cb	[#316] Added RPM dependencies.	2023-09-21 23:20:06 +02:00
Fabio Manganiello	a6efaad26d	[#306] Removed Travis CI integration. ... I've tried my best to keep it around, but the endpoints seem to be broken, they no longer have a link to their API v3 documentation, and the API Explorer that was supposed to be in the dashboard is gone.	2023-09-20 23:31:58 +02:00
Fabio Manganiello	3e8d6bb01e	Removed typing-extensions dependency. ... The package isn't present on older versions of Debian.	2023-09-18 14:05:39 +02:00
Fabio Manganiello	07c88c9530	Added logic to prevent socket leakage from paho-mqtt upon client stop.	2023-09-17 23:43:56 +02:00
Fabio Manganiello	3f4168eb69	Support for new Quality enumeration constants on tidalapi.	2023-09-17 22:40:26 +02:00
Fabio Manganiello	c6cda86b1c	LINT/regex fix for ESP plugin.	2023-09-17 17:10:40 +02:00
Fabio Manganiello	2e004c1a1e	get_all_plugins should use Message.Encoder for encoding the response. ... Some plugin information may include enums, tuples and other types that aren't serializable by the default JSON serializer.	2023-09-17 02:11:52 +02:00
Fabio Manganiello	2fcd623c51	Migrated zwave.mqtt integration. ... Merged the zwave.mqtt plugin with the listener and removed the leftovers of the legacy zwave plugin.	2023-09-16 03:58:19 +02:00
Fabio Manganiello	2c93049ee5	Catch all the exceptions in a plugin action wrapper. ... The @action decorator should capture all the exceptions, log them and return them on `Response.errors`. This ensures that uncaught exceptions from plugin actions won't unwind out of control, and also that they are logged and treated consistently across all the integrations.	2023-09-14 23:08:23 +02:00
Fabio Manganiello	ac72b2f7a8	Fixed management of state on zigbee.mqtt. ... Before the merge of the plugin and the listener those components used to have their own separate state, which led to inconsistencies.	2023-09-14 23:05:27 +02:00
Fabio Manganiello	5a514fdcce	Only support the run_topic logic on the MQTT plugin. ... Plugins that extend `MqttPlugin` shouldn't run messages as requests, even if the parent MQTT plugin is configured to do so.	2023-09-14 01:09:03 +02:00
Fabio Manganiello	4cb5aa7acb	Prepend the class name to the string used to generate the MQTT client_id hash. ... If we include the class name by default then we won't have to explicitly modify the client_id in the implementation classes in order to prevent clashes.	2023-09-14 01:06:53 +02:00
Fabio Manganiello	ddd8f1afdc	base_topic param in zigbee.mqtt renamed to topic_prefix. ... This is for sake of consistency with other integrations (like `zwave.mqtt`) that also use the same parameter name for the MQTT topic prefix.	2023-09-07 21:32:56 +02:00
Fabio Manganiello	3de510da68	Migrated zigbee.mqtt integration. ... The plugin has been migrated to the new `mqtt` API and the legacy listener that extended `MqttBackend` has been removed and merged into the plugin.	2023-09-06 02:44:56 +02:00
Fabio Manganiello	33a1ef39e4	Refactored and merged backend.mqtt logic into mqtt plugin.	2023-09-06 02:43:45 +02:00
Fabio Manganiello	b6c0ff799b	Rewritten the http.webpage plugin.	2023-09-03 17:33:25 +02:00
Fabio Manganiello	669f2eb2d2	LINT/black for tts.mimic3 plugin.	2023-09-02 12:40:34 +02:00
Fabio Manganiello	35416f3ee3	Some LINT on the old http.request.rss plugin.	2023-09-02 01:01:16 +02:00
Fabio Manganiello	2c46b6fe14	Added git a required manifest dependency when needed. ... It is needed for packages that install pip packages via git.	2023-08-20 21:19:15 +02:00
Fabio Manganiello	a99ffea37c	Fixed apt dependencies for mpd plugin.	2023-08-19 21:46:08 +02:00
Fabio Manganiello	69706eaabe	s/logger/_logger/ in the plugins module. ... The `logger` name may clash with the context of an action, where `logger` may have been set to something else.	2023-08-19 13:32:19 +02:00
Fabio Manganiello	1cb686bdab	Updated the inspect plugin to the new manifest utils interface.	2023-08-19 13:31:48 +02:00
Fabio Manganiello	1825b492b3	Replaced Config.workdir with Config.get_workdir(). ... Again, Python < 3.9 doesn't like class properties.	2023-08-19 13:21:24 +02:00
Fabio Manganiello	c2b3ec8ce3	Fixed manifest files with outdated formats.	2023-08-19 12:54:33 +02:00
Fabio Manganiello	2cab836bdf	Entity.columns class property replaced by Entity.get_columns method. ... Again, Python < 3.9 doesn't like the combination of `@property` + `@classmethod`.	2023-08-18 17:20:53 +02:00
Fabio Manganiello	a9cdff900e	_variable should be an external global function rather than a class property. ... The combination of `@property` + `@classmethod` isn't supported on Python < 3.9.	2023-08-18 16:16:47 +02:00
Fabio Manganiello	ca95490412	Added timeout parameter to requests.get in the rss plugin.	2023-08-18 15:53:30 +02:00
Fabio Manganiello	5dd7345c0b	Sync the latest parse timestamps in main instead of __init__ in rss. ... We should load the latest timestamps from the db when the thread starts instead of doing it in the constructor. The constructor may be invoked when the entities engine hasn't been initialized yet, and result in deadlocks.	2023-08-18 15:51:11 +02:00
Fabio Manganiello	bf7d060b81	Added @ensure_initialized decorator to actions in variable. ... The `variable` plugin may break in the constructor the first time the application is started. That's because it tries to initialize the cache of stored variables, but the local database hasn't yet been initialized. That's because plugins are registered _before_ the entities engine is initialized, as the entities engine assumes that it already has plugins to scan for entities. Therefore, the initialization of the `variable` plugin's cache should be lazy (only done upon the first call to `get`/`set` etc.), in order to prevent deadlock situations where the plugin waits for the engine to start, but the engine will be initialized only after the plugin is ready. And the lazy initialization logic should also ensure that the entities engine has been properly started (and emit a `TimeoutError` if that's not the case), in order to prevent race conditions.	2023-08-17 02:47:30 +02:00
Fabio Manganiello	98e9abde18	Extended manifest files with Python system packages (if available). ... - If a Python optional dependency is available as a system package on the target system, try and install it that route rather than pip. It's usually faster and it decreases the risk of breaking system packages. - Added support for apk dependencies in manifest files. This brings the number of distros officially supported by all the extensions to four: - Alpine - Arch - Debian - Ubuntu	2023-08-16 22:43:51 +02:00
Fabio Manganiello	00863a176e	Added application plugin.	2023-08-15 11:20:24 +02:00
Fabio Manganiello	f51beb271e	Large refactor + stability fixes for the external process control logic.	2023-08-15 11:12:21 +02:00
Fabio Manganiello	4062ddbcf0	More improvements to the inspect plugin. ... - Support for distinct `type` field on constructor and method arguments. - Added `has_varargs` field. - Added `required` field. - Better logic for parsing arguments `default` values.	2023-08-08 20:47:27 +02:00
Fabio Manganiello	9beb0a7af3	Skip :type: annotations from docstring args documentation.	2023-08-08 20:47:27 +02:00
Fabio Manganiello	2a30f060b4	Parse the arguments list from obj_type.__init__.__doc__ too.	2023-08-08 20:47:27 +02:00
Fabio Manganiello	e9a568fdd2	Unified interface to retrieve the Redis bus configuration. ... A common `utils.get_redis_conf` has been created to handle the cascade fallback logic used to retrive the default Redis configuration.	2023-07-24 01:04:13 +02:00
Fabio Manganiello	0dc380fa94	Removed dependency from prctl. ... Also, black'd and LINT-fixed some files that hadn't been touched in a while.	2023-07-23 19:17:30 +02:00
Fabio Manganiello	66981bd00b	Updated email addresses and black'd some old source files.	2023-07-22 23:02:44 +02:00
Fabio Manganiello	3eda0c6f17	[#268] Implemented XMPP integration.	2023-07-22 22:36:36 +02:00
Fabio Manganiello	cb04af0bbd	Catch TypeError when execution an action. ... Most of TypeError are due to the user passing wrong data. It usually doesn't mean that we have to fail hard and reload the plugin, nor retry the call with the same parameters.	2023-07-14 22:20:27 +02:00
Fabio Manganiello	27cf1bec52	Removed some optional top-level imports. ... Optional top-level imports in Tornado route declarations will trigger `ImportError`. While this will just mean that those routes will be skipped, it will also generate a lot of noise on the logs.	2023-07-01 03:13:38 +02:00
Fabio Manganiello	5e5403287e	The inspect plugin should not fail hard if the cache couldn't be loaded. ... This can happen for many reasons - not only if the cache file is not accessible, but also if the structure/signature of some pickled objects has changed. In that case, we should invalidate the current cache and re-initialize it instead of failing.	2023-06-28 01:33:12 +02:00
Fabio Manganiello	ba827b0248	Added sound plugin documentation. ... - Added example of how to stream audio over HTTP. - Added Portaudio to the apt/pacman dependencies.	2023-06-28 00:10:36 +02:00
Fabio Manganiello	a103ea49f1	Fixed ffmpeg/audio consumer synchronization upon timeout.	2023-06-27 15:12:15 +02:00
Fabio Manganiello	77f7cd8b90	Don't use typing.Self (yet). ... typing.Self has only been introduced in Python 3.10.	2023-06-27 14:17:04 +02:00
Fabio Manganiello	f2540437b7	Sound plugin rewritten almost from scratch.	2023-06-27 13:31:38 +02:00
Fabio Manganiello	2f4229d7b1	pylint fixes for the camera plugin.	2023-06-16 15:40:05 +02:00
Fabio Manganiello	9aa8e4538a	Better termination logic for the ffmpeg audio converter.	2023-06-16 11:47:37 +02:00
Fabio Manganiello	a6351dddd4	Extracted AudioRecorder out of SoundPlugin.	2023-06-16 03:12:55 +02:00
Fabio Manganiello	da93f1b3b0	[Chore] pylint	2023-06-14 01:44:36 +02:00
Fabio Manganiello	c8786b75de	sound.recordplay merged into sound.record.	2023-06-12 22:15:02 +02:00
Fabio Manganiello	be794316a8	Merged sound.stream_recording and sound.record.	2023-06-12 13:06:02 +02:00
Fabio Manganiello	a415c5b231	Merged outfile/fifo logic in sound.stream_recording.	2023-06-12 12:33:14 +02:00
Fabio Manganiello	e238fcb6e4	Refactoring the sound plugin to use ffmpeg as a stream converter.	2023-06-11 12:48:49 +02:00
Fabio Manganiello	4587b262b0	Stream camera frames over HTTP using a Redis pub/sub mechanism.	2023-06-05 20:40:12 +02:00
Fabio Manganiello	d7208c6bbc	Refactored Tornado routes for native pub/sub support. ... The Redis pub/sub mechanism is now a native feature for Tornado routes through the `PubSubMixin`. (Plus, lint/black chore for the sound plugin)	2023-05-30 21:58:27 +02:00
Fabio Manganiello	8b5eb82497	Camera stream writer fixes. ... - The readiness condition should be `multiprocessing.Condition`, not `threading.Condition` - in most of the cases it will be checked in a multiprocess environment. - Fixed parameter name for `write`.	2023-05-30 11:06:48 +02:00
Fabio Manganiello	4fffabd82a	Revert "Removed camera.gstreamer." ... This reverts commit b4d714df8a.	2023-05-29 22:13:24 +02:00
Fabio Manganiello	4bf9c01ac9	Moved camera routes. ... Camera routes migrated from Flask blueprints to Tornado handlers.	2023-05-27 22:24:45 +02:00
Fabio Manganiello	b4d714df8a	Removed camera.gstreamer. ... Too much of a pain in the ass to handle, too many format options to think of, too many combinations of pipelines to support, and if I don't think of those beforehand then I'll have to offload all of that complexity on the user.	2023-05-27 22:19:50 +02:00
Fabio Manganiello	1c40e5e843	Black'd the camera plugin and writer. ... Also, proper fix for the multi-inheritance problem of the ffmpeg writers.	2023-05-23 20:42:59 +02:00
Fabio Manganiello	a7aabd7c52	Fixed handling of :meth: docstring annotations with relative paths.	2023-05-22 16:32:30 +02:00
Fabio Manganiello	7eca1c27c9	Blackened the qrcode and pushbullet plugins	2023-05-22 02:33:54 +02:00
Fabio Manganiello	d7405ad05d	Added multiple parsers for the entities referenced in docstrings. ... The `inspect` plugin can now detect references to plugins, backends, events, responses and schemas in docstrings and replace them either with links to the documentation or auto-generated examples.	2023-05-22 02:20:58 +02:00
Fabio Manganiello	b91c1eba6d	Parse :return: definitions from action docstrings too.	2023-05-21 03:05:19 +02:00
Fabio Manganiello	27d4a20418	Use reflection to infer the arguments of a Python user procedure	2023-05-17 17:17:59 +02:00
Fabio Manganiello	61ea3d79e4	Large refactor for the inspect plugin. ... More common logic has been extracted and all the methods and classes have been documented and black'd.	2023-05-17 00:05:22 +02:00
Fabio Manganiello	8447f9a854	Improved rendering of actions/arguments documentation. ... The frontend now calls `utils.rst_to_html` to render the docstrings as HTML instead of dumping them as raw text. Also, actions and arguments are now cached to improve performance.	2023-05-14 15:06:34 +02:00
Fabio Manganiello	5f2d6dfeb5	Added utils.rst_to_html action.	2023-05-14 15:05:24 +02:00
Fabio Manganiello	3c83e7f412	A faster implementation for the inspect.get_* methods. ... Plugin/backend lookup is now done by inspecting the manifest files instead of searching all the subpackages.	2023-05-13 13:44:46 +02:00
Fabio Manganiello	cfedcd701e	Performance improvements when loading the Tensorflow plugin. ... The Tensorflow module may take a few seconds to load the first time and slow down the first scan of the plugins. All the Tensorflow imports should therefore be placed close to where they are used instead of being defined at the top of the module.	2023-05-11 19:48:22 +02:00
Fabio Manganiello	41233138ff	Blackened inspect module and extracted model defs to adjacent module.	2023-05-09 21:58:02 +02:00
Fabio Manganiello	56dc8d0972	Migrated the webapp to Tornado. ... It was just too painful to find a combination of versions of gunicorn, gevent, eventlet, pyuwsgi etc. that could work on all of my systems. On the other hand, Tornado works out of the box with no headaches. Also in this commit: - Updated a bunch of outdated/required integration dependencies. - Black'd and LINTed a couple of old plugins.	2023-05-08 02:06:45 +02:00
Fabio Manganiello	d1f0e1976c	Exclude squashfs/loopback mounts from system.disk_info.	2023-05-06 18:53:16 +02:00
Fabio Manganiello	373788377b	Created two separate actions under variable to delete/unset. ... `delete` will actually remove the record from the database (same as `unset`'s new behaviour), while `unset` will set it to null without deleting it (same as the `unset`'s previous behaviour).	2023-05-05 02:21:18 +02:00
Fabio Manganiello	2c254e8eb9	numpy and PIL should be required dependencies for all camera plugins.	2023-05-04 23:44:42 +02:00
Fabio Manganiello	3febfabdd7	Bluetooth LE blacklisted device notices moved info -> debug.	2023-05-01 22:10:06 +02:00
Fabio Manganiello	9d82ce6ea9	Noisy beacons notice back to debug level. ... There's just too many of them and it ends up polluting the logs.	2023-05-01 21:25:42 +02:00
Fabio Manganiello	5d4bffa119	Fixed retrieval of entities plugin.	2023-04-30 10:42:05 +02:00
Fabio Manganiello	3d7755159f	Improved compatibility for traceback.format_exception. ... The new syntax, that only requires an `Exception` instance to be passed to the function, is only compatible with Python >= 3.10.	2023-04-30 00:38:17 +02:00
Fabio Manganiello	6b28d16ccf	Exclude more noisy Bluetooth beacons. ... Exclude any beacons from devices with no name, no children other than services, and with none of those services being public/known.	2023-04-29 23:34:24 +02:00
Fabio Manganiello	f764d1b4fb	Noisy Bluetooth beacons notices should be logged on info level.	2023-04-29 23:18:12 +02:00
Fabio Manganiello	e96885a805	Delete the entity on variable.unset instead of setting it to null.	2023-04-29 18:21:57 +02:00
Fabio Manganiello	a3888be216	The robustness check in case of missing fields should also apply to other system entities.	2023-04-29 16:08:38 +02:00
Fabio Manganiello	8c9768b05e	Robustness check for system disk entities. ... When the system information is still loading it may happen that the device associated to the disk hasn't been loaded yet.	2023-04-29 16:04:57 +02:00
Fabio Manganiello	a20065c649	Exposed _entities utility property in Plugin. ... It can be used by other plugins to easily access the `entities` plugin, along the lines of `db` and `redis`.	2023-04-29 15:50:31 +02:00
Fabio Manganiello	68d8befa34	Removed some vestigial commented code.	2023-04-29 15:28:44 +02:00
Fabio Manganiello	23b851e9d7	variable.status robustness fix. ... `entities.transform_entities` will pass back an empty list instead of an empty dict if no entities were found, and the function should be able to handle it.	2023-04-29 15:24:58 +02:00
Fabio Manganiello	e919bf95ad	Print the full stack trace if a plugin failed in entities.scan	2023-04-29 15:14:13 +02:00
Fabio Manganiello	f40f956507	Migrated variable table to the new entities framework.	2023-04-29 11:36:55 +02:00
Fabio Manganiello	8fe61217ce	Added _db and _redis properties to the Plugin class. ... Plugins can now access the database and Redis APIs directly without having to run their own `get_plugin` validation logic.	2023-04-29 11:35:57 +02:00
Fabio Manganiello	87db5ca5f3	Exclude all iBeacon devices by default (it's not only Apple, it's everyone)	2023-04-26 14:17:59 +02:00
Fabio Manganiello	10d587efd0	FIX: Possible assert evaluation error. ... Some versions/configurations of Python may throw `Boolean value of this clause is not defined` here.	2023-04-26 02:25:28 +02:00
Fabio Manganiello	cb288deb71	Exclude more noisy BLE beacons. ... Excluding Apple iBeacons and devices with no name and no services.	2023-04-25 16:19:11 +02:00
Fabio Manganiello	dd60b8924d	Wrap the PRAGMA statement in sqlalchemy.text. ... SQLAlchemy 2 no longer supports raw strings passed to `.execute()` methods.	2023-04-25 10:41:37 +02:00
Fabio Manganiello	4cc88fcf5f	Rewritten the variable plugin to use SQLAlchemy's ORM. ... This removes the need for raw SQL statements and CREATE TABLE statements that may be engine-specific.	2023-04-25 10:35:12 +02:00
Fabio Manganiello	e1cd22121a	Removed connection.begin() pattern from the db plugin. ... SQLAlchemy should automatically begin a transaction on connection/session creation. Plus, `.begin()` messes up things with SQLAlchemy 2, which has `autobegin` enabled with no easy way of disabling it.	2023-04-25 10:31:49 +02:00
Fabio Manganiello	37722d12cd	No need for session.begin in db.create_all.	2023-04-24 23:55:50 +02:00
Fabio Manganiello	6fa179e769	LINT fixes	2023-04-24 23:49:31 +02:00
Fabio Manganiello	91df18f7b5	Better way to import declarative_base from SQLAlchemy. ... Import `declarative_base` in a way that is compatible with any SQLAlchemy version between 1.3 and 2.x.	2023-04-24 23:21:39 +02:00
Fabio Manganiello	87889142e0	Fixed compatibility with SQLAlchemy >= 2.0 in the db plugin.	2023-04-24 22:52:17 +02:00
Fabio Manganiello	dc3392c11d	Disk I/O stats are not always available and should therefore be optional.	2023-04-23 22:25:24 +02:00
Fabio Manganiello	0cd28f1040	libbluetooth-dev is a required dependency to build pybluez on Debian-derived distros	2023-04-23 18:59:37 +02:00
Fabio Manganiello	9c1855e4c0	Fixed docstring for zigbee.mqtt plugin.	2023-04-23 13:03:10 +02:00
Fabio Manganiello	27b1048789	Converted system.processes to the new data model.	2023-04-23 02:08:43 +02:00
Fabio Manganiello	387616ea96	Convert system.connected_users to the new data model.	2023-04-23 01:12:07 +02:00
Fabio Manganiello	763d9e06ec	Increased default poll_interval for system plugin to 60 seconds.	2023-04-23 00:42:44 +02:00
Fabio Manganiello	a72c32cb00	Added battery entity support to system plugin.	2023-04-23 00:41:21 +02:00
Fabio Manganiello	b3440ab96b	Added support for fan sensors on the system plugin.	2023-04-23 00:08:27 +02:00
Fabio Manganiello	45d5f439be	Added support for system temperature sensor entities.	2023-04-22 22:42:11 +02:00
Fabio Manganiello	1b048e1952	s/net_connections/network_connections/g	2023-04-22 17:19:24 +02:00
Fabio Manganiello	374f936c1f	Merged network_stats into NetworkInterface model.	2023-04-22 17:19:24 +02:00
Fabio Manganiello	e213941791	s/net_io_counters/network_info/g	2023-04-22 17:19:23 +02:00
Fabio Manganiello	977b55dea9	Merged network addresses into NetworkInterface model.	2023-04-22 17:19:23 +02:00
Fabio Manganiello	ebe79ac29a	Refactored system schema dataclasses. ... - `percent_field` should be declared on `platypush.schemas.dataclasses` level, since it's not specific to the `system` plugin. - Added a common `SystemBaseSchema` that takes care of calling `_asdict()` if the object is passed as a `psutil` object instead of a dict.	2023-04-22 17:19:23 +02:00
Fabio Manganiello	2d618188c8	Print the full exception stack trace if .status fails.	2023-04-22 17:19:23 +02:00
Fabio Manganiello	b3a0896485	Converted NetworkConnection schema/response.	2023-04-22 17:19:22 +02:00
Fabio Manganiello	98a300c4b1	Added NetworkInterface entities to system plugin. ... Plus, `platypush.schemas.system` has now been split into multiple submodules to avoid a single-file mega-module with all the system schemas definitions.	2023-04-21 00:45:15 +02:00
Fabio Manganiello	44b8fd4b34	Support for disk entities in the system integration.	2023-04-20 16:26:51 +02:00
Fabio Manganiello	153d03d43f	Moved CPU percentage on the level of the CPU entity instead of a child entity.	2023-04-19 01:48:05 +02:00
Fabio Manganiello	4ebfbf3851	Added memory stats entities.	2023-04-19 01:31:11 +02:00
Fabio Manganiello	0073239a40	Support for CPU load_average entity.	2023-04-18 18:26:02 +02:00
Fabio Manganiello	1cee0459cf	Added CpuFrequency entity to system.	2023-04-18 01:49:36 +02:00
Fabio Manganiello	a5b0a524f6	Added CpuStats entity to system.	2023-04-18 01:19:06 +02:00
Fabio Manganiello	b4fbd3e915	Added percent entity to cpu.	2023-04-17 02:25:04 +02:00
Fabio Manganiello	b9286f50b0	Added support for CpuTimes as an entity of the system plugin. ... Also, there is now a single `Cpu` entity being exported, with a nested hierarchy structured like: ``` cpu -> cpu_info -> cpu_times -> idle -> user -> system -> ... -> cpu_load -> ... ```	2023-04-17 02:25:03 +02:00
Fabio Manganiello	b43017ef01	Refactoring the system plugin to support entities.	2023-04-17 02:25:02 +02:00
Fabio Manganiello	74aeca5c34	Trigger a sensor event only if abs(old_data - new_data) > tolerance ... Not if abs(old_data - new_data) >= tolerance, otherwise events will always be triggered when tolerance=0, even if the data hasn't changed.	2023-04-17 02:25:01 +02:00
Fabio Manganiello	a499b7bc2f	Deprecated poll_seconds in light.hue. ... For sake of naming consistency with other plugins, we should use `poll_interval` instead.	2023-04-03 01:36:12 +02:00
Fabio Manganiello	10955dad72	Fixed some documentation glitches in switchbot.	2023-04-03 01:36:12 +02:00
Fabio Manganiello	6e5f746dbe	Removed deprecated gpio.sensor base plugin. ... Now all the plugins that used to implement it have been moved to `SensorPlugin`.	2023-04-03 01:36:12 +02:00
Fabio Manganiello	8852cb8db4	Fixed new class name for sensor.mcp3008 plugin.	2023-04-03 01:36:12 +02:00
Fabio Manganiello	d5ddc0c65e	Migrated arduino integration to the new SensorPlugin API.	2023-04-03 01:36:12 +02:00
Fabio Manganiello	ac2ec58f89	Migrated mcp3008 integration to the new SensorPlugin API.	2023-04-03 01:36:11 +02:00
Fabio Manganiello	45e5ca47e7	Fallback for sensor._has_changes	2023-04-02 15:38:49 +02:00
Fabio Manganiello	962c55937d	Migrated sensor.distance integration. ... Remove `backend.sensor.distance` and `gpio.sensor.distance`. They are now replaced by the `sensor.hcsr04` integration, which is compatible with the new `SensorPlugin` API.	2023-04-02 14:20:12 +02:00
Fabio Manganiello	92578a17c9	Added small docstring portion	2023-04-02 13:55:00 +02:00
Fabio Manganiello	beff88986a	Migrated dht integration. ... Removed `backend.sensor.dht` and `gpio.sensor.dht`. They have been merged into the new `sensor.dht` integration, which supports the new `SensorPlugin` API.	2023-04-02 13:38:53 +02:00
Fabio Manganiello	8f604445a2	Migrated old sensor.accelerometer integration. ... Removed `backend.sensor.accelerometer` and `gpio.sensor.accelerometer`. The logic has now been merged in the new `sensor.lis3dh` integration, which is compatible with the new `SensorPlugin` API.	2023-04-02 13:22:28 +02:00
Fabio Manganiello	44cf25271c	Migrated pmw3901 integration. ... Removed legacy `backend.sensor.motion.pmw3901` and `gpio.sensor.motion.pmw3901`. They have been merged in the new `sensor.pmw3901` integration, compatible with the new `SensorPlugin` API.	2023-04-02 12:36:08 +02:00
Fabio Manganiello	fcdda40c4a	Update the _last_measurement only if some events were processed from the new data.	2023-04-02 12:09:45 +02:00
Fabio Manganiello	88784985e1	Should be abs(old_data - new_data) >= tolerance. ... Not `abs(old_data - new_data) > tolerance`.	2023-04-02 12:08:40 +02:00
Fabio Manganiello	7697c1c6ad	Migrated envirophat to the new SensorPlugin API. ... Removed `backend.sensor.envirophat` and `gpio.sensor.envirophat` plugin. They have now been merged into the new `sensor.envirophat` plugin.	2023-04-02 02:49:08 +02:00
Fabio Manganiello	d964167631	s/TimeDurationSensor/TimeDuration/g	2023-04-02 00:57:48 +02:00
Fabio Manganiello	f24d0773d1	No need for sensor.vl53l1x.transform_entities to call the parent.	2023-04-01 23:54:43 +02:00
Fabio Manganiello	99572f9731	Sanity check to prevent empty objects from being propagated to sensor.transform_entities	2023-04-01 23:41:28 +02:00
Fabio Manganiello	3f3726c50a	Fixed another occurrence of "Subscripted generics cannot be used" etc. error	2023-04-01 23:34:22 +02:00
Fabio Manganiello	e2e73d0fdb	Fix another Python < 3.10 subscripted generic issue.	2023-04-01 23:23:51 +02:00
Fabio Manganiello	c1d0f21ead	Migrated ltr559 integration to the new API. ... Merged `backend.sensor.ltr559` and `gpio.sensor.ltr559` into the new `sensor.ltr559` plugin, which extends the new `SensorPlugin` API.	2023-04-01 23:16:03 +02:00
Fabio Manganiello	8e0f88ea16	Don't swap the argument of SensorPlugin.publish_entities with a list if not required	2023-04-01 23:06:37 +02:00
Fabio Manganiello	0047d85b9d	Dirty fix for "Subscripted generics cannot be used with class and instance checks" on Python < 3.10	2023-04-01 22:52:24 +02:00
Fabio Manganiello	98ec018292	Replaced NoneType reference. ... `types.NoneType` is not always available on all Python versions, so we have to make our own type for it.	2023-04-01 22:42:13 +02:00
Fabio Manganiello	5dabfed365	Migrated sensor.bme280 to the new SensorPlugin interface. ... Removed the old `backend.sensor.bme280` and the old `gpio.sensor.bme280` plugin. They have now been merged into the new `sensor.bme280` runnable plugin, which extends the `SensorPlugin` API and supports entities.	2023-04-01 22:31:24 +02:00
Fabio Manganiello	6f237a1500	Support the deprecated poll_seconds option on RunnablePlugin	2023-04-01 22:02:59 +02:00
Fabio Manganiello	c23e8867e2	Added enabled_sensors to the sensor plugin	2023-04-01 21:56:56 +02:00
Fabio Manganiello	7912a59ff8	vl53l1x plugin migrated to the new SensorPlugin interface.	2023-04-01 19:31:13 +02:00
Fabio Manganiello	6a5a5de03e	serial plugin migrated to the new SensorPlugin interface.	2023-04-01 19:29:56 +02:00
Fabio Manganiello	bf4db76830	Legacy sensor backend replaced by an extended sensor runnable plugin.	2023-04-01 19:24:35 +02:00
Fabio Manganiello	7bdd877e49	Support the binary flag both on serial.read and serial.write.	2023-03-31 14:31:45 +02:00
Fabio Manganiello	1efaff878e	Rewritten serial plugin. ... `backend.serial` has been removed and the polling logic merged into the `serial` plugin. The `serial` plugin now supports the new entity engine as well.	2023-03-31 14:31:45 +02:00
Fabio Manganiello	a3e8c7c155	Rewritten vl53l1x integration as a runnable plugin with entity support	2023-03-31 14:25:05 +02:00
Fabio Manganiello	6fb362a6fb	gpio.sensor.distance.vl53l1x -> sensor.distance.vl53l1x	2023-03-31 14:21:48 +02:00
Fabio Manganiello	2781eb1fb1	Merge branch 'master' into 29-generic-entities-support	2023-03-27 00:36:50 +02:00
Fabio Manganiello	7a368ebbb8	[#240] Migrated clipboard plugin from pyperclip to pyclip. ... Closes: #240	2023-03-26 23:52:15 +02:00
Fabio Manganiello	bce2fdee25	Replaced deprecated asyncio.wait([]) with asyncio.gather(*[]).	2023-03-26 23:15:53 +02:00
Fabio Manganiello	c0251ef2f7	s/instance/instance_name/g in LinodeInstanceStatusChanged. ... For sake of consistency - we also have `instance_id` and having the instance name assigned to the `instance` attribute is quite ambiguous.	2023-03-26 22:58:20 +02:00
Fabio Manganiello	6d674fef21	Fixed small JSON syntax error in the docstring of ntfy.send_message.	2023-03-26 22:53:42 +02:00
Fabio Manganiello	295758bb20	Added frontend components for cloud instances.	2023-03-26 12:27:17 +02:00
Fabio Manganiello	bc2730c841	Rewritten linode integration. ... - Support for cloud instances as native entities. - Using Marshmallow dataclasses+schemas instead of custom `Response` objects. - Merge `linode` backend into `linode` plugin.	2023-03-26 11:23:33 +02:00
Fabio Manganiello	3c355352c5	Using the new StoppableThread API.	2023-03-24 16:39:30 +01:00
Fabio Manganiello	998793e94f	Added OBEX_FILE_TRANSFER constant to directory stub.	2023-03-24 15:41:20 +01:00
Fabio Manganiello	2f49ddf33a	Fallback logic that uses DBus to disconnect from a BT device. ... This logic will be used if the connection wasn't opened by the current process and therefore a call to DBus is required to terminate it.	2023-03-24 01:57:05 +01:00
Fabio Manganiello	913ef6f8cd	Refresh BluetoothDevice.reachable when a device is found/lost.	2023-03-24 01:56:19 +01:00
Fabio Manganiello	d46d4e2300	Added support for Bluetooth devices blacklist. ... Based on device address, name or manufacturer.	2023-03-24 01:52:39 +01:00
Fabio Manganiello	0cebcf4f9b	switchbot.bluetooth integration migrated to a bluetooth plugin.	2023-03-23 17:46:54 +01:00
Fabio Manganiello	4fac110bb8	Added bluetooth.set method, whose execution is delegated to the plugins.	2023-03-23 17:45:02 +01:00
Fabio Manganiello	cd219f44c4	Pass the list of plugins when creating Bluetooth managers.	2023-03-23 17:42:16 +01:00
Fabio Manganiello	43289a3b55	Scan always at least for 10 seconds before failing on get_device.	2023-03-23 17:41:37 +01:00
Fabio Manganiello	6267943786	Wrap BleakError exceptions into AssertionError.	2023-03-23 17:40:30 +01:00
Fabio Manganiello	d6805a8b18	Added support for custom Bluetooth device plugins.	2023-03-23 17:10:37 +01:00
Fabio Manganiello	af125347d6	If no matching services are found when connecting to a device, default to BLEManager. ... GATT characteristics are not necessarily exposed as services.	2023-03-23 13:00:26 +01:00
Fabio Manganiello	a2a5fce6cb	Added Apple Continuity to the list of blacklisted manufacturers/models	2023-03-22 22:55:19 +01:00
Fabio Manganiello	5c23d3aa87	metadata and rssi fields on BLEDevice have been deprecated. ... Changed the BLE beacon parsing logic to read those fields from `AdvertisementData` instead of `BLEDevice`.	2023-03-22 22:39:01 +01:00
Fabio Manganiello	65bc3ae06d	Noisy beacons device configuration should look both at manufacturer and model.	2023-03-22 22:37:46 +01:00
Fabio Manganiello	e10bec88c0	Noisy beacons logging trace moved from info to debug.	2023-03-22 16:31:57 +01:00
Fabio Manganiello	99cfd247a5	A more effective logic to exclude noisy BLE beacons. ... This includes BLE beacons sent from all Google/Apple/Microsoft/Samsung beacon networks in all of their variants.	2023-03-22 15:35:02 +01:00
Fabio Manganiello	01d323fad0	Passing a boolean exclude_known_noisy_beacons to bluetooth plugin. ... The logic to exclude BLE beacons from randomized devices needs to be a bit more granular and not limited only to the reported device manufacturer.	2023-03-22 15:29:19 +01:00