Commit graph

2973 commits

Author SHA1 Message Date
Fabio Manganiello 07c2eee890
Changed (fixed) default location for config dir if not existing.
Following some common UNIX conventions, if no configuration file is
specified and none exists under the default locations, then a new
configuration directory should be created under:

```
- if root: /etc/platypush
- else:
  - if XDG_CONFIG_HOME:
    - $XDG_CONFIG_HOME/platypush
  - else:
    - ~/.config/platypush
```
2023-09-04 02:19:13 +02:00
Fabio Manganiello b6c0ff799b
Rewritten the http.webpage plugin. 2023-09-03 17:33:25 +02:00
Fabio Manganiello 669f2eb2d2
LINT/black for tts.mimic3 plugin. 2023-09-02 12:40:34 +02:00
Fabio Manganiello 35416f3ee3
Some LINT on the old http.request.rss plugin. 2023-09-02 01:01:16 +02:00
Fabio Manganiello 9aaf2559fa
Added utils.is_root method. 2023-08-31 23:21:32 +02:00
Fabio Manganiello e6b5abe909
Added SIGTERM handler for clean termination in Docker contexts.
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-31 02:09:23 +02:00
Fabio Manganiello 4dd713ffd2
Refactored Platydock and Platyvenv.
The two scripts now share the same command interface, behaviour and base
class.

Also, Platydock now builds a Docker image instead of just printing a
Dockerfile, unless the `--print` option is passed.
2023-08-31 01:16:56 +02:00
Fabio Manganiello 429658e7c8
Refactored PackageManager classes.
All checks were successful
continuous-integration/drone/push Build is passing
Instead of having a custom `get_installed` callable field, with
replicated code for each package manager, the field has now been
promoted to a class method containing the common logic, and the
instances now expect a `list` field (base command to list the installed
packages using the specified package manager) and a `parse_list_line`
callback field (to extract the base package name given a raw line from
the command above).

Also, we shouldn't run the list command if we're running within a Docker
context - the host and container environments will be different.
2023-08-28 01:26:19 +02:00
Fabio Manganiello 86e5f74645
platyvenv should generate (and document) a run.sh helper script. 2023-08-27 23:20:03 +02:00
Fabio Manganiello 9e6430a9ac Clone git repo if platyvenv is not running from a srcdir 2023-08-23 20:02:04 +02:00
Fabio Manganiello dafd65dc21
Added new platyvenv Python script. 2023-08-23 11:55:25 +02:00
Fabio Manganiello f230fa79bb
to_pkg_install_commands should skip already installed sys packages. 2023-08-23 11:51:53 +02:00
Fabio Manganiello 449821673c
Added PackageManager.get_installed. 2023-08-23 10:50:29 +02:00
Fabio Manganiello b10ccdb313
Added get_src_root utility function. 2023-08-23 02:53:31 +02:00
Fabio Manganiello 1ef0d804db
Added full_command argument to to_pip_install_commands.
This is useful if we just want to get the list of pip dependencies and
create our own pip command.
2023-08-23 02:19:54 +02:00
Fabio Manganiello 2bff4c9cf1
Exclude python-* system packages when installing in a venv. 2023-08-23 02:17:19 +02:00
Fabio Manganiello cddf318fa7
Dependencies.from_config should include the base system deps. 2023-08-23 02:16:33 +02:00
Fabio Manganiello 8f39231d31
Added new utility methods to the Dependencies class. 2023-08-23 02:15:50 +02:00
Fabio Manganiello ce68250b4d
Refactor/documentation round for platydock. 2023-08-22 02:49:05 +02:00
Fabio Manganiello 10c0e5fcad
Added default_os field to PackageManagers enum elements.
This is useful to determine which is the default set of scripts that
should be used by the installer depending on the detected installed
package manager.
2023-08-20 21:21:37 +02:00
Fabio Manganiello 2c46b6fe14
Added git a required manifest dependency when needed.
It is needed for packages that install pip packages via git.
2023-08-20 21:19:15 +02:00
Fabio Manganiello a28dcb7a8d
Remove /var/cache/apk from container image after installation. 2023-08-20 14:19:04 +02:00
Fabio Manganiello a6752ed034
Fixed wrong event path in a manifest file. 2023-08-20 14:08:22 +02:00
Fabio Manganiello 700b8e1d16
Added header and footer to generated Dockerfile. 2023-08-20 14:05:22 +02:00
Fabio Manganiello a6f8021150
PackageManager has both install and uninstall.
All checks were successful
continuous-integration/drone/push Build is passing
2023-08-20 13:33:49 +02:00
Fabio Manganiello 5efcae64c1
Better Dockerfile logic to retrieve sources.
If the /install folder on the container doesn't contain a copy of the
source files, then the git repository will be cloned under that folder.

The user can specify via `-r/--ref` option which tag/branch/commit they
want to install.
2023-08-20 13:31:13 +02:00
Fabio Manganiello 28ba042810
If no configuration file is passed to platydock generate a minimal Dockerfile.
Some checks reported errors
continuous-integration/drone/push Build was killed
2023-08-20 03:36:34 +02:00
Fabio Manganiello 199ac5f0f7
New way of managing installation scripts and dependencies.
Created `platypush/install` folder that contains:

- Dockerfiles for the supported distros
- Lists of required base dependencies for the supported distros
- Install and run scripts
- Added Debian to supported base images
2023-08-20 02:28:10 +02:00
Fabio Manganiello 71c5291190
Refactored the interface of Platydock and manifest utils. 2023-08-19 22:46:37 +02:00
Fabio Manganiello a99ffea37c
Fixed apt dependencies for mpd plugin. 2023-08-19 21:46:08 +02:00
Fabio Manganiello 980af16984
Rewritten platydock utility.
Platydock now will only print out a Dockerfile given a configuration
file.

No more maintaining the state of containers, storing separate workdirs
and configuration directories etc. - that introduced way too much
overhead over Docker.
2023-08-19 13:47:43 +02:00
Fabio Manganiello 9002f3034a
Tweaked package managers install command arguments. 2023-08-19 13:46:08 +02:00
Fabio Manganiello 69706eaabe
s/logger/_logger/ in the plugins module.
The `logger` name may clash with the context of an action, where
`logger` may have been set to something else.
2023-08-19 13:32:19 +02:00
Fabio Manganiello 1cb686bdab
Updated the inspect plugin to the new manifest utils interface. 2023-08-19 13:31:48 +02:00
Fabio Manganiello 043f303761
s/HttpBackend._DEFAULT_HTTP_PORT/HttpBackend.DEFAULT_HTTP_PORT/g 2023-08-19 13:30:51 +02:00
Fabio Manganiello dd3a701a2e
Full rewrite of platypush.utils.manifest.
The new version encapsulates all the utility functions into three
classes - `Manifest`, `Manifests` and `Dependencies`.
2023-08-19 13:28:40 +02:00
Fabio Manganiello a8255f3621
Pass the configuration file used by the application to the Alembic process.
The database settings could also be overridden in the configuration file
besides the command line.

We should therefore pass the path to the runtime configuration file, so
the Alembic process can initialize its configuration from the same file
and use the same settings.
2023-08-19 13:23:20 +02:00
Fabio Manganiello 1825b492b3
Replaced Config.workdir with Config.get_workdir().
Again, Python < 3.9 doesn't like class properties.
2023-08-19 13:21:24 +02:00
Fabio Manganiello a8836f95f5
Support explicit workdir parameter override in Config constructor. 2023-08-19 13:15:29 +02:00
Fabio Manganiello 5bc82dfe64
s/Config._cfgfile/Config.config_file/g 2023-08-19 13:13:36 +02:00
Fabio Manganiello 181da63c89
Pass the database engine to the Alembic process as an extra argument.
If the path of the default database engine is overridden via `--workdir`
option then it won't be visible to the new `python` subprocess spawned
for Alembic.
2023-08-19 13:02:05 +02:00
Fabio Manganiello c2b3ec8ce3
Fixed manifest files with outdated formats. 2023-08-19 12:54:33 +02:00
Fabio Manganiello e463a52435
Use sys.executable rather than 'python' to launch the application. 2023-08-19 12:40:57 +02:00
Fabio Manganiello 0a5fc40dc5
Merge branch 'master' into 276/better-docker 2023-08-18 17:34:56 +02:00
Fabio Manganiello 2cab836bdf
Entity.columns class property replaced by Entity.get_columns method.
All checks were successful
continuous-integration/drone/push Build is passing
Again, Python < 3.9 doesn't like the combination of `@property` +
`@classmethod`.
2023-08-18 17:20:53 +02:00
Fabio Manganiello a9cdff900e
_variable should be an external global function rather than a class property.
The combination of `@property` + `@classmethod` isn't supported on
Python < 3.9.
2023-08-18 16:16:47 +02:00
Fabio Manganiello d0c89f88a8
Fixed parenthesised multi-with statement.
This syntax is not supported on Python < 3.9, and therefore it should be
broken down as a multi-line statement.
2023-08-18 16:10:31 +02:00
Fabio Manganiello ca95490412
Added timeout parameter to requests.get in the rss plugin.
All checks were successful
continuous-integration/drone/push Build is passing
2023-08-18 15:53:30 +02:00
Fabio Manganiello 5dd7345c0b
Sync the latest parse timestamps in main instead of __init__ in rss.
We should load the latest timestamps from the db when the thread starts
instead of doing it in the constructor.

The constructor may be invoked when the entities engine hasn't been
initialized yet, and result in deadlocks.
2023-08-18 15:51:11 +02:00
Fabio Manganiello ec64b0ef8b
Added --device_id command line option.
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-17 23:16:24 +02:00
Fabio Manganiello ac83b43f98
Support for custom key-value overrides on Config.init. 2023-08-17 22:03:39 +02:00
Fabio Manganiello 657b2cc87d
Create the default configuration file even if --config is supplied but the file doesn't exist.
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-17 11:25:49 +02:00
Fabio Manganiello 24b04d9103
s/--config-file/--config/ option in Dockerfile.
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-17 10:35:40 +02:00
Fabio Manganiello bf7d060b81
Added @ensure_initialized decorator to actions in variable.
All checks were successful
continuous-integration/drone/push Build is passing
The `variable` plugin may break in the constructor the first time the
application is started.

That's because it tries to initialize the cache of stored variables, but
the local database hasn't yet been initialized.

That's because plugins are registered _before_ the entities engine is
initialized, as the entities engine assumes that it already has plugins
to scan for entities.

Therefore, the initialization of the `variable` plugin's cache should be
lazy (only done upon the first call to `get`/`set` etc.), in order to
prevent deadlock situations where the plugin waits for the engine to
start, but the engine will be initialized only after the plugin is
ready.

And the lazy initialization logic should also ensure that the entities
engine has been properly started (and emit a `TimeoutError` if that's
not the case), in order to prevent race conditions.
2023-08-17 02:47:30 +02:00
Fabio Manganiello adfedfa2dd
s/TimeoutError/AssertionError/ if get_entities_engine times out. 2023-08-17 02:36:40 +02:00
Fabio Manganiello ec2b8da983
Ignore Redis errors when a backend sends an unregister notify event.
When that happens, it's most likely that the application is already
stopping and the Redis service has already been terminated.
2023-08-17 01:49:41 +02:00
Fabio Manganiello 3bf068e0b2
The __main__ function should take no arguments.
setup.py won't pass any arguments to `main()`, so the default entry
point should get them itself from `sys.argv`.
2023-08-17 01:35:39 +02:00
Fabio Manganiello 98e9abde18
Extended manifest files with Python system packages (if available).
All checks were successful
continuous-integration/drone/push Build is passing
- If a Python optional dependency is available as a system package on
  the target system, try and install it that route rather than pip. It's
  usually faster and it decreases the risk of breaking system packages.

- Added support for apk dependencies in manifest files. This brings the
  number of distros officially supported by all the extensions to four:

  - Alpine
  - Arch
  - Debian
  - Ubuntu
2023-08-16 22:43:51 +02:00
Fabio Manganiello 00863a176e
Added application plugin.
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-15 11:20:24 +02:00
Fabio Manganiello f51beb271e
Large refactor + stability fixes for the external process control logic. 2023-08-15 11:12:21 +02:00
Fabio Manganiello 46245e851f
Synchronize with the currently running stop thread (if any) in Backend.wait_stop. 2023-08-15 02:08:47 +02:00
Fabio Manganiello a8a7ceb2ac
Implemented HttpBackend._stop_workers.
The Tornado WSGI container won't guarantee the termination of the
spawned workers upon termination, so the code of the backend has to take
care of it and terminate all the children processes of the server
process when it terminates.

This also means that `psutil` is now a required base dependency, as we
need to expand the process subtree under the webserver launcher.
2023-08-15 00:13:34 +02:00
Fabio Manganiello 04921c759f
Added wait_stop method to the entities engine and cron scheduler. 2023-08-14 23:49:47 +02:00
Fabio Manganiello ceb9d6d1ba
Removed redundant Backend._get_redis() method.
It was used only once, and it could easily be replaced by
`platypush.utils.get_redis()`.
2023-08-14 23:37:38 +02:00
Fabio Manganiello 7157936b87
Added get_remaining_timeout utility function. 2023-08-14 23:17:50 +02:00
Fabio Manganiello dc1a152433
Added get_default_pid_file utility method. 2023-08-14 10:47:25 +02:00
Fabio Manganiello c11bc69a66
Handle KeyboardInterrupt and process return code in the main. 2023-08-13 23:55:40 +02:00
Fabio Manganiello 8819a0ed4c
Added CommandStream to the main Application.
The stream will be used to communicate command messages from the
application to its runner.
2023-08-13 23:51:21 +02:00
Fabio Manganiello 06111587f7
Added CommandStream class.
This abstraction is used to write and read commands over a UNIX socket.
2023-08-13 23:37:51 +02:00
Fabio Manganiello c89f992867
Added StopCommand and RestartCommand. 2023-08-13 23:36:36 +02:00
Fabio Manganiello b1f244a812
Added ControllableProcess class.
This class can be used to easily control the execution of underlying
processes.
2023-08-13 23:33:08 +02:00
Fabio Manganiello efef9d7bc0
Added commands module. 2023-08-13 23:21:36 +02:00
Fabio Manganiello 97adc3f775
Moved application argument parser to an external platypush.cli module. 2023-08-13 22:27:16 +02:00
Fabio Manganiello 1819ee75ef
s/Application.stop_app/Application.stop/g 2023-08-13 22:23:15 +02:00
Fabio Manganiello 827b564006
Using a single constant for DEFAULT_REDIS_QUEUE.
Also, catch `AttributeError` on `self._proc.terminate` in the
`HttpBackend`, since the process may already have been terminated and
set to null by another worker process.
2023-08-13 22:20:16 +02:00
Fabio Manganiello 4062ddbcf0
More improvements to the inspect plugin.
All checks were successful
continuous-integration/drone/push Build is passing
- Support for distinct `type` field on constructor and method arguments.

- Added `has_varargs` field.

- Added `required` field.

- Better logic for parsing arguments `default` values.
2023-08-08 20:47:27 +02:00
Fabio Manganiello 9beb0a7af3
Skip :type: annotations from docstring args documentation. 2023-08-08 20:47:27 +02:00
Fabio Manganiello 2a30f060b4
Parse the arguments list from obj_type.__init__.__doc__ too. 2023-08-08 20:47:27 +02:00
Fabio Manganiello 1474685cc0
[Automatic] Updated UI files
All checks were successful
continuous-integration/drone/push Build is passing
2023-08-06 22:51:57 +00:00
Fabio Manganiello 26b4dcba46
Reduced padding of ConfirmDialog body element.
All checks were successful
continuous-integration/drone/push Build is passing
2023-08-07 00:50:10 +02:00
Fabio Manganiello 5184543a3e
Normalize the size of the camera frame container from window size.
Some checks failed
continuous-integration/drone/push Build is failing
If the window is smaller than the camera resolution, then we should
scale down the container size accordingly.
2023-08-07 00:38:19 +02:00
Fabio Manganiello dd313c6371
[Automatic] Updated UI files
Some checks reported errors
continuous-integration/drone/push Build was killed
2023-08-06 21:50:25 +00:00
Fabio Manganiello 2f007888fc
[Automatic] Updated UI files
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-06 21:48:33 +00:00
Fabio Manganiello 609a8c6052
[Automatic] Updated UI files
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-06 21:46:44 +00:00
Fabio Manganiello 276a9a7421
[Automatic] Updated UI files
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-06 21:44:51 +00:00
Fabio Manganiello 715cdcfcff
More consistent style for the buttons on the tokens settings page.
Some checks failed
continuous-integration/drone/push Build is failing
2023-08-06 23:42:50 +02:00
Fabio Manganiello bd49e460d5
More style improvements for the user settings panel.
All checks were successful
continuous-integration/drone/push Build is passing
2023-08-06 23:36:06 +02:00
Fabio Manganiello 6eed558342
More improvements on the users settings panel.
Some checks reported errors
continuous-integration/drone/push Build was killed
2023-08-06 23:14:54 +02:00
Fabio Manganiello 1d47b35219
Updated UI dependencies 2023-08-06 20:38:46 +02:00
Fabio Manganiello 237e0c47cb
Redesigned config panel UI. 2023-08-06 18:49:03 +02:00
Fabio Manganiello 53aeb0b3b1
Better documentation for the Redis server + LINT fixes.
All checks were successful
continuous-integration/drone/push Build is passing
1. Added documentation to the README on the possible options to run the
   Redis service.

2. Show a relevant message to the user if the application is run with
   `--start-redis` and Redis couldn't start.

3. Some LINT/black chores on some files that hadn't been touched in a
   while.
2023-08-02 22:17:11 +02:00
Fabio Manganiello e4756eb639
Replaced deprecated flask.escape with markupsafe.escape.
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2023-07-25 00:26:15 +02:00
Fabio Manganiello 91cd08cdff
Adapted tests to use a locally started Redis instance. 2023-07-24 10:37:07 +02:00
Fabio Manganiello 7a20fec52f
[#60] Added --workdir and --logsdir command-line options.
Also, the application is now using `XDG_CONFIG_HOME` and
`XDG_DATA_HOME` if available to lookup the configuration file and
working directory.

Closes: #60
2023-07-24 03:29:08 +02:00
Fabio Manganiello c1d66abb89
[#60] Added --workdir and --logsdir command-line options.
Closes: #60
2023-07-24 03:20:18 +02:00
Fabio Manganiello 5fd40af285 Revert "Added explicit --help option."
This reverts commit 1304be0718.
2023-07-24 02:38:12 +02:00
Fabio Manganiello 1304be0718 Added explicit --help option.
argparse.ArgumentParser doesn't seem to add the option automatically
anymore.
2023-07-24 02:11:42 +02:00
Fabio Manganiello e9a568fdd2
Unified interface to retrieve the Redis bus configuration.
A common `utils.get_redis_conf` has been created to handle the cascade
fallback logic used to retrive the default Redis configuration.
2023-07-24 01:04:13 +02:00
Fabio Manganiello 77ffefdccb
Added a Config.set method.
A useful interface to change configuration values at runtime.
2023-07-24 00:48:07 +02:00
Fabio Manganiello dd862db29a
[chore] LINT fixes 2023-07-24 00:35:03 +02:00