Psion-fanatic/platypush
1
0
Fork 0
forked from platypush/platypush
Code Pull requests Activity
3542 commits 6 branches 83 tags 114 MiB
Commit graph

2966 commits

Author SHA1 Message Date
Fabio Manganiello
429658e7c8
Refactored PackageManager classes. 
Instead of having a custom `get_installed` callable field, with
replicated code for each package manager, the field has now been
promoted to a class method containing the common logic, and the
instances now expect a `list` field (base command to list the installed
packages using the specified package manager) and a `parse_list_line`
callback field (to extract the base package name given a raw line from
the command above).

Also, we shouldn't run the list command if we're running within a Docker
context - the host and container environments will be different.
 2023-08-28 01:26:19 +02:00
Fabio Manganiello
86e5f74645
platyvenv should generate (and document) a run.sh helper script. 2023-08-27 23:20:03 +02:00
Fabio Manganiello
9e6430a9ac Clone git repo if platyvenv is not running from a srcdir 2023-08-23 20:02:04 +02:00
Fabio Manganiello
dafd65dc21
Added new platyvenv Python script. 2023-08-23 11:55:25 +02:00
Fabio Manganiello
f230fa79bb
to_pkg_install_commands should skip already installed sys packages. 2023-08-23 11:51:53 +02:00
Fabio Manganiello
449821673c
Added PackageManager.get_installed. 2023-08-23 10:50:29 +02:00
Fabio Manganiello
b10ccdb313
Added get_src_root utility function. 2023-08-23 02:53:31 +02:00
Fabio Manganiello
1ef0d804db
Added full_command argument to to_pip_install_commands. 
This is useful if we just want to get the list of pip dependencies and
create our own pip command.
 2023-08-23 02:19:54 +02:00
Fabio Manganiello
2bff4c9cf1
Exclude python-* system packages when installing in a venv. 2023-08-23 02:17:19 +02:00
Fabio Manganiello
cddf318fa7
Dependencies.from_config should include the base system deps. 2023-08-23 02:16:33 +02:00
Fabio Manganiello
8f39231d31
Added new utility methods to the Dependencies class. 2023-08-23 02:15:50 +02:00
Fabio Manganiello
ce68250b4d
Refactor/documentation round for platydock. 2023-08-22 02:49:05 +02:00
Fabio Manganiello
10c0e5fcad
Added default_os field to PackageManagers enum elements. 
This is useful to determine which is the default set of scripts that
should be used by the installer depending on the detected installed
package manager.
 2023-08-20 21:21:37 +02:00
Fabio Manganiello
2c46b6fe14
Added git a required manifest dependency when needed. 
It is needed for packages that install pip packages via git.
 2023-08-20 21:19:15 +02:00
Fabio Manganiello
a28dcb7a8d
Remove /var/cache/apk from container image after installation. 2023-08-20 14:19:04 +02:00
Fabio Manganiello
a6752ed034
Fixed wrong event path in a manifest file. 2023-08-20 14:08:22 +02:00
Fabio Manganiello
700b8e1d16
Added header and footer to generated Dockerfile. 2023-08-20 14:05:22 +02:00
Fabio Manganiello
a6f8021150
PackageManager has both install and uninstall. 2023-08-20 13:33:49 +02:00
Fabio Manganiello
5efcae64c1
Better Dockerfile logic to retrieve sources. 
If the /install folder on the container doesn't contain a copy of the
source files, then the git repository will be cloned under that folder.

The user can specify via `-r/--ref` option which tag/branch/commit they
want to install.
 2023-08-20 13:31:13 +02:00
Fabio Manganiello
28ba042810
If no configuration file is passed to platydock generate a minimal Dockerfile. 2023-08-20 03:36:34 +02:00
Fabio Manganiello
199ac5f0f7
New way of managing installation scripts and dependencies. 
Created `platypush/install` folder that contains:

- Dockerfiles for the supported distros
- Lists of required base dependencies for the supported distros
- Install and run scripts
- Added Debian to supported base images
 2023-08-20 02:28:10 +02:00
Fabio Manganiello
71c5291190
Refactored the interface of Platydock and manifest utils. 2023-08-19 22:46:37 +02:00
Fabio Manganiello
a99ffea37c
Fixed apt dependencies for mpd plugin. 2023-08-19 21:46:08 +02:00
Fabio Manganiello
980af16984
Rewritten platydock utility. 
Platydock now will only print out a Dockerfile given a configuration
file.

No more maintaining the state of containers, storing separate workdirs
and configuration directories etc. - that introduced way too much
overhead over Docker.
 2023-08-19 13:47:43 +02:00
Fabio Manganiello
9002f3034a
Tweaked package managers install command arguments. 2023-08-19 13:46:08 +02:00
Fabio Manganiello
69706eaabe
s/logger/_logger/ in the plugins module. 
The `logger` name may clash with the context of an action, where
`logger` may have been set to something else.
 2023-08-19 13:32:19 +02:00
Fabio Manganiello
1cb686bdab
Updated the inspect plugin to the new manifest utils interface. 2023-08-19 13:31:48 +02:00
Fabio Manganiello
043f303761
s/HttpBackend._DEFAULT_HTTP_PORT/HttpBackend.DEFAULT_HTTP_PORT/g 2023-08-19 13:30:51 +02:00
Fabio Manganiello
dd3a701a2e
Full rewrite of platypush.utils.manifest. 
The new version encapsulates all the utility functions into three
classes - `Manifest`, `Manifests` and `Dependencies`.
 2023-08-19 13:28:40 +02:00
Fabio Manganiello
a8255f3621
Pass the configuration file used by the application to the Alembic process. 
The database settings could also be overridden in the configuration file
besides the command line.

We should therefore pass the path to the runtime configuration file, so
the Alembic process can initialize its configuration from the same file
and use the same settings.
 2023-08-19 13:23:20 +02:00
Fabio Manganiello
1825b492b3
Replaced Config.workdir with Config.get_workdir(). 
Again, Python < 3.9 doesn't like class properties.
 2023-08-19 13:21:24 +02:00
Fabio Manganiello
a8836f95f5
Support explicit workdir parameter override in Config constructor. 2023-08-19 13:15:29 +02:00
Fabio Manganiello
5bc82dfe64
s/Config._cfgfile/Config.config_file/g 2023-08-19 13:13:36 +02:00
Fabio Manganiello
181da63c89
Pass the database engine to the Alembic process as an extra argument. 
If the path of the default database engine is overridden via `--workdir`
option then it won't be visible to the new `python` subprocess spawned
for Alembic.
 2023-08-19 13:02:05 +02:00
Fabio Manganiello
c2b3ec8ce3
Fixed manifest files with outdated formats. 2023-08-19 12:54:33 +02:00
Fabio Manganiello
e463a52435
Use sys.executable rather than 'python' to launch the application. 2023-08-19 12:40:57 +02:00
Fabio Manganiello
0a5fc40dc5
Merge branch 'master' into 276/better-docker 2023-08-18 17:34:56 +02:00
Fabio Manganiello
2cab836bdf
Entity.columns class property replaced by Entity.get_columns method. 
Again, Python < 3.9 doesn't like the combination of `@property` +
`@classmethod`.
 2023-08-18 17:20:53 +02:00
Fabio Manganiello
a9cdff900e
_variable should be an external global function rather than a class property. 
The combination of `@property` + `@classmethod` isn't supported on
Python < 3.9.
 2023-08-18 16:16:47 +02:00
Fabio Manganiello
d0c89f88a8
Fixed parenthesised multi-with statement. 
This syntax is not supported on Python < 3.9, and therefore it should be
broken down as a multi-line statement.
 2023-08-18 16:10:31 +02:00
Fabio Manganiello
ca95490412
Added timeout parameter to requests.get in the rss plugin. 2023-08-18 15:53:30 +02:00
Fabio Manganiello
5dd7345c0b
Sync the latest parse timestamps in main instead of __init__ in rss. 
We should load the latest timestamps from the db when the thread starts
instead of doing it in the constructor.

The constructor may be invoked when the entities engine hasn't been
initialized yet, and result in deadlocks.
 2023-08-18 15:51:11 +02:00
Fabio Manganiello
ec64b0ef8b
Added --device_id command line option. 2023-08-17 23:16:24 +02:00
Fabio Manganiello
ac83b43f98
Support for custom key-value overrides on Config.init. 2023-08-17 22:03:39 +02:00
Fabio Manganiello
657b2cc87d
Create the default configuration file even if --config is supplied but the file doesn't exist. 2023-08-17 11:25:49 +02:00
Fabio Manganiello
24b04d9103
s/--config-file/--config/ option in Dockerfile. 2023-08-17 10:35:40 +02:00
Fabio Manganiello
bf7d060b81
Added @ensure_initialized decorator to actions in variable. 
The `variable` plugin may break in the constructor the first time the
application is started.

That's because it tries to initialize the cache of stored variables, but
the local database hasn't yet been initialized.

That's because plugins are registered _before_ the entities engine is
initialized, as the entities engine assumes that it already has plugins
to scan for entities.

Therefore, the initialization of the `variable` plugin's cache should be
lazy (only done upon the first call to `get`/`set` etc.), in order to
prevent deadlock situations where the plugin waits for the engine to
start, but the engine will be initialized only after the plugin is
ready.

And the lazy initialization logic should also ensure that the entities
engine has been properly started (and emit a `TimeoutError` if that's
not the case), in order to prevent race conditions.
 2023-08-17 02:47:30 +02:00
Fabio Manganiello
adfedfa2dd
s/TimeoutError/AssertionError/ if get_entities_engine times out. 2023-08-17 02:36:40 +02:00
Fabio Manganiello
ec2b8da983
Ignore Redis errors when a backend sends an unregister notify event. 
When that happens, it's most likely that the application is already
stopping and the Redis service has already been terminated.
 2023-08-17 01:49:41 +02:00
Fabio Manganiello
3bf068e0b2
The __main__ function should take no arguments. 
setup.py won't pass any arguments to `main()`, so the default entry
point should get them itself from `sys.argv`.
 2023-08-17 01:35:39 +02:00
Fabio Manganiello
98e9abde18
Extended manifest files with Python system packages (if available). 
- If a Python optional dependency is available as a system package on
  the target system, try and install it that route rather than pip. It's
  usually faster and it decreases the risk of breaking system packages.

- Added support for apk dependencies in manifest files. This brings the
  number of distros officially supported by all the extensions to four:

  - Alpine
  - Arch
  - Debian
  - Ubuntu
 2023-08-16 22:43:51 +02:00
Fabio Manganiello
00863a176e
Added application plugin. 2023-08-15 11:20:24 +02:00
Fabio Manganiello
f51beb271e
Large refactor + stability fixes for the external process control logic. 2023-08-15 11:12:21 +02:00
Fabio Manganiello
46245e851f
Synchronize with the currently running stop thread (if any) in Backend.wait_stop. 2023-08-15 02:08:47 +02:00
Fabio Manganiello
a8a7ceb2ac
Implemented HttpBackend._stop_workers. 
The Tornado WSGI container won't guarantee the termination of the
spawned workers upon termination, so the code of the backend has to take
care of it and terminate all the children processes of the server
process when it terminates.

This also means that `psutil` is now a required base dependency, as we
need to expand the process subtree under the webserver launcher.
 2023-08-15 00:13:34 +02:00
Fabio Manganiello
04921c759f
Added wait_stop method to the entities engine and cron scheduler. 2023-08-14 23:49:47 +02:00
Fabio Manganiello
ceb9d6d1ba
Removed redundant Backend._get_redis() method. 
It was used only once, and it could easily be replaced by
`platypush.utils.get_redis()`.
 2023-08-14 23:37:38 +02:00
Fabio Manganiello
7157936b87
Added get_remaining_timeout utility function. 2023-08-14 23:17:50 +02:00
Fabio Manganiello
dc1a152433
Added get_default_pid_file utility method. 2023-08-14 10:47:25 +02:00
Fabio Manganiello
c11bc69a66
Handle KeyboardInterrupt and process return code in the main. 2023-08-13 23:55:40 +02:00
Fabio Manganiello
8819a0ed4c
Added CommandStream to the main Application. 
The stream will be used to communicate command messages from the
application to its runner.
 2023-08-13 23:51:21 +02:00
Fabio Manganiello
06111587f7
Added CommandStream class. 
This abstraction is used to write and read commands over a UNIX socket.
 2023-08-13 23:37:51 +02:00
Fabio Manganiello
c89f992867
Added StopCommand and RestartCommand. 2023-08-13 23:36:36 +02:00
Fabio Manganiello
b1f244a812
Added ControllableProcess class. 
This class can be used to easily control the execution of underlying
processes.
 2023-08-13 23:33:08 +02:00
Fabio Manganiello
efef9d7bc0
Added commands module. 2023-08-13 23:21:36 +02:00
Fabio Manganiello
97adc3f775
Moved application argument parser to an external platypush.cli module. 2023-08-13 22:27:16 +02:00
Fabio Manganiello
1819ee75ef
s/Application.stop_app/Application.stop/g 2023-08-13 22:23:15 +02:00
Fabio Manganiello
827b564006
Using a single constant for DEFAULT_REDIS_QUEUE. 
Also, catch `AttributeError` on `self._proc.terminate` in the
`HttpBackend`, since the process may already have been terminated and
set to null by another worker process.
 2023-08-13 22:20:16 +02:00
Fabio Manganiello
4062ddbcf0
More improvements to the inspect plugin. 
- Support for distinct `type` field on constructor and method arguments.

- Added `has_varargs` field.

- Added `required` field.

- Better logic for parsing arguments `default` values.
 2023-08-08 20:47:27 +02:00
Fabio Manganiello
9beb0a7af3
Skip :type: annotations from docstring args documentation. 2023-08-08 20:47:27 +02:00
Fabio Manganiello
2a30f060b4
Parse the arguments list from obj_type.__init__.__doc__ too. 2023-08-08 20:47:27 +02:00
Fabio Manganiello
1474685cc0
[Automatic] Updated UI files 2023-08-06 22:51:57 +00:00
Fabio Manganiello
26b4dcba46
Reduced padding of ConfirmDialog body element. 2023-08-07 00:50:10 +02:00
Fabio Manganiello
5184543a3e
Normalize the size of the camera frame container from window size. 
If the window is smaller than the camera resolution, then we should
scale down the container size accordingly.
 2023-08-07 00:38:19 +02:00
Fabio Manganiello
dd313c6371
[Automatic] Updated UI files 2023-08-06 21:50:25 +00:00
Fabio Manganiello
2f007888fc
[Automatic] Updated UI files 2023-08-06 21:48:33 +00:00
Fabio Manganiello
609a8c6052
[Automatic] Updated UI files 2023-08-06 21:46:44 +00:00
Fabio Manganiello
276a9a7421
[Automatic] Updated UI files 2023-08-06 21:44:51 +00:00
Fabio Manganiello
715cdcfcff
More consistent style for the buttons on the tokens settings page. 2023-08-06 23:42:50 +02:00
Fabio Manganiello
bd49e460d5
More style improvements for the user settings panel. 2023-08-06 23:36:06 +02:00
Fabio Manganiello
6eed558342
More improvements on the users settings panel. 2023-08-06 23:14:54 +02:00
Fabio Manganiello
1d47b35219
Updated UI dependencies 2023-08-06 20:38:46 +02:00
Fabio Manganiello
237e0c47cb
Redesigned config panel UI. 2023-08-06 18:49:03 +02:00
Fabio Manganiello
53aeb0b3b1
Better documentation for the Redis server + LINT fixes. 
1. Added documentation to the README on the possible options to run the
   Redis service.

2. Show a relevant message to the user if the application is run with
   `--start-redis` and Redis couldn't start.

3. Some LINT/black chores on some files that hadn't been touched in a
   while.
 2023-08-02 22:17:11 +02:00
Fabio Manganiello
e4756eb639
Replaced deprecated flask.escape with markupsafe.escape. 2023-07-25 00:26:15 +02:00
Fabio Manganiello
91cd08cdff
Adapted tests to use a locally started Redis instance. 2023-07-24 10:37:07 +02:00
Fabio Manganiello
7a20fec52f
[#60] Added --workdir and --logsdir command-line options. 
Also, the application is now using `XDG_CONFIG_HOME` and
`XDG_DATA_HOME` if available to lookup the configuration file and
working directory.

Closes: #60
 2023-07-24 03:29:08 +02:00
Fabio Manganiello
c1d66abb89
[#60] Added --workdir and --logsdir command-line options. 
Closes: #60
 2023-07-24 03:20:18 +02:00
Fabio Manganiello
5fd40af285 Revert "Added explicit --help option." 
This reverts commit 1304be0718.
 2023-07-24 02:38:12 +02:00
Fabio Manganiello
1304be0718 Added explicit --help option. 
argparse.ArgumentParser doesn't seem to add the option automatically
anymore.
 2023-07-24 02:11:42 +02:00
Fabio Manganiello
e9a568fdd2
Unified interface to retrieve the Redis bus configuration. 
A common `utils.get_redis_conf` has been created to handle the cascade
fallback logic used to retrive the default Redis configuration.
 2023-07-24 01:04:13 +02:00
Fabio Manganiello
77ffefdccb
Added a Config.set method. 
A useful interface to change configuration values at runtime.
 2023-07-24 00:48:07 +02:00
Fabio Manganiello
dd862db29a
[chore] LINT fixes 2023-07-24 00:35:03 +02:00
Fabio Manganiello
3e3d47aa44
Check first the configuration of the Redis plugin and then that of the backend. 2023-07-24 00:26:49 +02:00
Fabio Manganiello
315a89fb65
Added a default parameter to Config.get. 2023-07-23 23:31:57 +02:00
Fabio Manganiello
37dcaba7a1
Refactored structure of main application and startup scripts. 
The main application class has been moved from __init__ to the app
module.

__init__ will contain instead the relevant global variables and the
modules and objects exposed to external integrations - such as
`get_plugin` and `get_backend`, or the `main` itself.

This will make future integrations much easier - the global __init__
doesn't contain any business logic now, it can import anything without
fearing circular dependencies, and it can limit its exposed objects to
those that we want to expose to 3rd-party integrations and scripts.

It will also make it easier to extend the main entry point with
additional logic - such as a supervisor or an embedded Redis server.
 2023-07-23 23:12:36 +02:00
Fabio Manganiello
0dc380fa94
Removed dependency from prctl. 
Also, black'd and LINT-fixed some files that hadn't been touched in a
while.
 2023-07-23 19:17:30 +02:00
Fabio Manganiello
15a34ebac9
Upgraded Docker base image. 
Debian base image bumped from Bullseye -> Bookworm.
 2023-07-23 17:25:52 +02:00
Fabio Manganiello
cfdecd5703
[#267] Don't show the PWA popup again after the user dismisses it. 
Plus, upgraded a bunch of frontend libraries.

Closes: #267
 2023-07-23 02:44:17 +02:00
Fabio Manganiello
454228c56e
Upgraded UI dependencies 2023-07-23 02:42:14 +02:00