Commit Graph

3550 Commits

Author SHA1 Message Date
Fabio Manganiello f66c4aa071
Ignore the Dockerfile in the root folder
continuous-integration/drone/push Build is passing Details
2023-08-20 13:48:55 +02:00
Fabio Manganiello a6f8021150
`PackageManager` has both `install` and `uninstall`.
continuous-integration/drone/push Build is passing Details
2023-08-20 13:33:49 +02:00
Fabio Manganiello 5efcae64c1
Better Dockerfile logic to retrieve sources.
If the /install folder on the container doesn't contain a copy of the
source files, then the git repository will be cloned under that folder.

The user can specify via `-r/--ref` option which tag/branch/commit they
want to install.
2023-08-20 13:31:13 +02:00
Fabio Manganiello 28ba042810
If no configuration file is passed to platydock generate a minimal Dockerfile.
continuous-integration/drone/push Build was killed Details
2023-08-20 03:36:34 +02:00
Fabio Manganiello 199ac5f0f7
New way of managing installation scripts and dependencies.
Created `platypush/install` folder that contains:

- Dockerfiles for the supported distros
- Lists of required base dependencies for the supported distros
- Install and run scripts
- Added Debian to supported base images
2023-08-20 02:28:10 +02:00
Fabio Manganiello 71c5291190
Refactored the interface of Platydock and manifest utils. 2023-08-19 22:46:37 +02:00
Fabio Manganiello a99ffea37c
Fixed apt dependencies for `mpd` plugin. 2023-08-19 21:46:08 +02:00
Fabio Manganiello 7889b2f1db
Updated `generate_missing_docs` to use the new manifest API. 2023-08-19 13:49:31 +02:00
Fabio Manganiello 980af16984
Rewritten platydock utility.
Platydock now will only print out a Dockerfile given a configuration
file.

No more maintaining the state of containers, storing separate workdirs
and configuration directories etc. - that introduced way too much
overhead over Docker.
2023-08-19 13:47:43 +02:00
Fabio Manganiello 9002f3034a
Tweaked package managers install command arguments. 2023-08-19 13:46:08 +02:00
Fabio Manganiello 69706eaabe
`s/logger/_logger/` in the `plugins` module.
The `logger` name may clash with the context of an action, where
`logger` may have been set to something else.
2023-08-19 13:32:19 +02:00
Fabio Manganiello 1cb686bdab
Updated the `inspect` plugin to the new manifest utils interface. 2023-08-19 13:31:48 +02:00
Fabio Manganiello 043f303761
`s/HttpBackend._DEFAULT_HTTP_PORT/HttpBackend.DEFAULT_HTTP_PORT/g` 2023-08-19 13:30:51 +02:00
Fabio Manganiello dd3a701a2e
Full rewrite of `platypush.utils.manifest`.
The new version encapsulates all the utility functions into three
classes - `Manifest`, `Manifests` and `Dependencies`.
2023-08-19 13:28:40 +02:00
Fabio Manganiello a8255f3621
Pass the configuration file used by the application to the Alembic process.
The database settings could also be overridden in the configuration file
besides the command line.

We should therefore pass the path to the runtime configuration file, so
the Alembic process can initialize its configuration from the same file
and use the same settings.
2023-08-19 13:23:20 +02:00
Fabio Manganiello 1825b492b3
Replaced `Config.workdir` with `Config.get_workdir()`.
Again, Python < 3.9 doesn't like class properties.
2023-08-19 13:21:24 +02:00
Fabio Manganiello a8836f95f5
Support explicit `workdir` parameter override in `Config` constructor. 2023-08-19 13:15:29 +02:00
Fabio Manganiello 5bc82dfe64
s/Config._cfgfile/Config.config_file/g 2023-08-19 13:13:36 +02:00
Fabio Manganiello 181da63c89
Pass the database engine to the Alembic process as an extra argument.
If the path of the default database engine is overridden via `--workdir`
option then it won't be visible to the new `python` subprocess spawned
for Alembic.
2023-08-19 13:02:05 +02:00
Fabio Manganiello c2b3ec8ce3
Fixed manifest files with outdated formats. 2023-08-19 12:54:33 +02:00
Fabio Manganiello e463a52435
Use `sys.executable` rather than `'python'` to launch the application. 2023-08-19 12:40:57 +02:00
Fabio Manganiello 0a5fc40dc5
Merge branch 'master' into 276/better-docker 2023-08-18 17:34:56 +02:00
Fabio Manganiello 3236e235ef Merge pull request '278/fix-variable-errors-on-python-3-8' (#279) from 278/fix-variable-errors-on-python-3-8 into master
continuous-integration/drone/push Build is failing Details
continuous-integration/drone Build is running Details
Reviewed-on: #279
2023-08-18 17:25:08 +02:00
Fabio Manganiello 2cab836bdf
`Entity.columns` class property replaced by `Entity.get_columns` method.
continuous-integration/drone/push Build is passing Details
Again, Python < 3.9 doesn't like the combination of `@property` +
`@classmethod`.
2023-08-18 17:20:53 +02:00
Fabio Manganiello a9cdff900e
_variable should be an external global function rather than a class property.
The combination of `@property` + `@classmethod` isn't supported on
Python < 3.9.
2023-08-18 16:16:47 +02:00
Fabio Manganiello d0c89f88a8
Fixed parenthesised multi-with statement.
This syntax is not supported on Python < 3.9, and therefore it should be
broken down as a multi-line statement.
2023-08-18 16:10:31 +02:00
Fabio Manganiello ca95490412
Added timeout parameter to requests.get in the rss plugin.
continuous-integration/drone/push Build is passing Details
2023-08-18 15:53:30 +02:00
Fabio Manganiello 5dd7345c0b
Sync the latest parse timestamps in main instead of __init__ in rss.
We should load the latest timestamps from the db when the thread starts
instead of doing it in the constructor.

The constructor may be invoked when the entities engine hasn't been
initialized yet, and result in deadlocks.
2023-08-18 15:51:11 +02:00
Fabio Manganiello ec64b0ef8b
Added `--device_id` command line option.
continuous-integration/drone/push Build is failing Details
2023-08-17 23:16:24 +02:00
Fabio Manganiello ac83b43f98
Support for custom key-value overrides on `Config.init`. 2023-08-17 22:03:39 +02:00
Fabio Manganiello 657b2cc87d
Create the default configuration file even if --config is supplied but the file doesn't exist.
continuous-integration/drone/push Build is failing Details
2023-08-17 11:25:49 +02:00
Fabio Manganiello afa4de5673
Dockerfile moved to application root
continuous-integration/drone/push Build is passing Details
2023-08-17 10:38:21 +02:00
Fabio Manganiello 24b04d9103
s/--config-file/--config/ option in Dockerfile.
continuous-integration/drone/push Build is failing Details
2023-08-17 10:35:40 +02:00
Fabio Manganiello f07b774e75
A better Dockerfile.
continuous-integration/drone/push Build is passing Details
2023-08-17 03:04:34 +02:00
Fabio Manganiello bf7d060b81
Added `@ensure_initialized` decorator to actions in `variable`.
continuous-integration/drone/push Build is passing Details
The `variable` plugin may break in the constructor the first time the
application is started.

That's because it tries to initialize the cache of stored variables, but
the local database hasn't yet been initialized.

That's because plugins are registered _before_ the entities engine is
initialized, as the entities engine assumes that it already has plugins
to scan for entities.

Therefore, the initialization of the `variable` plugin's cache should be
lazy (only done upon the first call to `get`/`set` etc.), in order to
prevent deadlock situations where the plugin waits for the engine to
start, but the engine will be initialized only after the plugin is
ready.

And the lazy initialization logic should also ensure that the entities
engine has been properly started (and emit a `TimeoutError` if that's
not the case), in order to prevent race conditions.
2023-08-17 02:47:30 +02:00
Fabio Manganiello adfedfa2dd
s/TimeoutError/AssertionError/ if get_entities_engine times out. 2023-08-17 02:36:40 +02:00
Fabio Manganiello ec2b8da983
Ignore Redis errors when a backend sends an unregister notify event.
When that happens, it's most likely that the application is already
stopping and the Redis service has already been terminated.
2023-08-17 01:49:41 +02:00
Fabio Manganiello 3bf068e0b2
The __main__ function should take no arguments.
setup.py won't pass any arguments to `main()`, so the default entry
point should get them itself from `sys.argv`.
2023-08-17 01:35:39 +02:00
Fabio Manganiello a52de0e086
Removed tz dependency. 2023-08-17 00:35:39 +02:00
Fabio Manganiello e9dbcff1a8
Removed old `bin/platypush` script.
It's already been replaced by the setup.py entry point.
2023-08-16 23:52:54 +02:00
Fabio Manganiello 98e9abde18
Extended manifest files with Python system packages (if available).
continuous-integration/drone/push Build is passing Details
- If a Python optional dependency is available as a system package on
  the target system, try and install it that route rather than pip. It's
  usually faster and it decreases the risk of breaking system packages.

- Added support for apk dependencies in manifest files. This brings the
  number of distros officially supported by all the extensions to four:

  - Alpine
  - Arch
  - Debian
  - Ubuntu
2023-08-16 22:43:51 +02:00
Fabio Manganiello c2a17f0d75 Merge pull request '[#272] Support for external stop/restart control on the application' (#273) from 272/external-proc-controller into master
continuous-integration/drone/push Build is passing Details
Reviewed-on: #273
2023-08-15 21:34:43 +02:00
Fabio Manganiello fef6513cc8
Alpine Linux requires the linux-headers package to build psutil.
continuous-integration/drone/push Build is passing Details
2023-08-15 11:40:02 +02:00
Fabio Manganiello 00863a176e
Added `application` plugin.
continuous-integration/drone/push Build is failing Details
2023-08-15 11:20:24 +02:00
Fabio Manganiello f51beb271e
Large refactor + stability fixes for the external process control logic. 2023-08-15 11:12:21 +02:00
Fabio Manganiello 46245e851f
Synchronize with the currently running stop thread (if any) in `Backend.wait_stop`. 2023-08-15 02:08:47 +02:00
Fabio Manganiello a8a7ceb2ac
Implemented `HttpBackend._stop_workers`.
The Tornado WSGI container won't guarantee the termination of the
spawned workers upon termination, so the code of the backend has to take
care of it and terminate all the children processes of the server
process when it terminates.

This also means that `psutil` is now a required base dependency, as we
need to expand the process subtree under the webserver launcher.
2023-08-15 00:13:34 +02:00
Fabio Manganiello 04921c759f
Added `wait_stop` method to the entities engine and cron scheduler. 2023-08-14 23:49:47 +02:00
Fabio Manganiello ceb9d6d1ba
Removed redundant `Backend._get_redis()` method.
It was used only once, and it could easily be replaced by
`platypush.utils.get_redis()`.
2023-08-14 23:37:38 +02:00
Fabio Manganiello 7157936b87
Added get_remaining_timeout utility function. 2023-08-14 23:17:50 +02:00