[Backend] Expose procedures as entities #426

Merged
blacklight merged 38 commits from 341/procedure-entities into master 2024-09-23 03:45:31 +02:00
Owner

See #341

See #341
blacklight added this to the v1.3.0 milestone 2024-09-01 18:15:45 +02:00
blacklight added the
architecture
backend
labels 2024-09-01 18:15:45 +02:00
blacklight self-assigned this 2024-09-01 18:15:45 +02:00
blacklight added 15 commits 2024-09-01 18:15:45 +02:00
[WIP]
All checks were successful
continuous-integration/drone/push Build is passing
af21ff13ff
[#341] Support for procedure reconciliation.
All checks were successful
continuous-integration/drone/push Build is passing
0f186c44ef
If some procedures are removed either from the configuration or from the
loaded scripts, then their associated entities should also be removed
from the database when the `procedures` plugin is loaded.
[procedures] Store actions for YAML procedures.
All checks were successful
continuous-integration/drone/push Build is passing
d9916873cb
[#341] Added procedures.save and procedures.delete actions.
All checks were successful
continuous-integration/drone/push Build is passing
457333929f
Messages can be quite big and verbose, and they can anyway be subscribed
over Websockets.

Full dumps are anyway enabled when Platypush is started in verbose mode.

This commit replaces the dumps on INFO level with a quick summary
containing the message ID, request/event type and response time.
[core] Fix support for custom SQLAlchemy engine options on db conf.
All checks were successful
continuous-integration/drone/push Build is passing
a3eedc6adc
Earlier any extra parameters passed to the `db` configuration other than
`engine` where ignored.

This enables engine-level configurations such as:

```yaml
db:
  # Display all SQL queries
  echo: true
```
This is the default behaviour on basically any other supported RDBMS.
It seems to clash with something and cause plugin actions to return
random `ImportError`.
[#341] More procedures features.
All checks were successful
continuous-integration/drone/push Build is passing
1369848114
- `procedures.exec` now supports running procedures "on the fly" given a
  definition with a list of actions.

- Fixed procedure renaming/overwrite logic.

- Access to `_all_procedures` should always be guarded by a lock.
[#341] More improvements on the procedures plugin.
All checks were successful
continuous-integration/drone/push Build is passing
861e7e7c52
- `procedures.status` should always sync with the db to ensure that the
  action returns the most up-to-date version of the procedures.

- Store, return and propagate entity procedure metadata.
blacklight added this to the core project 2024-09-01 18:15:46 +02:00
blacklight added 1 commit 2024-09-01 20:01:00 +02:00
Merge branch 'master' into 341/procedure-entities
Some checks failed
continuous-integration/drone/push Build is failing
6df699359c
blacklight changed title from [Backend] Expose procedures as entities to WIP: [Backend] Expose procedures as entities 2024-09-01 20:06:39 +02:00
blacklight added 9 commits 2024-09-02 02:24:51 +02:00
[#341] Improvements on procedures.save.
Some checks failed
continuous-integration/drone/push Build is failing
26f491025a
- Update the cached representation of the procedure asynchronously on
  the `publish_entities` callback. This prevents stale records from
  being loaded from the db before the entities engine has persisted the
  new ones.

- Don't re-publish all entities when calling `procedures.status` at the
  end of `procedures.save`. This is both for performance reasons and to
  avoid sending to the entities engine stale representation of the data.
blacklight added 2 commits 2024-09-02 02:33:16 +02:00
[CI/CD] A more resilient github-mirror script.
All checks were successful
continuous-integration/drone/push Build is passing
e39e36e5f6
- Fail immediately if no branches are checked out.

- Rebase only if we're pushing on master (don't bother for feature
  branches).

- Do a push force to Github.
blacklight added 1 commit 2024-09-05 01:14:00 +02:00
[#341] Added utility procedures.to_yaml action.
All checks were successful
continuous-integration/drone/push Build is passing
c54269e3d2
blacklight added 2 commits 2024-09-06 11:55:51 +02:00
[procedures] Don't validate the presence of the actions field in procedures.save.
All checks were successful
continuous-integration/drone/push Build is passing
05b1fcd43a
When saving procedures with if/else/for blocks, some blocks aren't
supposed to have the `actions` field.
blacklight added 3 commits 2024-09-10 19:57:01 +02:00
This is for UI compatibility purposes.
The reason is that an `id` specified on procedure level will be applied
to all the child requests.

This means that the first response from the first completed request will
be sent to Redis and mistakenly interpreted by HTTP listeners as the
return value of the whole procedure.

`Procedure.build` should instead calculate its own ID for the procedure,
and apply different IDs to the child requests.
[procedure] Added support for custom values on the return statement.
All checks were successful
continuous-integration/drone/push Build is passing
1e9f7fb2c6
This enables constructs like this in procedures:

```yaml
- return

- return 1

- return: ${output}
```
blacklight added 1 commit 2024-09-12 02:16:28 +02:00
[procedures] Fixed if queue flushing logic.
All checks were successful
continuous-integration/drone/push Build is passing
853fce2521
Any pending `if`s in the parsing queue of a procedure should also be
cleared if the current statement in the procedure is a
break/continue/return.

In such case we should terminate the current branch, and that involves
ensuring that any `if`s branches that are still being parsed are
inserted before the branch-terminating statement.
blacklight added 1 commit 2024-09-13 18:22:09 +02:00
[#341] procedure._serialize_action should also support strings.
All checks were successful
continuous-integration/drone/push Build is passing
771e32e368
blacklight added 1 commit 2024-09-16 03:19:58 +02:00
[procedure] Several improvements to the procedure engine.
All checks were successful
continuous-integration/drone/push Build is passing
be8140ddb5
- Add `set` statement, which can be used to set context variables within
  YAML procedures. Example:

```yaml
procedure.test:
  - set:
      foo: bar

  - action: logger.info
    args:
      msg: ${bar}
```

- More reliable flow control for nested break/continue/return.

- Propagate changes to context variables also to upstream procedures.
blacklight added 1 commit 2024-09-23 03:21:25 +02:00
[alarm] Don't fail if no audio file is provided.
All checks were successful
continuous-integration/drone/push Build is passing
38c1ebb90c
blacklight added 1 commit 2024-09-23 03:44:16 +02:00
Updated CHANGELOG
All checks were successful
continuous-integration/drone/push Build is passing
8d8e1878bb
blacklight changed title from WIP: [Backend] Expose procedures as entities to [Backend] Expose procedures as entities 2024-09-23 03:44:34 +02:00
blacklight merged commit 62737b5a95 into master 2024-09-23 03:45:31 +02:00
blacklight deleted branch 341/procedure-entities 2024-09-23 03:45:31 +02:00
Sign in to join this conversation.
No description provided.