Compare commits

...

10 Commits

Author SHA1 Message Date
Fabio Manganiello 795754f858
Added PWA support 2023-05-18 03:12:48 +02:00
Fabio Manganiello 27d4a20418
Use reflection to infer the arguments of a Python user procedure 2023-05-17 17:17:59 +02:00
Fabio Manganiello 0a1209fe6e
Updated webapp dist files 2023-05-17 10:56:37 +02:00
Fabio Manganiello 33e2879413
Various UI improvements for the execute tab. 2023-05-17 10:41:02 +02:00
Fabio Manganiello 91daec579d
Reverted to the previous style for entities on mobile.
Better to use screen width wisely and avoid unnecessary padding.
2023-05-17 01:13:09 +02:00
Fabio Manganiello 61ea3d79e4
Large refactor for the `inspect` plugin.
More common logic has been extracted and all the methods and classes
have been documented and black'd.
2023-05-17 00:05:22 +02:00
Fabio Manganiello 2cba504e3b
Improvements for the autocomplete component. 2023-05-14 15:07:54 +02:00
Fabio Manganiello 8447f9a854
Improved rendering of actions/arguments documentation.
The frontend now calls `utils.rst_to_html` to render the docstrings as
HTML instead of dumping them as raw text.

Also, actions and arguments are now cached to improve performance.
2023-05-14 15:06:34 +02:00
Fabio Manganiello 5f2d6dfeb5
Added `utils.rst_to_html` action. 2023-05-14 15:05:24 +02:00
Fabio Manganiello 3c83e7f412
A faster implementation for the `inspect.get_*` methods.
Plugin/backend lookup is now done by inspecting the manifest files
instead of searching all the subpackages.
2023-05-13 13:44:46 +02:00
74 changed files with 3509 additions and 585 deletions

View File

@ -641,11 +641,7 @@ of Platypush to your fingertips.
## Tests
To run the tests simply run `pytest` either from the project root folder or the
`tests/` folder. Or run the following command from the project root folder:
```shell
python -m tests
```
`tests/` folder.
---

View File

@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" href="/favicon.ico"><link rel="stylesheet" href="/fonts/poppins.css"><title>platypush</title><script defer="defer" type="module" src="/static/js/chunk-vendors.95bedba1.js"></script><script defer="defer" type="module" src="/static/js/app.5b73356c.js"></script><link href="/static/css/chunk-vendors.0fcd36f0.css" rel="stylesheet"><link href="/static/css/app.7cb6eac2.css" rel="stylesheet"><script defer="defer" src="/static/js/chunk-vendors-legacy.79dede0c.js" nomodule></script><script defer="defer" src="/static/js/app-legacy.d7eee501.js" nomodule></script></head><body><noscript><strong>We're sorry but platypush doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript><div id="app"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" href="/favicon.ico"><link rel="stylesheet" href="/fonts/poppins.css"><title>platypush</title><script defer="defer" type="module" src="/static/js/chunk-vendors.95bedba1.js"></script><script defer="defer" type="module" src="/static/js/app.8fd4b02d.js"></script><link href="/static/css/chunk-vendors.0fcd36f0.css" rel="stylesheet"><link href="/static/css/app.7cb6eac2.css" rel="stylesheet"><script defer="defer" src="/static/js/chunk-vendors-legacy.79dede0c.js" nomodule></script><script defer="defer" src="/static/js/app-legacy.2bd8b862.js" nomodule></script></head><body><noscript><strong>We're sorry but platypush doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript><div id="app"></div></body></html>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

View File

@ -13,6 +13,7 @@
"core-js": "^3.23.4",
"lato-font": "^3.0.0",
"mitt": "^2.1.0",
"register-service-worker": "^1.7.2",
"sass": "^1.53.0",
"sass-loader": "^10.3.1",
"vue": "^3.2.13",
@ -25,6 +26,7 @@
"@babel/eslint-parser": "^7.12.16",
"@vue/cli-plugin-babel": "~5.0.0",
"@vue/cli-plugin-eslint": "~5.0.0",
"@vue/cli-plugin-pwa": "~5.0.0",
"@vue/cli-service": "~5.0.0",
"eslint": "^7.32.0",
"eslint-plugin-vue": "^8.0.3"

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 799 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

View File

@ -0,0 +1,3 @@
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M8.00251 14.9297L0 1.07422H6.14651L8.00251 4.27503L9.84583 1.07422H16L8.00251 14.9297Z" fill="black"/>
</svg>

After

Width:  |  Height:  |  Size: 215 B

View File

@ -2,19 +2,26 @@ function autocomplete(inp, arr, listener) {
/*the autocomplete function takes two arguments,
the text field element and an array of possible autocompleted values:*/
let currentFocus;
/*execute a function when someone writes in the text field:*/
inp.addEventListener("input", function() {
let a, b, i, val = this.value;
/*close any already open lists of autocompleted values*/
closeAllLists();
if (!val) { return false;}
if (!val) {
return false;
}
currentFocus = -1;
/*create a DIV element that will contain the items (values):*/
a = document.createElement("DIV");
a.setAttribute("id", this.id + "autocomplete-list");
a.setAttribute("class", "autocomplete-items");
/*append the DIV element as a child of the autocomplete container:*/
this.parentNode.appendChild(a);
/*for each item in the array...*/
for (i = 0; i < arr.length; i++) {
/*check if the item starts with the same letters as the text field value:*/
@ -43,10 +50,13 @@ function autocomplete(inp, arr, listener) {
}
});
inp.addEventListener("keydown", function(e) {
if (e.keyCode === 9) {
/*Reset the list if tab has been pressed*/
closeAllLists();
inp.addEventListener("keyup", function(e) {
if (["ArrowUp", "ArrowDown", "Tab", "Enter"].indexOf(e.key) >= 0) {
e.stopPropagation();
}
if (e.key === "Enter") {
this.blur();
}
});
@ -54,19 +64,21 @@ function autocomplete(inp, arr, listener) {
inp.addEventListener("keydown", function(e) {
let x = document.getElementById(this.id + "autocomplete-list");
if (x) x = x.getElementsByTagName("div");
if (e.keyCode === 40) {
if (e.key === 'ArrowDown' || (e.key === 'Tab' && !e.shiftKey)) {
/*If the arrow DOWN key is pressed,
increase the currentFocus variable:*/
currentFocus++;
/*and and make the current item more visible:*/
addActive(x);
} else if (e.keyCode === 38) { //up
e.preventDefault();
} else if (e.key === 'ArrowUp' || (e.key === 'Tab' && e.shiftKey)) { //up
/*If the arrow UP key is pressed,
decrease the currentFocus variable:*/
currentFocus--;
/*and and make the current item more visible:*/
addActive(x);
} else if (e.keyCode === 13) {
e.preventDefault();
} else if (e.key === 'Enter') {
/*If the ENTER key is pressed, prevent the form from being submitted,*/
if (currentFocus > -1 && x && x.length) {
e.preventDefault();
@ -77,6 +89,7 @@ function autocomplete(inp, arr, listener) {
}
}
});
function addActive(x) {
/*a function to classify an item as "active":*/
if (!x) return false;
@ -87,12 +100,14 @@ function autocomplete(inp, arr, listener) {
/*add class "autocomplete-active":*/
x[currentFocus].classList.add("autocomplete-active");
}
function removeActive(x) {
/*a function to remove the "active" class from all autocomplete items:*/
for (let i = 0; i < x.length; i++) {
x[i].classList.remove("autocomplete-active");
}
}
function closeAllLists(elmnt) {
/*close all autocomplete lists in the document,
except the one passed as an argument:*/
@ -103,6 +118,7 @@ function autocomplete(inp, arr, listener) {
}
}
}
/*execute a function when someone clicks in the document:*/
document.addEventListener("click", function (e) {
closeAllLists(e.target);

View File

@ -509,6 +509,7 @@ export default {
@include until(#{$tablet - 1}) {
padding: 0;
margin-bottom: $main-margin;
}
@include from($tablet) {
@ -524,11 +525,8 @@ export default {
flex-direction: column;
flex-grow: 1;
position: relative;
@include from($tablet) {
border-radius: 1em;
box-shadow: $group-shadow;
}
border-radius: 1em;
box-shadow: $group-shadow;
}
.header {
@ -579,10 +577,6 @@ export default {
overflow: auto;
flex-grow: 1;
@include until(#{$tablet - 1}) {
background: $default-bg-4;
}
@include from($tablet) {
background: $default-bg-2;
}
@ -590,16 +584,8 @@ export default {
.entity-frame {
background: $background-color;
@include until(#{$tablet - 1}) {
margin: 0.75em 0.25em;
border: $default-border-2;
border-radius: 1em;
}
@include from($tablet) {
&:last-child {
border-radius: 0 0 1em 1em;
}
&:last-child {
border-radius: 0 0 1em 1em;
}
}
}

View File

@ -6,10 +6,6 @@ $collapse-toggler-width: 2em;
background: $selected-bg;
font-weight: bold;
box-shadow: 0 0 3px 2px $default-shadow-color;
@include until(#{$tablet - 1}) {
border-radius: 1em;
}
}
.entity-container {
@ -18,18 +14,7 @@ $collapse-toggler-width: 2em;
align-items: center;
position: relative;
padding: 0 !important;
@include until(#{$tablet - 1}) {
border-radius: 1em;
&:not(:last-child) {
border-bottom: $default-border-3;
}
}
@include from($tablet) {
border-bottom: $default-border-3;
}
border-bottom: $default-border-3;
&.with-children:not(.collapsed) {
@include expanded-entity();
@ -50,12 +35,6 @@ $collapse-toggler-width: 2em;
}
@include until(#{$tablet - 1}) {
.entity-container {
.children {
border-radius: 0 0 1em 1em;
}
}
.child {
&:not(:last-child) {
.entity-container {
@ -239,38 +218,8 @@ $collapse-toggler-width: 2em;
&.with-children:not(.collapsed) {
box-shadow: 0 3px 4px 0 $default-shadow-color;
@include until(#{$tablet - 1}) {
border-radius: 1em;
}
@include from($tablet) {
.children .child:last-child {
box-shadow: 0 3px 4px 0 $default-shadow-color;
}
}
}
@include until(#{$tablet - 1}) {
box-shadow: 0 3px 4px 0 $default-shadow-color;
&.collapsed {
border-radius: 1em;
}
.children {
.entity-container-wrapper {
border-radius: 0;
box-shadow: none;
&.with-children:not(.collapsed) {
border-radius: 1em;
box-shadow: 0 3px 4px 0 $default-shadow-color;
}
&:last-child {
border-radius: 0 0 1em 1em;
}
}
.children .child:last-child {
box-shadow: 0 3px 4px 0 $default-shadow-color;
}
}
}

View File

@ -21,7 +21,8 @@
@change="actionChanged=true" @blur="updateAction">
</label>
</div>
<button type="submit" class="run-btn btn-primary" :disabled="running" title="Run">
<button type="submit" class="run-btn btn-primary"
:disabled="running || !action?.name?.length" title="Run">
<i class="fas fa-play" />
</button>
@ -30,8 +31,10 @@
Action documentation
</div>
<div class="doc html" v-html="selectedDoc" v-if="htmlDoc" />
<div class="doc raw" v-text="selectedDoc" v-else />
<div class="doc html">
<Loading v-if="docLoading" />
<span v-html="selectedDoc" v-else />
</div>
</div>
<div class="options" v-if="action.name in actions && (Object.keys(action.args).length ||
@ -51,8 +54,10 @@
Attribute: <div class="attr-name" v-text="selectedAttr" />
</div>
<div class="doc html" v-html="selectedAttrDoc" v-if="htmlDoc" />
<div class="doc raw" v-text="selectedAttrDoc" v-else />
<div class="doc html">
<Loading v-if="docLoading" />
<span v-html="selectedAttrDoc" v-else />
</div>
</div>
</div>
@ -62,11 +67,11 @@
<input type="text" class="action-extra-param-name" :disabled="running"
placeholder="Name" v-model="action.extraArgs[i].name">
</label>
<label class="col-5">
<label class="col-6">
<input type="text" class="action-extra-param-value" :disabled="running"
placeholder="Value" v-model="action.extraArgs[i].value">
</label>
<label class="col-2 buttons">
<label class="col-1 buttons">
<button type="button" class="action-extra-param-del" title="Remove parameter"
@click="removeParameter(i)">
<i class="fas fa-trash" />
@ -87,22 +92,24 @@
Attribute: <div class="attr-name" v-text="selectedAttr" />
</div>
<div class="doc html" v-html="selectedAttrDoc" v-if="htmlDoc" />
<div class="doc raw" v-text="selectedAttrDoc" v-else />
<div class="doc html">
<Loading v-if="docLoading" />
<span v-html="selectedAttrDoc" v-else />
</div>
</div>
</div>
<div class="output-container">
<div class="title" v-text="error != null ? 'Error' : 'Output'" v-if="error != null || response != null" />
<div class="response" v-html="response" v-if="response != null" />
<div class="error" v-html="error" v-else-if="error != null" />
</div>
<div class="output-container">
<div class="title" v-text="error != null ? 'Error' : 'Output'" v-if="error != null || response != null" />
<div class="response" v-html="response" v-if="response != null" />
<div class="error" v-html="error" v-else-if="error != null" />
</div>
</div>
<div class="request raw-request" :class="structuredInput ? 'hidden' : ''">
<div class="first-row">
<label>
<textarea v-model="rawRequest" placeholder="Raw JSON request" />
<textarea v-model="rawRequest" ref="rawAction" :placeholder="rawRequestPlaceholder" />
</label>
<button type="submit" :disabled="running" class="run-btn btn-primary" title="Run">
<i class="fas fa-play" />
@ -163,6 +170,7 @@ export default {
return {
loading: false,
running: false,
docLoading: false,
structuredInput: true,
actionChanged: false,
selectedDoc: undefined,
@ -175,11 +183,13 @@ export default {
response: undefined,
error: undefined,
htmlDoc: false,
rawRequest: undefined,
rawRequestPlaceholder: 'Raw JSON request. Example:\n\n' +
'{"type": "request", "action": "file.list", "args": {"path": "/"}}',
actions: {},
plugins: {},
procedures: {},
actionDocsCache: {},
action: {
name: undefined,
args: {},
@ -195,15 +205,12 @@ export default {
try {
this.procedures = await this.request('inspect.get_procedures')
this.plugins = await this.request('inspect.get_all_plugins', {html_doc: false})
this.plugins = await this.request('inspect.get_all_plugins')
} finally {
this.loading = false
}
for (const plugin of Object.values(this.plugins)) {
if (plugin.html_doc)
this.htmlDoc = true
for (const action of Object.values(plugin.actions)) {
action.name = plugin.name + '.' + action.name
action.supportsExtraArgs = !!action.has_kwargs
@ -213,20 +220,20 @@ export default {
}
const self = this
autocomplete(this.$refs.actionName, Object.keys(this.actions).sort(), (evt, value) => {
autocomplete(this.$refs.actionName, Object.keys(this.actions).sort(), (_, value) => {
this.action.name = value
self.updateAction()
})
},
updateAction() {
async updateAction() {
if (!(this.action.name in this.actions))
this.selectedDoc = undefined
if (!this.actionChanged || !(this.action.name in this.actions))
return
this.loading = true
this.docLoading = true
try {
this.action = {
...this.actions[this.action.name],
@ -241,32 +248,27 @@ export default {
extraArgs: [],
}
} finally {
this.loading = false
this.docLoading = false
}
this.selectedDoc = this.parseDoc(this.action.doc)
this.selectedDoc =
this.actionDocsCache[this.action.name]?.html ||
await this.parseDoc(this.action.doc)
if (!this.actionDocsCache[this.action.name])
this.actionDocsCache[this.action.name] = {}
this.actionDocsCache[this.action.name].html = this.selectedDoc
this.actionChanged = false
this.response = undefined
this.error = undefined
},
parseDoc(docString) {
if (!docString?.length || this.htmlDoc)
async parseDoc(docString) {
if (!docString?.length)
return docString
let lineNo = 0
let trailingSpaces = 0
return docString.split('\n').reduce((doc, line) => {
if (++lineNo === 2)
trailingSpaces = line.match(/^(\s*)/)[1].length
if (line.trim().startsWith('.. code-block'))
return doc
doc += line.slice(trailingSpaces).replaceAll('``', '') + '\n'
return doc
}, '')
return await this.request('utils.rst_to_html', {text: docString})
},
updateProcedure(name, event) {
@ -308,11 +310,16 @@ export default {
this.action.extraArgs.pop(i)
},
selectAttrDoc(name) {
this.response = undefined
this.error = undefined
async selectAttrDoc(name) {
this.selectedAttr = name
this.selectedAttrDoc = this.parseDoc(this.action.args[name].doc)
this.selectedAttrDoc =
this.actionDocsCache[this.action.name]?.[name]?.html ||
await this.parseDoc(this.action.args[name].doc)
if (!this.actionDocsCache[this.action.name])
this.actionDocsCache[this.action.name] = {}
this.actionDocsCache[this.action.name][name] = {html: this.selectedAttrDoc}
},
resetAttrDoc() {
@ -326,6 +333,13 @@ export default {
this.structuredInput = structuredInput
this.response = undefined
this.error = undefined
this.$nextTick(() => {
if (structuredInput) {
this.$refs.actionName.focus()
} else {
this.$refs.rawAction.focus()
}
})
},
onResponse(response) {
@ -422,6 +436,10 @@ export default {
},
mounted() {
this.$nextTick(() => {
this.$refs.actionName.focus()
})
this.refresh()
},
}
@ -450,11 +468,12 @@ $params-tablet-width: 20em;
}
.action-form {
background: $default-bg-2;
padding: 1em .5em;
}
.title {
background: $title-bg;
background: $header-bg;
padding: .5em;
border: $title-border;
box-shadow: $title-shadow;
@ -544,12 +563,26 @@ $params-tablet-width: 20em;
display: flex;
margin-bottom: .5em;
label {
margin-left: 0.25em;
}
.buttons {
display: flex;
flex-grow: 1;
justify-content: right;
}
.action-extra-param-del {
border: 0;
text-align: right;
padding: 0 .5em;
}
input[type=text] {
width: 100%;
}
.buttons {
display: flex;
align-items: center;
@ -569,11 +602,6 @@ $params-tablet-width: 20em;
.doc-container,
.output-container {
margin-top: .5em;
.doc {
&.raw {
white-space: pre;
}
}
}
.output-container {
@ -590,7 +618,6 @@ $params-tablet-width: 20em;
}
.doc {
white-space: pre-line;
width: 100%;
overflow: auto;
}
@ -613,11 +640,6 @@ $params-tablet-width: 20em;
.attr-doc-container {
.doc {
padding: 1em !important;
&.raw {
font-family: monospace;
font-size: .8em;
}
}
}
@ -747,11 +769,30 @@ $params-tablet-width: 20em;
}
.run-btn {
border-radius: 2em;
padding: .5em .75em;
background: $background-color;
border-radius: .25em;
padding: .5em 1.5em;
box-shadow: $primary-btn-shadow;
cursor: pointer;
&:hover {
opacity: .8;
background: $hover-bg;
box-shadow: none;
}
&:disabled {
opacity: 0.7;
color: $default-fg-2;
cursor: initial;
box-shadow: none;
&:hover {
background: $background-color;
box-shadow: none;
}
}
&:not(disabled) {
}
}
}

View File

@ -7,7 +7,7 @@ $response-bg: #edfff2;
$response-border: 1px dashed #98ff98;
$error-bg: #ffbcbc;
$error-border: 1px dashed #ff5353;
$doc-bg: #e8feff;
$doc-border: 1px dashed #84f9ff;
$doc-bg: $background-color;
$doc-border: 1px dashed $border-color-2;
$procedure-submit-btn-bg: #ebffeb;
$section-title-bg: rgba(0, 0, 0, .04);

View File

@ -1,6 +1,7 @@
import { createApp } from 'vue'
import App from '@/App.vue'
import router from '@/router'
import './registerServiceWorker'
const app = createApp(App)
app.config.globalProperties._config = window.config

View File

@ -0,0 +1,32 @@
/* eslint-disable no-console */
import { register } from 'register-service-worker'
if (process.env.NODE_ENV === 'production') {
register(`${process.env.BASE_URL}service-worker.js`, {
ready () {
console.log(
'App is being served from cache by a service worker.\n' +
'For more details, visit https://goo.gl/AFskqB'
)
},
registered () {
console.log('Service worker has been registered.')
},
cached () {
console.log('Content has been cached for offline use.')
},
updatefound () {
console.log('New content is downloading.')
},
updated () {
console.log('New content is available; please refresh.')
},
offline () {
console.log('No internet connection found. App is running in offline mode.')
},
error (error) {
console.error('Error during service worker registration:', error)
}
})
}

View File

@ -22,11 +22,11 @@
background-color: $background-color;
&:hover {
background-color: $hover-bg;
background-color: $hover-bg-2;
}
}
}
.autocomplete-active {
background-color: $selected-bg !important;
background-color: $hover-bg-2 !important;
}

View File

@ -56,6 +56,7 @@ $border-shadow-right: 2.5px 0 4px 0 $default-shadow-color;
$border-shadow-bottom-right: 2.5px 2.5px 3px 0 $default-shadow-color;
$header-shadow: 0px 1px 3px 1px #bbb !default;
$group-shadow: 3px -2px 6px 1px #98b0a0;
$primary-btn-shadow: 1px 1px 0.5px 0.75px #32b64640 !default;
//// Modals
$modal-header-bg: #e0e0e0 !default;
@ -81,6 +82,7 @@ $default-hover-fg: #35b870 !default;
$default-hover-fg-2: #38cf80 !default;
$hover-fg: $default-hover-fg !default;
$hover-bg: linear-gradient(90deg, rgba(190,246,218,1) 0%, rgba(229,251,240,1) 100%) !default;
$hover-bg-2: rgb(190,246,218) !default;
$active-bg: #8fefb7 !default;
/// Disabled

View File

@ -1,28 +1,36 @@
from collections import defaultdict
import importlib
import inspect
import json
import os
import pathlib
import pickle
import pkgutil
import threading
from typing import Optional
import platypush.backend # lgtm [py/import-and-import-from]
import platypush.plugins # lgtm [py/import-and-import-from]
import platypush.message.event # lgtm [py/import-and-import-from]
import platypush.message.response # lgtm [py/import-and-import-from]
from types import ModuleType
from typing import Callable, Dict, Generator, Optional, Type, Union
from platypush.backend import Backend
from platypush.config import Config
from platypush.plugins import Plugin, action
from platypush.message.event import Event
from platypush.message.response import Response
from platypush.utils import (
get_backend_class_by_name,
get_backend_name_by_class,
get_plugin_class_by_name,
get_plugin_name_by_class,
)
from platypush.utils.manifest import Manifest, scan_manifests
from ._context import ComponentContext
from ._model import (
BackendModel,
EventModel,
Model,
PluginModel,
ProcedureEncoder,
ResponseModel,
)
from ._serialize import ProcedureEncoder
class InspectPlugin(Plugin):
@ -32,183 +40,326 @@ class InspectPlugin(Plugin):
def __init__(self, **kwargs):
super().__init__(**kwargs)
self._plugins = {}
self._backends = {}
self._events = {}
self._responses = {}
self._plugins_lock = threading.RLock()
self._backends_lock = threading.RLock()
self._events_lock = threading.RLock()
self._responses_lock = threading.RLock()
self._html_doc = False
self._components_cache_file = os.path.join(
Config.get('workdir'), # type: ignore
'components.cache', # type: ignore
)
self._components_context: Dict[type, ComponentContext] = defaultdict(
ComponentContext
)
self._components_cache: Dict[type, dict] = defaultdict(dict)
self._load_components_cache()
def _load_components_cache(self):
"""
Loads the components cache from disk.
"""
try:
with open(self._components_cache_file, 'rb') as f:
self._components_cache = pickle.load(f)
except OSError:
return
def _flush_components_cache(self):
"""
Flush the current components cache to disk.
"""
with open(self._components_cache_file, 'wb') as f:
pickle.dump(self._components_cache, f)
def _get_cached_component(
self, base_type: type, comp_type: type
) -> Optional[Model]:
"""
Retrieve a cached component's ``Model``.
:param base_type: The base type of the component (e.g. ``Plugin`` or
``Backend``).
:param comp_type: The specific type of the component (e.g.
``MusicMpdPlugin`` or ``HttpBackend``).
:return: The cached component's ``Model`` if it exists, otherwise null.
"""
return self._components_cache.get(base_type, {}).get(comp_type)
def _cache_component(
self,
base_type: type,
comp_type: type,
model: Model,
index_by_module: bool = False,
):
"""
Cache the ``Model`` object for a component.
:param base_type: The base type of the component (e.g. ``Plugin`` or
``Backend``).
:param comp_type: The specific type of the component (e.g.
``MusicMpdPlugin`` or ``HttpBackend``).
:param model: The ``Model`` object to cache.
:param index_by_module: If ``True``, the ``Model`` object will be
indexed according to the ``base_type -> module -> comp_type``
mapping, otherwise ``base_type -> comp_type``.
"""
if index_by_module:
if not self._components_cache.get(base_type, {}).get(model.package):
self._components_cache[base_type][model.package] = {}
self._components_cache[base_type][model.package][comp_type] = model
else:
self._components_cache[base_type][comp_type] = model
def _scan_integrations(self, base_type: type):
"""
A generator that scans the manifest files given a ``base_type``
(``Plugin`` or ``Backend``) and yields the parsed submodules.
"""
for mf_file in scan_manifests(base_type):
manifest = Manifest.from_file(mf_file)
try:
yield importlib.import_module(manifest.package)
except Exception as e:
self.logger.debug(
'Could not import module %s: %s',
manifest.package,
e,
)
continue
def _scan_modules(self, base_type: type) -> Generator[ModuleType, None, None]:
"""
A generator that scan the modules given a ``base_type`` (e.g. ``Event``).
Unlike :meth:`._scan_integrations`, this method recursively scans the
modules using ``pkgutil`` instead of using the information provided in
the integrations' manifest files.
"""
prefix = base_type.__module__ + '.'
path = str(pathlib.Path(inspect.getfile(base_type)).parent)
for _, modname, _ in pkgutil.walk_packages(
path=[path], prefix=prefix, onerror=lambda _: None
):
try:
yield importlib.import_module(modname)
except Exception as e:
self.logger.debug('Could not import module %s: %s', modname, e)
continue
def _init_component(
self,
base_type: type,
comp_type: type,
model_type: Type[Model],
index_by_module: bool = False,
) -> Model:
"""
Initialize a component's ``Model`` object and cache it.
:param base_type: The base type of the component (e.g. ``Plugin`` or
``Backend``).
:param comp_type: The specific type of the component (e.g.
``MusicMpdPlugin`` or ``HttpBackend``).
:param model_type: The type of the ``Model`` object that should be
created.
:param index_by_module: If ``True``, the ``Model`` object will be
indexed according to the ``base_type -> module -> comp_type``
mapping, otherwise ``base_type -> comp_type``.
:return: The initialized component's ``Model`` object.
"""
prefix = base_type.__module__ + '.'
comp_file = inspect.getsourcefile(comp_type)
model = None
mtime = None
if comp_file:
mtime = os.stat(comp_file).st_mtime
cached_model = self._get_cached_component(base_type, comp_type)
# Only update the component model if its source file was
# modified since the last time it was scanned
if (
cached_model
and cached_model.last_modified
and mtime <= cached_model.last_modified
):
model = cached_model
if not model:
self.logger.info('Scanning component %s', comp_type.__name__)
model = model_type(comp_type, prefix=prefix, last_modified=mtime)
self._cache_component(
base_type, comp_type, model, index_by_module=index_by_module
)
return model
def _init_modules(
self,
base_type: type,
model_type: Type[Model],
):
"""
Initializes, parses and caches all the components of a given type.
Unlike :meth:`._scan_integrations`, this method inspects all the
members of a ``module`` for those that match the given ``base_type``
instead of relying on the information provided in the manifest.
It is a bit more inefficient, but it works fine for simple components
(like entities and messages) that don't require extra recursive parsing
logic for their docs (unlike plugins).
"""
for module in self._scan_modules(base_type):
for _, obj_type in inspect.getmembers(module):
if (
inspect.isclass(obj_type)
and issubclass(obj_type, base_type)
# Exclude the base_type itself
and obj_type != base_type
):
self._init_component(
base_type=base_type,
comp_type=obj_type,
model_type=model_type,
index_by_module=True,
)
def _init_integrations(
self,
base_type: Type[Union[Plugin, Backend]],
model_type: Type[Union[PluginModel, BackendModel]],
class_by_name: Callable[[str], Optional[type]],
):
"""
Initializes, parses and caches all the integrations of a given type.
:param base_type: The base type of the component (e.g. ``Plugin`` or
``Backend``).
:param model_type: The type of the ``Model`` objects that should be
created.
:param class_by_name: A function that returns the class of a given
integration given its qualified name.
"""
for module in self._scan_integrations(base_type):
comp_name = '.'.join(module.__name__.split('.')[2:])
comp_type = class_by_name(comp_name)
if not comp_type:
continue
self._init_component(
base_type=base_type,
comp_type=comp_type,
model_type=model_type,
)
self._flush_components_cache()
def _init_plugins(self):
package = platypush.plugins
prefix = package.__name__ + '.'
for _, modname, _ in pkgutil.walk_packages(
path=package.__path__, prefix=prefix, onerror=lambda _: None
):
try:
module = importlib.import_module(modname)
except Exception as e:
self.logger.warning('Could not import module %s: %s', modname, e)
continue
for _, obj in inspect.getmembers(module):
if inspect.isclass(obj) and issubclass(obj, Plugin):
model = PluginModel(
plugin=obj, prefix=prefix, html_doc=self._html_doc
)
if model.name:
self._plugins[model.name] = model
"""
Initializes and caches all the available plugins.
"""
self._init_integrations(
base_type=Plugin,
model_type=PluginModel,
class_by_name=get_plugin_class_by_name,
)
def _init_backends(self):
package = platypush.backend
prefix = package.__name__ + '.'
for _, modname, _ in pkgutil.walk_packages(
path=package.__path__, prefix=prefix, onerror=lambda _: None
):
try:
module = importlib.import_module(modname)
except Exception as e:
self.logger.debug('Could not import module %s: %s', modname, e)
continue
for _, obj in inspect.getmembers(module):
if inspect.isclass(obj) and issubclass(obj, Backend):
model = BackendModel(
backend=obj, prefix=prefix, html_doc=self._html_doc
)
if model.name:
self._backends[model.name] = model
"""
Initializes and caches all the available backends.
"""
self._init_integrations(
base_type=Backend,
model_type=BackendModel,
class_by_name=get_backend_class_by_name,
)
def _init_events(self):
package = platypush.message.event
prefix = package.__name__ + '.'
for _, modname, _ in pkgutil.walk_packages(
path=package.__path__, prefix=prefix, onerror=lambda _: None
):
try:
module = importlib.import_module(modname)
except Exception as e:
self.logger.debug('Could not import module %s: %s', modname, e)
continue
for _, obj in inspect.getmembers(module):
if type(obj) == Event: # pylint: disable=unidiomatic-typecheck
continue
if inspect.isclass(obj) and issubclass(obj, Event) and obj != Event:
event = EventModel(
event=obj, html_doc=self._html_doc, prefix=prefix
)
if event.package not in self._events:
self._events[event.package] = {event.name: event}
else:
self._events[event.package][event.name] = event
"""
Initializes and caches all the available events.
"""
self._init_modules(
base_type=Event,
model_type=EventModel,
)
def _init_responses(self):
package = platypush.message.response
prefix = package.__name__ + '.'
"""
Initializes and caches all the available responses.
"""
self._init_modules(
base_type=Response,
model_type=ResponseModel,
)
for _, modname, _ in pkgutil.walk_packages(
path=package.__path__, prefix=prefix, onerror=lambda _: None
):
try:
module = importlib.import_module(modname)
except Exception as e:
self.logger.debug('Could not import module %s: %s', modname, e)
continue
for _, obj in inspect.getmembers(module):
if type(obj) == Response: # pylint: disable=unidiomatic-typecheck
continue
if (
inspect.isclass(obj)
and issubclass(obj, Response)
and obj != Response
):
response = ResponseModel(
response=obj, html_doc=self._html_doc, prefix=prefix
)
if response.package not in self._responses:
self._responses[response.package] = {response.name: response}
else:
self._responses[response.package][response.name] = response
def _init_components(self, base_type: type, initializer: Callable[[], None]):
"""
Context manager boilerplate for the other ``_init_*`` methods.
"""
ctx = self._components_context[base_type]
with ctx.init_lock:
if not ctx.refreshed.is_set():
initializer()
ctx.refreshed.set()
@action
def get_all_plugins(self, html_doc: Optional[bool] = None):
def get_all_plugins(self):
"""
:param html_doc: If True then the docstring will be parsed into HTML (default: False)
Get information about all the available plugins.
"""
with self._plugins_lock:
if not self._plugins or (
html_doc is not None and html_doc != self._html_doc
):
self._html_doc = html_doc
self._init_plugins()
return json.dumps(
{name: dict(plugin) for name, plugin in self._plugins.items()}
)
self._init_components(Plugin, self._init_plugins)
return json.dumps(
{
get_plugin_name_by_class(cls): dict(plugin)
for cls, plugin in self._components_cache.get(Plugin, {}).items()
}
)
@action
def get_all_backends(self, html_doc: Optional[bool] = None):
def get_all_backends(self):
"""
:param html_doc: If True then the docstring will be parsed into HTML (default: False)
Get information about all the available backends.
"""
with self._backends_lock:
if not self._backends or (
html_doc is not None and html_doc != self._html_doc
):
self._html_doc = html_doc
self._init_backends()
return json.dumps(
{name: dict(backend) for name, backend in self._backends.items()}
)
self._init_components(Backend, self._init_backends)
return json.dumps(
{
get_backend_name_by_class(cls): dict(backend)
for cls, backend in self._components_cache.get(Backend, {}).items()
}
)
@action
def get_all_events(self, html_doc: Optional[bool] = None):
def get_all_events(self):
"""
:param html_doc: If True then the docstring will be parsed into HTML (default: False)
Get information about all the available events.
"""
with self._events_lock:
if not self._events or (
html_doc is not None and html_doc != self._html_doc
):
self._html_doc = html_doc
self._init_events()
return json.dumps(
{
package: {name: dict(event) for name, event in events.items()}
for package, events in self._events.items()
self._init_components(Event, self._init_events)
return json.dumps(
{
package: {
obj_type.__name__: dict(event_model)
for obj_type, event_model in events.items()
}
)
for package, events in self._components_cache.get(Event, {}).items()
}
)
@action
def get_all_responses(self, html_doc: Optional[bool] = None):
def get_all_responses(self):
"""
:param html_doc: If True then the docstring will be parsed into HTML (default: False)
Get information about all the available responses.
"""
with self._responses_lock:
if not self._responses or (
html_doc is not None and html_doc != self._html_doc
):
self._html_doc = html_doc
self._init_responses()
return json.dumps(
{
package: {name: dict(event) for name, event in responses.items()}
for package, responses in self._responses.items()
self._init_components(Response, self._init_responses)
return json.dumps(
{
package: {
obj_type.__name__: dict(response_model)
for obj_type, response_model in responses.items()
}
)
for package, responses in self._components_cache.get(
Response, {}
).items()
}
)
@action
def get_procedures(self) -> dict:
@ -228,8 +379,7 @@ class InspectPlugin(Plugin):
if entry:
return Config.get(entry)
cfg = Config.get()
return cfg
return Config.get()
# vim:sw=4:ts=4:et:

View File

@ -0,0 +1,12 @@
from dataclasses import dataclass, field
import threading
@dataclass
class ComponentContext:
"""
This class is used to store the context of a component type.
"""
init_lock: threading.RLock = field(default_factory=threading.RLock)
refreshed: threading.Event = field(default_factory=threading.Event)

View File

@ -1,123 +1,131 @@
from abc import ABC, abstractmethod
import inspect
import json
import re
from typing import Optional
from typing import Optional, Type
from platypush.backend import Backend
from platypush.message.event import Event
from platypush.message.response import Response
from platypush.plugins import Plugin
from platypush.utils import get_decorators
class Model(ABC):
class Model:
"""
Base class for component models.
"""
def __init__(
self,
obj_type: type,
name: Optional[str] = None,
doc: Optional[str] = None,
prefix: str = '',
last_modified: Optional[float] = None,
) -> None:
"""
:param obj_type: Type of the component.
:param name: Name of the component.
:param doc: Documentation of the component.
:param last_modified: Last modified timestamp of the component.
"""
self._obj_type = obj_type
self.package = obj_type.__module__[len(prefix) :]
self.name = name or self.package
self.doc = doc or obj_type.__doc__
self.last_modified = last_modified
def __str__(self):
"""
:return: JSON string representation of the model.
"""
return json.dumps(dict(self), indent=2, sort_keys=True)
def __repr__(self):
"""
:return: JSON string representation of the model.
"""
return json.dumps(dict(self))
@staticmethod
def to_html(doc):
try:
import docutils.core # type: ignore
except ImportError:
# docutils not found
return doc
return docutils.core.publish_parts(doc, writer_name='html')['html_body']
@abstractmethod
def __iter__(self):
raise NotImplementedError()
class ProcedureEncoder(json.JSONEncoder):
def default(self, o):
if callable(o):
return {
'type': 'native_function',
}
return super().default(o)
class BackendModel(Model):
def __init__(self, backend, prefix='', html_doc: Optional[bool] = False):
self.name = backend.__module__[len(prefix) :]
self.html_doc = html_doc
self.doc = (
self.to_html(backend.__doc__)
if html_doc and backend.__doc__
else backend.__doc__
)
def __iter__(self):
for attr in ['name', 'doc', 'html_doc']:
"""
Iterator for the model public attributes/values pairs.
"""
for attr in ['name', 'doc']:
yield attr, getattr(self, attr)
# pylint: disable=too-few-public-methods
class BackendModel(Model):
"""
Model for backend components.
"""
def __init__(self, obj_type: Type[Backend], *args, **kwargs):
super().__init__(obj_type, *args, **kwargs)
# pylint: disable=too-few-public-methods
class PluginModel(Model):
def __init__(self, plugin, prefix='', html_doc: Optional[bool] = False):
self.name = plugin.__module__[len(prefix) :]
self.html_doc = html_doc
self.doc = (
self.to_html(plugin.__doc__)
if html_doc and plugin.__doc__
else plugin.__doc__
"""
Model for plugin components.
"""
def __init__(self, obj_type: Type[Plugin], prefix: str = '', **kwargs):
super().__init__(
obj_type,
name=re.sub(r'\._plugin$', '', obj_type.__module__[len(prefix) :]),
**kwargs,
)
self.actions = {
action_name: ActionModel(
getattr(plugin, action_name), html_doc=html_doc or False
)
for action_name in get_decorators(plugin, climb_class_hierarchy=True).get(
action_name: ActionModel(getattr(obj_type, action_name))
for action_name in get_decorators(obj_type, climb_class_hierarchy=True).get(
'action', []
)
}
def __iter__(self):
for attr in ['name', 'actions', 'doc', 'html_doc']:
"""
Overrides the default implementation of ``__iter__`` to also include
plugin actions.
"""
for attr in ['name', 'actions', 'doc']:
if attr == 'actions':
# noinspection PyShadowingNames
yield attr, {
name: dict(action) for name, action in self.actions.items()
},
}
else:
yield attr, getattr(self, attr)
class EventModel(Model):
def __init__(self, event, prefix='', html_doc: Optional[bool] = False):
self.package = event.__module__[len(prefix) :]
self.name = event.__name__
self.html_doc = html_doc
self.doc = (
self.to_html(event.__doc__) if html_doc and event.__doc__ else event.__doc__
)
"""
Model for event components.
"""
def __iter__(self):
for attr in ['name', 'doc', 'html_doc']:
yield attr, getattr(self, attr)
def __init__(self, obj_type: Type[Event], **kwargs):
super().__init__(obj_type, **kwargs)
class ResponseModel(Model):
def __init__(self, response, prefix='', html_doc: Optional[bool] = False):
self.package = response.__module__[len(prefix) :]
self.name = response.__name__
self.html_doc = html_doc
self.doc = (
self.to_html(response.__doc__)
if html_doc and response.__doc__
else response.__doc__
)
"""
Model for response components.
"""
def __iter__(self):
for attr in ['name', 'doc', 'html_doc']:
yield attr, getattr(self, attr)
def __init__(self, obj_type: Type[Response], **kwargs):
super().__init__(obj_type, **kwargs)
class ActionModel(Model):
# noinspection PyShadowingNames
def __init__(self, action, html_doc: bool = False):
self.name = action.__name__
self.doc, argsdoc = self._parse_docstring(action.__doc__, html_doc=html_doc)
"""
Model for plugin action components.
"""
def __init__(self, action, **kwargs):
doc, argsdoc = self._parse_docstring(action.__doc__)
super().__init__(action, name=action.__name__, doc=doc, **kwargs)
self.args = {}
self.has_kwargs = False
@ -134,7 +142,7 @@ class ActionModel(Model):
}
@classmethod
def _parse_docstring(cls, docstring: str, html_doc: bool = False):
def _parse_docstring(cls, docstring: str):
new_docstring = ''
params = {}
cur_param = None
@ -147,11 +155,7 @@ class ActionModel(Model):
m = re.match(r'^\s*:param ([^:]+):\s*(.*)', line)
if m:
if cur_param:
params[cur_param] = (
cls.to_html(cur_param_docstring)
if html_doc
else cur_param_docstring
)
params[cur_param] = cur_param_docstring
cur_param = m.group(1)
cur_param_docstring = m.group(2)
@ -160,11 +164,7 @@ class ActionModel(Model):
else:
if cur_param:
if not line.strip():
params[cur_param] = (
cls.to_html(cur_param_docstring)
if html_doc
else cur_param_docstring
)
params[cur_param] = cur_param_docstring
cur_param = None
cur_param_docstring = ''
else:
@ -173,14 +173,9 @@ class ActionModel(Model):
new_docstring += line.rstrip() + '\n'
if cur_param:
params[cur_param] = (
cls.to_html(cur_param_docstring) if html_doc else cur_param_docstring
)
params[cur_param] = cur_param_docstring
return (
new_docstring.strip() if not html_doc else cls.to_html(new_docstring),
params,
)
return new_docstring.strip(), params
def __iter__(self):
for attr in ['name', 'args', 'doc', 'has_kwargs']:

View File

@ -0,0 +1,23 @@
import inspect
import json
class ProcedureEncoder(json.JSONEncoder):
"""
Encoder for the Procedure model.
"""
def default(self, o):
if callable(o):
return {
'type': 'native_function',
'module': o.__module__,
'source': inspect.getsourcefile(o),
'args': [
name
for name, arg in inspect.signature(o).parameters.items()
if arg.kind != arg.VAR_KEYWORD
],
}
return super().default(o)

View File

@ -344,5 +344,24 @@ class UtilsPlugin(Plugin):
return plugins
@action
def rst_to_html(self, text: str):
"""
Utility action to convert RST to HTML.
It is mostly used by the frontend to render the docstring of the
available plugins and actions.
"""
try:
import docutils.core # type: ignore
except ImportError:
self.logger.warning(
"docutils is not installed. "
"Please install docutils to convert RST to HTML."
)
return text
return docutils.core.publish_parts(text, writer_name='html')['html_body']
# vim:sw=4:ts=4:et:

View File

@ -73,6 +73,17 @@ def get_plugin_module_by_name(plugin_name):
return None
def get_backend_module_by_name(backend_name):
"""Gets the module of a backend by name (e.g. "backend.http" or "backend.mqtt")"""
module_name = 'platypush.backend.' + backend_name
try:
return importlib.import_module('platypush.backend.' + backend_name)
except ImportError as e:
logger.error('Cannot import %s: %s', module_name, e)
return None
def get_plugin_class_by_name(plugin_name):
"""Gets the class of a plugin by name (e.g. "music.mpd" or "media.vlc")"""
@ -110,6 +121,34 @@ def get_plugin_name_by_class(plugin) -> Optional[str]:
return '.'.join(class_tokens)
def get_backend_class_by_name(backend_name: str):
"""Gets the class of a backend by name (e.g. "backend.http" or "backend.mqtt")"""
module = get_backend_module_by_name(backend_name)
if not module:
return
class_name = getattr(
module,
''.join(
[
token.capitalize()
for i, token in enumerate(backend_name.split('.'))
if not (i == 0 and token == 'backend')
]
)
+ 'Backend',
)
try:
return getattr(
module,
''.join([_.capitalize() for _ in backend_name.split('.')]) + 'Backend',
)
except Exception as e:
logger.error('Cannot import class %s: %s', class_name, e)
return None
def get_backend_name_by_class(backend) -> Optional[str]:
"""Gets the common name of a backend (e.g. "http" or "mqtt") given its class."""