platypush/platypush
platypush
/
platypush
2
2
Fork 4
Code Issues 43 Pull Requests Packages Projects 4 Releases Wiki Activity

Merge branch 'master' into snyk-upgrade-7f1672a9074c3d844aa231ad5ba0e90d

This commit is contained in:
Fabio Manganiello 2023-11-03 22:11:01 +01:00 committed by GitHub
parent 26c9694664 4a43f5ebf6
commit 8acb4156e0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
885 changed files with 26631 additions and 20339 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.dockerignore Normal file
View File

@ -0,0 +1,6 @@
**/.git
**/node_modules
**/__pycache__
**/venv
**/.mypy_cache
**/build

1012
.drone.yml
View File

File diff suppressed because it is too large Load Diff

4
.gitignore vendored
View File

@ -10,7 +10,7 @@ package.sh
platypush/backend/http/static/resources/*
docs/build
.idea/
config
/config
platypush/backend/http/static/css/*/.sass-cache/
.vscode
platypush/backend/http/static/js/lib/vue.js
@ -24,3 +24,5 @@ coverage.xml
Session.vim
/jsconfig.json
/package.json
/Dockerfile
/docs/source/wiki

1
MANIFEST.in
View File

@ -1,4 +1,5 @@
recursive-include platypush/backend/http/webapp/dist *
recursive-include platypush/install *
include platypush/plugins/http/webpage/mercury-parser.js
include platypush/config/*.yaml
global-include manifest.yaml

145
README.md
View File

@ -2,7 +2,6 @@ Platypush
=========
[![Build Status](https://ci-cd.platypush.tech/api/badges/platypush/platypush/status.svg)](https://ci-cd.platypush.tech/platypush/platypush)
[![Documentation Status](https://ci.platypush.tech/docs/status.svg)](https://ci.platypush.tech/docs/latest.log)
[![pip version](https://img.shields.io/pypi/v/platypush.svg?style=flat)](https://pypi.python.org/pypi/platypush/)
[![License](https://img.shields.io/github/license/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/src/branch/master/LICENSE.txt)
[![Last Commit](https://img.shields.io/github/last-commit/BlackLight/platypush.svg)](https://git.platypush.tech/platypush/platypush/commits/branch/master)
@ -11,20 +10,22 @@ Platypush
<!-- toc -->
- [Useful links](#useful-links)
- [Introduction](#introduction)
+ [What it can do](#what-it-can-do)
- [Installation](#installation)
* [System installation](#system-installation)
+ [Install through `pip`](#install-through-pip)
+ [Install through a system package manager](#install-through-a-system-package-manager)
+ [Install from sources](#install-from-sources)
* [Prerequisites](#prerequisites)
+ [Docker installation](#docker-installation)
+ [Use an external service](#use-an-external-service)
+ [Manual installation](#manual-installation)
* [Install through `pip`](#install-through-pip)
* [Install through a system package manager](#install-through-a-system-package-manager)
* [Install from sources](#install-from-sources)
* [Installing the dependencies for your extensions](#installing-the-dependencies-for-your-extensions)
+ [Install via `extras` name](#install-via-extras-name)
+ [Install via `manifest.yaml`](#install-via-manifestyaml)
+ [Check the instructions reported in the documentation](#check-the-instructions-reported-in-the-documentation)
* [Virtual environment installation](#virtual-environment-installation)
* [Docker installation](#docker-installation)
* [Docker installation](#docker-installation-1)
- [Architecture](#architecture)
* [Plugins](#plugins)
* [Actions](#actions)
@ -44,31 +45,11 @@ Platypush
+ [PWA support](#pwa-support)
- [Mobile app](#mobile-app)
- [Tests](#tests)
- [Useful links](#useful-links)
- [Funding](#funding)
<!-- tocstop -->
## Useful links
- Recommended read: [**Getting started with Platypush**](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush).
- The [blog](https://blog.platypush.tech) is a good place to get more insights
and inspiration on what you can build.
- The [wiki](https://git.platypush.tech/platypush/platypush/wiki) also
contains many resources on getting started.
- Extensive documentation for all the available integrations and messages [is
available](https://docs.platypush.tech/).
- If you have issues/feature requests/enhancements please [create an
issue](https://git.platypush.tech/platypush/platypush/issues).
- A [Matrix instance](https://matrix.to/#/#platypush:matrix.platypush.tech) is
available if you are looking for interactive support.
- A [Lemmy instance](https://lemmy.platypush.tech/c/platypush) is available for
general questions.
## Introduction
Platypush is a general-purpose extensible platform for automation across
@ -124,26 +105,82 @@ You can use Platypush to do things like:
## Installation
### System installation
### Prerequisites
Platypush uses Redis to deliver and store requests and temporary messages:
Platypush uses [Redis](https://redis.io/) to dispatch requests, responses,
events and custom messages across several processes and integrations.
#### Docker installation
You can run Redis on the fly on your local machine using a Docker image:
```bash
# Expose a Redis server on port 6379 (default)
docker run --rm -p 6379:6379 --name redis redis
```
#### Use an external service
You can let Platypush use an external Redis service, if you wish to avoid
running one on the same machine.
In such scenario, simply start the application by passing custom values for
`--redis-host` and `--redis-port`, or configure these values in its
configuration file:
```yaml
# Example for Debian-based distributions
[sudo] apt-get install redis-server
redis:
host: some-ip
port: some-port
```
If you wish to run multiple instances that use the same Redis server, you may
also want to customize the name of the default queue that they use
(`--redis-queue` command-line option) in order to avoid conflicts.
#### Manual installation
Unless you are running Platypush in a Docker container, or you are running
Redis in a Docker container, or you want to use a remote Redis service, the
Redis server should be installed on the same machine where Platypush runs:
```bash
# On Debian-based distributions
sudo apt install redis-server
# On Arch-based distributions
# The hiredis package is also advised
sudo pacman -S redis
# On MacOS
brew install redis
```
Once Redis is installed, you have two options:
1. Run it a separate service. This depends on your operating system and
supervisor/service controller. For example, on systemd:
```bash
# Enable and start the service
[sudo] systemctl enable redis
[sudo] systemctl start redis
sudo systemctl enable redis
sudo systemctl start redis
```
#### Install through `pip`
2. Let Platypush run and control the Redis service. This is a good option if
you want Platypush to run its own service, separate from any other one
running on the same machine, and terminate it as soon as the application
ends. In this case, simply launch the application with the `--start-redis`
option (and optionally `--redis-port <any-num>` to customize the listen
port).
```shell
[sudo] pip3 install platypush
### Install through `pip`
```bash
[sudo] pip install platypush
```
#### Install through a system package manager
### Install through a system package manager
Note: currently only Arch Linux and derived distributions are supported.
@ -154,7 +191,7 @@ latest stable version) or the
(for the latest git version) through your favourite AUR package manager. For
example, using `yay`:
```shell
```bash
yay platypush
# Or
yay platypush-git
@ -163,14 +200,12 @@ yay platypush-git
The Arch Linux packages on AUR are automatically updated upon new git commits
or tags.
#### Install from sources
### Install from sources
```shell
git clone https://git.platypush.tech/platypush/platypush.git
cd platypush
[sudo] pip install .
# Or
[sudo] python3 setup.py install
```
### Installing the dependencies for your extensions
@ -224,6 +259,8 @@ You can then start the service by simply running:
platypush
```
See `platypush --help` for a full list of options.
It's advised to run it as a systemd service though - simply copy the provided
[`.service`
file](https://git.platypush.tech/platypush/platypush/src/branch/master/examples/systemd/platypush.service)
@ -751,6 +788,30 @@ of Platypush to your fingertips.
To run the tests simply run `pytest` either from the project root folder or the
`tests/` folder.
## Useful links
- Recommended read: [**Getting started with Platypush**](https://blog.platypush.tech/article/Ultimate-self-hosted-automation-with-Platypush).
- The [blog](https://blog.platypush.tech) is a good place to get more insights
and inspiration on what you can build.
- The [wiki](https://git.platypush.tech/platypush/platypush/wiki) also
contains many resources on getting started.
- Extensive documentation for all the available integrations and messages [is
available](https://docs.platypush.tech/).
- If you have issues/feature requests/enhancements please [create an
issue](https://git.platypush.tech/platypush/platypush/issues).
- A [Matrix instance](https://matrix.to/#/#platypush:matrix.platypush.tech) is
available if you are looking for interactive support.
- An IRC channel is also available at `#platypush@irc.platypush.tech:6697` (SSL
only).
- A [Lemmy instance](https://lemmy.platypush.tech/c/platypush) is available for
general questions.
---
## Funding

13
bin/platypush
View File

@ -1,13 +0,0 @@
#!python3
import sys
from platypush.app import Application
if __name__ == '__main__':
app = Application.build(*sys.argv[1:])
app.run()
# vim:sw=4:ts=4:et:

249
bin/platyvenv
View File

@ -1,249 +0,0 @@
#!/bin/bash
##############################################################################
# This script allows you to easily manage Platypush instances through Python #
# virtual environment. You can build environments from a config.yaml file #
# and automatically managed the required dependencies, as well as start, #
# stop and remove them #
# #
# @author: Fabio Manganiello <fabio@platypush.tech> #
# @licence: MIT #
##############################################################################
workdir="$HOME/.local/share/platypush/venv"
function build {
cfgfile=
while getopts ':c:' opt; do
case ${opt} in
c)
cfgfile=$OPTARG;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1;;
:)
echo "Option -$OPTARG requires the path to a Platypush configuration file" >&2
exit 1;;
esac
done
if [[ -z "$cfgfile" ]]; then
echo "Usage: $0 build -c <path-to-platypush-config-file>" >&2
exit 1
fi
echo "Parsing configuration file"
pip_cmd=
pkg_cmd=
includes=()
cmd_exec=()
while read -r line; do
if echo "$line" | grep -E "^pip:\s*"; then
pip_cmd="$(echo "$line" | sed -r -e 's/^pip:\s*(.*)'/\\1/)"
elif echo "$line" | grep -E "^packages:\s*"; then
pkg_cmd="$(echo "$line" | sed -r -e 's/^packages:\s*(.*)'/\\1/)"
elif echo "$line" | grep -E "^exec:\s*"; then
cmd_exec+=("$(echo "$line" | sed -r -e 's/^exec:\s*(.*)'/\\1/)")
elif echo "$line" | grep -E "^include:\s*"; then
includes+=("$(echo "$line" | sed -r -e 's/^include:\s*(.*)'/\\1/)")
elif echo "$line" | grep -E "^device_id:\s*"; then
device_id="$(echo "$line" | sed -r -e 's/^device_id:\s*(.*)'/\\1/)"
fi
done <<< "$(python <<EOF
from platypush.config import Config
from platypush.utils.manifest import get_install_commands_from_conf
deps = get_install_commands_from_conf('$(realpath "${cfgfile}")')
print(f'device_id: {Config.get("device_id")}')
if deps.get('pip'):
print(f'pip: {deps["pip"]}')
if deps.get('packages'):
print(f'packages: {deps["packages"]}')
for cmd in deps.get('exec', []):
print(f'exec: {cmd}')
for include in Config._included_files:
print(f'include: {include}')
EOF
)"
envdir="${workdir}/${device_id}"
etcdir="${envdir}/etc/platypush"
echo "Preparing virtual environment for device $device_id"
mkdir -p "$envdir"
mkdir -p "$etcdir"
srcdir=$(dirname "$cfgfile")
for ((i=0; i < ${#includes[@]}; i++)); do
incdir=$(dirname "${includes[$i]}")
incdir=$(realpath --relative-to="$srcdir" "$incdir")
destdir="$etcdir/$incdir"
mkdir -p "$destdir"
cp "${includes[$i]}" "$destdir"
done
cp "$cfgfile" "$etcdir/config.yaml"
cfgfile="${etcdir}/config.yaml"
python3 -m venv "${envdir}"
cd "${envdir}" || exit 1
source "${envdir}/bin/activate"
echo "Installing required dependencies"
# shellcheck disable=SC2086
[ -n "${pkg_cmd}" ] && sudo ${pkg_cmd}
[ -n "${pip_cmd}" ] && ${pip_cmd}
for ((i=0; i < ${#cmd_exec[@]}; i++)); do
${cmd_exec[$i]}
done
pip install --upgrade git+https://git.platypush.tech/platypush/platypush.git
echo "Platypush virtual environment prepared under $envdir"
}
function start {
if [[ -z "$1" ]]; then
echo "Usage: $0 start <env-name>" >&2
exit 1
fi
env=$1
envdir="${workdir}/${env}"
rundir="${envdir}/var/run"
pidfile="${rundir}/platypush.pid"
cfgfile="${envdir}/etc/platypush/config.yaml"
if [[ ! -d "$envdir" ]]; then
echo "No such directory: $envdir" >&2
exit 1
fi
mkdir -p "${rundir}"
if [[ -f "$pidfile" ]]; then
if pgrep -F "${pidfile}"; then
echo "Another instance (PID $(cat "${pidfile}")) is running, please stop that instance first"
exit 1
fi
echo "A PID file was found but the process does not seem to be running, starting anyway"
rm -f "$pidfile"
fi
python3 -m venv "${envdir}"
cd "${envdir}" || exit 1
source bin/activate
bin/platypush -c "$cfgfile" -P "$pidfile" &
start_time=$(date +'%s')
timeout=30
while :; do
[[ -f "$pidfile" ]] && break
now=$(date +'%s')
elapsed=$(( now-start_time ))
if (( elapsed >= timeout )); then
echo "Platypush instance '$env' did not start within $timeout seconds" >&2
exit 1
fi
echo -n '.'
sleep 1
done
pid=$(cat "$pidfile")
echo
echo "Platypush environment $env started with PID $pid"
wait "${pid}"
echo "Platypush environment $env terminated"
}
function stop {
if [[ -z "$1" ]]; then
echo "Usage: $0 stop <env-name>" >&2
exit 1
fi
env=$1
envdir="${workdir}/${env}"
rundir="${envdir}/var/run"
pidfile="${rundir}/platypush.pid"
if [[ ! -d "$envdir" ]]; then
echo "No such directory: $envdir" >&2
exit 1
fi
if [[ ! -f "$pidfile" ]]; then
echo "No pidfile found for instance \"${env}\""
exit 1
fi
pid=$(cat "$pidfile")
pids="$pid $(ps --no-headers -o pid= --ppid "$pid")"
# shellcheck disable=SC2086
kill -9 ${pids}
rm -f "$pidfile"
echo "Instance '$env' with PID $pid stopped"
}
function rme {
if [[ -z "$1" ]]; then
echo "Usage: $0 rm <env-name>" >&2
exit 1
fi
envdir="${workdir}/$1"
rundir="${envdir}/var/run"
pidfile="${rundir}/platypush.pid"
if [[ ! -d "$envdir" ]]; then
echo "No such directory: $envdir" >&2
exit 1
fi
if [[ -f "$pidfile" ]]; then
if pgrep -F "${pidfile}"; then
echo "Another instance (PID $(cat "$pidfile")) is running, please stop that instance first"
exit 1
fi
echo "A PID file was found but the process does not seem to be running, removing anyway"
fi
echo "WARNING: This operation will permanently remove the Platypush environment $1"
echo -n "Are you sure you want to continue? (y/N) "
IFS= read -r answer
echo "$answer" | grep -E '^[yY]' >/dev/null || exit 0
rm -rf "$envdir"
echo "$envdir removed"
}
function usage {
echo "Usage: $0 <build|start|stop|rm> [options]" >&2
exit 1
}
if (( $# < 1 )); then
usage
fi
action=$1
shift
mkdir -p "${workdir}"
# shellcheck disable=SC2048,SC2086
case ${action} in
'build') build $*;;
'start') start $*;;
'stop') stop $*;;
'rm') rme $*;;
*) usage;;
esac

186
docs/source/_ext/add_dependencies.py Normal file
View File

@ -0,0 +1,186 @@
import inspect
import os
import re
import sys
import textwrap as tw
from sphinx.application import Sphinx
base_path = os.path.abspath(
os.path.join(os.path.dirname(os.path.relpath(__file__)), '..', '..', '..')
)
sys.path.insert(0, base_path)
from platypush.common.reflection import Integration # noqa: E402
from platypush.utils import get_plugin_name_by_class # noqa: E402
from platypush.utils.mock import auto_mocks # noqa: E402
class IntegrationEnricher:
@staticmethod
def add_events(source: list[str], manifest: Integration, idx: int) -> int:
if not manifest.events:
return idx
source.insert(
idx,
'Triggered events\n----------------\n\n'
+ '\n'.join(
f'\t- :class:`{event.__module__}.{event.__qualname__}`'
for event in manifest.events
)
+ '\n\n',
)
return idx + 1
@staticmethod
def add_actions(source: list[str], manifest: Integration, idx: int) -> int:
if not (manifest.actions and manifest.cls):
return idx
source.insert(
idx,
'Actions\n-------\n\n'
+ '\n'.join(
f'\t- `{get_plugin_name_by_class(manifest.cls)}.{action} '
+ f'<#{manifest.cls.__module__}.{manifest.cls.__qualname__}.{action}>`_'
for action in sorted(manifest.actions.keys())
)
+ '\n\n',
)
return idx + 1
@staticmethod
def _shellify(title: str, cmd: str) -> str:
return (
f'**{title}**\n\n'
+ '.. code-block:: bash\n\n'
+ tw.indent(cmd, '\t')
+ '\n\n'
)
@classmethod
def add_install_deps(
cls, source: list[str], manifest: Integration, idx: int
) -> int:
deps = manifest.deps
parsed_deps = {
'before': deps.before,
'pip': deps.pip,
'after': deps.after,
}
if not (any(parsed_deps.values()) or deps.by_pkg_manager):
return idx
source.insert(idx, 'Dependencies\n------------\n\n')
idx += 1
if parsed_deps['before']:
source.insert(idx, cls._shellify('Pre-install', '\n'.join(deps.before)))
idx += 1
if parsed_deps['pip']:
source.insert(
idx, cls._shellify('pip', 'pip install ' + ' '.join(deps.pip))
)
idx += 1
for pkg_manager, sys_deps in deps.by_pkg_manager.items():
if not sys_deps:
continue
source.insert(
idx,
cls._shellify(
pkg_manager.value.default_os.value.description,
pkg_manager.value.install_doc + ' ' + ' '.join(sys_deps),
),
)
idx += 1
if parsed_deps['after']:
source.insert(idx, cls._shellify('Post-install', '\n'.join(deps.after)))
idx += 1
return idx
@classmethod
def add_description(cls, source: list[str], manifest: Integration, idx: int) -> int:
docs = (
doc
for doc in (
inspect.getdoc(manifest.cls) or '',
manifest.constructor.doc if manifest.constructor else '',
)
if doc
)
if not docs:
return idx
docstring = '\n\n'.join(docs)
source.insert(idx, f"Description\n-----------\n\n{docstring}\n\n")
return idx + 1
@classmethod
def add_conf_snippet(
cls, source: list[str], manifest: Integration, idx: int
) -> int:
source.insert(
idx,
tw.dedent(
f"""
Configuration
-------------
.. code-block:: yaml
{tw.indent(manifest.config_snippet, ' ')}
"""
),
)
return idx + 1
def __call__(self, _: Sphinx, doc: str, source: list[str]):
if not (source and re.match(r'^platypush/(backend|plugins)/.*', doc)):
return
src = [src.split('\n') for src in source][0]
if len(src) < 3:
return
manifest_file = os.path.join(
base_path,
*doc.split(os.sep)[:-1],
*doc.split(os.sep)[-1].split('.'),
'manifest.yaml',
)
if not os.path.isfile(manifest_file):
return
with auto_mocks():
manifest = Integration.from_manifest(manifest_file)
idx = self.add_description(src, manifest, idx=3)
idx = self.add_conf_snippet(src, manifest, idx=idx)
idx = self.add_install_deps(src, manifest, idx=idx)
idx = self.add_events(src, manifest, idx=idx)
idx = self.add_actions(src, manifest, idx=idx)
src.insert(idx, '\n\nModule reference\n----------------\n\n')
source[0] = '\n'.join(src)
def setup(app: Sphinx):
app.connect('source-read', IntegrationEnricher())
return {
'version': '0.1',
'parallel_read_safe': True,
'parallel_write_safe': True,
}

196
docs/source/_static/scripts/custom.js Normal file
View File

@ -0,0 +1,196 @@
const processList = (list, level, addTitle) => {
const title = list.parentElement.querySelector('a')
list.classList.add('grid')
if (addTitle)
title.classList.add('grid-title')
list.querySelectorAll(`li.toctree-l${level}`).forEach((item) => {
const link = item.querySelector('a')
if (link) {
item.style.cursor = 'pointer'
item.addEventListener('click', () => link.click())
}
const name = item.querySelector('a').innerText
const img = document.createElement('img')
img.src = `https://static.platypush.tech/icons/${name.toLowerCase()}-64.png`
img.alt = ' '
item.prepend(img)
})
}
const addClipboard = (parent) => {
const pre = parent.tagName === 'PRE' ? parent : parent.querySelector('pre')
if (!pre)
return
const clipboard = document.createElement('i')
const setClipboard = (img, text) => {
clipboard.innerHTML = `<img src="https://static.platypush.tech/icons/${img}-64.png" alt="${text}">`
}
clipboard.classList.add('clipboard')
setClipboard('clipboard-bw', 'Copy')
clipboard.onclick = () => {
if (navigator && navigator.clipboard && navigator.clipboard.writeText) {
setClipboard('ok', 'Copied!')
setTimeout(() => setClipboard('clipboard-bw', 'Copy'), 2000)
return navigator.clipboard.writeText(pre.innerText.trim())
}
return Promise.reject('The Clipboard API is not available.');
}
pre.style.position = 'relative'
pre.appendChild(clipboard)
}
const Tabs = () => {
let selectedTab = null
let parent = null
let data = {}
const init = (obj) => {
data = obj
if (Object.keys(data).length && selectedTab == null)
selectedTab = Object.keys(data)[0]
}
const select = (name) => {
if (!parent) {
console.warn('Cannot select tab: parent not set')
return
}
if (!data[name]) {
console.warn(`Cannot select tab: invalid name: ${name}`)
return
}
const tabsBody = parent.querySelector('.body')
selectedTab = name
tabsBody.innerHTML = data[selectedTab]
parent.querySelectorAll('.tabs li').forEach(
(tab) => tab.classList.remove('selected')
)
const tab = [...parent.querySelectorAll('.tabs li')].find(
(t) => t.innerText === name
)
if (!tab) {
console.warn(`Cannot select tab: invalid name: ${name}`)
return
}
addClipboard(tabsBody)
tab.classList.add('selected')
}
const mount = (p) => {
const tabs = document.createElement('div')
tabs.classList.add('tabs')
parent = p
const tabsList = document.createElement('ul')
Object.keys(data).forEach((title) => {
const tab = document.createElement('li')
tab.innerText = title
tab.onclick = (event) => {
event.stopPropagation()
select(title)
},
tabsList.appendChild(tab)
})
const tabsBody = document.createElement('div')
tabsBody.classList.add('body')
tabs.appendChild(tabsList)
tabs.appendChild(tabsBody)
parent.innerHTML = ''
parent.appendChild(tabs)
select(selectedTab)
}
return {
init,
select,
mount,
}
}
const depsTabs = Tabs()
const convertDepsToTabs = () => {
const depsContainer = document.getElementById('dependencies')
if (!depsContainer)
return
const blocks = [...depsContainer.querySelectorAll('.highlight-bash')].map((block) => block.outerHTML)
const titles = [...depsContainer.querySelectorAll('p strong')].map((title) => title.innerText)
if (!(blocks.length && titles.length && blocks.length === titles.length))
return
const title = depsContainer.querySelector('h2')
const tabsData = titles.reduce((obj, title, i) => {
obj[title] = blocks[i]
return obj
}, {})
depsTabs.init(tabsData)
depsTabs.mount(depsContainer)
depsContainer.prepend(title)
}
const generateComponentsGrid = () => {
const tocWrappers = document.querySelectorAll('.toctree-wrapper.compound')
if (!tocWrappers.length) {
return
}
if (window.location.pathname.endsWith('/index.html')) {
if (tocWrappers.length < 2) {
return
}
const referenceLists = [
...tocWrappers[1].querySelectorAll('ul li.toctree-l1 ul')
].slice(0, 4)
referenceLists.forEach((list) => processList(list, 2, true))
} else if (window.location.pathname.endsWith('/plugins.html') || window.location.pathname.endsWith('/backends.html')) {
if (tocWrappers.length < 1) {
return
}
const list = tocWrappers[0].querySelector('ul')
if (list)
processList(list, 1, false)
}
}
const addClipboardToCodeBlocks = () => {
document.querySelectorAll('pre').forEach((pre) => addClipboard(pre))
}
const renderActionsList = () => {
const actionsList = document.getElementById('actions')?.querySelector('ul')
if (!actionsList)
return
[...actionsList.querySelectorAll('li')].forEach((li) => {
const link = li.querySelector('a')
link.innerHTML = `<code class="docutils literal notranslate"><span class="pre">${link.innerText}</span></code>`
})
}
document.addEventListener("DOMContentLoaded", function() {
generateComponentsGrid()
convertDepsToTabs()
addClipboardToCodeBlocks()
renderActionsList()
})

130
docs/source/_static/styles/custom.css Normal file
View File

@ -0,0 +1,130 @@
a, a:visited {
/* Don't change the color for visited links */
color: var(--pst-color-link) !important;
}
ul.grid {
display: grid;
@media screen and (max-width: 500px) {
grid-template-columns: repeat(1, 1fr);
}
@media screen and (min-width: 501px) and (max-width: 699px) {
grid-template-columns: repeat(2, 1fr);
}
@media screen and (min-width: 700px) {
grid-template-columns: repeat(3, 1fr);
}
}
a.grid-title {
width: 100%;
display: block;
margin: 1.5em 0;
font-size: 1.5em !important;
border-bottom: 1px solid #ccc;
}
ul.grid li {
display: flex;
align-items: center;
margin: 0 10px 10px 0;
padding: 10px;
border: 1px solid #ccc;
border-radius: 15px;
}
ul.grid img {
width: 32px;
margin-right: 5px;
}
ul.grid li code {
width: 100%;
}
ul.grid li code .pre {
width: 100%;
display: block;
white-space: pre-wrap;
}
ul.grid li:hover {
background: linear-gradient(0deg, #e0ffe8, #e3ffff);
}
ul.grid li a {
width: calc(100% - 35px);
display: flex;
justify-content: center;
}
ul.grid li a code {
background: none;
border: none;
}
ul.grid .icon {
width: 32px;
}
/* Clipboard button */
.clipboard {
position: absolute;
display: inline-block;
width: 32px;
top: 0.5em;
right: 0.5em;
cursor: pointer;
}
/* Tabs */
.tabs {
margin: 0 0 1em 0;
padding: 0;
list-style: none;
}
.tabs ul {
display: flex;
flex-direction: row;
justify-content: space-between;
align-items: center;
margin: 0 0 1em 0;
padding: 0;
list-style: none;
border-bottom: 1px solid #ccc;
}
.tabs ul li {
display: inline-flex;
max-width: 25%;
margin: 0;
padding: 0.25em 0.5em;
list-style: none;
cursor: pointer;
flex-grow: 1;
justify-content: center;
align-items: center;
border-radius: 0.75em 0.75em 0 0;
border: 1px solid #ddd;
}
.tabs ul li.selected {
background: rgb(200,255,208);
}
.tabs ul li:hover {
background: rgb(190,246,218);
}
.tabs .body {
margin-top: -1em;
padding: 1em;
border: 1px solid #ccc;
border-top: none;
border-radius: 0 0 0.75em 0.75em;
}

10
docs/source/backends.rst
View File

@ -8,12 +8,9 @@ Backends
platypush/backend/adafruit.io.rst
platypush/backend/alarm.rst
platypush/backend/assistant.google.rst
platypush/backend/assistant.snowboy.rst
platypush/backend/button.flic.rst
platypush/backend/camera.pi.rst
platypush/backend/chat.telegram.rst
platypush/backend/covid19.rst
platypush/backend/file.monitor.rst
platypush/backend/foursquare.rst
platypush/backend/github.rst
@ -21,17 +18,13 @@ Backends
platypush/backend/google.pubsub.rst
platypush/backend/gps.rst
platypush/backend/http.rst
platypush/backend/http.poll.rst
platypush/backend/inotify.rst
platypush/backend/joystick.rst
platypush/backend/joystick.jstest.rst
platypush/backend/joystick.linux.rst
platypush/backend/kafka.rst
platypush/backend/light.hue.rst
platypush/backend/log.http.rst
platypush/backend/mail.rst
platypush/backend/midi.rst
platypush/backend/mqtt.rst
platypush/backend/music.mopidy.rst
platypush/backend/music.mpd.rst
platypush/backend/music.snapcast.rst
@ -50,11 +43,8 @@ Backends
platypush/backend/stt.picovoice.speech.rst
platypush/backend/tcp.rst
platypush/backend/todoist.rst
platypush/backend/travisci.rst
platypush/backend/trello.rst
platypush/backend/weather.buienradar.rst
platypush/backend/weather.darksky.rst
platypush/backend/weather.openweathermap.rst
platypush/backend/wiimote.rst
platypush/backend/zwave.rst
platypush/backend/zwave.mqtt.rst

152
docs/source/conf.py
View File

@ -15,17 +15,14 @@ import sys
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath("./_ext"))
# -- Project information -----------------------------------------------------
project = 'Platypush'
copyright = '2017-2021, Fabio Manganiello'
author = 'Fabio Manganiello'
copyright = '2017-2023, Fabio Manganiello'
author = 'Fabio Manganiello <fabio@manganiello.tech>'
# The short X.Y version
version = ''
@ -43,6 +40,7 @@ release = ''
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'myst_parser',
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx',
'sphinx.ext.todo',
@ -52,6 +50,7 @@ extensions = [
'sphinx.ext.githubpages',
'sphinx_rtd_theme',
'sphinx_marshmallow',
'add_dependencies',
]
# Add any paths that contain templates here, relative to this directory.
@ -60,8 +59,8 @@ templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
source_suffix = ['.rst', '.md']
# source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
@ -113,7 +112,14 @@ html_theme_options = {
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ['_static']
html_static_path = ['_static']
html_css_files = [
'styles/custom.css',
]
html_js_files = [
'scripts/custom.js',
]
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
@ -165,9 +171,9 @@ latex_documents = [
man_pages = [(master_doc, 'platypush', 'platypush Documentation', [author], 1)]
# -- Options for Texinfo output ----------------------------------------------
# -- Options for TexInfo output ----------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# Grouping the document tree into TexInfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
@ -190,136 +196,30 @@ texinfo_documents = [
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
# -- Options for todo extension ----------------------------------------------
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = True
autodoc_default_options = {
'members': True,
'show-inheritance': True,
}
autodoc_mock_imports = [
'gunicorn',
'googlesamples.assistant.grpc.audio_helpers',
'google.assistant.embedded',
'google.assistant.library',
'google.assistant.library.event',
'google.assistant.library.file_helpers',
'google.oauth2.credentials',
'oauth2client',
'apiclient',
'tenacity',
'smartcard',
'Leap',
'oauth2client',
'rtmidi',
'bluetooth',
'gevent.wsgi',
'Adafruit_IO',
'pyclip',
'pydbus',
'inputs',
'inotify',
'omxplayer',
'plexapi',
'cwiid',
'sounddevice',
'soundfile',
'numpy',
'cv2',
'nfc',
'ndef',
'bcrypt',
'google',
'feedparser',
'kafka',
'googlesamples',
'icalendar',
'httplib2',
'mpd',
'serial',
'pyHS100',
'grpc',
'envirophat',
'gps',
'picamera',
'pmw3901',
'PIL',
'croniter',
'pyaudio',
'avs',
'PyOBEX',
'PyOBEX.client',
'todoist',
'trello',
'telegram',
'telegram.ext',
'pyfirmata2',
'cups',
'graphyte',
'cpuinfo',
'psutil',
'openzwave',
'deepspeech',
'wave',
'pvporcupine ',
'pvcheetah',
'pyotp',
'linode_api4',
'pyzbar',
'tensorflow',
'keras',
'pandas',
'samsungtvws',
'paramiko',
'luma',
'zeroconf',
'dbus',
'gi',
'gi.repository',
'twilio',
'Adafruit_Python_DHT',
'RPi.GPIO',
'RPLCD',
'imapclient',
'pysmartthings',
'aiohttp',
'watchdog',
'pyngrok',
'irc',
'irc.bot',
'irc.strings',
'irc.client',
'irc.connection',
'irc.events',
'defusedxml',
'nio',
'aiofiles',
'aiofiles.os',
'async_lru',
'bleak',
'bluetooth_numbers',
'TheengsDecoder',
'simple_websocket',
'uvicorn',
'websockets',
'docutils',
'aioxmpp',
]
sys.path.insert(0, os.path.abspath('../..'))
from platypush.utils.mock.modules import mock_imports # noqa
def skip(app, what, name, obj, skip, options):
autodoc_mock_imports = [*mock_imports]
# _ = app
# __ = what
# ___ = obj
# ____ = options
def _skip(_, __, name, ___, skip, ____):
if name == "__init__":
return False
return skip
def setup(app):
app.connect("autodoc-skip-member", skip)
app.connect("autodoc-skip-member", _skip)
# vim:sw=4:ts=4:et:

2
docs/source/events.rst
View File

@ -16,7 +16,6 @@ Events
platypush/events/chat.slack.rst
platypush/events/chat.telegram.rst
platypush/events/clipboard.rst
platypush/events/covid19.rst
platypush/events/custom.rst
platypush/events/dbus.rst
platypush/events/distance.rst
@ -70,7 +69,6 @@ Events
platypush/events/tensorflow.rst
platypush/events/todoist.rst
platypush/events/torrent.rst
platypush/events/travisci.rst
platypush/events/trello.rst
platypush/events/video.rst
platypush/events/weather.rst

49
docs/source/index.rst
View File

@ -1,23 +1,50 @@
Platypush
#########
Welcome to the Platypush reference of available plugins, backends and event types.
Description
===========
For more information on Platypush check out:
This is the main documentation hub for Platypush. It includes both the wiki and
the complete reference of the available integrations.
* The `main page`_ of the project
* The `Gitea page`_ of the project
* The `online wiki`_ for quickstart and examples
* The `Blog articles`_ for inspiration on use-cases possible projects
Platypush is a general-purpose automation framework that can be used to cover
all the cases where you'd use a home automation hub, a media center, a smart
assistant, some IFTTT recipes, and a variety of other products and services.
.. _main page: https://platypush.tech
.. _Gitea page: https://git.platypush.tech/platypush/platypush
.. _online wiki: https://git.platypush.tech/platypush/platypush/wiki
.. _Blog articles: https://blog.platypush.tech
It draws inspiration from the following projects, and it aims to cover all of
their use-cases:
* `Home Assistant <https://www.home-assistant.io/>`_
* `Homebridge <https://homebridge.io/>`_
* `OpenHAB <https://www.openhab.org/>`_
* `IFTTT <https://ifttt.com/>`_
* `Tasker <https://tasker.joaoapps.com/>`_
Useful links
============
* The `main page <https://platypush.tech>`_ of the project.
* The `Gitea page <https://git.platypush.tech/platypush/platypush>`_.
* The `blog <https://blog.platypush.tech>`_, for articles showing how to use
Platypush in real-world scenarios.
Wiki
====
.. toctree::
:maxdepth: 3
wiki/index
wiki/Installation
wiki/Configuration
wiki/Installing-extensions
wiki/A-configuration-example
Reference
=========
.. toctree::
:maxdepth: 2
:caption: Contents:
backends
plugins

6
docs/source/platypush/backend/assistant.google.rst
View File

@ -1,6 +0,0 @@
``assistant.google``
======================================
.. automodule:: platypush.backend.assistant.google
:members:

6
docs/source/platypush/backend/assistant.snowboy.rst
View File

@ -1,6 +0,0 @@
``assistant.snowboy``
=======================================
.. automodule:: platypush.backend.assistant.snowboy
:members:

5
docs/source/platypush/backend/covid19.rst
View File

@ -1,5 +0,0 @@
``covid19``
=============================
.. automodule:: platypush.backend.covid19
:members:

6
docs/source/platypush/backend/http.poll.rst
View File

@ -1,6 +0,0 @@
``http.poll``
===============================
.. automodule:: platypush.backend.http.poll
:members:

6
docs/source/platypush/backend/inotify.rst
View File

@ -1,6 +0,0 @@
``inotify``
=============================
.. automodule:: platypush.backend.inotify
:members:

6
docs/source/platypush/backend/light.hue.rst
View File

@ -1,6 +0,0 @@
``light.hue``
===============================
.. automodule:: platypush.backend.light.hue
:members:

6
docs/source/platypush/backend/mqtt.rst
View File

@ -1,6 +0,0 @@
``mqtt``
==========================
.. automodule:: platypush.backend.mqtt
:members:

5
docs/source/platypush/backend/travisci.rst
View File

@ -1,5 +0,0 @@
``travisci``
==============================
.. automodule:: platypush.backend.travisci
:members:

5
docs/source/platypush/backend/zwave.mqtt.rst
View File

@ -1,5 +0,0 @@
``zwave.mqtt``
================================
.. automodule:: platypush.backend.zwave.mqtt
:members:

5
docs/source/platypush/backend/zwave.rst
View File

@ -1,5 +0,0 @@
``zwave``
===========================
.. automodule:: platypush.backend.zwave
:members:

5
docs/source/platypush/events/covid19.rst
View File

@ -1,5 +0,0 @@
``covid19``
===================================
.. automodule:: platypush.message.event.covid19
:members:

5
docs/source/platypush/events/travisci.rst
View File

@ -1,5 +0,0 @@
``travisci``
====================================
.. automodule:: platypush.message.event.travisci
:members:

4
docs/source/platypush/events/xmpp.rst
View File

@ -1,5 +1,5 @@
``event.xmpp``
==============
``xmpp``
========
.. automodule:: platypush.message.event.xmpp
:members:

5
docs/source/platypush/plugins/application.rst Normal file
View File

@ -0,0 +1,5 @@
``application``
===============
.. automodule:: platypush.plugins.application
:members:

6
docs/source/platypush/plugins/assistant.echo.rst
View File

@ -1,6 +0,0 @@
``assistant.echo``
====================================
.. automodule:: platypush.plugins.assistant.echo
:members:

6
docs/source/platypush/plugins/assistant.google.pushtotalk.rst
View File

@ -1,6 +0,0 @@
``assistant.google.pushtotalk``
=================================================
.. automodule:: platypush.plugins.assistant.google.pushtotalk
:members:

5
docs/source/platypush/plugins/covid19.rst
View File

@ -1,5 +0,0 @@
``covid19``
=============================
.. automodule:: platypush.plugins.covid19
:members:

6
docs/source/platypush/plugins/http.request.rss.rst
View File

@ -1,6 +0,0 @@
``http.request.rss``
======================================
.. automodule:: platypush.plugins.http.request.rss
:members:

5
docs/source/platypush/plugins/travisci.rst
View File

@ -1,5 +0,0 @@
``travisci``
==============================
.. automodule:: platypush.plugins.travisci
:members:

5
docs/source/platypush/plugins/zwave.rst
View File

@ -1,5 +0,0 @@
``zwave``
===========================
.. automodule:: platypush.plugins.zwave
:members:

7
docs/source/plugins.rst
View File

@ -8,10 +8,9 @@ Plugins
platypush/plugins/adafruit.io.rst
platypush/plugins/alarm.rst
platypush/plugins/application.rst
platypush/plugins/arduino.rst
platypush/plugins/assistant.echo.rst
platypush/plugins/assistant.google.rst
platypush/plugins/assistant.google.pushtotalk.rst
platypush/plugins/autoremote.rst
platypush/plugins/bluetooth.rst
platypush/plugins/calendar.rst
@ -26,7 +25,6 @@ Plugins
platypush/plugins/chat.telegram.rst
platypush/plugins/clipboard.rst
platypush/plugins/config.rst
platypush/plugins/covid19.rst
platypush/plugins/csv.rst
platypush/plugins/db.rst
platypush/plugins/dbus.rst
@ -50,7 +48,6 @@ Plugins
platypush/plugins/graphite.rst
platypush/plugins/hid.rst
platypush/plugins/http.request.rst
platypush/plugins/http.request.rss.rst
platypush/plugins/http.webpage.rst
platypush/plugins/ifttt.rst
platypush/plugins/inputs.rst
@ -128,7 +125,6 @@ Plugins
platypush/plugins/tensorflow.rst
platypush/plugins/todoist.rst
platypush/plugins/torrent.rst
platypush/plugins/travisci.rst
platypush/plugins/trello.rst
platypush/plugins/tts.rst
platypush/plugins/tts.google.rst
@ -148,5 +144,4 @@ Plugins
platypush/plugins/xmpp.rst
platypush/plugins/zeroconf.rst
platypush/plugins/zigbee.mqtt.rst
platypush/plugins/zwave.rst
platypush/plugins/zwave.mqtt.rst

379
examples/conf/config.yaml
View File

@ -1,379 +0,0 @@
#################################################################################
# Sample platypush configuration file.
# Edit it and copy it to /etc/platypush/config.yaml for system installation or to
# ~/.config/platypush/config.yaml for user installation (recommended).
#################################################################################
# --
# include directive example
# --
#
# You can split your configuration over multiple files
# and use the include directive to import them in your configuration.
# Relative paths are also supported, and computed using the config.yaml
# installation directory as base folder. Symlinks are also supported.
#
# Using multiple files is encouraged in the case of large configurations
# that can easily end up in a messy config.yaml file, as they help you
# keep your configuration more organized.
#include:
# - include/logging.yaml
# - include/media.yaml
# - include/sensors.yaml
# platypush logs on stdout by default. You can use the logging section to specify
# an alternative file or change the logging level.
#logging:
# filename: ~/.local/log/platypush/platypush.log
# level: INFO
# The device_id is used by many components of platypush and it should uniquely
# identify a device in your network. If nothing is specified then the hostname
# will be used.
#device_id: my_device
## --
## Plugin configuration examples
## --
#
# Plugins configuration is very straightforward. Each plugin is mapped to
# a plugin class. The methods of the class with @action annotation will
# be exported as runnable actions, while the __init__ parameters are
# configuration attributes that you can initialize in your config.yaml.
# Plugin classes are documented at https://docs.platypush.tech/en/latest/plugins.html
#
# In this example we'll configure the light.hue plugin, see
# https://docs.platypush.tech/en/latest/platypush/plugins/light.hue.html
# for reference. You can easily install the required dependencies for the plugin through
# pip install 'platypush[hue]'
light.hue:
# IP address or hostname of the Hue bridge
bridge: 192.168.1.10
# Groups that will be handled by default if nothing is specified on the request
groups:
- Living Room
# Example configuration of music.mpd plugin, see
# https://docs.platypush.tech/en/latest/platypush/plugins/music.mpd.html
# You can easily install the dependencies through pip install 'platypush[mpd]'
music.mpd:
host: localhost
port: 6600
# Example configuration of media.chromecast plugin, see
# https://docs.platypush.tech/en/latest/platypush/plugins/media.chromecast.html
# You can easily install the dependencies through pip install 'platypush[chromecast]'
media.chromecast:
chromecast: Living Room TV
# Plugins with empty configuration can also be explicitly enabled by specifying
# enabled=True or disabled=False (it's a good practice if you want the
# corresponding web panel to be enabled, if available)
camera.pi:
enabled: True
# Support for calendars - in this case Google and Facebook calendars
# Installing the dependencies: pip install 'platypush[ical,google]'
calendar:
calendars:
- type: platypush.plugins.google.calendar.GoogleCalendarPlugin
- type: platypush.plugins.calendar.ical.CalendarIcalPlugin
url: https://www.facebook.com/events/ical/upcoming/?uid=your_user_id&key=your_key
## --
## Backends configuration examples
## --
#
# Backends are basically threads that run in the background and listen for something
# to happen and either trigger events or provide additional services on top of platypush.
# Just like plugins, backends are classes whose configuration matches one-to-one the
# supported parameters on the __init__ methods. You can check the documentation for the
# available backends here: https://docs.platypush.tech/en/latest/backends.html.
# Moreover, most of the backends will generate events that you can react to through custom
# event hooks. Check here for the events documentation:
# https://docs.platypush.tech/en/latest/events.html
#
# You may usually want to enable the HTTP backend, as it provides many useful features on
# top of platypush. Among those:
#
# - Expose the /execute endpoint, that allows you to send requests to platypush through a
# JSON-RPC interface.
# - Web panel, one of the key additiona features of platypush. Many plugins will expose web
# panel tabs for e.g. accessing and controlling lights, music, media and sensors.
# - Dashboard: platypush can be configured to show a custom dashboard on your screens with
# e.g. music platypush and weather info, news, upcoming calendar events and photo carousel.
# - Streaming support - the HTTP backend makes it possible to stream local media to other
# devices - e.g. Chromecasts and external browsers.
#
# To install the HTTP backend dependencies simply run 'pip install "platypush[http]"'
backend.http:
# Listening port
port: 8008
# Through resource_dirs you can specify external folders whose content can be accessed on
# the web server through a custom URL. In the case below we have a Dropbox folder containing
# our pictures and we mount it to the '/carousel' endpoint.
resource_dirs:
carousel: /mnt/hd/photos/carousel
# The HTTP poll backend is a versatile backend that can monitor for HTTP-based resources and
# trigger events whenever new entries are available. In the example below we show how to use
# the backend to listen for changes on a set of RSS feeds. New content will be stored by default
# on a SQLite database under ~/.local/share/platypush/feeds/rss.db.
# Install the required dependencies through 'pip install "platypush[rss,db]"'
backend.http.poll:
requests:
- type: platypush.backend.http.request.rss.RssUpdates # HTTP poll type (RSS)
# Remote URL
url: http://www.theguardian.com/rss/world
# Custom title
title: The Guardian - World News
# How often we should check for changes
poll_seconds: 600
# Maximum number of new entries to be processed
max_entries: 10
- type: platypush.backend.http.request.rss.RssUpdates
url: http://www.physorg.com/rss-feed
title: Phys.org
poll_seconds: 600
max_entries: 10
- type: platypush.backend.http.request.rss.RssUpdates
url: http://feeds.feedburner.com/Techcrunch
title: Tech Crunch
poll_seconds: 600
max_entries: 10
- type: platypush.backend.http.request.rss.RssUpdates
url: http://www.nytimes.com/services/xml/rss/nyt/HomePage.xml
title: The New York Times
poll_seconds: 300
max_entries: 10
# MQTT backend. Installed required dependencies through 'pip install "platypush[mqtt]"'
backend.mqtt:
# Remote MQTT server IP or hostname
host: mqtt-server
# By default the backend will listen for messages on the platypush_bus_mq/device_id
# topic, but you can change the prefix using the topic attribute
# topic: MyBus
# Raw TCP socket backend. It can run commands sent as JSON over telnet or netcat
#backend.tcp:
# port: 3333
## --
## Assistant configuration examples
## --
#
# Both Google Assistant and Alexa voice assistant interfaces are supported by platypush.
# You can easily make your custom voice assistant with a RaspberryPi and a USB microphone,
# or on your laptop. Note however that the Alexa integration is still experimental
# (mostly because of glitches and bugs on the avs package provided by Amazon), while the
# Google Assistant support should be more robust. The recommended way of triggering a
# hotword ('OK Google', 'Alexa' or any custom hotword you like) is through the snowboy
# backend (install it through 'pip install "platypush[hotword]"'). You can download custom
# voice model files (.umdl) from https://snowboy.kitt.ai.
backend.assistant.snowboy:
# Microphone audio gain
audio_gain: 1.1
models:
# "Computer" hotword model
computer:
# UMDL file path
voice_model_file: ~/.local/share/snowboy/models/computer.umdl
# Plugin to use (Google Assistant)
assistant_plugin: assistant.google.pushtotalk
# Language assistant (Italian)
assistant_language: it-IT
# Sound to play when the hotword is detected
detect_sound: ~/.local/share/sounds/hotword.wav
# Model sensitivity
sensitivity: 0.4
# "OK Google" hotword model
ok_google:
voice_model_file: ~/.local/share/snowboy/models/OK Google.pmdl
assistant_plugin: assistant.google.pushtotalk
assistant_language: en-US
detect_sound: ~/.local/share/sounds/sci-fi/PremiumBeat_0013_cursor_selection_16.wav
sensitivity: 0.4
# "Alexa" voice model
alexa:
voice_model_file: ~/.local/share/snowboy/models/Alexa.pmdl
assistant_plugin: assistant.echo
assistant_language: en-US
detect_sound: ~/.local/share/sounds/sci-fi/PremiumBeat_0013_cursor_selection_16.wav
sensitivity: 0.5
# Install Alexa dependencies with 'pip install "platypush[alexa]"'
assistant.echo:
audio_player: mplayer
# Install Google Assistant dependencies with 'pip install "platypush[google-assistant-legacy]"'
assistant.google:
enabled: True
backend.assistant.google:
enabled: True
## --
## Procedure examples
## --
#
# Procedures are lists of actions that can be executed synchronously (default) or in parallel
# (procedure.async. prefix). Basic flow control operators (if/else/for) are also available.
# You can also access Python variables and evaluate Python expressions by using the ${} expressions.
# The 'context' special variable is a name->value dictionary containing the items returned from
# previous actions - for example if an action returned '{"status": "ok", "temperature":21.5}' then
# the following actions can access those variables through ${status} and ${temperature} respectively,
# and you can also add things like '- if ${temperature > 20.0}' or '- for ${temp in temperature_values}'.
# Alternatively, you can access those variable also through ${context.get('status')} or ${context.get('temperature')}.
# Other special variables that you can use in your procedures:
#
# - output: Will contain the parsed output of the previous action
# - errors: Will contain the errors of the previous action
# - event: If the procedure is executed within an event hook, it contains the event that triggered the hook
#
# An example procedure that can be called when you arrive home. You can run this procedure by sending a JSON
# message like this on whichever backend you like (HTTP, websocket, TCP, Redis, MQTT, Node-RED, Pushbullet...)
# {"type":"request", "action":"procedure.at_home"}
# You can for instance install Tasker+AutoLocation on your mobile and send this message whenever you enter
# your home area.
procedure.at_home:
# Set the db variable HOME to 1
- action: variable.set
args:
HOME: 1
# Check the luminosity level from a connected LTR559 sensor
- action: gpio.sensor.ltr559.get_data
# If it's below a certain threshold turn on the lights
- if ${int(light or 0) < 110}:
- action: light.hue.on
# Say a welcome home message. Install dependencies through 'pip install "platypush[google-tts]"'
- action: tts.google.say
args:
text: Welcome home
# Start the music
- action: music.mpd.play
# Procedure that will be execute when you're outside of home
procedure.outside_home:
# Unset the db variable HOME
- action: variable.unset
args:
name: HOME
# Stop the music
- action: music.mpd.stop
# Turn off the lights
- action: light.hue.off
# Start the camera streaming. Install the Pi Camera dependencies through
# 'pip install "platypush[picamera]"'
- action: camera.pi.start_streaming
args:
listen_port: 2222
# Procedures can also take optional arguments. The example below show a
# generic procedure to send a request to another platypush host over MQTT
# given target, action and args
procedure.send_request(target, action, args):
- action: mqtt.send_message
args:
topic: platypush_bus_mq/${target}
host: mqtt-server
port: 1883
msg:
type: request
target: ${target}
action: ${action}
args: ${args}
## --
## Event hook examples
## --
#
# Event hooks are procedures that are run when a certain condition is met.
# Check the documentation of the backends to see which events they can trigger.
# An event hook consists of two parts: an 'if' field that specifies on which
# event the hook will be triggered (type and attributes content), and a 'then'
# field that uses the same syntax as procedures to specify a list of actions to
# execute when the event is matched.
#
# The example below plays the music on mpd/mopidy when your voice assistant
# triggers a speech recognized event with "play the music" content.
event.hook.PlayMusicAssistantCommand:
if:
type: platypush.message.event.assistant.SpeechRecognizedEvent
# Note that basic regexes are supported, so the hook will be triggered
# both if you say "play the music" and "play music"
phrase: "play (the)? music"
then:
- action: music.mpd.play
# This will turn on the lights when you say "turn on the lights"
event.hook.TurnOnLightsCommand:
if:
type: platypush.message.event.assistant.SpeechRecognizedEvent
phrase: "turn on (the)? lights?"
then:
- action: light.hue.on
# This will play a song by a specified artist
event.hook.SearchSongVoiceCommand:
if:
type: platypush.message.event.assistant.SpeechRecognizedEvent
# Note that you can use the ${} operator in event matching to
# extract part of the matched string into context variables that
# can be accessed in your event hook.
phrase: "play ${title} by ${artist}"
then:
- action: music.mpd.clear
- action: music.mpd.search
args:
filter:
artist: ${artist}
title: ${title}
# Play the first search result
- action: music.mpd.play
args:
resource: ${output[0]['file']}
# This event will scrobble newly listened tracks on mpd/mopidy to last.fm
event.hook.ScrobbleNewTrack:
if:
type: platypush.message.event.music.NewPlayingTrackEvent
then:
- action: lastfm.scrobble
args:
artist: ${track['artist']}
title: ${track['title']}
- action: lastfm.update_now_playing
args:
artist: ${track['artist']}
title: ${track['title']}
## --
## Cron examples
## --
#
# Cronjobs allow you to execute procedures at periodic intervals.
# Standard UNIX cron syntax is supported, plus an optional 6th indicator
# at the end of the expression to run jobs with second granularity.
# The example below executes a script at intervals of 1 minute.
cron.TestCron:
cron_expression: '* * * * *'
actions:
- action: shell.exec
args:
cmd: ~/bin/myscript.sh

1
examples/config/config.yaml Symbolic link
View File

@ -0,0 +1 @@
../../platypush/config/config.yaml

0
examples/conf/dashboard.xml → examples/config/dashboard.xml
View File

16
examples/conf/hook.py → examples/config/hook.py
View File

@ -12,7 +12,10 @@ from platypush.utils import run
from platypush.event.hook import hook
# Event types that you want to react to
from platypush.message.event.assistant import ConversationStartEvent, SpeechRecognizedEvent
from platypush.message.event.assistant import (
ConversationStartEvent,
SpeechRecognizedEvent,
)
@hook(SpeechRecognizedEvent, phrase='play ${title} by ${artist}')
@ -23,10 +26,13 @@ def on_music_play_command(event, title=None, artist=None, **context):
Note that in this specific case we can leverage the token-extraction feature of SpeechRecognizedEvent through
${} that operates on regex-like principles to extract any text that matches the pattern into context variables.
"""
results = run('music.mpd.search', filter={
'artist': artist,
'title': title,
})
results = run(
'music.mpd.search',
filter={
'artist': artist,
'title': title,
},
)
if results:
run('music.mpd.play', results[0]['file'])

19
examples/docker/Dockerfile
View File

@ -1,19 +0,0 @@
# Sample Dockerfile. Use platydock -c /path/to/custom/config.yaml
# to generate your custom Dockerfile.
FROM python:3.11-alpine
RUN mkdir -p /install /app
COPY . /install
RUN apk add --update --no-cache redis
RUN apk add --update --no-cache --virtual build-base g++ rust
RUN pip install -U pip
RUN cd /install && pip install .
RUN apk del build-base g++ rust
EXPOSE 8008
VOLUME /app/config
VOLUME /app/workdir
CMD python -m platypush --start-redis --config-file /app/config/config.yaml --workdir /app/workdir

35
examples/nginx/nginx.sample.conf
View File

@ -1,12 +1,17 @@
# An nginx configuration that can be used to reverse proxy connections to your
# Platypush' HTTP service.
server {
server_name my-platypush-host.domain.com;
upstream platypush {
# The address and port where the HTTP backend is listening
server 127.0.0.1:8008;
}
# Proxy standard HTTP connections to your Platypush IP
server {
server_name platypush.example.com;
# Proxy standard HTTP connections
location / {
proxy_pass http://my-platypush-host:8008/;
proxy_pass http://platypush;
client_max_body_size 5M;
proxy_read_timeout 60;
@ -18,21 +23,33 @@ server {
}
# Proxy websocket connections
location ~ ^/ws/(.*)$ {
proxy_pass http://10.0.0.2:8008/ws/$1;
location /ws/ {
proxy_pass http://platypush;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_http_version 1.1;
client_max_body_size 200M;
client_max_body_size 5M;
proxy_set_header Host $http_host;
}
# Optional SSL configuration - using Let's Encrypt certificates in this case
# listen 443 ssl;
# ssl_certificate /etc/letsencrypt/live/my-platypush-host.domain.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/my-platypush-host.domain.com/privkey.pem;
# ssl_certificate /etc/letsencrypt/live/platypush.example.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/platypush.example.com/privkey.pem;
# include /etc/letsencrypt/options-ssl-nginx.conf;
# ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
# Uncomment if you are using SSL and you want to force an HTTPS upgrade to
# clients connecting over the port 80
# server {
# if ($host = platypush.example.com) {
# return 301 https://$host$request_uri;
# }
#
# server_name platypush.example.com;
# listen 80;
# return 404;
# }

64
generate_missing_docs.py
View File

@ -1,32 +1,57 @@
import importlib
import inspect
import os
import sys
from typing import Iterable, Optional
import pkgutil
from platypush.backend import Backend
from platypush.context import get_plugin
from platypush.message.event import Event
from platypush.message.response import Response
from platypush.plugins import Plugin
from platypush.utils.manifest import get_manifests
def _get_inspect_plugin():
p = get_plugin('inspect')
assert p, 'Could not load the `inspect` plugin'
return p
from platypush.utils.manifest import Manifests
from platypush.utils.mock import auto_mocks
def get_all_plugins():
return sorted([mf.component_name for mf in get_manifests(Plugin)])
return sorted([mf.component_name for mf in Manifests.by_base_class(Plugin)])
def get_all_backends():
return sorted([mf.component_name for mf in get_manifests(Backend)])
return sorted([mf.component_name for mf in Manifests.by_base_class(Backend)])
def get_all_events():
return _get_inspect_plugin().get_all_events().output
return _get_modules(Event)
def get_all_responses():
return _get_inspect_plugin().get_all_responses().output
return _get_modules(Response)
def _get_modules(base_type: type):
ret = set()
base_dir = os.path.dirname(inspect.getfile(base_type))
package = base_type.__module__
for _, mod_name, _ in pkgutil.walk_packages([base_dir], prefix=package + '.'):
try:
module = importlib.import_module(mod_name)
except Exception:
print('Could not import module', mod_name, file=sys.stderr)
continue
for _, obj_type in inspect.getmembers(module):
if (
inspect.isclass(obj_type)
and issubclass(obj_type, base_type)
# Exclude the base_type itself
and obj_type != base_type
):
ret.add(obj_type.__module__.replace(package + '.', '', 1))
return list(ret)
def _generate_components_doc(
@ -122,7 +147,7 @@ def generate_events_doc():
_generate_components_doc(
index_name='events',
package_name='message.event',
components=sorted(event for event in get_all_events().keys() if event),
components=sorted(event for event in get_all_events() if event),
)
@ -130,17 +155,16 @@ def generate_responses_doc():
_generate_components_doc(
index_name='responses',
package_name='message.response',
components=sorted(
response for response in get_all_responses().keys() if response
),
components=sorted(response for response in get_all_responses() if response),
)
def main():
generate_plugins_doc()
generate_backends_doc()
generate_events_doc()
generate_responses_doc()
with auto_mocks():
generate_plugins_doc()
generate_backends_doc()
generate_events_doc()
generate_responses_doc()
if __name__ == '__main__':

5
platypush/__init__.py
View File

@ -5,12 +5,14 @@ Platypush
.. license: MIT
"""
from .app import Application, main
from .app import Application
from .config import Config
from .context import get_backend, get_bus, get_plugin
from .message.event import Event
from .message.request import Request
from .message.response import Response
from .runner import main
from .utils import run
__author__ = 'Fabio Manganiello <fabio@manganiello.tech>'
@ -25,6 +27,7 @@ __all__ = [
'get_bus',
'get_plugin',
'main',
'run',
]

11
platypush/__main__.py
View File

@ -1,10 +1,3 @@
import sys
from platypush.runner import main
from platypush.app import main
if __name__ == '__main__':
main(*sys.argv[1:])
# vim:sw=4:ts=4:et:
main()

4
platypush/app/__init__.py Normal file
View File

@ -0,0 +1,4 @@
from ._app import Application, main
__all__ = ["Application", "main"]

5
platypush/app/__main__.py Normal file
View File

@ -0,0 +1,5 @@
import sys
from ._app import main
sys.exit(main(*sys.argv[1:]))

321
platypush/app.py → platypush/app/_app.py
View File

@ -1,23 +1,26 @@
import argparse
from contextlib import contextmanager
import logging
import os
import signal
import subprocess
import sys
from typing import Optional
from typing import Optional, Sequence
from .bus import Bus
from .bus.redis import RedisBus
from .config import Config
from .context import register_backends, register_plugins
from .cron.scheduler import CronScheduler
from .entities import init_entities_engine, EntitiesEngine
from .event.processor import EventProcessor
from .logger import Logger
from .message.event import Event
from .message.event.application import ApplicationStartedEvent
from .message.request import Request
from .message.response import Response
from .utils import get_enabled_plugins, get_redis_conf
from platypush.bus import Bus
from platypush.bus.redis import RedisBus
from platypush.cli import parse_cmdline
from platypush.commands import CommandStream
from platypush.config import Config
from platypush.context import register_backends, register_plugins
from platypush.cron.scheduler import CronScheduler
from platypush.entities import init_entities_engine, EntitiesEngine
from platypush.event.processor import EventProcessor
from platypush.logger import Logger
from platypush.message.event import Event
from platypush.message.event.application import ApplicationStartedEvent
from platypush.message.request import Request
from platypush.message.response import Response
from platypush.utils import get_enabled_plugins, get_redis_conf
log = logging.getLogger('platypush')
@ -25,9 +28,6 @@ log = logging.getLogger('platypush')
class Application:
"""Main class for the Platypush application."""
# Default bus queue name
_default_redis_queue = 'platypush/bus'
# Default Redis port
_default_redis_port = 6379
@ -42,6 +42,8 @@ class Application:
config_file: Optional[str] = None,
workdir: Optional[str] = None,
logsdir: Optional[str] = None,
cachedir: Optional[str] = None,
device_id: Optional[str] = None,
pidfile: Optional[str] = None,
requests_to_process: Optional[int] = None,
no_capture_stdout: bool = False,
@ -51,6 +53,7 @@ class Application:
start_redis: bool = False,
redis_host: Optional[str] = None,
redis_port: Optional[int] = None,
ctrl_sock: Optional[str] = None,
):
"""
:param config_file: Configuration file override (default: None).
@ -60,6 +63,12 @@ class Application:
``filename`` setting under the ``logging`` section of the
configuration file is used. If not set, logging will be sent to
stdout and stderr.
:param cachedir: Overrides the ``cachedir`` setting in the configuration
file (default: None).
:param device_id: Override the device ID used to identify this
instance. If not passed here, it is inferred from the configuration
(device_id field). If not present there either, it is inferred from
the hostname.
:param pidfile: File where platypush will store its PID upon launch,
useful if you're planning to integrate the application within a
service or a launcher script (default: None).
@ -82,24 +91,31 @@ class Application:
the settings in the ``redis`` section of the configuration file.
:param redis_port: Port of the local Redis server. It overrides the
settings in the ``redis`` section of the configuration file.
:param ctrl_sock: If set, it identifies a path to a UNIX domain socket
that the application can use to send control messages (e.g. STOP
and RESTART) to its parent.
"""
self.pidfile = pidfile
if pidfile:
with open(pidfile, 'w') as f:
f.write(str(os.getpid()))
self.bus: Optional[Bus] = None
self.redis_queue = redis_queue or self._default_redis_queue
self.redis_queue = redis_queue or RedisBus.DEFAULT_REDIS_QUEUE
self.config_file = config_file
self._verbose = verbose
self._logsdir = (
os.path.abspath(os.path.expanduser(logsdir)) if logsdir else None
)
Config.init(self.config_file)
if workdir:
Config.set('workdir', os.path.abspath(os.path.expanduser(workdir)))
Config.init(
self.config_file,
device_id=device_id,
workdir=os.path.abspath(os.path.expanduser(workdir)) if workdir else None,
cachedir=os.path.abspath(os.path.expanduser(cachedir))
if cachedir
else None,
ctrl_sock=os.path.abspath(os.path.expanduser(ctrl_sock))
if ctrl_sock
else None,
)
self.no_capture_stdout = no_capture_stdout
self.no_capture_stderr = no_capture_stderr
@ -113,6 +129,7 @@ class Application:
self.redis_port = redis_port
self.redis_conf = {}
self._redis_proc: Optional[subprocess.Popen] = None
self.cmd_stream = CommandStream(ctrl_sock)
self._init_bus()
self._init_logging()
@ -153,16 +170,27 @@ class Application:
port = self._redis_conf['port']
log.info('Starting local Redis instance on %s', port)
self._redis_proc = subprocess.Popen( # pylint: disable=consider-using-with
[
'redis-server',
'--bind',
'localhost',
'--port',
str(port),
],
stdout=subprocess.PIPE,
)
redis_cmd_args = [
'redis-server',
'--bind',
'localhost',
'--port',
str(port),
]
try:
self._redis_proc = subprocess.Popen( # pylint: disable=consider-using-with
redis_cmd_args,
stdout=subprocess.PIPE,
)
except Exception as e:
log.error(
'Failed to start local Redis instance: "%s": %s',
' '.join(redis_cmd_args),
e,
)
sys.exit(1)
log.info('Waiting for Redis to start')
for line in self._redis_proc.stdout: # type: ignore
@ -176,137 +204,17 @@ class Application:
self._redis_proc = None
@classmethod
def build(cls, *args: str):
def from_cmdline(cls, args: Sequence[str]) -> "Application":
"""
Build the app from command line arguments.
"""
from . import __version__
parser = argparse.ArgumentParser()
parser.add_argument(
'--config',
'-c',
dest='config',
required=False,
default=None,
help='Custom location for the configuration file',
)
parser.add_argument(
'--workdir',
'-w',
dest='workdir',
required=False,
default=None,
help='Custom working directory to be used for the application',
)
parser.add_argument(
'--logsdir',
'-l',
dest='logsdir',
required=False,
default=None,
help='Store logs in the specified directory. By default, the '
'`[logging.]filename` configuration option will be used. If not '
'set, logging will be sent to stdout and stderr.',
)
parser.add_argument(
'--version',
dest='version',
required=False,
action='store_true',
help="Print the current version and exit",
)
parser.add_argument(
'--verbose',
'-v',
dest='verbose',
required=False,
action='store_true',
help="Enable verbose/debug logging",
)
parser.add_argument(
'--pidfile',
'-P',
dest='pidfile',
required=False,
default=None,
help="File where platypush will "
+ "store its PID, useful if you're planning to "
+ "integrate it in a service",
)
parser.add_argument(
'--no-capture-stdout',
dest='no_capture_stdout',
required=False,
action='store_true',
help="Set this flag if you have max stack depth "
+ "exceeded errors so stdout won't be captured by "
+ "the logging system",
)
parser.add_argument(
'--no-capture-stderr',
dest='no_capture_stderr',
required=False,
action='store_true',
help="Set this flag if you have max stack depth "
+ "exceeded errors so stderr won't be captured by "
+ "the logging system",
)
parser.add_argument(
'--redis-queue',
dest='redis_queue',
required=False,
default=cls._default_redis_queue,
help="Name of the Redis queue to be used to internally deliver messages "
"(default: platypush/bus)",
)
parser.add_argument(
'--start-redis',
dest='start_redis',
required=False,
action='store_true',
help="Set this flag if you want to run and manage Redis internally "
"from the app rather than using an external server. It requires the "
"redis-server executable to be present in the path",
)
parser.add_argument(
'--redis-host',
dest='redis_host',
required=False,
default=None,
help="Overrides the host specified in the redis section of the "
"configuration file",
)
parser.add_argument(
'--redis-port',
dest='redis_port',
required=False,
default=None,
help="Overrides the port specified in the redis section of the "
"configuration file",
)
opts, _ = parser.parse_known_args(args)
if opts.version:
print(__version__)
sys.exit(0)
opts = parse_cmdline(args)
return cls(
config_file=opts.config,
workdir=opts.workdir,
cachedir=opts.cachedir,
logsdir=opts.logsdir,
device_id=opts.device_id,
pidfile=opts.pidfile,
no_capture_stdout=opts.no_capture_stdout,
no_capture_stderr=opts.no_capture_stderr,
@ -315,6 +223,7 @@ class Application:
start_redis=opts.start_redis,
redis_host=opts.redis_host,
redis_port=opts.redis_port,
ctrl_sock=opts.ctrl_sock,
)
def on_message(self):
@ -340,7 +249,7 @@ class Application:
self.requests_to_process
and self.processed_requests >= self.requests_to_process
):
self.stop_app()
self.stop()
elif isinstance(msg, Response):
msg.log()
elif isinstance(msg, Event):
@ -349,36 +258,68 @@ class Application:
return _f
def stop_app(self):
def stop(self):
"""Stops the backends and the bus."""
from .plugins import RunnablePlugin
from platypush.plugins import RunnablePlugin
if self.backends:
for backend in self.backends.values():
backend.stop()
log.info('Stopping the application')
backends = (self.backends or {}).copy().values()
runnable_plugins = [
plugin
for plugin in get_enabled_plugins().values()
if isinstance(plugin, RunnablePlugin)
]
for plugin in get_enabled_plugins().values():
if isinstance(plugin, RunnablePlugin):
plugin.stop()
for backend in backends:
backend.stop()
for plugin in runnable_plugins:
plugin.stop()
for backend in backends:
backend.wait_stop()
for plugin in runnable_plugins:
plugin.wait_stop()
if self.entities_engine:
self.entities_engine.stop()
self.entities_engine.wait_stop()
self.entities_engine = None
if self.cron_scheduler:
self.cron_scheduler.stop()
self.cron_scheduler.wait_stop()
self.cron_scheduler = None
if self.bus:
self.bus.stop()
self.bus = None
if self.cron_scheduler:
self.cron_scheduler.stop()
self.cron_scheduler = None
if self.entities_engine:
self.entities_engine.stop()
self.entities_engine = None
if self.start_redis:
self._stop_redis()
def run(self):
"""Start the daemon."""
from . import __version__
log.info('Exiting application')
@contextmanager
def _open_pidfile(self):
if self.pidfile:
try:
with open(self.pidfile, 'w') as f:
f.write(str(os.getpid()))
except OSError as e:
log.warning('Failed to create PID file %s: %s', self.pidfile, e)
yield
if self.pidfile:
try:
os.remove(self.pidfile)
except OSError as e:
log.warning('Failed to remove PID file %s: %s', self.pidfile, e)
def _run(self):
from platypush import __version__
if not self.no_capture_stdout:
sys.stdout = Logger(log.info)
@ -417,16 +358,30 @@ class Application:
self.bus.poll()
except KeyboardInterrupt:
log.info('SIGINT received, terminating application')
# Ignore other SIGINT signals
signal.signal(signal.SIGINT, signal.SIG_IGN)
finally:
self.stop_app()
self.stop()
def run(self):
"""Run the application."""
with self._open_pidfile():
self._run()
def main(*args: str):
"""
Application entry point.
"""
app = Application.build(*args)
app.run()
app = Application.from_cmdline(args)
try:
app.run()
except KeyboardInterrupt:
pass
return 0
# vim:sw=4:ts=4:et:

91
platypush/backend/__init__.py
View File

@ -6,27 +6,28 @@ import time
from threading import Thread, Event as ThreadEvent, get_ident
from typing import Optional, Dict
from platypush import __version__
from platypush.bus import Bus
from platypush.common import ExtensionWithManifest
from platypush.config import Config
from platypush.context import get_backend
from platypush.event import EventGenerator
from platypush.message import Message
from platypush.message.event import Event
from platypush.message.event.zeroconf import (
ZeroconfServiceAddedEvent,
ZeroconfServiceRemovedEvent,
)
from platypush.utils import (
set_timeout,
clear_timeout,
get_redis_queue_name_by_message,
get_backend_name_by_class,
)
from platypush import __version__
from platypush.event import EventGenerator
from platypush.message import Message
from platypush.message.event import Event
from platypush.message.request import Request
from platypush.message.response import Response
from platypush.utils import (
clear_timeout,
get_backend_name_by_class,
get_redis,
get_redis_queue_name_by_message,
get_remaining_timeout,
set_timeout,
)
class Backend(Thread, EventGenerator, ExtensionWithManifest):
@ -68,6 +69,7 @@ class Backend(Thread, EventGenerator, ExtensionWithManifest):
self.device_id = Config.get('device_id')
self.thread_id = None
self._stop_event = ThreadEvent()
self._stop_thread: Optional[Thread] = None
self._kwargs = kwargs
self.logger = logging.getLogger(
'platypush:backend:' + get_backend_name_by_class(self.__class__)
@ -299,30 +301,38 @@ class Backend(Thread, EventGenerator, ExtensionWithManifest):
self._stop_event.set()
self.unregister_service()
self.on_stop()
self._stop_thread = None
Thread(target=_async_stop).start()
if not (self._stop_thread and self._stop_thread.is_alive()):
self._stop_thread = Thread(target=_async_stop)
self._stop_thread.start()
def should_stop(self):
"""
:return: True if the backend thread should be stopped, False otherwise.
"""
return self._stop_event.is_set()
def wait_stop(self, timeout=None) -> bool:
return self._stop_event.wait(timeout)
"""
Waits for the backend thread to stop.
def _get_redis(self):
import redis
:param timeout: The maximum time to wait for the backend thread to stop (default: None)
:return: True if the backend thread has stopped, False otherwise.
"""
start = time.time()
redis_backend = get_backend('redis')
if not redis_backend:
self.logger.warning(
'Redis backend not configured - some '
'web server features may not be working properly'
)
redis_args = {}
else:
redis_args = redis_backend.redis_args
if self._stop_thread:
try:
self._stop_thread.join(
get_remaining_timeout(timeout=timeout, start=start)
)
except AttributeError:
pass
redis = redis.Redis(**redis_args)
return redis
return self._stop_event.wait(
get_remaining_timeout(timeout=timeout, start=start)
)
def get_message_response(self, msg):
queue = get_redis_queue_name_by_message(msg)
@ -331,7 +341,7 @@ class Backend(Thread, EventGenerator, ExtensionWithManifest):
return None
try:
redis = self._get_redis()
redis = get_redis()
response = redis.blpop(queue, timeout=60)
if response and len(response) > 1:
response = Message.build(response[1])
@ -431,6 +441,8 @@ class Backend(Thread, EventGenerator, ExtensionWithManifest):
"""
Unregister the Zeroconf service configuration if available.
"""
from redis import exceptions
if self.zeroconf and self.zeroconf_info:
try:
self.zeroconf.unregister_service(self.zeroconf_info)
@ -448,17 +460,22 @@ class Backend(Thread, EventGenerator, ExtensionWithManifest):
except TimeoutError:
pass
if self.zeroconf_info:
self.bus.post(
ZeroconfServiceRemovedEvent(
service_type=self.zeroconf_info.type,
service_name=self.zeroconf_info.name,
try:
if self.zeroconf_info:
self.bus.post(
ZeroconfServiceRemovedEvent(
service_type=self.zeroconf_info.type,
service_name=self.zeroconf_info.name,
)
)
)
else:
self.bus.post(
ZeroconfServiceRemovedEvent(service_type=None, service_name=None)
)
else:
self.bus.post(
ZeroconfServiceRemovedEvent(
service_type=None, service_name=None
)
)
except exceptions.ConnectionError:
pass
self.zeroconf_info = None
self.zeroconf = None

26
platypush/backend/adafruit/io/__init__.py
View File

@ -2,23 +2,17 @@ from typing import Optional
from platypush.backend import Backend
from platypush.context import get_plugin
from platypush.message.event.adafruit import ConnectedEvent, DisconnectedEvent, \
FeedUpdateEvent
from platypush.message.event.adafruit import (
ConnectedEvent,
DisconnectedEvent,
FeedUpdateEvent,
)
class AdafruitIoBackend(Backend):
"""
Backend that listens to messages received over the Adafruit IO message queue
Triggers:
* :class:`platypush.message.event.adafruit.ConnectedEvent` when the
backend connects to the Adafruit queue
* :class:`platypush.message.event.adafruit.DisconnectedEvent` when the
backend disconnects from the Adafruit queue
* :class:`platypush.message.event.adafruit.FeedUpdateEvent` when an
update event is received on a monitored feed
Requires:
* The :class:`platypush.plugins.adafruit.io.AdafruitIoPlugin` plugin to
@ -33,6 +27,7 @@ class AdafruitIoBackend(Backend):
super().__init__(*args, **kwargs)
from Adafruit_IO import MQTTClient
self.feeds = feeds
self._client: Optional[MQTTClient] = None
@ -41,6 +36,7 @@ class AdafruitIoBackend(Backend):
return
from Adafruit_IO import MQTTClient
plugin = get_plugin('adafruit.io')
if not plugin:
raise RuntimeError('Adafruit IO plugin not configured')
@ -80,8 +76,11 @@ class AdafruitIoBackend(Backend):
def run(self):
super().run()
self.logger.info(('Initialized Adafruit IO backend, listening on ' +
'feeds {}').format(self.feeds))
self.logger.info(
('Initialized Adafruit IO backend, listening on ' + 'feeds {}').format(
self.feeds
)
)
while not self.should_stop():
try:
@ -94,4 +93,5 @@ class AdafruitIoBackend(Backend):
self.logger.exception(e)
self._client = None
# vim:sw=4:ts=4:et:

113
platypush/backend/alarm/__init__.py
View File

@ -11,7 +11,11 @@ from dateutil.tz import gettz
from platypush.backend import Backend
from platypush.context import get_bus, get_plugin
from platypush.message.event.alarm import AlarmStartedEvent, AlarmDismissedEvent, AlarmSnoozedEvent
from platypush.message.event.alarm import (
AlarmStartedEvent,
AlarmDismissedEvent,
AlarmSnoozedEvent,
)
from platypush.plugins.media import MediaPlugin, PlayerState
from platypush.procedure import Procedure
@ -28,10 +32,17 @@ class Alarm:
_alarms_count = 0
_id_lock = threading.RLock()
def __init__(self, when: str, actions: Optional[list] = None, name: Optional[str] = None,
audio_file: Optional[str] = None, audio_plugin: Optional[str] = None,
audio_volume: Optional[Union[int, float]] = None,
snooze_interval: float = 300.0, enabled: bool = True):
def __init__(
self,
when: str,
actions: Optional[list] = None,
name: Optional[str] = None,
audio_file: Optional[str] = None,
audio_plugin: Optional[str] = None,
audio_volume: Optional[Union[int, float]] = None,
snooze_interval: float = 300.0,
enabled: bool = True,
):
with self._id_lock:
self._alarms_count += 1
self.id = self._alarms_count
@ -42,20 +53,26 @@ class Alarm:
if audio_file:
self.audio_file = os.path.abspath(os.path.expanduser(audio_file))
assert os.path.isfile(self.audio_file), 'No such audio file: {}'.format(self.audio_file)
assert os.path.isfile(self.audio_file), 'No such audio file: {}'.format(
self.audio_file
)
self.audio_plugin = audio_plugin
self.audio_volume = audio_volume
self.snooze_interval = snooze_interval
self.state: Optional[AlarmState] = None
self.timer: Optional[threading.Timer] = None
self.actions = Procedure.build(name=name, _async=False, requests=actions or [], id=self.id)
self.actions = Procedure.build(
name=name, _async=False, requests=actions or [], id=self.id
)
self._enabled = enabled
self._runtime_snooze_interval = snooze_interval
def get_next(self) -> float:
now = datetime.datetime.now().replace(tzinfo=gettz()) # lgtm [py/call-to-non-callable]
now = datetime.datetime.now().replace(
tzinfo=gettz()
) # lgtm [py/call-to-non-callable]
try:
cron = croniter.croniter(self.when, now)
@ -63,10 +80,14 @@ class Alarm:
except (AttributeError, croniter.CroniterBadCronError):
try:
timestamp = datetime.datetime.fromisoformat(self.when).replace(
tzinfo=gettz()) # lgtm [py/call-to-non-callable]
tzinfo=gettz()
) # lgtm [py/call-to-non-callable]
except (TypeError, ValueError):
timestamp = (datetime.datetime.now().replace(tzinfo=gettz()) + # lgtm [py/call-to-non-callable]
datetime.timedelta(seconds=int(self.when)))
timestamp = datetime.datetime.now().replace(
tzinfo=gettz()
) + datetime.timedelta( # lgtm [py/call-to-non-callable]
seconds=int(self.when)
)
return timestamp.timestamp() if timestamp >= now else None
@ -88,7 +109,9 @@ class Alarm:
self._runtime_snooze_interval = interval or self.snooze_interval
self.state = AlarmState.SNOOZED
self.stop_audio()
get_bus().post(AlarmSnoozedEvent(name=self.name, interval=self._runtime_snooze_interval))
get_bus().post(
AlarmSnoozedEvent(name=self.name, interval=self._runtime_snooze_interval)
)
def start(self):
if self.timer:
@ -159,7 +182,9 @@ class Alarm:
break
if not sleep_time:
sleep_time = self.get_next() - time.time() if self.get_next() else 10
sleep_time = (
self.get_next() - time.time() if self.get_next() else 10
)
time.sleep(sleep_time)
@ -179,18 +204,15 @@ class Alarm:
class AlarmBackend(Backend):
"""
Backend to handle user-configured alarms.
Triggers:
* :class:`platypush.message.event.alarm.AlarmStartedEvent` when an alarm starts.
* :class:`platypush.message.event.alarm.AlarmSnoozedEvent` when an alarm is snoozed.
* :class:`platypush.message.event.alarm.AlarmTimeoutEvent` when an alarm times out.
* :class:`platypush.message.event.alarm.AlarmDismissedEvent` when an alarm is dismissed.
"""
def __init__(self, alarms: Optional[Union[list, Dict[str, Any]]] = None, audio_plugin: str = 'media.mplayer',
*args, **kwargs):
def __init__(
self,
alarms: Optional[Union[list, Dict[str, Any]]] = None,
audio_plugin: str = 'media.mplayer',
*args,
**kwargs
):
"""
:param alarms: List or name->value dict with the configured alarms. Example:
@ -231,13 +253,29 @@ class AlarmBackend(Backend):
alarms = [{'name': name, **alarm} for name, alarm in alarms.items()]
self.audio_plugin = audio_plugin
alarms = [Alarm(**{'audio_plugin': self.audio_plugin, **alarm}) for alarm in alarms]
alarms = [
Alarm(**{'audio_plugin': self.audio_plugin, **alarm}) for alarm in alarms
]
self.alarms: Dict[str, Alarm] = {alarm.name: alarm for alarm in alarms}
def add_alarm(self, when: str, actions: list, name: Optional[str] = None, audio_file: Optional[str] = None,
audio_volume: Optional[Union[int, float]] = None, enabled: bool = True) -> Alarm:
alarm = Alarm(when=when, actions=actions, name=name, enabled=enabled, audio_file=audio_file,
audio_plugin=self.audio_plugin, audio_volume=audio_volume)
def add_alarm(
self,
when: str,
actions: list,
name: Optional[str] = None,
audio_file: Optional[str] = None,
audio_volume: Optional[Union[int, float]] = None,
enabled: bool = True,
) -> Alarm:
alarm = Alarm(
when=when,
actions=actions,
name=name,
enabled=enabled,
audio_file=audio_file,
audio_plugin=self.audio_plugin,
audio_volume=audio_volume,
)
if alarm.name in self.alarms:
self.logger.info('Overwriting existing alarm {}'.format(alarm.name))
@ -274,10 +312,15 @@ class AlarmBackend(Backend):
alarm.snooze(interval=interval)
def get_alarms(self) -> List[Alarm]:
return sorted([alarm for alarm in self.alarms.values()], key=lambda alarm: alarm.get_next())
return sorted(
self.alarms.values(),
key=lambda alarm: alarm.get_next(),
)
def get_running_alarm(self) -> Optional[Alarm]:
running_alarms = [alarm for alarm in self.alarms.values() if alarm.state == AlarmState.RUNNING]
running_alarms = [
alarm for alarm in self.alarms.values() if alarm.state == AlarmState.RUNNING
]
return running_alarms[0] if running_alarms else None
def __enter__(self):
@ -285,9 +328,11 @@ class AlarmBackend(Backend):
alarm.stop()
alarm.start()
self.logger.info('Initialized alarm backend with {} alarms'.format(len(self.alarms)))
self.logger.info(
'Initialized alarm backend with {} alarms'.format(len(self.alarms))
)
def __exit__(self, exc_type, exc_val, exc_tb):
def __exit__(self, *_, **__):
for alarm in self.alarms.values():
alarm.stop()
@ -295,7 +340,9 @@ class AlarmBackend(Backend):
def loop(self):
for name, alarm in self.alarms.copy().items():
if not alarm.timer or (not alarm.timer.is_alive() and alarm.state == AlarmState.SHUTDOWN):
if not alarm.timer or (
not alarm.timer.is_alive() and alarm.state == AlarmState.SHUTDOWN
):
del self.alarms[name]
time.sleep(10)

39
platypush/backend/assistant/__init__.py
View File

@ -1,39 +0,0 @@
from abc import ABC
import threading
from typing import Optional, Dict, Any, Tuple
from platypush.backend import Backend
from platypush.context import get_plugin
from platypush.plugins.tts import TtsPlugin
class AssistantBackend(Backend):
def __init__(self, tts_plugin: Optional[str] = None, tts_args: Optional[Dict[str, Any]] = None, **kwargs):
"""
Default assistant backend constructor.
:param tts_plugin: If set, and if the assistant returns the processed response as text, then the processed
response will be played through the selected text-to-speech plugin (can be e.g. "``tts``",
"``tts.google``" or any other implementation of :class:`platypush.plugins.tts.TtsPlugin`).
:param tts_args: Extra parameters to pass to the ``say`` method of the selected TTS plugin (e.g.
language, voice or gender).
"""
super().__init__(**kwargs)
self._detection_paused = threading.Event()
self.tts_plugin = tts_plugin
self.tts_args = tts_args or {}
def pause_detection(self):
self._detection_paused.set()
def resume_detection(self):
self._detection_paused.clear()
def is_detecting(self):
return not self._detection_paused.is_set()
def _get_tts_plugin(self) -> Tuple[Optional[TtsPlugin], Dict[str, Any]]:
return get_plugin(self.tts_plugin) if self.tts_plugin else None, self.tts_args
# vim:sw=4:ts=4:et:

225
platypush/backend/assistant/google/__init__.py
View File

@ -1,225 +0,0 @@
import json
import os
import time
from platypush.backend.assistant import AssistantBackend
from platypush.message.event.assistant import (
ConversationStartEvent,
ConversationEndEvent,
ConversationTimeoutEvent,
ResponseEvent,
NoResponseEvent,
SpeechRecognizedEvent,
AlarmStartedEvent,
AlarmEndEvent,
TimerStartedEvent,
TimerEndEvent,
AlertStartedEvent,
AlertEndEvent,
MicMutedEvent,
MicUnmutedEvent,
)
class AssistantGoogleBackend(AssistantBackend):
"""
Google Assistant backend.
It listens for voice commands and post conversation events on the bus.
**WARNING**: The Google Assistant library used by this backend has officially been deprecated:
https://developers.google.com/assistant/sdk/reference/library/python/. This backend still works on most of the
devices where I use it, but its correct functioning is not guaranteed as the assistant library is no longer
maintained.
Triggers:
* :class:`platypush.message.event.assistant.ConversationStartEvent` \
when a new conversation starts
* :class:`platypush.message.event.assistant.SpeechRecognizedEvent` \
when a new voice command is recognized
* :class:`platypush.message.event.assistant.NoResponse` \
when a conversation returned no response
* :class:`platypush.message.event.assistant.ResponseEvent` \
when the assistant is speaking a response
* :class:`platypush.message.event.assistant.ConversationTimeoutEvent` \
when a conversation times out
* :class:`platypush.message.event.assistant.ConversationEndEvent` \
when a new conversation ends
* :class:`platypush.message.event.assistant.AlarmStartedEvent` \
when an alarm starts
* :class:`platypush.message.event.assistant.AlarmEndEvent` \
when an alarm ends
* :class:`platypush.message.event.assistant.TimerStartedEvent` \
when a timer starts
* :class:`platypush.message.event.assistant.TimerEndEvent` \
when a timer ends
* :class:`platypush.message.event.assistant.MicMutedEvent` \
when the microphone is muted.
* :class:`platypush.message.event.assistant.MicUnmutedEvent` \
when the microphone is un-muted.
Requires:
* **google-assistant-library** (``pip install google-assistant-library``)
* **google-assistant-sdk[samples]** (``pip install google-assistant-sdk[samples]``)
* **google-auth** (``pip install google-auth``)
"""
_default_credentials_file = os.path.join(
os.path.expanduser('~/.config'), 'google-oauthlib-tool', 'credentials.json'
)
def __init__(
self,
credentials_file=_default_credentials_file,
device_model_id='Platypush',
**kwargs
):
"""
:param credentials_file: Path to the Google OAuth credentials file
(default: ~/.config/google-oauthlib-tool/credentials.json).
See
https://developers.google.com/assistant/sdk/guides/library/python/embed/install-sample#generate_credentials
for instructions to get your own credentials file.
:type credentials_file: str
:param device_model_id: Device model ID to use for the assistant
(default: Platypush)
:type device_model_id: str
"""
super().__init__(**kwargs)
self.credentials_file = credentials_file
self.device_model_id = device_model_id
self.credentials = None
self.assistant = None
self._has_error = False
self._is_muted = False
self.logger.info('Initialized Google Assistant backend')
def _process_event(self, event):
from google.assistant.library.event import EventType, AlertType
self.logger.info('Received assistant event: {}'.format(event))
self._has_error = False
if event.type == EventType.ON_CONVERSATION_TURN_STARTED:
self.bus.post(ConversationStartEvent(assistant=self))
elif event.type == EventType.ON_CONVERSATION_TURN_FINISHED:
if not event.args.get('with_follow_on_turn'):
self.bus.post(ConversationEndEvent(assistant=self))
elif event.type == EventType.ON_CONVERSATION_TURN_TIMEOUT:
self.bus.post(ConversationTimeoutEvent(assistant=self))
elif event.type == EventType.ON_NO_RESPONSE:
self.bus.post(NoResponseEvent(assistant=self))
elif (
hasattr(EventType, 'ON_RENDER_RESPONSE')
and event.type == EventType.ON_RENDER_RESPONSE
):
self.bus.post(
ResponseEvent(assistant=self, response_text=event.args.get('text'))
)
tts, args = self._get_tts_plugin()
if tts and 'text' in event.args:
self.stop_conversation()
tts.say(text=event.args['text'], **args)
elif (
hasattr(EventType, 'ON_RESPONDING_STARTED')
and event.type == EventType.ON_RESPONDING_STARTED
and event.args.get('is_error_response', False) is True
):
self.logger.warning('Assistant response error')
elif event.type == EventType.ON_RECOGNIZING_SPEECH_FINISHED:
phrase = event.args['text'].lower().strip()
self.logger.info('Speech recognized: {}'.format(phrase))
self.bus.post(SpeechRecognizedEvent(assistant=self, phrase=phrase))
elif event.type == EventType.ON_ALERT_STARTED:
if event.args.get('alert_type') == AlertType.ALARM:
self.bus.post(AlarmStartedEvent(assistant=self))
elif event.args.get('alert_type') == AlertType.TIMER:
self.bus.post(TimerStartedEvent(assistant=self))
else:
self.bus.post(AlertStartedEvent(assistant=self))
elif event.type == EventType.ON_ALERT_FINISHED:
if event.args.get('alert_type') == AlertType.ALARM:
self.bus.post(AlarmEndEvent(assistant=self))
elif event.args.get('alert_type') == AlertType.TIMER:
self.bus.post(TimerEndEvent(assistant=self))
else:
self.bus.post(AlertEndEvent(assistant=self))
elif event.type == EventType.ON_ASSISTANT_ERROR:
self._has_error = True
if event.args.get('is_fatal'):
self.logger.error('Fatal assistant error')
else:
self.logger.warning('Assistant error')
if event.type == EventType.ON_MUTED_CHANGED:
self._is_muted = event.args.get('is_muted')
event = MicMutedEvent() if self._is_muted else MicUnmutedEvent()
self.bus.post(event)
def start_conversation(self):
"""Starts an assistant conversation"""
if self.assistant:
self.assistant.start_conversation()
def stop_conversation(self):
"""Stops an assistant conversation"""
if self.assistant:
self.assistant.stop_conversation()
def set_mic_mute(self, muted):
if not self.assistant:
self.logger.warning('Assistant not running')
return
self.assistant.set_mic_mute(muted)
def is_muted(self) -> bool:
return self._is_muted
def send_text_query(self, query):
if not self.assistant:
self.logger.warning('Assistant not running')
return
self.assistant.send_text_query(query)
def run(self):
import google.oauth2.credentials
from google.assistant.library import Assistant
super().run()
with open(self.credentials_file, 'r') as f:
self.credentials = google.oauth2.credentials.Credentials(
token=None, **json.load(f)
)
while not self.should_stop():
self._has_error = False
with Assistant(self.credentials, self.device_model_id) as assistant:
self.assistant = assistant
for event in assistant.start():
if not self.is_detecting():
self.logger.info(
'Assistant event received but detection is currently paused'
)
continue
self._process_event(event)
if self._has_error:
self.logger.info(
'Restarting the assistant after an unrecoverable error'
)
time.sleep(5)
break
# vim:sw=4:ts=4:et:

27
platypush/backend/assistant/google/manifest.yaml
View File

@ -1,27 +0,0 @@
manifest:
events:
platypush.message.event.assistant.AlarmEndEvent: when an alarm ends
platypush.message.event.assistant.AlarmStartedEvent: when an alarm starts
platypush.message.event.assistant.ConversationEndEvent: when a new conversation
ends
platypush.message.event.assistant.ConversationStartEvent: when a new conversation
starts
platypush.message.event.assistant.ConversationTimeoutEvent: when a conversation
times out
platypush.message.event.assistant.MicMutedEvent: when the microphone is muted.
platypush.message.event.assistant.MicUnmutedEvent: when the microphone is un-muted.
platypush.message.event.assistant.NoResponse: when a conversation returned no
response
platypush.message.event.assistant.ResponseEvent: when the assistant is speaking
a response
platypush.message.event.assistant.SpeechRecognizedEvent: when a new voice command
is recognized
platypush.message.event.assistant.TimerEndEvent: when a timer ends
platypush.message.event.assistant.TimerStartedEvent: when a timer starts
install:
pip:
- google-assistant-library
- google-assistant-sdk[samples]
- google-auth
package: platypush.backend.assistant.google
type: backend

193
platypush/backend/assistant/snowboy/__init__.py
View File

@ -1,193 +0,0 @@
import os
import threading
from platypush.backend.assistant import AssistantBackend
from platypush.context import get_plugin
from platypush.message.event.assistant import HotwordDetectedEvent
class AssistantSnowboyBackend(AssistantBackend):
"""
Backend for detecting custom voice hotwords through Snowboy. The purpose of
this component is only to detect the hotword specified in your Snowboy voice
model. If you want to trigger proper assistant conversations or custom
speech recognition, you should create a hook in your configuration on
HotwordDetectedEvent to trigger the conversation on whichever assistant
plugin you're using (Google, Alexa...)
Triggers:
* :class:`platypush.message.event.assistant.HotwordDetectedEvent` \
whenever the hotword has been detected
Requires:
* **snowboy** (``pip install snowboy``)
Manual installation for snowboy and its Python bindings if the command above fails::
$ [sudo] apt-get install libatlas-base-dev swig
$ [sudo] pip install pyaudio
$ git clone https://github.com/Kitt-AI/snowboy
$ cd snowboy/swig/Python3
$ make
$ cd ../..
$ python3 setup.py build
$ [sudo] python setup.py install
You will also need a voice model for the hotword detection. You can find
some under the ``resources/models`` directory of the Snowboy repository,
or train/download other models from https://snowboy.kitt.ai.
"""
def __init__(self, models, audio_gain=1.0, **kwargs):
"""
:param models: Map (name -> configuration) of voice models to be used by
the assistant. See https://snowboy.kitt.ai/ for training/downloading
models. Sample format::
ok_google: # Hotword model name
voice_model_file: /path/models/OK Google.pmdl # Voice model file location
sensitivity: 0.5 # Model sensitivity, between 0 and 1 (default: 0.5)
assistant_plugin: assistant.google.pushtotalk # When the hotword is detected trigger the Google
# push-to-talk assistant plugin (optional)
assistant_language: en-US # The assistant will conversate in English when this hotword is
detected (optional)
detect_sound: /path/to/bell.wav # Sound file to be played when the hotword is detected (optional)
ciao_google: # Hotword model name
voice_model_file: /path/models/Ciao Google.pmdl # Voice model file location
sensitivity: 0.5 # Model sensitivity, between 0 and 1 (default: 0.5)
assistant_plugin: assistant.google.pushtotalk # When the hotword is detected trigger the Google
# push-to-talk assistant plugin (optional)
assistant_language: it-IT # The assistant will conversate in Italian when this hotword is
# detected (optional)
detect_sound: /path/to/bell.wav # Sound file to be played when the hotword is detected (optional)
:type models: dict
:param audio_gain: Audio gain, between 0 and 1. Default: 1
:type audio_gain: float
"""
try:
import snowboydecoder
except ImportError:
import snowboy.snowboydecoder as snowboydecoder
super().__init__(**kwargs)
self.models = {}
self._init_models(models)
self.audio_gain = audio_gain
self.detector = snowboydecoder.HotwordDetector(
[model['voice_model_file'] for model in self.models.values()],
sensitivity=[model['sensitivity'] for model in self.models.values()],
audio_gain=self.audio_gain,
)
self.logger.info(
'Initialized Snowboy hotword detection with {} voice model configurations'.format(
len(self.models)
)
)
def _init_models(self, models):
if not models:
raise AttributeError('Please specify at least one voice model')
self.models = {}
for name, conf in models.items():
if name in self.models:
raise AttributeError('Duplicate model key {}'.format(name))
model_file = conf.get('voice_model_file')
detect_sound = conf.get('detect_sound')
if not model_file:
raise AttributeError(
'No voice_model_file specified for model {}'.format(name)
)
model_file = os.path.abspath(os.path.expanduser(model_file))
assistant_plugin_name = conf.get('assistant_plugin')
if detect_sound:
detect_sound = os.path.abspath(os.path.expanduser(detect_sound))
if not os.path.isfile(model_file):
raise FileNotFoundError(
'Voice model file {} does not exist or it not a regular file'.format(
model_file
)
)
self.models[name] = {
'voice_model_file': model_file,
'sensitivity': conf.get('sensitivity', 0.5),
'detect_sound': detect_sound,
'assistant_plugin': get_plugin(assistant_plugin_name)
if assistant_plugin_name
else None,
'assistant_language': conf.get('assistant_language'),
'tts_plugin': conf.get('tts_plugin'),
'tts_args': conf.get('tts_args', {}),
}
def hotword_detected(self, hotword):
"""
Callback called on hotword detection
"""
try:
import snowboydecoder
except ImportError:
import snowboy.snowboydecoder as snowboydecoder
def sound_thread(sound):
snowboydecoder.play_audio_file(sound)
def callback():
if not self.is_detecting():
self.logger.info(
'Hotword detected but assistant response currently paused'
)
return
self.bus.post(HotwordDetectedEvent(hotword=hotword))
model = self.models[hotword]
detect_sound = model.get('detect_sound')
assistant_plugin = model.get('assistant_plugin')
assistant_language = model.get('assistant_language')
tts_plugin = model.get('tts_plugin')
tts_args = model.get('tts_args')
if detect_sound:
threading.Thread(target=sound_thread, args=(detect_sound,)).start()
if assistant_plugin:
assistant_plugin.start_conversation(
language=assistant_language,
tts_plugin=tts_plugin,
tts_args=tts_args,
)
return callback
def on_stop(self):
super().on_stop()
if self.detector:
self.detector.terminate()
self.detector = None
def run(self):
super().run()
self.detector.start(
detected_callback=[
self.hotword_detected(hotword) for hotword in self.models.keys()
]
)
# vim:sw=4:ts=4:et:

9
platypush/backend/assistant/snowboy/manifest.yaml
View File

@ -1,9 +0,0 @@
manifest:
events:
platypush.message.event.assistant.HotwordDetectedEvent: whenever the hotword has
been detected
install:
pip:
- snowboy
package: platypush.backend.assistant.snowboy
type: backend

54
platypush/backend/button/flic/__init__.py
View File

@ -12,15 +12,12 @@ class ButtonFlicBackend(Backend):
Backend that listen for events from the Flic (https://flic.io/) bluetooth
smart buttons.
Triggers:
* :class:`platypush.message.event.button.flic.FlicButtonEvent` when a button is pressed.
The event will also contain the press sequence
(e.g. ``["ShortPressEvent", "LongPressEvent", "ShortPressEvent"]``)
Requires:
* **fliclib** (https://github.com/50ButtonsEach/fliclib-linux-hci). For the backend to work properly you need to have the ``flicd`` daemon from the fliclib running, and you have to first pair the buttons with your device using any of the scanners provided by the library.
* **fliclib** (https://github.com/50ButtonsEach/fliclib-linux-hci). For
the backend to work properly you need to have the ``flicd`` daemon
from the fliclib running, and you have to first pair the buttons with
your device using any of the scanners provided by the library.
"""
@ -29,16 +26,23 @@ class ButtonFlicBackend(Backend):
ShortPressEvent = "ShortPressEvent"
LongPressEvent = "LongPressEvent"
def __init__(self, server='localhost', long_press_timeout=_long_press_timeout,
btn_timeout=_btn_timeout, **kwargs):
def __init__(
self,
server='localhost',
long_press_timeout=_long_press_timeout,
btn_timeout=_btn_timeout,
**kwargs
):
"""
:param server: flicd server host (default: localhost)
:type server: str
:param long_press_timeout: How long you should press a button for a press action to be considered "long press" (default: 0.3 secohds)
:param long_press_timeout: How long you should press a button for a
press action to be considered "long press" (default: 0.3 secohds)
:type long_press_timeout: float
:param btn_timeout: How long since the last button release before considering the user interaction completed (default: 0.5 seconds)
:param btn_timeout: How long since the last button release before
considering the user interaction completed (default: 0.5 seconds)
:type btn_timeout: float
"""
@ -55,15 +59,16 @@ class ButtonFlicBackend(Backend):
self._btn_addr = None
self._down_pressed_time = None
self._cur_sequence = []
self.logger.info('Initialized Flic buttons backend on {}'.format(self.server))
self.logger.info('Initialized Flic buttons backend on %s', self.server)
def _got_button(self):
def _f(bd_addr):
cc = ButtonConnectionChannel(bd_addr)
cc.on_button_up_or_down = \
lambda channel, click_type, was_queued, time_diff: \
self._on_event()(bd_addr, channel, click_type, was_queued, time_diff)
cc.on_button_up_or_down = (
lambda channel, click_type, was_queued, time_diff: self._on_event()(
bd_addr, channel, click_type, was_queued, time_diff
)
)
self.client.add_connection_channel(cc)
return _f
@ -72,23 +77,27 @@ class ButtonFlicBackend(Backend):
def _f(items):
for bd_addr in items["bd_addr_of_verified_buttons"]:
self._got_button()(bd_addr)
return _f
def _on_btn_timeout(self):
def _f():
self.logger.info('Flic event triggered from {}: {}'.format(
self._btn_addr, self._cur_sequence))
self.logger.info(
'Flic event triggered from %s: %s', self._btn_addr, self._cur_sequence
)
self.bus.post(FlicButtonEvent(
btn_addr=self._btn_addr, sequence=self._cur_sequence))
self.bus.post(
FlicButtonEvent(btn_addr=self._btn_addr, sequence=self._cur_sequence)
)
self._cur_sequence = []
return _f
def _on_event(self):
# noinspection PyUnusedLocal
def _f(bd_addr, channel, click_type, was_queued, time_diff):
# _ = channel
# __ = time_diff
def _f(bd_addr, _, click_type, was_queued, __):
if was_queued:
return
@ -120,4 +129,3 @@ class ButtonFlicBackend(Backend):
# vim:sw=4:ts=4:et:

67
platypush/backend/camera/pi/__init__.py
View File

@ -15,10 +15,6 @@ class CameraPiBackend(Backend):
the :class:`platypush.plugins.camera.pi` plugin. Note that the Redis backend
must be configured and running to enable camera control.
Requires:
* **picamera** (``pip install picamera``)
This backend is **DEPRECATED**. Use the plugin :class:`platypush.plugins.camera.pi.CameraPiPlugin` instead to run
Pi camera actions. If you want to start streaming the camera on application start then simply create an event hook
on :class:`platypush.message.event.application.ApplicationStartedEvent` that runs ``camera.pi.start_streaming``.
@ -33,15 +29,32 @@ class CameraPiBackend(Backend):
return self.value == other
# noinspection PyUnresolvedReferences,PyPackageRequirements
def __init__(self, listen_port, bind_address='0.0.0.0', x_resolution=640, y_resolution=480,
redis_queue='platypush/camera/pi',
start_recording_on_startup=True,
framerate=24, hflip=False, vflip=False,
sharpness=0, contrast=0, brightness=50,
video_stabilization=False, iso=0, exposure_compensation=0,
exposure_mode='auto', meter_mode='average', awb_mode='auto',
image_effect='none', color_effects=None, rotation=0,
crop=(0.0, 0.0, 1.0, 1.0), **kwargs):
def __init__(
self,
listen_port,
bind_address='0.0.0.0',
x_resolution=640,
y_resolution=480,
redis_queue='platypush/camera/pi',
start_recording_on_startup=True,
framerate=24,
hflip=False,
vflip=False,
sharpness=0,
contrast=0,
brightness=50,
video_stabilization=False,
iso=0,
exposure_compensation=0,
exposure_mode='auto',
meter_mode='average',
awb_mode='auto',
image_effect='none',
color_effects=None,
rotation=0,
crop=(0.0, 0.0, 1.0, 1.0),
**kwargs
):
"""
See https://www.raspberrypi.org/documentation/usage/camera/python/README.md
for a detailed reference about the Pi camera options.
@ -58,7 +71,9 @@ class CameraPiBackend(Backend):
self.bind_address = bind_address
self.listen_port = listen_port
self.server_socket = socket.socket()
self.server_socket.bind((self.bind_address, self.listen_port)) # lgtm [py/bind-socket-all-network-interfaces]
self.server_socket.bind(
(self.bind_address, self.listen_port)
) # lgtm [py/bind-socket-all-network-interfaces]
self.server_socket.listen(0)
import picamera
@ -87,10 +102,7 @@ class CameraPiBackend(Backend):
self._recording_thread = None
def send_camera_action(self, action, **kwargs):
action = {
'action': action.value,
**kwargs
}
action = {'action': action.value, **kwargs}
self.redis.send_message(msg=json.dumps(action), queue_name=self.redis_queue)
@ -127,7 +139,9 @@ class CameraPiBackend(Backend):
else:
while not self.should_stop():
connection = self.server_socket.accept()[0].makefile('wb')
self.logger.info('Accepted client connection on port {}'.format(self.listen_port))
self.logger.info(
'Accepted client connection on port {}'.format(self.listen_port)
)
try:
self.camera.start_recording(connection, format=format)
@ -138,12 +152,16 @@ class CameraPiBackend(Backend):
try:
self.stop_recording()
except Exception as e:
self.logger.warning('Could not stop recording: {}'.format(str(e)))
self.logger.warning(
'Could not stop recording: {}'.format(str(e))
)
try:
connection.close()
except Exception as e:
self.logger.warning('Could not close connection: {}'.format(str(e)))
self.logger.warning(
'Could not close connection: {}'.format(str(e))
)
self.send_camera_action(self.CameraAction.START_RECORDING)
@ -152,12 +170,13 @@ class CameraPiBackend(Backend):
return
self.logger.info('Starting camera recording')
self._recording_thread = Thread(target=recording_thread,
name='PiCameraRecorder')
self._recording_thread = Thread(
target=recording_thread, name='PiCameraRecorder'
)
self._recording_thread.start()
def stop_recording(self):
""" Stops recording """
"""Stops recording"""
self.logger.info('Stopping camera recording')

18
platypush/backend/camera/pi/manifest.yaml
View File

@ -1,9 +1,21 @@
manifest:
events: {}
install:
apk:
- py3-numpy
- py3-pillow
dnf:
- python-numpy
- python-pillow
pacman:
- python-numpy
- python-pillow
apt:
- python3-numpy
- python3-pillow
pip:
- picamera
- numpy
- Pillow
- picamera
- numpy
- Pillow
package: platypush.backend.camera.pi
type: backend

113
platypush/backend/chat/telegram/__init__.py
View File

@ -4,9 +4,17 @@ from typing import Type, Optional, Union, List
from platypush.backend import Backend
from platypush.context import get_plugin
from platypush.message.event.chat.telegram import MessageEvent, CommandMessageEvent, TextMessageEvent, \
PhotoMessageEvent, VideoMessageEvent, ContactMessageEvent, DocumentMessageEvent, LocationMessageEvent, \
GroupChatCreatedEvent
from platypush.message.event.chat.telegram import (
MessageEvent,
CommandMessageEvent,
TextMessageEvent,
PhotoMessageEvent,
VideoMessageEvent,
ContactMessageEvent,
DocumentMessageEvent,
LocationMessageEvent,
GroupChatCreatedEvent,
)
from platypush.plugins.chat.telegram import ChatTelegramPlugin
@ -14,24 +22,15 @@ class ChatTelegramBackend(Backend):
"""
Telegram bot that listens for messages and updates.
Triggers:
* :class:`platypush.message.event.chat.telegram.TextMessageEvent` when a text message is received.
* :class:`platypush.message.event.chat.telegram.PhotoMessageEvent` when a photo is received.
* :class:`platypush.message.event.chat.telegram.VideoMessageEvent` when a video is received.
* :class:`platypush.message.event.chat.telegram.LocationMessageEvent` when a location is received.
* :class:`platypush.message.event.chat.telegram.ContactMessageEvent` when a contact is received.
* :class:`platypush.message.event.chat.telegram.DocumentMessageEvent` when a document is received.
* :class:`platypush.message.event.chat.telegram.CommandMessageEvent` when a command message is received.
* :class:`platypush.message.event.chat.telegram.GroupCreatedEvent` when the bot is invited to a new group.
Requires:
* The :class:`platypush.plugins.chat.telegram.ChatTelegramPlugin` plugin configured
"""
def __init__(self, authorized_chat_ids: Optional[List[Union[str, int]]] = None, **kwargs):
def __init__(
self, authorized_chat_ids: Optional[List[Union[str, int]]] = None, **kwargs
):
"""
:param authorized_chat_ids: Optional list of chat_id/user_id which are authorized to send messages to
the bot. If nothing is specified then no restrictions are applied.
@ -39,40 +38,52 @@ class ChatTelegramBackend(Backend):
super().__init__(**kwargs)
self.authorized_chat_ids = set(authorized_chat_ids or [])
self._plugin: ChatTelegramPlugin = get_plugin('chat.telegram')
self._plugin: ChatTelegramPlugin = get_plugin('chat.telegram') # type: ignore
def _authorize(self, msg):
if not self.authorized_chat_ids:
return
if msg.chat.type == 'private' and msg.chat.id not in self.authorized_chat_ids:
self.logger.info('Received message from unauthorized chat_id {}'.format(msg.chat.id))
self._plugin.send_message(chat_id=msg.chat.id, text='You are not allowed to send messages to this bot')
self.logger.info(
'Received message from unauthorized chat_id %s', msg.chat.id
)
self._plugin.send_message(
chat_id=msg.chat.id,
text='You are not allowed to send messages to this bot',
)
raise PermissionError
def _msg_hook(self, cls: Type[MessageEvent]):
# noinspection PyUnusedLocal
def hook(update, context):
def hook(update, _):
msg = update.effective_message
try:
self._authorize(msg)
self.bus.post(cls(chat_id=update.effective_chat.id,
message=self._plugin.parse_msg(msg).output,
user=self._plugin.parse_user(update.effective_user).output))
self.bus.post(
cls(
chat_id=update.effective_chat.id,
message=self._plugin.parse_msg(msg).output,
user=self._plugin.parse_user(update.effective_user).output,
)
)
except PermissionError:
pass
return hook
def _group_hook(self):
# noinspection PyUnusedLocal
def hook(update, context):
msg = update.effective_message
if msg.group_chat_created:
self.bus.post(GroupChatCreatedEvent(chat_id=update.effective_chat.id,
message=self._plugin.parse_msg(msg).output,
user=self._plugin.parse_user(update.effective_user).output))
self.bus.post(
GroupChatCreatedEvent(
chat_id=update.effective_chat.id,
message=self._plugin.parse_msg(msg).output,
user=self._plugin.parse_user(update.effective_user).output,
)
)
elif msg.photo:
self._msg_hook(PhotoMessageEvent)(update, context)
elif msg.video:
@ -92,27 +103,33 @@ class ChatTelegramBackend(Backend):
return hook
def _command_hook(self):
# noinspection PyUnusedLocal
def hook(update, context):
def hook(update, _):
msg = update.effective_message
m = re.match('\s*/([0-9a-zA-Z_-]+)\s*(.*)', msg.text)
m = re.match(r'\s*/([0-9a-zA-Z_-]+)\s*(.*)', msg.text)
if not m:
self.logger.warning('Invalid command: %s', msg.text)
return
cmd = m.group(1).lower()
args = [arg for arg in re.split('\s+', m.group(2)) if len(arg)]
args = [arg for arg in re.split(r'\s+', m.group(2)) if len(arg)]
try:
self._authorize(msg)
self.bus.post(CommandMessageEvent(chat_id=update.effective_chat.id,
command=cmd,
cmdargs=args,
message=self._plugin.parse_msg(msg).output,
user=self._plugin.parse_user(update.effective_user).output))
self.bus.post(
CommandMessageEvent(
chat_id=update.effective_chat.id,
command=cmd,
cmdargs=args,
message=self._plugin.parse_msg(msg).output,
user=self._plugin.parse_user(update.effective_user).output,
)
)
except PermissionError:
pass
return hook
def run(self):
# noinspection PyPackageRequirements
from telegram.ext import MessageHandler, Filters
super().run()
@ -120,12 +137,24 @@ class ChatTelegramBackend(Backend):
dispatcher = telegram.dispatcher
dispatcher.add_handler(MessageHandler(Filters.group, self._group_hook()))
dispatcher.add_handler(MessageHandler(Filters.text, self._msg_hook(TextMessageEvent)))
dispatcher.add_handler(MessageHandler(Filters.photo, self._msg_hook(PhotoMessageEvent)))
dispatcher.add_handler(MessageHandler(Filters.video, self._msg_hook(VideoMessageEvent)))
dispatcher.add_handler(MessageHandler(Filters.contact, self._msg_hook(ContactMessageEvent)))
dispatcher.add_handler(MessageHandler(Filters.location, self._msg_hook(LocationMessageEvent)))
dispatcher.add_handler(MessageHandler(Filters.document, self._msg_hook(DocumentMessageEvent)))
dispatcher.add_handler(
MessageHandler(Filters.text, self._msg_hook(TextMessageEvent))
)
dispatcher.add_handler(
MessageHandler(Filters.photo, self._msg_hook(PhotoMessageEvent))
)
dispatcher.add_handler(
MessageHandler(Filters.video, self._msg_hook(VideoMessageEvent))
)
dispatcher.add_handler(
MessageHandler(Filters.contact, self._msg_hook(ContactMessageEvent))
)
dispatcher.add_handler(
MessageHandler(Filters.location, self._msg_hook(LocationMessageEvent))
)
dispatcher.add_handler(
MessageHandler(Filters.document, self._msg_hook(DocumentMessageEvent))
)
dispatcher.add_handler(MessageHandler(Filters.command, self._command_hook()))
self.logger.info('Initialized Telegram backend')

2
platypush/backend/chat/telegram/manifest.yaml
View File

@ -5,7 +5,7 @@ manifest:
platypush.message.event.chat.telegram.ContactMessageEvent: when a contact is received.
platypush.message.event.chat.telegram.DocumentMessageEvent: when a document is
received.
platypush.message.event.chat.telegram.GroupCreatedEvent: when the bot is invited
platypush.message.event.chat.telegram.GroupChatCreatedEvent: when the bot is invited
to a new group.
platypush.message.event.chat.telegram.LocationMessageEvent: when a location is
received.

137
platypush/backend/covid19/__init__.py
View File

@ -1,137 +0,0 @@
import datetime
import os
from typing import Optional, Union, List, Dict, Any
from sqlalchemy import create_engine, Column, Integer, String, DateTime
from sqlalchemy.orm import sessionmaker, scoped_session
from platypush.backend import Backend
from platypush.common.db import declarative_base
from platypush.config import Config
from platypush.context import get_plugin
from platypush.message.event.covid19 import Covid19UpdateEvent
from platypush.plugins.covid19 import Covid19Plugin
Base = declarative_base()
Session = scoped_session(sessionmaker())
class Covid19Update(Base):
"""Models the Covid19Data table"""
__tablename__ = 'covid19data'
__table_args__ = {'sqlite_autoincrement': True}
country = Column(String, primary_key=True)
confirmed = Column(Integer, nullable=False, default=0)
deaths = Column(Integer, nullable=False, default=0)
recovered = Column(Integer, nullable=False, default=0)
last_updated_at = Column(DateTime, nullable=False, default=datetime.datetime.utcnow)
class Covid19Backend(Backend):
"""
This backend polls new data about the Covid-19 pandemic diffusion and triggers events when new data is available.
Triggers:
- :class:`platypush.message.event.covid19.Covid19UpdateEvent` when new data is available.
"""
# noinspection PyProtectedMember
def __init__(
self,
country: Optional[Union[str, List[str]]],
poll_seconds: Optional[float] = 3600.0,
**kwargs
):
"""
:param country: Default country (or list of countries) to retrieve the stats for. It can either be the full
country name or the country code. Special values:
- ``world``: Get worldwide stats.
- ``all``: Get all the available stats.
Default: either the default configured on the :class:`platypush.plugins.covid19.Covid19Plugin` plugin or
``world``.
:param poll_seconds: How often the backend should check for new check-ins (default: one hour).
"""
super().__init__(poll_seconds=poll_seconds, **kwargs)
self._plugin: Covid19Plugin = get_plugin('covid19')
self.country: List[str] = self._plugin._get_countries(country)
self.workdir = os.path.join(
os.path.expanduser(Config.get('workdir')), 'covid19'
)
self.dbfile = os.path.join(self.workdir, 'data.db')
os.makedirs(self.workdir, exist_ok=True)
def __enter__(self):
self.logger.info('Started Covid19 backend')
def __exit__(self, exc_type, exc_val, exc_tb):
self.logger.info('Stopped Covid19 backend')
def _process_update(self, summary: Dict[str, Any], session: Session):
update_time = datetime.datetime.fromisoformat(
summary['Date'].replace('Z', '+00:00')
)
self.bus.post(
Covid19UpdateEvent(
country=summary['Country'],
country_code=summary['CountryCode'],
confirmed=summary['TotalConfirmed'],
deaths=summary['TotalDeaths'],
recovered=summary['TotalRecovered'],
update_time=update_time,
)
)
session.merge(
Covid19Update(
country=summary['CountryCode'],
confirmed=summary['TotalConfirmed'],
deaths=summary['TotalDeaths'],
recovered=summary['TotalRecovered'],
last_updated_at=update_time,
)
)
def loop(self):
# noinspection PyUnresolvedReferences
summaries = self._plugin.summary(self.country).output
if not summaries:
return
engine = create_engine(
'sqlite:///{}'.format(self.dbfile),
connect_args={'check_same_thread': False},
)
Base.metadata.create_all(engine)
Session.configure(bind=engine)
session = Session()
last_records = {
record.country: record
for record in session.query(Covid19Update)
.filter(Covid19Update.country.in_(self.country))
.all()
}
for summary in summaries:
country = summary['CountryCode']
last_record = last_records.get(country)
if (
not last_record
or summary['TotalConfirmed'] != last_record.confirmed
or summary['TotalDeaths'] != last_record.deaths
or summary['TotalRecovered'] != last_record.recovered
):
self._process_update(summary=summary, session=session)
session.commit()
# vim:sw=4:ts=4:et:

7
platypush/backend/covid19/manifest.yaml
View File

@ -1,7 +0,0 @@
manifest:
events:
platypush.message.event.covid19.Covid19UpdateEvent: when new data is available.
install:
pip: []
package: platypush.backend.covid19
type: backend

29
platypush/backend/file/monitor/__init__.py
View File

@ -10,17 +10,6 @@ from .entities.resources import MonitoredResource, MonitoredPattern, MonitoredRe
class FileMonitorBackend(Backend):
"""
This backend monitors changes to local files and directories using the Watchdog API.
Triggers:
* :class:`platypush.message.event.file.FileSystemCreateEvent` if a resource is created.
* :class:`platypush.message.event.file.FileSystemDeleteEvent` if a resource is removed.
* :class:`platypush.message.event.file.FileSystemModifyEvent` if a resource is modified.
Requires:
* **watchdog** (``pip install watchdog``)
"""
class EventHandlerFactory:
@ -29,20 +18,28 @@ class FileMonitorBackend(Backend):
"""
@staticmethod
def from_resource(resource: Union[str, Dict[str, Any], MonitoredResource]) -> EventHandler:
def from_resource(
resource: Union[str, Dict[str, Any], MonitoredResource]
) -> EventHandler:
if isinstance(resource, str):
resource = MonitoredResource(resource)
elif isinstance(resource, dict):
if 'regexes' in resource or 'ignore_regexes' in resource:
resource = MonitoredRegex(**resource)
elif 'patterns' in resource or 'ignore_patterns' in resource or 'ignore_directories' in resource:
elif (
'patterns' in resource
or 'ignore_patterns' in resource
or 'ignore_directories' in resource
):
resource = MonitoredPattern(**resource)
else:
resource = MonitoredResource(**resource)
return EventHandler.from_resource(resource)
def __init__(self, paths: Iterable[Union[str, Dict[str, Any], MonitoredResource]], **kwargs):
def __init__(
self, paths: Iterable[Union[str, Dict[str, Any], MonitoredResource]], **kwargs
):
"""
:param paths: List of paths to monitor. Paths can either be expressed in any of the following ways:
@ -113,7 +110,9 @@ class FileMonitorBackend(Backend):
for path in paths:
handler = self.EventHandlerFactory.from_resource(path)
self._observer.schedule(handler, handler.resource.path, recursive=handler.resource.recursive)
self._observer.schedule(
handler, handler.resource.path, recursive=handler.resource.recursive
)
def run(self):
super().run()

10
platypush/backend/file/monitor/manifest.yaml
View File

@ -4,7 +4,15 @@ manifest:
platypush.message.event.file.FileSystemDeleteEvent: if a resource is removed.
platypush.message.event.file.FileSystemModifyEvent: if a resource is modified.
install:
apk:
- py3-watchdog
apt:
- python3-watchdog
dnf:
- python-watchdog
pacman:
- python-watchdog
pip:
- watchdog
- watchdog
package: platypush.backend.file.monitor
type: backend

16
platypush/backend/foursquare/__init__.py
View File

@ -14,10 +14,6 @@ class FoursquareBackend(Backend):
* The :class:`platypush.plugins.foursquare.FoursquarePlugin` plugin configured and enabled.
Triggers:
- :class:`platypush.message.event.foursquare.FoursquareCheckinEvent` when a new check-in occurs.
"""
_last_created_at_varname = '_foursquare_checkin_last_created_at'
@ -30,8 +26,12 @@ class FoursquareBackend(Backend):
self._last_created_at = None
def __enter__(self):
self._last_created_at = int(get_plugin('variable').get(self._last_created_at_varname).
output.get(self._last_created_at_varname) or 0)
self._last_created_at = int(
get_plugin('variable')
.get(self._last_created_at_varname)
.output.get(self._last_created_at_varname)
or 0
)
self.logger.info('Started Foursquare backend')
def loop(self):
@ -46,7 +46,9 @@ class FoursquareBackend(Backend):
self.bus.post(FoursquareCheckinEvent(checkin=last_checkin))
self._last_created_at = last_checkin_created_at
get_plugin('variable').set(**{self._last_created_at_varname: self._last_created_at})
get_plugin('variable').set(
**{self._last_created_at_varname: self._last_created_at}
)
# vim:sw=4:ts=4:et:

21
platypush/backend/github/__init__.py
View File

@ -60,27 +60,6 @@ class GithubBackend(Backend):
- ``notifications``
- ``read:org`` if you want to access repositories on organization level.
Triggers:
- :class:`platypush.message.event.github.GithubPushEvent` when a new push is created.
- :class:`platypush.message.event.github.GithubCommitCommentEvent` when a new commit comment is created.
- :class:`platypush.message.event.github.GithubCreateEvent` when a tag or branch is created.
- :class:`platypush.message.event.github.GithubDeleteEvent` when a tag or branch is deleted.
- :class:`platypush.message.event.github.GithubForkEvent` when a user forks a repository.
- :class:`platypush.message.event.github.GithubWikiEvent` when new activity happens on a repository wiki.
- :class:`platypush.message.event.github.GithubIssueCommentEvent` when new activity happens on an issue comment.
- :class:`platypush.message.event.github.GithubIssueEvent` when new repository issue activity happens.
- :class:`platypush.message.event.github.GithubMemberEvent` when new repository collaborators activity happens.
- :class:`platypush.message.event.github.GithubPublicEvent` when a repository goes public.
- :class:`platypush.message.event.github.GithubPullRequestEvent` when new pull request related activity happens.
- :class:`platypush.message.event.github.GithubPullRequestReviewCommentEvent` when activity happens on a pull
request commit.
- :class:`platypush.message.event.github.GithubReleaseEvent` when a new release happens.
- :class:`platypush.message.event.github.GithubSponsorshipEvent` when new sponsorship related activity happens.
- :class:`platypush.message.event.github.GithubWatchEvent` when someone stars/starts watching a repository.
- :class:`platypush.message.event.github.GithubEvent` for any event that doesn't fall in the above categories
(``event_type`` will be set accordingly).
"""
_base_url = 'https://api.github.com'

83
platypush/backend/google/fit/__init__.py
View File

@ -13,28 +13,30 @@ class GoogleFitBackend(Backend):
measurements, new fitness activities etc.) on the specified data streams and
fire an event upon new data.
Triggers:
* :class:`platypush.message.event.google.fit.GoogleFitEvent` when a new
data point is received on one of the registered streams.
Requires:
* The **google.fit** plugin
(:class:`platypush.plugins.google.fit.GoogleFitPlugin`) enabled.
* The **db** plugin (:class:`platypush.plugins.db`) configured
(:class:`platypush.plugins.google.fit.GoogleFitPlugin`) enabled.
"""
_default_poll_seconds = 60
_default_user_id = 'me'
_last_timestamp_varname = '_GOOGLE_FIT_LAST_TIMESTAMP_'
def __init__(self, data_sources, user_id=_default_user_id,
poll_seconds=_default_poll_seconds, *args, **kwargs):
def __init__(
self,
data_sources,
user_id=_default_user_id,
poll_seconds=_default_poll_seconds,
*args,
**kwargs
):
"""
:param data_sources: Google Fit data source IDs to monitor. You can
get a list of the available data sources through the
:meth:`platypush.plugins.google.fit.get_data_sources` action
:meth:`platypush.plugins.google.fit.GoogleFitPlugin.get_data_sources`
action
:type data_sources: list[str]
:param user_id: Google user ID to track (default: 'me')
@ -53,23 +55,31 @@ class GoogleFitBackend(Backend):
def run(self):
super().run()
self.logger.info('Started Google Fit backend on data sources {}'.format(
self.data_sources))
self.logger.info(
'Started Google Fit backend on data sources {}'.format(self.data_sources)
)
while not self.should_stop():
try:
for data_source in self.data_sources:
varname = self._last_timestamp_varname + data_source
last_timestamp = float(get_plugin('variable').
get(varname).output.get(varname) or 0)
last_timestamp = float(
get_plugin('variable').get(varname).output.get(varname) or 0
)
new_last_timestamp = last_timestamp
self.logger.info('Processing new entries from data source {}, last timestamp: {}'.
format(data_source,
str(datetime.datetime.fromtimestamp(last_timestamp))))
self.logger.info(
'Processing new entries from data source {}, last timestamp: {}'.format(
data_source,
str(datetime.datetime.fromtimestamp(last_timestamp)),
)
)
data_points = get_plugin('google.fit').get_data(
user_id=self.user_id, data_source_id=data_source).output
data_points = (
get_plugin('google.fit')
.get_data(user_id=self.user_id, data_source_id=data_source)
.output
)
new_data_points = 0
for dp in data_points:
@ -78,25 +88,34 @@ class GoogleFitBackend(Backend):
del dp['dataSourceId']
if dp_time > last_timestamp:
self.bus.post(GoogleFitEvent(
user_id=self.user_id, data_source_id=data_source,
data_type=dp.pop('dataTypeName'),
start_time=dp_time,
end_time=dp.pop('endTime'),
modified_time=dp.pop('modifiedTime'),
values=dp.pop('values'),
**{camel_case_to_snake_case(k): v
for k, v in dp.items()}
))
self.bus.post(
GoogleFitEvent(
user_id=self.user_id,
data_source_id=data_source,
data_type=dp.pop('dataTypeName'),
start_time=dp_time,
end_time=dp.pop('endTime'),
modified_time=dp.pop('modifiedTime'),
values=dp.pop('values'),
**{
camel_case_to_snake_case(k): v
for k, v in dp.items()
}
)
)
new_data_points += 1
new_last_timestamp = max(dp_time, new_last_timestamp)
last_timestamp = new_last_timestamp
self.logger.info('Got {} new entries from data source {}, last timestamp: {}'.
format(new_data_points, data_source,
str(datetime.datetime.fromtimestamp(last_timestamp))))
self.logger.info(
'Got {} new entries from data source {}, last timestamp: {}'.format(
new_data_points,
data_source,
str(datetime.datetime.fromtimestamp(last_timestamp)),
)
)
get_plugin('variable').set(**{varname: last_timestamp})
except Exception as e:

14
platypush/backend/google/pubsub/__init__.py
View File

@ -10,18 +10,8 @@ from platypush.message.event.google.pubsub import GooglePubsubMessageEvent
class GooglePubsubBackend(Backend):
"""
Subscribe to a list of topics on a Google Pub/Sub instance. See
:class:`platypush.plugins.google.pubsub.GooglePubsubPlugin` for a reference on how to generate your
project and credentials file.
Triggers:
* :class:`platypush.message.event.google.pubsub.GooglePubsubMessageEvent` when a new message is received on
a subscribed topic.
Requires:
* **google-cloud-pubsub** (``pip install google-cloud-pubsub``)
:class:`platypush.plugins.google.pubsub.GooglePubsubPlugin` for a reference on how to generate your
project and credentials file.
"""
def __init__(

100
platypush/backend/gps/__init__.py
View File

@ -9,17 +9,6 @@ class GpsBackend(Backend):
"""
This backend can interact with a GPS device and listen for events.
Triggers:
* :class:`platypush.message.event.gps.GPSVersionEvent` when a GPS device advertises its version data
* :class:`platypush.message.event.gps.GPSDeviceEvent` when a GPS device is connected or updated
* :class:`platypush.message.event.gps.GPSUpdateEvent` when a GPS device has new data
Requires:
* **gps** (``pip install gps``)
* **gpsd** daemon running (``apt-get install gpsd`` or ``pacman -S gpsd`` depending on your distro)
Once installed gpsd you need to run it and associate it to your device. Example if your GPS device communicates
over USB and is available on /dev/ttyUSB0::
@ -52,41 +41,68 @@ class GpsBackend(Backend):
with self._session_lock:
if not self._session:
self._session = gps.gps(host=self.gpsd_server, port=self.gpsd_port, reconnect=True)
self._session = gps.gps(
host=self.gpsd_server, port=self.gpsd_port, reconnect=True
)
self._session.stream(gps.WATCH_ENABLE | gps.WATCH_NEWSTYLE)
return self._session
def _gps_report_to_event(self, report):
if report.get('class').lower() == 'version':
return GPSVersionEvent(release=report.get('release'),
rev=report.get('rev'),
proto_major=report.get('proto_major'),
proto_minor=report.get('proto_minor'))
return GPSVersionEvent(
release=report.get('release'),
rev=report.get('rev'),
proto_major=report.get('proto_major'),
proto_minor=report.get('proto_minor'),
)
if report.get('class').lower() == 'devices':
for device in report.get('devices', []):
if device.get('path') not in self._devices or device != self._devices.get('path'):
if device.get(
'path'
) not in self._devices or device != self._devices.get('path'):
# noinspection DuplicatedCode
self._devices[device.get('path')] = device
return GPSDeviceEvent(path=device.get('path'), activated=device.get('activated'),
native=device.get('native'), bps=device.get('bps'),
parity=device.get('parity'), stopbits=device.get('stopbits'),
cycle=device.get('cycle'), driver=device.get('driver'))
return GPSDeviceEvent(
path=device.get('path'),
activated=device.get('activated'),
native=device.get('native'),
bps=device.get('bps'),
parity=device.get('parity'),
stopbits=device.get('stopbits'),
cycle=device.get('cycle'),
driver=device.get('driver'),
)
if report.get('class').lower() == 'device':
# noinspection DuplicatedCode
self._devices[report.get('path')] = report
return GPSDeviceEvent(path=report.get('path'), activated=report.get('activated'),
native=report.get('native'), bps=report.get('bps'),
parity=report.get('parity'), stopbits=report.get('stopbits'),
cycle=report.get('cycle'), driver=report.get('driver'))
return GPSDeviceEvent(
path=report.get('path'),
activated=report.get('activated'),
native=report.get('native'),
bps=report.get('bps'),
parity=report.get('parity'),
stopbits=report.get('stopbits'),
cycle=report.get('cycle'),
driver=report.get('driver'),
)
if report.get('class').lower() == 'tpv':
return GPSUpdateEvent(device=report.get('device'), latitude=report.get('lat'), longitude=report.get('lon'),
altitude=report.get('alt'), mode=report.get('mode'), epv=report.get('epv'),
eph=report.get('eph'), sep=report.get('sep'))
return GPSUpdateEvent(
device=report.get('device'),
latitude=report.get('lat'),
longitude=report.get('lon'),
altitude=report.get('alt'),
mode=report.get('mode'),
epv=report.get('epv'),
eph=report.get('eph'),
sep=report.get('sep'),
)
def run(self):
super().run()
self.logger.info('Initialized GPS backend on {}:{}'.format(self.gpsd_server, self.gpsd_port))
self.logger.info(
'Initialized GPS backend on {}:{}'.format(self.gpsd_server, self.gpsd_port)
)
last_event = None
while not self.should_stop():
@ -94,15 +110,31 @@ class GpsBackend(Backend):
session = self._get_session()
report = session.next()
event = self._gps_report_to_event(report)
if event and (last_event is None or
abs((last_event.args.get('latitude') or 0) - (event.args.get('latitude') or 0)) >= self._lat_lng_tolerance or
abs((last_event.args.get('longitude') or 0) - (event.args.get('longitude') or 0)) >= self._lat_lng_tolerance or
abs((last_event.args.get('altitude') or 0) - (event.args.get('altitude') or 0)) >= self._alt_tolerance):
if event and (
last_event is None
or abs(
(last_event.args.get('latitude') or 0)
- (event.args.get('latitude') or 0)
)
>= self._lat_lng_tolerance
or abs(
(last_event.args.get('longitude') or 0)
- (event.args.get('longitude') or 0)
)
>= self._lat_lng_tolerance
or abs(
(last_event.args.get('altitude') or 0)
- (event.args.get('altitude') or 0)
)
>= self._alt_tolerance
):
self.bus.post(event)
last_event = event
except Exception as e:
if isinstance(e, StopIteration):
self.logger.warning('GPS service connection lost, check that gpsd is running')
self.logger.warning(
'GPS service connection lost, check that gpsd is running'
)
else:
self.logger.exception(e)

10
platypush/backend/gps/manifest.yaml
View File

@ -6,11 +6,15 @@ manifest:
platypush.message.event.gps.GPSVersionEvent: when a GPS device advertises its
version data
install:
pip:
- gps
pacman:
apk:
- gpsd
apt:
- gpsd
dnf:
- gpsd
pacman:
- gpsd
pip:
- gps
package: platypush.backend.gps
type: backend

121
platypush/backend/http/__init__.py
View File

@ -2,13 +2,17 @@ import asyncio
import os
import pathlib
import secrets
import signal
import threading
from functools import partial
from multiprocessing import Process
from time import time
from typing import List, Mapping, Optional
from tornado.httpserver import HTTPServer
from typing import Mapping, Optional
import psutil
from tornado.httpserver import HTTPServer
from tornado.netutil import bind_sockets
from tornado.process import cpu_count, fork_processes
from tornado.wsgi import WSGIContainer
@ -18,9 +22,9 @@ from platypush.backend import Backend
from platypush.backend.http.app import application
from platypush.backend.http.app.utils import get_streaming_routes, get_ws_routes
from platypush.backend.http.app.ws.events import WSEventProxy
from platypush.bus.redis import RedisBus
from platypush.config import Config
from platypush.utils import get_remaining_timeout
class HttpBackend(Backend):
@ -188,12 +192,15 @@ class HttpBackend(Backend):
"""
_DEFAULT_HTTP_PORT = 8008
DEFAULT_HTTP_PORT = 8008
"""The default listen port for the webserver."""
_STOP_TIMEOUT = 5
"""How long we should wait (in seconds) before killing the worker processes."""
def __init__(
self,
port: int = _DEFAULT_HTTP_PORT,
port: int = DEFAULT_HTTP_PORT,
bind_address: str = '0.0.0.0',
resource_dirs: Optional[Mapping[str, str]] = None,
secret_key_file: Optional[str] = None,
@ -227,7 +234,6 @@ class HttpBackend(Backend):
self.port = port
self._server_proc: Optional[Process] = None
self._workers: List[Process] = []
self._service_registry_thread = None
self.bind_address = bind_address
@ -254,35 +260,37 @@ class HttpBackend(Backend):
"""On backend stop"""
super().on_stop()
self.logger.info('Received STOP event on HttpBackend')
start_time = time()
timeout = 5
workers = self._workers.copy()
for i, worker in enumerate(workers[::-1]):
if worker and worker.is_alive():
worker.terminate()
worker.join(timeout=max(0, start_time + timeout - time()))
if worker and worker.is_alive():
worker.kill()
self._workers.pop(i)
start = time()
remaining_time: partial[float] = partial( # type: ignore
get_remaining_timeout, timeout=self._STOP_TIMEOUT, start=start
)
if self._server_proc:
self._server_proc.terminate()
self._server_proc.join(timeout=5)
self._server_proc = None
if self._server_proc.pid:
try:
os.kill(self._server_proc.pid, signal.SIGINT)
except OSError:
pass
if self._server_proc and self._server_proc.is_alive():
self._server_proc.join(timeout=remaining_time() / 2)
try:
self._server_proc.terminate()
self._server_proc.join(timeout=remaining_time() / 2)
except AttributeError:
pass
if self._server_proc and self._server_proc.is_alive():
self._server_proc.kill()
self._server_proc = None
self.logger.info('HTTP server terminated')
if self._service_registry_thread and self._service_registry_thread.is_alive():
self._service_registry_thread.join(timeout=5)
self._service_registry_thread.join(timeout=remaining_time())
self._service_registry_thread = None
self.logger.info('HTTP server terminated')
def notify_web_clients(self, event):
"""Notify all the connected web clients (over websocket) of a new event"""
WSEventProxy.publish(event) # noqa: E1120
@ -344,7 +352,10 @@ class HttpBackend(Backend):
try:
await asyncio.Event().wait()
except (asyncio.CancelledError, KeyboardInterrupt):
return
pass
finally:
server.stop()
await server.close_all_connections()
def _web_server_proc(self):
self.logger.info(
@ -371,7 +382,65 @@ class HttpBackend(Backend):
future = self._post_fork_main(sockets)
asyncio.run(future)
except (asyncio.CancelledError, KeyboardInterrupt):
return
pass
finally:
self._stop_workers()
def _stop_workers(self):
"""
Stop all the worker processes.
We have to run this manually on server termination because of a
long-standing issue with Tornado not being able to wind down the forked
workers when the server terminates:
https://github.com/tornadoweb/tornado/issues/1912.
"""
parent_pid = (
self._server_proc.pid
if self._server_proc and self._server_proc.pid
else None
)
if not parent_pid:
return
try:
cur_proc = psutil.Process(parent_pid)
except psutil.NoSuchProcess:
return
# Send a SIGTERM to all the children
children = cur_proc.children()
for child in children:
if child.pid != parent_pid and child.is_running():
try:
os.kill(child.pid, signal.SIGTERM)
except OSError as e:
self.logger.warning(
'Could not send SIGTERM to PID %d: %s', child.pid, e
)
# Initialize the timeout
start = time()
remaining_time: partial[int] = partial( # type: ignore
get_remaining_timeout, timeout=self._STOP_TIMEOUT, start=start, cls=int
)
# Wait for all children to terminate (with timeout)
for child in children:
if child.pid != parent_pid and child.is_running():
try:
child.wait(timeout=remaining_time())
except TimeoutError:
pass
# Send a SIGKILL to any child process that is still running
for child in children:
if child.pid != parent_pid and child.is_running():
try:
child.kill()
except OSError:
pass
def _start_web_server(self):
self._server_proc = Process(target=self._web_server_proc)

1
platypush/backend/http/app/mixins/__init__.py
View File

@ -117,6 +117,7 @@ class PubSubMixin:
"""
try:
with self.pubsub as pubsub:
pubsub.subscribe(*self._subscriptions)
for msg in pubsub.listen():
channel = msg.get('channel', b'').decode()
if msg.get('type') != 'message' or not (

46
platypush/backend/http/app/routes/plugins/tts/mimic3.py
View File

@ -1,46 +0,0 @@
import requests
from urllib.parse import urljoin
from flask import abort, request, Blueprint
from platypush.backend.http.app import template_folder
mimic3 = Blueprint('mimic3', __name__, template_folder=template_folder)
# Declare routes list
__routes__ = [
mimic3,
]
@mimic3.route('/tts/mimic3/say', methods=['GET'])
def proxy_tts_request():
"""
This route is used to proxy the POST request to the Mimic3 TTS server
through a GET, so it can be easily processed as a URL through a media
plugin.
"""
required_args = {
'text',
'server_url',
'voice',
}
missing_args = required_args.difference(set(request.args.keys()))
if missing_args:
abort(400, f'Missing parameters: {missing_args}')
args = {arg: request.args[arg] for arg in required_args}
rs = requests.post(
urljoin(args['server_url'], '/api/tts'),
data=args['text'],
params={
'voice': args['voice'],
},
)
return rs.content
# vim:sw=4:ts=4:et:

3
platypush/backend/http/app/streaming/_base.py
View File

@ -3,7 +3,6 @@ from http.client import responses
import json
from logging import getLogger
from typing import Optional
from typing_extensions import override
from tornado.web import RequestHandler, stream_request_body
@ -22,7 +21,6 @@ class StreamingRoute(RequestHandler, PubSubMixin, ABC):
super().__init__(*args, **kwargs)
self.logger = getLogger(__name__)
@override
def prepare(self):
"""
Request preparation logic. It performs user authentication if
@ -38,7 +36,6 @@ class StreamingRoute(RequestHandler, PubSubMixin, ABC):
'Client %s connected to %s', self.request.remote_ip, self.request.path
)
@override
def write_error(self, status_code: int, error: Optional[str] = None, **_):
"""
Make sure that errors are always returned in JSON format.

3
platypush/backend/http/app/streaming/plugins/camera.py
View File

@ -1,7 +1,6 @@
from enum import Enum
import json
from typing import Optional
from typing_extensions import override
from tornado.web import stream_request_body
from platypush.context import get_plugin
@ -37,7 +36,6 @@ class CameraRoute(StreamingRoute):
self._request_type = RequestType.UNKNOWN
self._extension: str = ''
@override
@classmethod
def path(cls) -> str:
return r"/camera/([a-zA-Z0-9_./]+)/([a-zA-Z0-9_]+)\.?([a-zA-Z0-9_]+)?"
@ -95,7 +93,6 @@ class CameraRoute(StreamingRoute):
return kwargs
@override
@classmethod
def _get_redis_queue(cls, camera: CameraPlugin, *_, **__) -> str:
plugin_name = get_plugin_name_by_class(camera.__class__)

3
platypush/backend/http/app/streaming/plugins/sound.py
View File

@ -1,7 +1,6 @@
from contextlib import contextmanager
import json
from typing import Generator, Optional
from typing_extensions import override
from tornado.web import stream_request_body
@ -24,7 +23,6 @@ class SoundRoute(StreamingRoute):
self._audio_headers_written: bool = False
"""Send the audio file headers before we send the first audio frame."""
@override
@classmethod
def path(cls) -> str:
return r"/sound/stream\.?([a-zA-Z0-9_]+)?"
@ -44,7 +42,6 @@ class SoundRoute(StreamingRoute):
yield
send_request('sound.stop_recording')
@override
@classmethod
def _get_redis_queue(cls, *_, device: Optional[str] = None, **__) -> str:
return '/'.join([cls._redis_queue_prefix, *([device] if device else [])])

3
platypush/backend/http/app/utils/__init__.py
View File

@ -4,7 +4,7 @@ from .auth import (
authenticate_user_pass,
get_auth_status,
)
from .bus import bus, get_message_response, send_message, send_request
from .bus import bus, send_message, send_request
from .logger import logger
from .routes import (
get_http_port,
@ -25,7 +25,6 @@ __all__ = [
'get_http_port',
'get_ip_or_hostname',
'get_local_base_url',
'get_message_response',
'get_remote_base_url',
'get_routes',
'get_streaming_routes',

25
platypush/backend/http/app/utils/bus.py
View File

@ -1,11 +1,9 @@
from redis import Redis
from platypush.bus.redis import RedisBus
from platypush.config import Config
from platypush.context import get_backend
from platypush.message import Message
from platypush.message.request import Request
from platypush.utils import get_redis_conf, get_redis_queue_name_by_message
from platypush.utils import get_redis_conf, get_message_response
from .logger import logger
@ -67,24 +65,3 @@ def send_request(action, wait_for_response=True, **kwargs):
msg['args'] = kwargs
return send_message(msg, wait_for_response=wait_for_response)
def get_message_response(msg):
"""
Get the response to the given message.
:param msg: The message to get the response for.
:return: The response to the given message.
"""
redis = Redis(**bus().redis_args)
redis_queue = get_redis_queue_name_by_message(msg)
if not redis_queue:
return None
response = redis.blpop(redis_queue, timeout=60)
if response and len(response) > 1:
response = Message.build(response[1])
else:
response = None
return response

2
platypush/backend/http/app/utils/routes.py
View File

@ -14,7 +14,7 @@ def get_http_port():
from platypush.backend.http import HttpBackend
http_conf = Config.get('backend.http') or {}
return http_conf.get('port', HttpBackend._DEFAULT_HTTP_PORT)
return http_conf.get('port', HttpBackend.DEFAULT_HTTP_PORT)
def get_routes():

6
platypush/backend/http/app/ws/_base.py
View File

@ -1,7 +1,6 @@
from abc import ABC, abstractmethod
from logging import getLogger
from threading import Thread
from typing_extensions import override
from tornado.ioloop import IOLoop
from tornado.websocket import WebSocketHandler
@ -24,7 +23,6 @@ class WSRoute(WebSocketHandler, Thread, PubSubMixin, ABC):
Thread.__init__(self)
self._io_loop = IOLoop.current()
@override
def open(self, *_, **__):
auth_status = get_auth_status(self.request)
if auth_status != AuthStatus.OK:
@ -37,11 +35,9 @@ class WSRoute(WebSocketHandler, Thread, PubSubMixin, ABC):
self.name = f'ws:{self.app_name()}@{self.request.remote_ip}'
self.start()
@override
def data_received(self, *_, **__):
pass
@override
def on_message(self, message):
return message
@ -63,12 +59,10 @@ class WSRoute(WebSocketHandler, Thread, PubSubMixin, ABC):
self.write_message, self._serialize(msg)
)
@override
def run(self) -> None:
super().run()
self.subscribe(*self._subscriptions)
@override
def on_close(self):
super().on_close()
for channel in self._subscriptions.copy():

66
platypush/backend/http/app/ws/cmd.py Normal file
View File

@ -0,0 +1,66 @@
from base64 import b64decode
import json
from typing import Optional
from platypush.common.cmd_stream import redis_topic
from . import WSRoute, logger
class WSCommandOutput(WSRoute):
"""
Websocket route that pushes the output of an executed command to the client
as it is generated. Mapped to ``/ws/shell``.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, subscriptions=[redis_topic], **kwargs)
self._id = None
@classmethod
def app_name(cls) -> str:
return 'shell'
@classmethod
def path(cls) -> str:
return f'/ws/{cls.app_name()}'
def _parse_msg(self, msg: bytes) -> Optional[bytes]:
parsed_msg = json.loads(msg)
cmd_id = parsed_msg.get('id')
output = parsed_msg.get('output')
if output is None: # End-of-stream
raise StopIteration()
if cmd_id != self._id:
return None
return b64decode(output)
def open(self, *args, **kwargs):
self._id = next(iter(self.request.arguments['id']), b'').decode() or None
super().open(*args, **kwargs)
def run(self) -> None:
super().run()
for msg in self.listen():
try:
output = self._parse_msg(msg.data)
if output is None:
continue
self.send(output)
except StopIteration:
break
except Exception as e:
logger.warning('Failed to parse message: %s', e)
logger.exception(e)
continue
self._io_loop.add_callback(self._ws_close)
def _ws_close(self):
if not self.ws_connection:
return
self.ws_connection.close(1000, 'Command terminated')

6
platypush/backend/http/app/ws/events.py
View File

@ -1,5 +1,3 @@
from typing_extensions import override
from platypush.backend.http.app.mixins import MessageType
from platypush.message.event import Event
@ -16,7 +14,6 @@ class WSEventProxy(WSRoute):
super().__init__(*args, subscriptions=[self.events_channel], **kwargs)
@classmethod
@override
def app_name(cls) -> str:
return 'events'
@ -25,12 +22,10 @@ class WSEventProxy(WSRoute):
def events_channel(cls) -> str:
return cls.get_channel('events')
@override
@classmethod
def publish(cls, data: MessageType, *_) -> None:
super().publish(data, cls.events_channel)
@override
def on_message(self, message):
try:
event = Event.build(message)
@ -42,7 +37,6 @@ class WSEventProxy(WSRoute):
send_message(event, wait_for_response=False)
@override
def run(self) -> None:
for msg in self.listen():
try:

2
platypush/backend/http/app/ws/requests.py
View File

@ -1,6 +1,5 @@
from threading import Thread, current_thread
from typing import Set
from typing_extensions import override
from platypush.backend.http.app.utils import send_message
from platypush.message.request import Request
@ -21,7 +20,6 @@ class WSRequestsProxy(WSRoute):
self._requests: Set[Thread] = set()
@classmethod
@override
def app_name(cls) -> str:
return 'requests'

3
platypush/backend/http/manifest.yaml
View File

@ -1,7 +1,6 @@
manifest:
events: {}
install:
pip:
- gunicorn
pip: []
package: platypush.backend.http
type: backend

88
platypush/backend/http/poll/__init__.py
View File

@ -1,88 +0,0 @@
import importlib
import time
from platypush.backend import Backend
from platypush.backend.http.request import HttpRequest
class HttpPollBackend(Backend):
"""
WARNING: This integration is deprecated, since it was practically only used for RSS subscriptions.
RSS feeds integration has been replaced by :class:`platypush.plugins.rss.RSSPlugin`.
This backend will poll multiple HTTP endpoints/services and return events
the bus whenever something new happened. Supported types:
:class:`platypush.backend.http.request.JsonHttpRequest` (for polling updates on
a JSON endpoint), :class:`platypush.backend.http.request.rss.RssUpdates`
(for polling updates on an RSS feed). Example configuration::
backend.http.poll:
requests:
-
# Poll for updates on a JSON endpoint
method: GET
type: platypush.backend.http.request.JsonHttpRequest
args:
url: https://host.com/api/v1/endpoint
headers:
Token: TOKEN
params:
updatedSince: 1m
timeout: 5 # Times out after 5 seconds (default)
poll_seconds: 60 # Check for updates on this endpoint every 60 seconds (default)
path: ${response['items']} # Path in the JSON to check for new items.
# Python expressions are supported.
# Note that 'response' identifies the JSON root.
# Default value: JSON root.
-
# Poll for updates on an RSS feed
type: platypush.backend.http.request.rss.RssUpdates
url: https://www.theguardian.com/rss/world
title: The Guardian - World News
poll_seconds: 120
max_entries: 10
Triggers: an update event for the relevant HTTP source if it contains new items. For example:
* :class:`platypush.message.event.http.rss.NewFeedEvent` if a feed contains new items
* :class:`platypush.message.event.http.HttpEvent` if a JSON endpoint contains new items
"""
def __init__(self, requests, *args, **kwargs):
"""
:param requests: Configuration of the requests to make (see class description for examples)
:type requests: dict
"""
super().__init__(*args, **kwargs)
self.requests = []
for request in requests:
if isinstance(request, dict):
req_type = request['type']
(module, name) = ('.'.join(req_type.split('.')[:-1]), req_type.split('.')[-1])
module = importlib.import_module(module)
request = getattr(module, name)(**request)
elif not isinstance(request, HttpRequest):
raise RuntimeError('Request should either be a dict or a ' +
'HttpRequest object, {} found'.format(type(request)))
request.bus = self.bus
self.requests.append(request)
def run(self):
super().run()
while not self.should_stop():
for request in self.requests:
if time.time() - request.last_request_timestamp > request.poll_seconds:
try:
request.execute()
except Exception as e:
self.logger.error('Error while executing request: {}'.format(request))
self.logger.exception(e)
time.sleep(0.1) # Prevent a tight loop
# vim:sw=4:ts=4:et:

6
platypush/backend/http/poll/manifest.yaml
View File

@ -1,6 +0,0 @@
manifest:
events: {}
install:
pip: []
package: platypush.backend.http.poll
type: backend

9
platypush/backend/http/request/rss/__init__.py
View File

@ -40,15 +40,6 @@ class RssUpdates(HttpRequest):
poll_seconds: 86400 # Poll once a day
digest_format: html # Generate an HTML feed with the new items
Triggers:
- :class:`platypush.message.event.http.rss.NewFeedEvent` when new items are parsed from a feed or a new digest
is available.
Requires:
* **feedparser** (``pip install feedparser``)
"""
user_agent = (

2
platypush/backend/http/webapp/dist/index.html vendored
View File

@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><!--[if IE]><link rel="icon" href="/favicon.ico"><![endif]--><link rel="stylesheet" href="/fonts/poppins.css"><title>platypush</title><script defer="defer" src="/static/js/chunk-vendors.22fb39a8.js"></script><script defer="defer" src="/static/js/app.ebd1a697.js"></script><link href="/static/css/chunk-vendors.d510eff2.css" rel="stylesheet"><link href="/static/css/app.0a781c41.css" rel="stylesheet"><link rel="icon" type="image/svg+xml" href="/img/icons/favicon.svg"><link rel="icon" type="image/png" sizes="32x32" href="/img/icons/favicon-32x32.png"><link rel="icon" type="image/png" sizes="16x16" href="/img/icons/favicon-16x16.png"><link rel="manifest" href="/manifest.json"><meta name="theme-color" content="#ffffff"><meta name="apple-mobile-web-app-capable" content="no"><meta name="apple-mobile-web-app-status-bar-style" content="default"><meta name="apple-mobile-web-app-title" content="Platypush"><link rel="apple-touch-icon" href="/img/icons/apple-touch-icon-152x152.png"><link rel="mask-icon" href="/img/icons/safari-pinned-tab.svg" color="#ffffff"><meta name="msapplication-TileImage" content="/img/icons/msapplication-icon-144x144.png"><meta name="msapplication-TileColor" content="#000000"></head><body><noscript><strong>We're sorry but platypush doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript><div id="app"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><!--[if IE]><link rel="icon" href="/favicon.ico"><![endif]--><link rel="stylesheet" href="/fonts/poppins.css"><title>platypush</title><script defer="defer" src="/static/js/chunk-vendors.650eb8e8.js"></script><script defer="defer" src="/static/js/app.6a696f12.js"></script><link href="/static/css/chunk-vendors.a2412607.css" rel="stylesheet"><link href="/static/css/app.eab47dab.css" rel="stylesheet"><link rel="icon" type="image/svg+xml" href="/img/icons/favicon.svg"><link rel="icon" type="image/png" sizes="32x32" href="/img/icons/favicon-32x32.png"><link rel="icon" type="image/png" sizes="16x16" href="/img/icons/favicon-16x16.png"><link rel="manifest" href="/manifest.json"><meta name="theme-color" content="#ffffff"><meta name="apple-mobile-web-app-capable" content="no"><meta name="apple-mobile-web-app-status-bar-style" content="default"><meta name="apple-mobile-web-app-title" content="Platypush"><link rel="apple-touch-icon" href="/img/icons/apple-touch-icon-152x152.png"><link rel="mask-icon" href="/img/icons/safari-pinned-tab.svg" color="#ffffff"><meta name="msapplication-TileImage" content="/img/icons/msapplication-icon-144x144.png"><meta name="msapplication-TileColor" content="#000000"></head><body><noscript><strong>We're sorry but platypush doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript><div id="app"></div></body></html>

2
platypush/backend/http/webapp/dist/service-worker.js vendored
View File

File diff suppressed because one or more lines are too long

2
platypush/backend/http/webapp/dist/service-worker.js.map vendored
View File

File diff suppressed because one or more lines are too long

1
platypush/backend/http/webapp/dist/static/css/1259.28135e7d.css vendored Normal file
View File

File diff suppressed because one or more lines are too long

2
platypush/backend/http/webapp/dist/static/css/9575.1b22f65c.css → platypush/backend/http/webapp/dist/static/css/1391.cb558276.css vendored
View File

File diff suppressed because one or more lines are too long

1
platypush/backend/http/webapp/dist/static/css/1449.64ebf9cc.css vendored Normal file
View File

File diff suppressed because one or more lines are too long

2
platypush/backend/http/webapp/dist/static/css/675.10cdb721.css → platypush/backend/http/webapp/dist/static/css/169.18c97596.css vendored
View File

File diff suppressed because one or more lines are too long

1
platypush/backend/http/webapp/dist/static/css/201.3ba92d09.css vendored
View File

File diff suppressed because one or more lines are too long

2
platypush/backend/http/webapp/dist/static/css/8444.95911650.css → platypush/backend/http/webapp/dist/static/css/2217.664973a5.css vendored
View File

File diff suppressed because one or more lines are too long

1
platypush/backend/http/webapp/dist/static/css/2246.9882445c.css vendored Normal file
View File

File diff suppressed because one or more lines are too long

2
platypush/backend/http/webapp/dist/static/css/9450.fd9ed6f2.css → platypush/backend/http/webapp/dist/static/css/2460.c44d2954.css vendored
View File

File diff suppressed because one or more lines are too long

2
platypush/backend/http/webapp/dist/static/css/7782.a6a32303.css → platypush/backend/http/webapp/dist/static/css/2471.750151be.css vendored
View File

File diff suppressed because one or more lines are too long

1
platypush/backend/http/webapp/dist/static/css/2790.a0725ecc.css vendored
View File

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More