blacklight/platypush
1
0
Fork 0
forked from platypush/platypush
Code Issues Pull requests Projects Releases Packages Wiki Activity

Several improvements for the Google integrations.

Browse source 
1. Improved documentation. Every plugin now reports the exact steps to
   get the integration up and running with the right API scopes.

2. All Google plugins now have a standard process to get (and reuse) the
   client secret. Except for PubSub, Translate and Maps (which have
   their own flows), all the Google plugins now read the client secrets
   from `<WORKDIR>/credentials/google/client_secret.json` by default.

3. Black/LINT for some of those plugins, which hadn't been touched in a
   while.

4. The interface to pass API scopes is now leaner. It's now possible to
   pass a scope directly as e.g. `calendar.readonly` rather than
   `https://www.googleapis.com/auth/calendar.readonly`.

5. Improved the logic to retrieve the right scope tokens file. If e.g.
   an integration requires the role `A`, and a credentials file exists
   for the roles `A` and `B`, then this file will be used rather than
   prompting the user to authenticate again.
This commit is contained in:
Fabio Manganiello 2023-10-01 15:37:20 +02:00
parent 5ca3757834
commit 2aefc4e5c8
Signed by: blacklight
GPG key ID: D90FBA7F76362774
10 changed files with 401 additions and 120 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

73
platypush/plugins/google/__init__.py
View file

@ -1,56 +1,93 @@
from typing import Collection, Optional
from platypush.plugins import Plugin from platypush.plugins import Plugin
class GooglePlugin(Plugin): class GooglePlugin(Plugin):
""" """
Executes calls to the Google APIs using the google-api-python-client. Integrates with the Google APIs using the google-api-python-client.
This class is extended by ``GoogleMailPlugin``, ``GoogleCalendarPlugin`` etc. This class is extended by ``GoogleMailPlugin``, ``GoogleCalendarPlugin`` etc.
In order to use Google services (like GMail, Maps, Calendar etc.) with In order to use Google services (like GMail, Maps, Calendar etc.) with
your account you need to: your account you need to:
1. Create your Google application, if you don't have one already, on 1. Create your Google application, if you don't have one already, on
the developers console, https://console.developers.google.com the `developers console <https://console.developers.google.com>`_.
2. Click on "Credentials", then "Create credentials" -> "OAuth client ID" 2. You may have to explicitly enable your user to use the app if the app
is created in test mode. Go to "OAuth consent screen" and add your user's
email address to the list of authorized users.
3 Select "Other", enter whichever description you like, and create 3. Select the scopes that you want to enable for your application, depending
on the integrations that you want to use.
See https://developers.google.com/identity/protocols/oauth2/scopes
for a list of the available scopes.
4. Click on the "Download JSON" icon next to your newly created client ID 4. Click on "Credentials", then "Create credentials" -> "OAuth client ID".
5. Generate a credentials file for the needed scope:: 5 Select "Desktop app", enter whichever name you like, and click "Create".
6. Click on the "Download JSON" icon next to your newly created client ID.
7. Generate a credentials file for the required scope:
.. code-block:: bash
mkdir -p <WORKDIR>/credentials/google
python -m platypush.plugins.google.credentials \ python -m platypush.plugins.google.credentials \
'https://www.googleapis.com/auth/gmail.compose' ~/client_secret.json 'calendar.readonly' \
<WORKDIR>/credentials/google/client_secret.json
""" """
def __init__(self, scopes=None, **kwargs): def __init__(
self,
scopes: Optional[Collection[str]] = None,
secrets_path: Optional[str] = None,
**kwargs
):
""" """
Initialized the Google plugin with the required scopes. :param scopes: List of scopes required by the API.
See https://developers.google.com/identity/protocols/oauth2/scopes
:param scopes: List of required scopes for a list of the available scopes. Override it in your configuration
:type scopes: list only if you need specific scopes that aren't normally required by the
plugin.
:param secrets_path: Path to the client secrets file.
You can create your secrets.json from https://console.developers.google.com.
Default: ``<PLATYPUSH_WORKDIR>/credentials/google/client_secret.json``.
""" """
from platypush.plugins.google.credentials import get_credentials from platypush.plugins.google.credentials import (
get_credentials,
default_secrets_file,
)
super().__init__(**kwargs) super().__init__(**kwargs)
self._scopes = scopes or [] self._scopes = scopes or []
self._secrets_path: str = secrets_path or default_secrets_file
if self._scopes: if self._scopes:
scopes = ' '.join(sorted(self._scopes)) scopes = " ".join(sorted(self._scopes))
self.credentials = {scopes: get_credentials(scopes)} try:
self.credentials = {
scopes: get_credentials(scopes, secrets_file=self._secrets_path)
}
except AssertionError as e:
self.logger.warning(str(e))
else: else:
self.credentials = {} self.credentials = {}
def get_service(self, service, version, scopes=None): def get_service(
self, service: str, version: str, scopes: Optional[Collection[str]] = None
):
import httplib2 import httplib2
from apiclient import discovery from apiclient import discovery
if scopes is None: if scopes is None:
scopes = getattr(self, 'scopes', []) scopes = getattr(self, "scopes", [])
scopes = ' '.join(sorted(scopes)) scopes = " ".join(sorted(scopes))
credentials = self.credentials[scopes] credentials = self.credentials[scopes]
http = credentials.authorize(httplib2.Http()) http = credentials.authorize(httplib2.Http())
return discovery.build(service, version, http=http, cache_discovery=False) return discovery.build(service, version, http=http, cache_discovery=False)

34
platypush/plugins/google/calendar/__init__.py
View file

@ -6,11 +6,41 @@ from platypush.plugins.calendar import CalendarInterface
class GoogleCalendarPlugin(GooglePlugin, CalendarInterface): class GoogleCalendarPlugin(GooglePlugin, CalendarInterface):
""" r"""
Google Calendar plugin. Google Calendar plugin.
In order to use this plugin:
1. Create your Google application, if you don't have one already, on
the `developers console <https://console.developers.google.com>`_.
2. You may have to explicitly enable your user to use the app if the app
is created in test mode. Go to "OAuth consent screen" and add your user's
email address to the list of authorized users.
3. Select the scopes that you want to enable for your application, depending
on the integrations that you want to use.
See https://developers.google.com/identity/protocols/oauth2/scopes
for a list of the available scopes.
4. Click on "Credentials", then "Create credentials" -> "OAuth client ID".
5 Select "Desktop app", enter whichever name you like, and click "Create".
6. Click on the "Download JSON" icon next to your newly created client ID.
7. Generate a credentials file for the required scope:
.. code-block:: bash
mkdir -p <WORKDIR>/credentials/google
python -m platypush.plugins.google.credentials \
'calendar.readonly' \
<WORKDIR>/credentials/google/client_secret.json
""" """
scopes = ['https://www.googleapis.com/auth/calendar.readonly'] scopes = ['calendar.readonly']
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super().__init__(scopes=self.scopes, *args, **kwargs) super().__init__(scopes=self.scopes, *args, **kwargs)

138
platypush/plugins/google/credentials.py
View file

@ -1,35 +1,93 @@
import argparse import argparse
import httplib2
import os import os
import re
import sys import sys
import textwrap as tw
from typing import List, Optional
import httplib2
from oauth2client import client from oauth2client import client
from oauth2client import tools from oauth2client import tools
from oauth2client.file import Storage from oauth2client.file import Storage
def get_credentials_filename(*scopes):
from platypush.config import Config from platypush.config import Config
scope_name = '-'.join([scope.split('/')[-1] for scope in scopes]) credentials_dir = os.path.join(Config.get_workdir(), "credentials", "google")
credentials_dir = os.path.join( default_secrets_file = os.path.join(credentials_dir, "client_secret.json")
Config.get('workdir'), 'credentials', 'google') """Default path for the Google API client secrets file"""
def _parse_scopes(*scopes: str) -> List[str]:
return sorted(
{
t.split("/")[-1].strip()
for scope in scopes
for t in re.split(r"[\s,]", scope)
if t
}
)
def get_credentials_filename(*scopes: str):
parsed_scopes = _parse_scopes(*scopes)
scope_name = "-".join([scope.split("/")[-1] for scope in parsed_scopes])
os.makedirs(credentials_dir, exist_ok=True) os.makedirs(credentials_dir, exist_ok=True)
return os.path.join(credentials_dir, scope_name + '.json') matching_scope_file = next(
iter(
os.path.join(credentials_dir, scopes_file)
for scopes_file in {
os.path.basename(file)
for file in os.listdir(credentials_dir)
if file.endswith(".json")
}
if not set(parsed_scopes).difference(
set(scopes_file.split(".json")[0].split("-"))
)
),
None,
)
if matching_scope_file:
return matching_scope_file
return os.path.join(credentials_dir, scope_name + ".json")
def get_credentials(scope): def get_credentials(scope: str, secrets_file: Optional[str] = None):
credentials_file = get_credentials_filename(*sorted(scope.split(' '))) scopes = _parse_scopes(scope)
if not os.path.exists(credentials_file): credentials_file = get_credentials_filename(*scopes)
raise RuntimeError(('Credentials file {} not found. Generate it through:\n' +
'\tpython -m platypush.plugins.google.credentials "{}" ' + # If we don't have a credentials file for the required set of scopes, but we have a secrets file,
'<path to client_secret.json>\n' + # then try and generate the credentials file from the stored secrets.
'\t\t[--auth_host_name AUTH_HOST_NAME]\n' + if (
'\t\t[--noauth_local_webserver]\n' + not os.path.isfile(credentials_file)
'\t\t[--auth_host_port [AUTH_HOST_PORT [AUTH_HOST_PORT ...]]]\n' + and secrets_file
'\t\t[--logging_level [DEBUG,INFO,WARNING,ERROR,CRITICAL]]\n'). and os.path.isfile(secrets_file)
format(credentials_file, scope)) ):
# If DISPLAY or BROWSER are set, then we can open the authentication URL in the browser.
# Otherwise, we'll have to use the --noauth_local_webserver flag and copy/paste the URL
args = (
["--noauth_local_webserver"]
if not (os.getenv("DISPLAY") or os.getenv("BROWSER"))
else []
)
generate_credentials(secrets_file, scope, *args)
assert os.path.isfile(credentials_file), tw.dedent(
f"""
Credentials file {credentials_file} not found. Generate it through:
python -m platypush.plugins.google.credentials "{','.join(scopes)}" /path/to/client_secret.json
[--auth_host_name AUTH_HOST_NAME]
[--noauth_local_webserver]
[--auth_host_port [AUTH_HOST_PORT [AUTH_HOST_PORT ...]]]
[--logging_level [DEBUG,INFO,WARNING,ERROR,CRITICAL]]
Specify --noauth_local_webserver if you're running this script on a headless machine.
You will then get an authentication URL on the logs.
Otherwise, the URL will be opened in the available browser.
"""
)
store = Storage(credentials_file) store = Storage(credentials_file)
credentials = store.get() credentials = store.get()
@ -40,16 +98,20 @@ def get_credentials(scope):
return credentials return credentials
def generate_credentials(client_secret_path, scope): def generate_credentials(client_secret_path: str, scope: str, *args: str):
credentials_file = get_credentials_filename(*sorted(scope.split(' '))) scopes = _parse_scopes(scope)
credentials_file = get_credentials_filename(*scopes)
store = Storage(credentials_file) store = Storage(credentials_file)
scope = ' '.join(
f'https://www.googleapis.com/auth/{scope}' for scope in _parse_scopes(scope)
)
flow = client.flow_from_clientsecrets(client_secret_path, scope) flow = client.flow_from_clientsecrets(client_secret_path, scope)
flow.user_agent = 'Platypush' flow.user_agent = "Platypush"
flow.access_type = 'offline' flow.access_type = "offline" # type: ignore
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args() flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args(args) # type: ignore
tools.run_flow(flow, store, flags) tools.run_flow(flow, store, flags)
print('Storing credentials to ' + credentials_file) print("Storing credentials to", credentials_file)
def main(): def main():
@ -57,23 +119,29 @@ def main():
Generates a Google API credentials file given client secret JSON and scopes. Generates a Google API credentials file given client secret JSON and scopes.
Usage:: Usage::
python -m platypush.plugins.google.credentials [client_secret.json location] [comma-separated list of scopes] python -m platypush.plugins.google.credentials \
[spaces/comma-separated list of scopes] \
[client_secret.json location]
""" """
scope = sys.argv.pop(1) if len(sys.argv) > 1 \ args = sys.argv[1:]
else input('Space separated list of OAuth scopes: ') scope = (
args.pop(0) if args else input("Space/comma separated list of OAuth scopes: ")
).strip()
client_secret_path = os.path.expanduser( if args:
sys.argv.pop(1) if len(sys.argv) > 1 client_secret_path = args.pop(0)
else input('Google credentials JSON file location: ')) elif os.path.isfile(default_secrets_file):
client_secret_path = default_secrets_file
else:
client_secret_path = input("Google credentials JSON file location: ")
# Uncomment to force headless (no browser spawned) authentication client_secret_path = os.path.abspath(os.path.expanduser(client_secret_path)).strip()
# sys.argv.append('--noauth_local_webserver') generate_credentials(client_secret_path, scope, *args)
generate_credentials(client_secret_path, scope)
if __name__ == '__main__': if __name__ == "__main__":
main() main()
# vim:sw=4:ts=4:et: # vim:sw=4:ts=4:et:

40
platypush/plugins/google/drive/__init__.py
View file

@ -8,8 +8,36 @@ from platypush.message.response.google.drive import GoogleDriveFile
class GoogleDrivePlugin(GooglePlugin): class GoogleDrivePlugin(GooglePlugin):
""" r"""
Google Drive plugin. Google Drive plugin.
1. Create your Google application, if you don't have one already, on
the `developers console <https://console.developers.google.com>`_.
2. You may have to explicitly enable your user to use the app if the app
is created in test mode. Go to "OAuth consent screen" and add your user's
email address to the list of authorized users.
3. Select the scopes that you want to enable for your application, depending
on the integrations that you want to use.
See https://developers.google.com/identity/protocols/oauth2/scopes
for a list of the available scopes.
4. Click on "Credentials", then "Create credentials" -> "OAuth client ID".
5 Select "Desktop app", enter whichever name you like, and click "Create".
6. Click on the "Download JSON" icon next to your newly created client ID.
7. Generate a credentials file for the required scope:
.. code-block:: bash
mkdir -p <WORKDIR>/credentials/google
python -m platypush.plugins.google.credentials \
'drive,drive.appfolder,drive.photos.readonly' \
<WORKDIR>/credentials/google/client_secret.json
""" """
scopes = [ scopes = [
@ -21,7 +49,7 @@ class GoogleDrivePlugin(GooglePlugin):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super().__init__(scopes=self.scopes, *args, **kwargs) super().__init__(scopes=self.scopes, *args, **kwargs)
def get_service(self, **kwargs): def get_service(self, **_):
return super().get_service(service='drive', version='v3') return super().get_service(service='drive', version='v3')
# noinspection PyShadowingBuiltins # noinspection PyShadowingBuiltins
@ -85,7 +113,7 @@ class GoogleDrivePlugin(GooglePlugin):
else: else:
filter += ' ' filter += ' '
filter += "'{}' in parents".format(folder_id) filter += f"'{folder_id}' in parents"
while True: while True:
results = ( results = (
@ -216,7 +244,7 @@ class GoogleDrivePlugin(GooglePlugin):
while not done: while not done:
status, done = downloader.next_chunk() status, done = downloader.next_chunk()
self.logger.info('Download progress: {}%'.format(status.progress())) self.logger.info('Download progress: %s%%', status.progress())
with open(path, 'wb') as f: with open(path, 'wb') as f:
f.write(fh.getbuffer().tobytes()) f.write(fh.getbuffer().tobytes())
@ -269,8 +297,8 @@ class GoogleDrivePlugin(GooglePlugin):
add_parents: Optional[List[str]] = None, add_parents: Optional[List[str]] = None,
remove_parents: Optional[List[str]] = None, remove_parents: Optional[List[str]] = None,
mime_type: Optional[str] = None, mime_type: Optional[str] = None,
starred: bool = None, starred: Optional[bool] = None,
trashed: bool = None, trashed: Optional[bool] = None,
) -> GoogleDriveFile: ) -> GoogleDriveFile:
""" """
Update the metadata or the content of a file. Update the metadata or the content of a file.

39
platypush/plugins/google/fit/__init__.py
View file

@ -3,8 +3,45 @@ from platypush.plugins.google import GooglePlugin
class GoogleFitPlugin(GooglePlugin): class GoogleFitPlugin(GooglePlugin):
""" r"""
Google Fit plugin. Google Fit plugin.
In order to use this plugin:
1. Create your Google application, if you don't have one already, on
the `developers console <https://console.developers.google.com>`_.
2. You may have to explicitly enable your user to use the app if the app
is created in test mode. Go to "OAuth consent screen" and add your user's
email address to the list of authorized users.
3. Select the scopes that you want to enable for your application, depending
on the integrations that you want to use.
See https://developers.google.com/identity/protocols/oauth2/scopes
for a list of the available scopes.
4. Click on "Credentials", then "Create credentials" -> "OAuth client ID".
5 Select "Desktop app", enter whichever name you like, and click "Create".
6. Click on the "Download JSON" icon next to your newly created client ID.
7. Generate a credentials file for the required scope:
.. code-block:: bash
$ mkdir -p <WORKDIR>/credentials/google
$ roles="
fitness.activity.read,
fitness.body.read,
fitness.body_temperature.read,
fitness.heart_rate.read,
fitness.sleep.read,
fitness.location.read
"
$ python -m platypush.plugins.google.credentials "$roles" \
<WORKDIR>/credentials/google/client_secret.json
""" """
scopes = [ scopes = [

37
platypush/plugins/google/mail/__init__.py
View file

@ -15,8 +15,38 @@ from platypush.plugins.google import GooglePlugin
class GoogleMailPlugin(GooglePlugin): class GoogleMailPlugin(GooglePlugin):
""" r"""
GMail plugin. It allows you to programmatically compose and (TODO) get emails GMail plugin. It allows you to programmatically compose and (TODO) get emails.
To use this plugin:
1. Create your Google application, if you don't have one already, on
the `developers console <https://console.developers.google.com>`_.
2. You may have to explicitly enable your user to use the app if the app
is created in test mode. Go to "OAuth consent screen" and add your user's
email address to the list of authorized users.
3. Select the scopes that you want to enable for your application, depending
on the integrations that you want to use.
See https://developers.google.com/identity/protocols/oauth2/scopes
for a list of the available scopes.
4. Click on "Credentials", then "Create credentials" -> "OAuth client ID".
5 Select "Desktop app", enter whichever name you like, and click "Create".
6. Click on the "Download JSON" icon next to your newly created client ID.
7. Generate a credentials file for the required scope:
.. code-block:: bash
mkdir -p <WORKDIR>/credentials/google
python -m platypush.plugins.google.credentials \
'gmail.modify' \
<WORKDIR>/credentials/google/client_secret.json
""" """
scopes = ['https://www.googleapis.com/auth/gmail.modify'] scopes = ['https://www.googleapis.com/auth/gmail.modify']
@ -64,8 +94,7 @@ class GoogleMailPlugin(GooglePlugin):
content = fp.read() content = fp.read()
if main_type == 'text': if main_type == 'text':
# noinspection PyUnresolvedReferences msg = MIMEText(str(content), _subtype=sub_type)
msg = mimetypes.MIMEText(content, _subtype=sub_type)
elif main_type == 'image': elif main_type == 'image':
msg = MIMEImage(content, _subtype=sub_type) msg = MIMEImage(content, _subtype=sub_type)
elif main_type == 'audio': elif main_type == 'audio':

32
platypush/plugins/google/maps/__init__.py
View file

@ -14,18 +14,24 @@ datetime_types = Union[str, int, float, datetime]
class GoogleMapsPlugin(GooglePlugin): class GoogleMapsPlugin(GooglePlugin):
""" """
Plugins that provides utilities to interact with Google Maps API services. Plugins that provides utilities to interact with Google Maps API services.
It requires you to create a Google application - you can create one at the
`developers console <https://console.developers.google.com>`_.
After that, you'll need to create a new API key from the _Credentials_ tab.
This integration doesn't require any additional scopes.
""" """
scopes = [] scopes = []
def __init__(self, api_key, *args, **kwargs): def __init__(self, api_key: str, **kwargs):
""" """
:param api_key: Server-side API key to be used for the requests, get one at :param api_key: Server-side API key to be used for the requests, get one at
https://console.developers.google.com https://console.developers.google.com
:type api_key: str
""" """
super().__init__(scopes=self.scopes, *args, **kwargs) super().__init__(scopes=self.scopes, **kwargs)
self.api_key = api_key self.api_key = api_key
@action @action
@ -43,7 +49,7 @@ class GoogleMapsPlugin(GooglePlugin):
response = requests.get( response = requests.get(
'https://maps.googleapis.com/maps/api/geocode/json', 'https://maps.googleapis.com/maps/api/geocode/json',
params={ params={
'latlng': '{},{}'.format(latitude, longitude), 'latlng': f'{latitude},{longitude}',
'key': self.api_key, 'key': self.api_key,
}, },
).json() ).json()
@ -59,9 +65,10 @@ class GoogleMapsPlugin(GooglePlugin):
if 'results' in response and response['results']: if 'results' in response and response['results']:
result = response['results'][0] result = response['results'][0]
self.logger.info( self.logger.info(
'Google Maps geocode response for latlng ({},{}): {}'.format( 'Google Maps geocode response for latlng (%f,%f): %s',
latitude, longitude, result latitude,
) longitude,
result,
) )
address['address'] = result['formatted_address'].split(',')[0] address['address'] = result['formatted_address'].split(',')[0]
@ -91,7 +98,7 @@ class GoogleMapsPlugin(GooglePlugin):
response = requests.get( response = requests.get(
'https://maps.googleapis.com/maps/api/elevation/json', 'https://maps.googleapis.com/maps/api/elevation/json',
params={ params={
'locations': '{},{}'.format(latitude, longitude), 'locations': f'{latitude},{longitude}',
'key': self.api_key, 'key': self.api_key,
}, },
).json() ).json()
@ -192,16 +199,21 @@ class GoogleMapsPlugin(GooglePlugin):
""" """
rs = requests.get( rs = requests.get(
'https://maps.googleapis.com/maps/api/distancematrix/json', 'https://maps.googleapis.com/maps/api/distancematrix/json',
timeout=20,
params={ params={
'origins': '|'.join(origins), 'origins': '|'.join(origins),
'destinations': '|'.join(destinations), 'destinations': '|'.join(destinations),
'units': units, 'units': units,
**( **(
{'departure_time': to_datetime(departure_time)} {'departure_time': to_datetime(departure_time).isoformat()}
if departure_time if departure_time
else {} else {}
), ),
**({'arrival_time': to_datetime(arrival_time)} if arrival_time else {}), **(
{'arrival_time': to_datetime(arrival_time).isoformat()}
if arrival_time
else {}
),
**({'avoid': '|'.join(avoid)} if avoid else {}), **({'avoid': '|'.join(avoid)} if avoid else {}),
**({'language': language} if language else {}), **({'language': language} if language else {}),
**({'mode': mode} if mode else {}), **({'mode': mode} if mode else {}),

36
platypush/plugins/google/pubsub/__init__.py
View file

@ -9,15 +9,18 @@ class GooglePubsubPlugin(Plugin):
Send messages over a Google pub/sub instance. Send messages over a Google pub/sub instance.
You'll need a Google Cloud active project and a set of credentials to use this plugin: You'll need a Google Cloud active project and a set of credentials to use this plugin:
1. Create a project on the `Google Cloud console <https://console.cloud.google.com/projectcreate>`_ if 1. Create a project on the `Google Cloud console
you don't have one already. <https://console.cloud.google.com/projectcreate>`_ if you don't have
one already.
2. In the `Google Cloud API console <https://console.cloud.google.com/apis/credentials/serviceaccountkey>`_ 2. In the `Google Cloud API console
create a new service account key. Select "New Service Account", choose the role "Pub/Sub Editor" and leave <https://console.cloud.google.com/apis/credentials/serviceaccountkey>`_
the key type as JSON. create a new service account key. Select "New Service Account", choose
the role "Pub/Sub Editor" and leave the key type as JSON.
3. Download the JSON service credentials file. By default platypush will look for the credentials file under 3. Download the JSON service credentials file. By default Platypush
~/.credentials/platypush/google/pubsub.json. will look for the credentials file under
``~/.credentials/platypush/google/pubsub.json``.
""" """
@ -29,8 +32,8 @@ class GooglePubsubPlugin(Plugin):
def __init__(self, credentials_file: str = default_credentials_file, **kwargs): def __init__(self, credentials_file: str = default_credentials_file, **kwargs):
""" """
:param credentials_file: Path to the JSON credentials file for Google pub/sub (default: :param credentials_file: Path to the JSON credentials file for Google
~/.credentials/platypush/google/pubsub.json) pub/sub (default: ``~/.credentials/platypush/google/pubsub.json``)
""" """
super().__init__(**kwargs) super().__init__(**kwargs)
self.credentials_file = credentials_file self.credentials_file = credentials_file
@ -52,10 +55,13 @@ class GooglePubsubPlugin(Plugin):
""" """
Sends a message to a topic Sends a message to a topic
:param topic: Topic/channel where the message will be delivered. You can either specify the full topic name in :param topic: Topic/channel where the message will be delivered. You
the format ``projects/<project_id>/topics/<topic_name>``, where ``<project_id>`` must be the ID of your can either specify the full topic name in the format
Google Pub/Sub project, or just ``<topic_name>`` - in such case it's implied that you refer to the ``projects/<project_id>/topics/<topic_name>``, where
``topic_name`` under the ``project_id`` of your service credentials. ``<project_id>`` must be the ID of your Google Pub/Sub project, or
just ``<topic_name>`` - in such case it's implied that you refer to
the ``topic_name`` under the ``project_id`` of your service
credentials.
:param msg: Message to be sent. It can be a list, a dict, or a Message object :param msg: Message to be sent. It can be a list, a dict, or a Message object
:param kwargs: Extra arguments to be passed to .publish() :param kwargs: Extra arguments to be passed to .publish()
""" """
@ -65,8 +71,8 @@ class GooglePubsubPlugin(Plugin):
credentials = self.get_credentials(self.publisher_audience) credentials = self.get_credentials(self.publisher_audience)
publisher = pubsub_v1.PublisherClient(credentials=credentials) publisher = pubsub_v1.PublisherClient(credentials=credentials)
if not topic.startswith('projects/{}/topics/'.format(self.project_id)): if not topic.startswith(f'projects/{self.project_id}/topics/'):
topic = 'projects/{}/topics/{}'.format(self.project_id, topic) topic = f'projects/{self.project_id}/topics/{topic}'
try: try:
publisher.create_topic(topic) publisher.create_topic(topic)

31
platypush/plugins/google/translate/__init__.py
View file

@ -1,7 +1,6 @@
import os import os
from typing import Optional, List from typing import Optional, List
# noinspection PyPackageRequirements
from google.cloud import translate_v2 as translate from google.cloud import translate_v2 as translate
from platypush.message.response.translate import TranslateResponse from platypush.message.response.translate import TranslateResponse
@ -13,16 +12,19 @@ class GoogleTranslatePlugin(Plugin):
Plugin to interact with the Google Translate API. Plugin to interact with the Google Translate API.
You'll need a Google Cloud active project and a set of credentials to use this plugin: You'll need a Google Cloud active project and a set of credentials to use this plugin:
1. Create a project on the `Google Cloud console <https://console.cloud.google.com/projectcreate>`_ if 1. Create a project on the `Google Cloud console
you don't have one already. <https://console.cloud.google.com/projectcreate>`_ if you don't
have one already.
2. In the menu navigate to the *Artificial Intelligence* section and select *Translations* and enable the API. 2. In the menu navigate to the *Artificial Intelligence* section and
select *Translations* and enable the API.
3. From the menu select *APIs & Services* and create a service account. You can leave role and permissions 3. From the menu select *APIs & Services* and create a service account.
empty. You can leave role and permissions empty.
4. Create a new private JSON key for the service account and download it. By default platypush will look for the 4. Create a new private JSON key for the service account and download
credentials file under ``~/.credentials/platypush/google/translate.json``. it. By default platypush will look for the credentials file under
``~/.credentials/platypush/google/translate.json``.
""" """
@ -39,11 +41,14 @@ class GoogleTranslatePlugin(Plugin):
): ):
""" """
:param target_language: Default target language (default: 'en'). :param target_language: Default target language (default: 'en').
:param credentials_file: Google service account JSON credentials file. If none is specified then the plugin will :param credentials_file: Google service account JSON credentials file.
search for the credentials file in the following order: If none is specified then the plugin will search for the credentials
file in the following order:
1. ``~/.credentials/platypush/google/translate.json`` 1. ``~/.credentials/platypush/google/translate.json``
2. Context from the ``GOOGLE_APPLICATION_CREDENTIALS`` environment variable. 2. Context from the ``GOOGLE_APPLICATION_CREDENTIALS``
environment variable.
""" """
super().__init__(**kwargs) super().__init__(**kwargs)
self.target_language = target_language self.target_language = target_language
@ -64,7 +69,7 @@ class GoogleTranslatePlugin(Plugin):
for i in range(min(pos, len(text) - 1), -1, -1): for i in range(min(pos, len(text) - 1), -1, -1):
if text[i] in [' ', '\t', ',', '.', ')', '>']: if text[i] in [' ', '\t', ',', '.', ')', '>']:
return i return i
elif text[i] in ['(', '<']: if text[i] in ['(', '<']:
return i - 1 if i > 0 else 0 return i - 1 if i > 0 else 0
return 0 return 0
@ -86,7 +91,6 @@ class GoogleTranslatePlugin(Plugin):
return parts return parts
# noinspection PyShadowingBuiltins
@action @action
def translate( def translate(
self, self,
@ -121,7 +125,6 @@ class GoogleTranslatePlugin(Plugin):
if not result: if not result:
result = response result = response
else: else:
# noinspection PyTypeChecker
result['translatedText'] += ' ' + response['translatedText'] result['translatedText'] += ' ' + response['translatedText']
return TranslateResponse( return TranslateResponse(

57
platypush/plugins/google/youtube/__init__.py
View file

@ -1,10 +1,41 @@
from typing import Collection, Optional, Union
from platypush.plugins import action from platypush.plugins import action
from platypush.plugins.google import GooglePlugin from platypush.plugins.google import GooglePlugin
class GoogleYoutubePlugin(GooglePlugin): class GoogleYoutubePlugin(GooglePlugin):
""" r"""
YouTube plugin. YouTube plugin.
Requirements:
1. Create your Google application, if you don't have one already, on
the `developers console <https://console.developers.google.com>`_.
2. You may have to explicitly enable your user to use the app if the app
is created in test mode. Go to "OAuth consent screen" and add your user's
email address to the list of authorized users.
3. Select the scopes that you want to enable for your application, depending
on the integrations that you want to use.
See https://developers.google.com/identity/protocols/oauth2/scopes
for a list of the available scopes.
4. Click on "Credentials", then "Create credentials" -> "OAuth client ID".
5 Select "Desktop app", enter whichever name you like, and click "Create".
6. Click on the "Download JSON" icon next to your newly created client ID.
7. Generate a credentials file for the required scope:
.. code-block:: bash
mkdir -p <WORKDIR>/credentials/google
python -m platypush.plugins.google.credentials \
'youtube.readonly' \
<WORKDIR>/credentials/google/client_secret.json
""" """
scopes = ['https://www.googleapis.com/auth/youtube.readonly'] scopes = ['https://www.googleapis.com/auth/youtube.readonly']
@ -19,28 +50,28 @@ class GoogleYoutubePlugin(GooglePlugin):
super().__init__(scopes=self.scopes, *args, **kwargs) super().__init__(scopes=self.scopes, *args, **kwargs)
@action @action
def search(self, parts=None, query='', types=None, max_results=25, **kwargs): def search(
self,
parts: Optional[Union[str, Collection[str]]] = None,
query: str = '',
types: Optional[Union[str, Collection[str]]] = None,
max_results: int = 25,
**kwargs
):
""" """
Search for YouTube content. Search for YouTube content.
:param parts: List of parts to get (default: snippet). :param parts: List of parts to get (default: snippet).
See the `Getting started - Part <https://developers.google.com/youtube/v3/getting-started#part>`_. See the `Getting started - Part
:type parts: list[str] or str <https://developers.google.com/youtube/v3/getting-started#part>`_.
:param query: Query string (default: empty string) :param query: Query string (default: empty string)
:type query: str
:param types: List of types to retrieve (default: video). :param types: List of types to retrieve (default: video).
See the `Getting started - Resources <https://developers.google.com/youtube/v3/getting-started#resources>`_. See the `Getting started - Resources
:type types: list[str] or str <https://developers.google.com/youtube/v3/getting-started#resources>`_.
:param max_results: Maximum number of items that will be returned (default: 25). :param max_results: Maximum number of items that will be returned (default: 25).
:type max_results: int
:param kwargs: Any extra arguments that will be transparently passed to the YouTube API. :param kwargs: Any extra arguments that will be transparently passed to the YouTube API.
See the `Getting started - parameters See the `Getting started - parameters
<https://developers.google.com/youtube/v3/docs/search/list#parameters>`_. <https://developers.google.com/youtube/v3/docs/search/list#parameters>`_.
:return: A list of YouTube resources. :return: A list of YouTube resources.
See the `Getting started - Resource See the `Getting started - Resource
<https://developers.google.com/youtube/v3/docs/search#resource>`_. <https://developers.google.com/youtube/v3/docs/search#resource>`_.