Commit graph

79 commits

Author SHA1 Message Date
7a8f30e5e0 [#384] Added assistant.openai and tts.openai plugins.
Closes: #384
2024-06-02 15:31:11 +00:00
f356fcd844 Added tts.stop method. 2024-06-02 15:31:11 +00:00
fcae7aa3ad Several improvements for assistant plugins.
- `stop_conversation_on_speech_match` should default to True.

- `render_response` should also handle conversation follow-ups, set the
  follow-up to True if the response ends with a question mark and the
  value of `with_follow_on_turn` is not set,

- Don't render responses if a `tts_plugin` is not set.
2024-06-02 15:31:11 +00:00
1cc2aaf5a4 [assistant.picovoice] _on_response_* methods should have varargs. 2024-06-02 15:31:11 +00:00
d6185ddb1e
[assistant] tts_plugin_args should include join=True by default.
The assistant by default should be configured to wait for response audio
to be fully rendered before proceeding.
2024-05-30 01:33:56 +02:00
f06233801b
[#394] Dynamically generate setup extras.
Also, convert all code that relied on `manifest.yaml` to use
`manifest.json` instead.

Closes: #394
2024-05-17 02:21:57 +02:00
59c693d6a0
[#394] All manifest.yaml converted to manifest.json.
YAML isn't part of the Python standard library, while JSON is.

If we want `setup.py` to dynamically parse the available integration
manifest files in order to populate the extra dependencies, then it's
better to rely on a JSON format for manifest files - the parser is part
of the standard library and it doesn't require the user to install
`pyyaml` before `platypush`.
2024-05-17 02:21:57 +02:00
7a849379f9
[#399] Added @when decorator as an alias for @hook.
Closes: #399
2024-05-08 21:58:58 +02:00
72bc697122 [assistant.picovoice] Extended documentation. 2024-05-02 00:51:09 +00:00
bd4b1d3e0f [assistant.picovoice] Sync between the speech and intent engines. 2024-05-02 00:51:09 +00:00
632d98703b New architecture for the assistant speech detection logic.
The assistant object now runs in its own thread and leverages an
external `SpeechProcessor` that uses two threads to scan for both
intents and speech in parallel on audio frames.
2024-05-02 00:51:09 +00:00
af1392b5b9 [assistant] Added ResponseEndEvent and IntentMatchedEvent 2024-05-02 00:51:09 +00:00
bb9b6cd319 [assistant.picovoice] Various improvements.
- Added `intent_model_path` parameter.

- Always apply `expanduser` to configuration paths.

- Better logic to infer the fallback model path.

- The Picovoice Leonardo object should always be removed after
  `assistant.picovoice.transcribe` is called.
2024-05-02 00:51:09 +00:00
f0a780b759 Added assistant.picovoice.transcribe and assistant.picovoice.say. 2024-05-02 00:51:09 +00:00
37d70d67ac [assistant.picovoice] Implemented assistant.picovoice.send_text_query. 2024-05-02 00:51:09 +00:00
a345b12244 [assistant.picovoice] Leverage upstream _on_mute_changed.
The plugin should leverage `AssistantPlugin._on_mute_changed` to handle
the boilerplate state managent on mute/unmute actions instead of
re-implementing the same logic.
2024-05-02 00:51:09 +00:00
2c197c275e [assistant.picovoice] Implemented mic mute/unmute handling. 2024-05-02 00:51:09 +00:00
9de49c71a1 [assistant.picovoice] Conversation flow improvements.
- The `Responding` state should be modelled as an extra event/binary
  flag, not as an assistant state. The assistant may be listening for
  hotwords even while the `tts` plugin is responding, and we don't want
  the two states to interfere with each either - neither to build a more
  complex state machine that also needs to take concurrent states into
  account.

- Stop any responses being rendered upon the `tts` plugin when a new
  hotword audio is detected. If e.g. I say "Ok Google", I should always
  be able to trigger the assistant and stop any concurrent audio
  process.

- `SpeechRecognizedEvent` should be emitted even if `cheetah`'s latest
  audio frame results weren't marked as final, and the speech detection
  window timed out. Cheetah's `is_final` detection seems to be quite
  buggy sometimes, and it may not properly detect the end of utterances,
  especially with non-native accents. The workaround is to flush out
  whatever text is available (if at least some speech was detected) into
  a `SpeechRecognizedEvent` upon timeout.
2024-05-02 00:51:09 +00:00
aa333db05c [assistant.picovoice] More features.
- Added wiring between `assistant.picovoice` and `tts.picovoice`.

- Added `RESPONDING` status to the assistant.

- Added ability to override the default speech model upon
  `start_conversation`.

- Better handling of conversation timeouts.

- Cache Cheetah objects in a `model -> object` map - at least the
  default model should be pre-loaded, since model loading at runtime
  seems to take a while, and that could impact the ability to detect the
  speech in the first seconds after a hotword is detected.
2024-05-02 00:51:09 +00:00
a4c911a5d7 Added ffmpeg requirement for assistant.picovoice. 2024-05-02 00:51:09 +00:00
8193c5702c s/partial_transcript/transcript/g 2024-05-02 00:51:09 +00:00
8378bee7c6 Refactored AssistantEvent.
`AssistantEvent.assistant` is now modelled as an opaque object that
behaves the following way:

- The underlying plugin name is saved under `event.args['_assistant']`.

- `event.assistant` is a property that returns the assistant instance
  via `get_plugin`.

- `event.assistant` is reported as a string (plugin qualified name) upon
  event dump.

This allows event hooks to easily use `event.assistant` to interact with
the underlying assistant and easily modify the conversation flow, while
event hook conditions can still be easily modelled as equality
operations between strings.
2024-05-02 00:51:09 +00:00
a670f01647 [assistant.picovoice] Better partial transcript + flush handling logic. 2024-05-02 00:51:09 +00:00
921025be0c picovoice -> assistant.picovoice 2024-05-02 00:51:09 +00:00
bb38f2439c Better integration with the native base API of the assistant plugin. 2024-05-02 00:51:09 +00:00
8f7aad2af7
google-assistant-sdk is no longer a dependency for the Google Assistant integration 2024-02-06 20:08:57 +01:00
171ed7f743
[Entities UI] Better refresh logic. 2023-12-29 03:45:17 +01:00
0de322fb95
[assistant.google] Propagate plugin name as a string to events.
This also makes it easier to programmatically stop conversations on
`SpeechRecognizedEvent` with a matched phrase.
2023-12-21 00:28:27 +01:00
869ef38364
Removed explicit numpy pip dependency in assistant.google.
It will be installed by the other dependencies anyway.
2023-11-18 16:01:24 +01:00
42912581f9 [Fix] Always initialize AssistantPlugin._conversation_start_sound. 2023-10-23 21:30:49 +00:00
cea3055369 Indent pre- and post-install shell scripts in docs. 2023-10-23 15:48:09 +00:00
a005958317
[Assistant] Added assistant entity type. 2023-10-23 16:02:05 +02:00
4c0e3a16b6
Removed Alexa/AVS integration.
The integration was based on my old fork of the AVS service, which is no
longer functional given the changes the the Amazon's backend side.

A new `avs-device-sdk` is now available, but it seems that it requires
lengthy compilation processes which are RaspberryPi-specific.

Further investigation is needed for a new Alexa plugin - see #334.
2023-10-23 00:23:10 +02:00
fb017a8b35
[assistant.google] Added doc on how to configure audio devices. 2023-10-22 22:49:41 +02:00
645e8c8f77
Added updated configuration snippet for assistant.google plugin. 2023-10-22 21:53:15 +02:00
b46c00f015
Removed deprecated google.assistant.pushtotalk plugin.
It only existed as a back-compatibility layer with armv6, since there
was no build of the assistant library that worked on Raspberry Pi Zero.

But that API layer has been discontinued by Google and it's no longer
functional, so only the `assistant.google` integration (on x86_64 and
armv7) is currently supported.
2023-10-22 19:57:55 +02:00
cce6c4c5ad
[#284] Merged assistant.google plugin and backend.
This removes the deprecated `assistant.google` backend and also adds a
new `conversation_start_sound` parameter.
2023-10-22 19:55:11 +02:00
c3337ccc6c
[#311] Docs deps autogen sphinx plugin.
Added an `add_dependencies` plugin to the Sphinx build process that
parses the manifest files of the scanned backends and plugins and
automatically generates the documentation for the required dependencies
and triggered events.

This means that those dependencies are no longer required to be listed
in the docstring of the class itself.

Also in this commit:

- Black/LINT for some integrations that hadn't been touched in a long
  time.

- Deleted some leftovers from previous refactors (deprecated
  `backend.mqtt`, `backend.zwave.mqtt`, `backend.http.request.rss`).

- Deleted deprecated `inotify` backend - replaced by `file.monitor` (see
  #289).
2023-09-24 17:00:08 +02:00
761f2768cb
[#316] Added RPM dependencies. 2023-09-21 23:20:06 +02:00
98e9abde18
Extended manifest files with Python system packages (if available).
- If a Python optional dependency is available as a system package on
  the target system, try and install it that route rather than pip. It's
  usually faster and it decreases the risk of breaking system packages.

- Added support for apk dependencies in manifest files. This brings the
  number of distros officially supported by all the extensions to four:

  - Alpine
  - Arch
  - Debian
  - Ubuntu
2023-08-16 22:43:51 +02:00
66981bd00b
Updated email addresses and black'd some old source files. 2023-07-22 23:02:44 +02:00
56dc8d0972
Migrated the webapp to Tornado.
It was just too painful to find a combination of versions of  gunicorn,
gevent, eventlet, pyuwsgi etc. that could work on all of my systems.

On the other hand, Tornado works out of the box with no headaches.

Also in this commit:

- Updated a bunch of outdated/required integration dependencies.
- Black'd and LINTed a couple of old plugins.
2023-05-08 02:06:45 +02:00
3bfc5b83ef Moved to manifest files for describing plugins and backends and their dependencies 2021-09-16 17:53:40 +02:00
2a78f81a7b Major LINT fixes 2021-04-05 00:58:44 +02:00
Fabio Manganiello
3c3ee09d90 Fixed auto-generated docs indentation and code blocks 2020-08-31 17:39:42 +02:00
Fabio Manganiello
aa631deb88 Fixed docs typo 2020-08-24 01:24:24 +02:00
Fabio Manganiello
2e6388f6f4 ReadTheDocs fixes 2020-08-24 01:14:40 +02:00
Fabio Manganiello
27f847eac6 Support for is_muted/toggle_muted on Google assistant 2020-03-05 18:13:44 +01:00
Fabio Manganiello
87a51b391c Support for custom TTS engine for rendering assistant response (see #86) 2020-02-24 20:23:22 +01:00
Fabio Manganiello
f38121d176 Always pause/resume speech detection on backend level 2020-01-19 16:32:41 +01:00