[#304] Rewrite Picovoice integrations #385

Merged
blacklight merged 36 commits from 304-new-picovoice-integration into master 2024-05-02 02:50:51 +02:00

36 Commits

Author SHA1 Message Date
Fabio Manganiello 50c6190a25
[assistant.picovoice] Extended documentation.
continuous-integration/drone/push Build is passing Details
2024-05-02 02:46:32 +02:00
Fabio Manganiello bd20c53c32
Fixed tests
continuous-integration/drone/push Build is passing Details
2024-04-24 01:25:02 +02:00
Fabio Manganiello 47926290a2
[music.mopidy] Handle the case where `add` returns a list.
continuous-integration/drone/push Build is failing Details
2024-04-24 01:13:03 +02:00
Fabio Manganiello 71e32e4775
[tts.picovoice] Convert digits before replacing other substrings. 2024-04-24 00:49:04 +02:00
Fabio Manganiello 6b20c2083b
🐛 The `EventMatchResult` object shouldn't be initialized with `args` from the event.
If there's a good use-case for overriding `Event._matches_condition`
with a logic that also parses the event arguments, then those arguments
should be accessed directly from the event object, not from the match
result.

Initializing `EventMatchResult` with the arguments from the event means
that, if `EventMatchResult.parsed_args` are populated with custom
extracted arguments, then the upstream event arguments will also be
modified.

If the event is matched against multiple conditions, this will result in
the extracted tokens getting modified by each `matches_condition`
iteration.
2024-04-24 00:18:30 +02:00
Fabio Manganiello a3972e55b6
[tts.picovoice] A more robust logic for replacing unsupported characteres on the input.
continuous-integration/drone/push Build is passing Details
2024-04-23 21:02:51 +02:00
Fabio Manganiello 8cb448623c
Added `openai` plugin. 2024-04-23 20:39:28 +02:00
Fabio Manganiello 99829a93cf
[assistant.picovoice] Sync between the speech and intent engines.
continuous-integration/drone/push Build is passing Details
2024-04-22 02:05:19 +02:00
Fabio Manganiello 862d56a338
Prevent duplicate hook runs.
Instead of being a list, the hooks in the hook processor should be
backed by by-name and by-value maps.

Don't insert a hook if its exact backing method has already been
inserted. This is actually very common when hooks are defined as Python
snippets imported in other scripts too.
2024-04-22 02:05:19 +02:00
Fabio Manganiello 4b41c5377a Merge branch 'master' into 304-new-picovoice-integration
continuous-integration/drone/push Build is failing Details
2024-04-20 17:27:15 +02:00
Fabio Manganiello 8df34a13f1
New architecture for the assistant speech detection logic.
continuous-integration/drone/push Build is passing Details
The assistant object now runs in its own thread and leverages an
external `SpeechProcessor` that uses two threads to scan for both
intents and speech in parallel on audio frames.
2024-04-20 17:24:03 +02:00
Fabio Manganiello cb2077eddc
Merge branch 'master' into 304-new-picovoice-integration 2024-04-17 04:08:49 +02:00
Fabio Manganiello 51a9b956bf Merge pull request '[#389] Fix for "Too many open files" media issue.' (#390) from fix-too-many-open-files-during-media into master
continuous-integration/drone/push Build is passing Details
Reviewed-on: #390
2024-04-17 04:08:10 +02:00
Fabio Manganiello 7c504685e2
Prevent a potential recursion error in `wait_for_either`.
continuous-integration/drone/push Build is passing Details
We shouldn't overwrite `event._set` and `event._clear` if those values
have already been set.

Those attributes hold the original references to `Event.set` and
`Event.clear` respectively, and the `OrEvent` logic overwrites them with
a callback-based logic.

This shouldn't happen if those attributes are already present.
2024-04-14 23:27:13 +02:00
Fabio Manganiello 699f92e42b
[assistant] Added `ResponseEndEvent` and `IntentMatchedEvent` 2024-04-14 23:26:59 +02:00
Fabio Manganiello 317241eb36
[assistant.picovoice] Various improvements.
- Added `intent_model_path` parameter.

- Always apply `expanduser` to configuration paths.

- Better logic to infer the fallback model path.

- The Picovoice Leonardo object should always be removed after
  `assistant.picovoice.transcribe` is called.
2024-04-14 21:24:06 +02:00
Fabio Manganiello 6305934f97
Added `assistant.picovoice.transcribe` and `assistant.picovoice.say`.
continuous-integration/drone/push Build is passing Details
2024-04-14 01:07:17 +02:00
Fabio Manganiello 1044f82a6a
[tts.picovoice] Better text pre-processing logic. 2024-04-14 00:43:28 +02:00
Fabio Manganiello ea1fa39cb7
[assistant.picovoice] Implemented `assistant.picovoice.send_text_query`.
continuous-integration/drone/push Build is passing Details
2024-04-14 00:09:25 +02:00
Fabio Manganiello 7e9b19d328
[tts.picovoice] Added text pre-processing workaround.
continuous-integration/drone/push Build is passing Details
This workaround is required until
https://github.com/Picovoice/orca/issues/10 is fixed.
2024-04-13 21:40:05 +02:00
Fabio Manganiello c86e3be0d1
[assistant.picovoice] Leverage upstream `_on_mute_changed`.
The plugin should leverage `AssistantPlugin._on_mute_changed` to handle
the boilerplate state managent on mute/unmute actions instead of
re-implementing the same logic.
2024-04-13 21:00:30 +02:00
Fabio Manganiello 8082123c58
[assistant.picovoice] Implemented mic mute/unmute handling.
continuous-integration/drone/push Build is passing Details
2024-04-13 20:51:34 +02:00
Fabio Manganiello 19a90ee045 Merge branch 'master' into 304-new-picovoice-integration
continuous-integration/drone/push Build is passing Details
2024-04-13 20:04:32 +02:00
Fabio Manganiello fa49db4107
[assistant.picovoice] Conversation flow improvements.
continuous-integration/drone/push Build is passing Details
- The `Responding` state should be modelled as an extra event/binary
  flag, not as an assistant state. The assistant may be listening for
  hotwords even while the `tts` plugin is responding, and we don't want
  the two states to interfere with each either - neither to build a more
  complex state machine that also needs to take concurrent states into
  account.

- Stop any responses being rendered upon the `tts` plugin when a new
  hotword audio is detected. If e.g. I say "Ok Google", I should always
  be able to trigger the assistant and stop any concurrent audio
  process.

- `SpeechRecognizedEvent` should be emitted even if `cheetah`'s latest
  audio frame results weren't marked as final, and the speech detection
  window timed out. Cheetah's `is_final` detection seems to be quite
  buggy sometimes, and it may not properly detect the end of utterances,
  especially with non-native accents. The workaround is to flush out
  whatever text is available (if at least some speech was detected) into
  a `SpeechRecognizedEvent` upon timeout.
2024-04-13 20:03:35 +02:00
Fabio Manganiello 24e93ad160
Added more default imports under the `platypush` module root.
These objects can now also be imported in scripts through
`from platypush import <name>`:

- `Variable`
- `cron`
- `hook`
- `procedure`
2024-04-10 23:33:48 +02:00
Fabio Manganiello 3b73b22db9
[assistant.picovoice] More features.
- Added wiring between `assistant.picovoice` and `tts.picovoice`.

- Added `RESPONDING` status to the assistant.

- Added ability to override the default speech model upon
  `start_conversation`.

- Better handling of conversation timeouts.

- Cache Cheetah objects in a `model -> object` map - at least the
  default model should be pre-loaded, since model loading at runtime
  seems to take a while, and that could impact the ability to detect the
  speech in the first seconds after a hotword is detected.
2024-04-10 22:26:45 +02:00
Fabio Manganiello 9761cc2eef
Added `tts.picovoice` plugin. 2024-04-10 20:32:32 +02:00
Fabio Manganiello 6bd20bfcf6
Added ffmpeg requirement for `assistant.picovoice`. 2024-04-10 20:31:38 +02:00
Fabio Manganiello 8702eaa25b
s/partial_transcript/transcript/g 2024-04-09 00:19:51 +02:00
Fabio Manganiello 6feb824c04
Refactored `AssistantEvent`.
`AssistantEvent.assistant` is now modelled as an opaque object that
behaves the following way:

- The underlying plugin name is saved under `event.args['_assistant']`.

- `event.assistant` is a property that returns the assistant instance
  via `get_plugin`.

- `event.assistant` is reported as a string (plugin qualified name) upon
  event dump.

This allows event hooks to easily use `event.assistant` to interact with
the underlying assistant and easily modify the conversation flow, while
event hook conditions can still be easily modelled as equality
operations between strings.
2024-04-09 00:15:51 +02:00
Fabio Manganiello fa70c91a67
[assistant.picovoice] Better partial transcript + flush handling logic.
continuous-integration/drone/push Build is passing Details
2024-04-08 13:33:07 +02:00
Fabio Manganiello c7094c7886
`picovoice` -> `assistant.picovoice` 2024-04-08 13:26:44 +02:00
Fabio Manganiello 0b8e1bb81b
Better integration with the native base API of the assistant plugin.
continuous-integration/drone/push Build is passing Details
2024-04-08 03:02:03 +02:00
Fabio Manganiello f021b471aa
[WIP] Added speech detection logic over Cheetah.
continuous-integration/drone/push Build is passing Details
2024-04-08 01:54:26 +02:00
Fabio Manganiello 01dec0b7a4
[WIP] Added initial hotword integration.
continuous-integration/drone/push Build is passing Details
2024-04-07 22:42:01 +02:00
Fabio Manganiello f0382c73ab
[#304] Removed old Picovoice integrations
continuous-integration/drone/push Build is passing Details
2024-04-06 00:11:46 +02:00