[#304] Rewrite Picovoice integrations #385

Merged

blacklight merged 36 commits from 304-new-picovoice-integration into master 2024-05-02 02:50:51 +02:00

Conversation 0 Commits 36 Files changed 49 +2838 -805

blacklight commented 2024-04-06 00:13:20 +02:00

Owner

Copy link

• Implement Porcupine hotword trigger interface
• Implement Cheetah speech-to-text interface
• Docs plugin icon
  UI plugin icon
• Test conversation sound
• Intents (Rhino)
• Offline transcriptions (Leopard)
• TTS
• Start/stop conversation actions
• Mic mute/unmute
• Implement with_follow_on_turn
• Implement send_text_query
• Add OpenAI plugin
• Implement temporary workaround for unsupported handling of digits and punctuation on orca

Closes: #304

* [x] Implement Porcupine hotword trigger interface * [x] Implement Cheetah speech-to-text interface * [x] Docs plugin icon ~~UI plugin icon~~ * [x] Test conversation sound * [x] Intents (Rhino) * [x] Offline transcriptions (Leopard) * [x] TTS * [x] Start/stop conversation actions * [x] Mic mute/unmute * [x] Implement `with_follow_on_turn` * [x] Implement `send_text_query` * [x] Add OpenAI plugin * [x] Implement temporary workaround for unsupported handling of digits and punctuation on `orca` Closes: #304

blacklight added the

enhancement

label 2024-04-06 00:13:20 +02:00

blacklight self-assigned this 2024-04-06 00:13:20 +02:00

blacklight added 1 commit 2024-04-06 00:13:21 +02:00

[#304] Removed old Picovoice integrations f0382c73ab

blacklight added this to the integrations project 2024-04-06 00:13:21 +02:00

blacklight added 1 commit 2024-04-07 22:42:23 +02:00

[WIP] Added initial hotword integration. 01dec0b7a4

blacklight added 1 commit 2024-04-08 01:54:55 +02:00

[WIP] Added speech detection logic over Cheetah. f021b471aa

blacklight added 1 commit 2024-04-08 03:02:37 +02:00

Better integration with the native base API of the assistant plugin. 0b8e1bb81b

blacklight added 2 commits 2024-04-08 13:33:20 +02:00

picovoice -> assistant.picovoice c7094c7886

[assistant.picovoice] Better partial transcript + flush handling logic. fa70c91a67

blacklight added 6 commits 2024-04-13 20:01:35 +02:00

Refactored AssistantEvent. ... 6feb824c04

`AssistantEvent.assistant` is now modelled as an opaque object that
behaves the following way:

- The underlying plugin name is saved under `event.args['_assistant']`.

- `event.assistant` is a property that returns the assistant instance
  via `get_plugin`.

- `event.assistant` is reported as a string (plugin qualified name) upon
  event dump.

This allows event hooks to easily use `event.assistant` to interact with
the underlying assistant and easily modify the conversation flow, while
event hook conditions can still be easily modelled as equality
operations between strings.

s/partial_transcript/transcript/g 8702eaa25b

Added ffmpeg requirement for assistant.picovoice. 6bd20bfcf6

Added tts.picovoice plugin. 9761cc2eef

[assistant.picovoice] More features. ... 3b73b22db9

- Added wiring between `assistant.picovoice` and `tts.picovoice`.

- Added `RESPONDING` status to the assistant.

- Added ability to override the default speech model upon
  `start_conversation`.

- Better handling of conversation timeouts.

- Cache Cheetah objects in a `model -> object` map - at least the
  default model should be pre-loaded, since model loading at runtime
  seems to take a while, and that could impact the ability to detect the
  speech in the first seconds after a hotword is detected.

Added more default imports under the platypush module root. ... 24e93ad160

These objects can now also be imported in scripts through
`from platypush import <name>`:

- `Variable`
- `cron`
- `hook`
- `procedure`

blacklight force-pushed 304-new-picovoice-integration from 2b287b569f to fa49db4107 2024-04-13 20:04:00 +02:00 Compare

blacklight added 1 commit 2024-04-13 20:04:35 +02:00

Merge branch 'master' into 304-new-picovoice-integration 19a90ee045

blacklight added 1 commit 2024-04-13 20:51:41 +02:00

[assistant.picovoice] Implemented mic mute/unmute handling. 8082123c58

blacklight added 2 commits 2024-04-13 21:40:49 +02:00

[assistant.picovoice] Leverage upstream _on_mute_changed. ... c86e3be0d1

The plugin should leverage `AssistantPlugin._on_mute_changed` to handle
the boilerplate state managent on mute/unmute actions instead of
re-implementing the same logic.

[tts.picovoice] Added text pre-processing workaround. ... 7e9b19d328

This workaround is required until
https://github.com/Picovoice/orca/issues/10 is fixed.

blacklight added 1 commit 2024-04-14 00:10:09 +02:00

[assistant.picovoice] Implemented assistant.picovoice.send_text_query. ea1fa39cb7

blacklight added 2 commits 2024-04-14 01:07:45 +02:00

[tts.picovoice] Better text pre-processing logic. 1044f82a6a

Added assistant.picovoice.transcribe and assistant.picovoice.say. 6305934f97

blacklight added 3 commits 2024-04-15 23:02:59 +02:00

[assistant.picovoice] Various improvements. ... 317241eb36

- Added `intent_model_path` parameter.

- Always apply `expanduser` to configuration paths.

- Better logic to infer the fallback model path.

- The Picovoice Leonardo object should always be removed after
  `assistant.picovoice.transcribe` is called.

[assistant] Added ResponseEndEvent and IntentMatchedEvent 699f92e42b

Prevent a potential recursion error in wait_for_either. ... 7c504685e2

We shouldn't overwrite `event._set` and `event._clear` if those values
have already been set.

Those attributes hold the original references to `Event.set` and
`Event.clear` respectively, and the `OrEvent` logic overwrites them with
a callback-based logic.

This shouldn't happen if those attributes are already present.

blacklight added 3 commits 2024-04-20 17:26:25 +02:00

Merge pull request '[#389] Fix for "Too many open files" media issue.' (#390) from fix-too-many-open-files-during-media into master ... 51a9b956bf

Reviewed-on: #390

Merge branch 'master' into 304-new-picovoice-integration cb2077eddc

New architecture for the assistant speech detection logic. ... 8df34a13f1

The assistant object now runs in its own thread and leverages an
external `SpeechProcessor` that uses two threads to scan for both
intents and speech in parallel on audio frames.

blacklight added 1 commit 2024-04-20 17:27:18 +02:00

Merge branch 'master' into 304-new-picovoice-integration 4b41c5377a

blacklight added 2 commits 2024-04-22 02:05:49 +02:00

Prevent duplicate hook runs. ... 862d56a338

Instead of being a list, the hooks in the hook processor should be
backed by by-name and by-value maps.

Don't insert a hook if its exact backing method has already been
inserted. This is actually very common when hooks are defined as Python
snippets imported in other scripts too.

[assistant.picovoice] Sync between the speech and intent engines. 99829a93cf

blacklight added 2 commits 2024-04-23 21:03:27 +02:00

Added openai plugin. 8cb448623c

[tts.picovoice] A more robust logic for replacing unsupported characteres on the input. a3972e55b6

blacklight added 3 commits 2024-04-24 01:13:47 +02:00

🐛 The EventMatchResult object shouldn't be initialized with args from the event. ... 6b20c2083b

If there's a good use-case for overriding `Event._matches_condition`
with a logic that also parses the event arguments, then those arguments
should be accessed directly from the event object, not from the match
result.

Initializing `EventMatchResult` with the arguments from the event means
that, if `EventMatchResult.parsed_args` are populated with custom
extracted arguments, then the upstream event arguments will also be
modified.

If the event is matched against multiple conditions, this will result in
the extracted tokens getting modified by each `matches_condition`
iteration.

[tts.picovoice] Convert digits before replacing other substrings. 71e32e4775

[music.mopidy] Handle the case where add returns a list. 47926290a2

blacklight added 1 commit 2024-04-24 01:28:21 +02:00

Fixed tests bd20c53c32

blacklight changed title from WIP: [#304] Rewrite Picovoice integrations to [#304] Rewrite Picovoice integrations 2024-04-26 00:47:06 +02:00

blacklight added 1 commit 2024-05-02 02:47:22 +02:00

[assistant.picovoice] Extended documentation. 50c6190a25

blacklight merged commit ee4cb09b52 into master 2024-05-02 02:50:51 +02:00

blacklight deleted branch 304-new-picovoice-integration 2024-05-02 02:50:52 +02:00

blacklight referenced this pull request from a commit 2024-05-02 02:50:53 +02:00

Merge pull request '[#304] Rewrite Picovoice integrations' (#385) from 304-new-picovoice-integration into master

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

architecture

  

authentication

  

backend

  

bug

  

ci/cd

  

cleanup

  

documentation

  

duplicate

  

enhancement

  

good first issue

  

help wanted

  

in progress

  

invalid

  

media

  

mobile

  

new feature

  

packaging

  

question

  

ui

  

voice

  

waiting user input

  

wontfix

No labels

architecture

authentication

backend

bug

ci/cd

cleanup

documentation

duplicate

enhancement

good first issue

help wanted

in progress

invalid

media

mobile

new feature

packaging

question

ui

voice

waiting user input

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

integrations

Assignees

Clear assignees

No assignees

blacklight

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: platypush/platypush#385

No description provided.