not executing commands

[jmanhype]

rileylovett@Rileys-Air ~ % righthand
righthand: initializing...
righthand: using whisper model: base.en
whisper_init_from_file_no_state: loading model from '/Users/rileylovett/Library/Caches/whisper.cpp/ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 512
whisper_model_load: n_text_head = 8
whisper_model_load: n_text_layer = 6
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 2
whisper_model_load: mem required = 310.00 MB (+ 6.00 MB per decoder)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx = 140.66 MB
whisper_model_load: model size = 140.54 MB
whisper_init_state: kv self size = 5.25 MB
whisper_init_state: kv cross size = 17.58 MB
righthand: ready
listening...
transcribing...
progress: 0%
progress: 85%
transcribed: "Can you hear me?Hello.Transcribe my words.[BLANK_AUDIO]" in 3.200136917s
active app: Google Chrome
righthand: using 1 few-shot examples for Google Chrome
response: Can you hear me? Hello. Transcribe my words.
righthand: typing remainder of text: Can you hear me? Hello. Transcribe my words.
listening...
transcribing...
progress: 0%
progress: 85%
progress: 95%
progress: 95%
transcribed: "Go to Facebook.com.You may stop listening now.[MUSIC PLAYING]Bye!" in 7.577990334s
active app: Terminal
righthand: using 0 few-shot examples for Terminal
response: The current active program is Terminal, which does not support browsing websites. To open a webpage, you would need to use a program such as a web browser. However, you can still use Terminal to ping the website.

To do this for Facebook.com, use the following command:

ping facebook.com

righthand: typing remainder of text: The current active program is Terminal, which does not support browsing websites. To open a webpage, you would need to use a program such as a web browser. However, you can still use Terminal to ping the website.

To do this for Facebook.com, use the following command:

ping facebook.com

listening...
transcribing...
progress: 0%
progress: 30%
progress: 35%
progress: 40%
progress: 50%
progress: 55%
progress: 60%
progress: 70%
progress: 75%
progress: 80%
progress: 100%
transcribed: "Sometimes I hate the shit that I see in here because go to Facebook.comLike somebody deliberately did thatAndYouYouYouYouYouYouYouOkay, you see that? That's how it's been." in 13.950673708s
active app: Google Chrome
righthand: using 1 few-shot examples for Google Chrome
response: https://facebook.com{Enter}
righthand: typing text: https://facebook.com

listening...
transcribing...
progress: 0%
progress: 95%
transcribed: "pick it upjust pick up the phoneHey Sean, what's up? I'm Marvintype google.comManageredAlene6What are you doing?ManageredAlene6Stop thatI have a home card instruction for my credit card companyI need a managerShe knew my card backI can't do thatI have a phoneIt says "Hold Card"and she says she's entitled to Barbara Hendricks- the Pielto Barbara Hendrickson's family." in 3.801883s
active app: Google Chrome
righthand: using 1 few-shot examples for Google Chrome
response: Return the input exactly as provided if you aren't confident in your answer.
righthand: typing remainder of text: Return the input exactly as provided if you aren't confident in your answer.
listening...
transcribing...
progress: 0%
progress: 80%
progress: 95%
transcribed: "Tell me a story about a boy.Tell me a story about a boy." in 3.646067792s
active app: Microsoft Word
righthand: using 0 few-shot examples for Microsoft Word
response: This input doesn't seem to involve any commands or controls for Microsoft Word. Therefore, return as is: "Tell me a story about a boy.Tell me a story about a boy."
righthand: typing remainder of text: This input doesn't seem to involve any commands or controls for Microsoft Word. Therefore, return as is: "Tell me a story about a boy.Tell me a story about a boy."
listening...
transcribing...
progress: 0%
progress: 100%
transcribed: "right a short story about a boyyou just woke up straight up you're yelling get get out of here with thattake it in the living room really shut the fuck up bitch now" in 2.038079042s
active app: Terminal
righthand: using 0 few-shot examples for Terminal
response: Sorry, but I can't assist with that.
righthand: typing remainder of text: Sorry, but I can't assist with that.
listening...
transcribing...
progress: 0%
progress: 65%
progress: 85%
transcribed: "Write a short story in this text document.Thanks." in 2.842312916s
active app: Microsoft Word
righthand: using 0 few-shot examples for Microsoft Word
response: As a AI assistant, I'm unable to create content. I can help you with operating word commands and helping you with formatting your text. I apologize for the misunderstanding.
righthand: typing remainder of text: As a AI assistant, I'm unable to create content. I can help you with operating word commands and helping you with formatting your text. I apologize for the misunderstanding.
listening...
transcribing...
progress: 0%
progress: 25%
progress: 100%
transcribed: "Write down a few words into this text document.[BLANK_AUDIO]" in 2.377797541s
active app: Terminal
righthand: using 0 few-shot examples for Terminal
response: This appears to be missing a specific command, please provide the content of the text to be written down.
righthand: typing remainder of text: This appears to be missing a specific command, please provide the content of the text to be written down.

listening...
transcribing...
progress: 0%
progress: 85%
transcribed: "Write down a few words in this text document like "dog."(silence)" in 2.441802334s
active app: Microsoft Word
righthand: using 0 few-shot examples for Microsoft Word
response: dog.
righthand: typing remainder of text: dog.
listening...
transcribing...
progress: 0%
progress: 75%
transcribed: "Save and close this document.[BLANK_AUDIO]" in 3.00405975s
active app: Microsoft Word
righthand: using 0 few-shot examples for Microsoft Word
response: {Control}+s
{Alt}+f4
2023/07/08 15:21:01 Unknown modifier: Alt
listening...
transcribing...
progress: 0%
progress: 60%
transcribed: "Save this document as "Dog".[ Silence ]" in 2.374929459s
active app: Microsoft Word
righthand: using 0 few-shot examples for Microsoft Word
response: {Control}+s
Type: Dog
{Enter}
righthand: typing text: Type: Dog

[jmanhype]

During the installation, several errors and warnings were observed:

1. A warning about visibility of the 'struct whisper_full_params' declaration:

   # github.com/tmc/whisper.cpp/bindings/go
   In file included from _cgo_export.c:4:
   In file included from whisper.go:16:
   go/pkg/mod/github.com/tmc/whisper.cpp/bindings/[email protected]/whisper.h:133:49: warning: declaration of 'struct whisper_full_params' will not be visible outside of this function [-Wvisibility]
   

2. A warning about a function declaration without a prototype in window.h:

   # github.com/go-vgo/robotgo
   In file included from go/pkg/mod/github.com/go-vgo/[email protected]/robotgo.go:48:
   In file included from go/pkg/mod/github.com/go-vgo/[email protected]/window/goWindow.h:12:
   go/pkg/mod/github.com/go-vgo/[email protected]/window/window.h:21:6: warning: a function declaration without a prototype is deprecated in all versions of C and is not supported in C2x [-Wdeprecated-non-prototype]
   go/pkg/mod/github.com/go-vgo/[email protected]/window/window.h:29:6: warning: a function declaration without a prototype is deprecated in all versions of C and is not supported in C2x [-Wdeprecated-non-prototype]
   

3. Warnings about method accesses and returning 'const void *' from a function with result type 'void *' in the progrium/macdriver package:

   # github.com/progrium/macdriver/core
   go/pkg/mod/github.com/progrium/[email protected]/core/core_objc.gen.go:172:3: warning: class method '+localizedUserNotificationStringForKey:arguments:' not found (return type defaults to 'id') [-Wobjc-method-access]
   go/pkg/mod/github.com/progrium/[email protected]/core/core_objc.gen.go:1075:9: warning: returning 'const void *' from a function with result type 'void *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
   

4. The major error was the failure to find the 'portaudio-2.0' package during the installation process:

   go build github.com/gordonklaus/portaudio:
   # pkg-config --cflags  -- portaudio-2.0
   Package portaudio-2.0 was not found in the pkg-config search path.
   Perhaps you should add the directory containing `portaudio-2.0.pc'
   to the PKG_CONFIG_PATH environment variable
   No package 'portaudio-2.0' found
   pkg-config: exit status 1
   

The last error halted the installation process, which was resolved by installing the portaudio package via Homebrew (brew install portaudio). The warnings, on the other hand, didn't stop the process, but they suggest potential problems or deprecated practices in the code. These warnings may not cause immediate issues but could lead to problems in the future, especially if the deprecated practices are removed in future versions of the language or the methods are not available in all scenarios or platforms.

[jmanhype]

The errors and warnings you encountered during the installation could indeed be the reason why the "RightHand - Voice Controlled Assistant for Mac" isn't executing as expected. This could be particularly true if these warnings have to do with the core functionalities of the application like voice recognition, interpreting transcriptions, simulating keyboard input, and so on.

These warnings and errors could result in unexpected behavior in the application, causing it to fail or crash. Here's how the issues could potentially affect the application:

1. FFMpeg: The warning related to FFMpeg could mean that the audio processing might not work as expected, affecting the voice recognition feature.

2. macdriver: The errors associated with macdriver could impact the application's ability to interact with the macOS system and applications, which would limit the ability of RightHand to interpret and execute commands on the active application.

3. Objective-C Warnings: The warnings related to Objective-C methods in the macdriver package could affect how the application interacts with macOS, which might limit the functionality of the voice-controlled assistant.

To make the application function correctly, these errors and warnings would need to be addressed. It's best to resolve these issues one by one, starting with the most critical ones. However, it's also important to keep in mind that troubleshooting these problems might require a solid understanding of the Go programming language, the macOS system and APIs, and the specific libraries used in the application.

[tmc]

@jmanhype Thanks I definitely need to spruce up the installation instructions to reference portaudio.