R2D2 Satellite - Voice Assistant Contest - Home Assistant Community = 40rem)" rel="stylesheet" data-target="desktop" /> = 40rem)" rel="stylesheet" data-target="chat_desktop" /> = 40rem)" rel="stylesheet" data-target="discourse-ai_desktop" /> = 40rem)" rel="stylesheet" data-target="discourse-reactions_desktop" /> = 40rem)" rel="stylesheet" data-target="poll_desktop" /> = 40rem)" rel="stylesheet" data-target="desktop_theme" data-theme-id="19" data-theme-name="scroll to top"/> R2D2 Satellite Voice Assistant Contest abrakhim February 15, 2024, 11:29pm 1 Hello all! I am working on my ‘smart’ R2D2 for some time and i just finished making it a Home Assistant Satellite. The project has two main goals - first, making the robot drive independently, patrol my house and follow me, recharge when battery is low, detect obstacles, map the rooms - much like a vacuum robot. The second goal is to make it smart and integrate with HA. IMG_20240215_233132__011920×2379 267 KB This goal is ready now, so i wanna share the current setup here: The 40cm remote controlled R2D2 toy was gutted and now has a Pi Motor HAT (…had an Arduino with MotorShield) and a Raspberry Pi 4(no, 3…) inside, with a Respeaker 4-mic array in the top dome. I tried other ways to turn it into an assistant, but recently things got a lot easier thanks to HA Focus on Assistants. I used: some r2d2-Sounds as wav-Files found on the net wyoming-openwakeword wyoming-satellite HA wakewords Collection - which luckily had a much better r2d2-Wakeword than the one i trained and parts of [wyoming Satellite - Tutorial]wyoming-satellite/docs/tutorial_2mic.md at 8850100ca658a80708e562e6741739ed27c91f99 · rhasspy/wyoming-satellite · GitHub So, what does it do? I listens locally to the wakeword ‘R2D2’, then acts as a remote satellite in Home Assistant and reacts to the wakeword with a nice R2-Sound, then records the voice command and HA tries to understand and execute the command. It does not play responses with TTS, it just plays another sound after finishing it’s task. Heres a Dropbox-Link to a short video demo: R2D2-Video Many thanks for making this so easy - now i can continue working on the robot part - currently i can stear it with a PS4 Controller, but sadly all my tinkering has shifted the center of gravity and it falls over when braking Keep on tinkering, Abrakhim R2D2 Voice Assistant jenova70 (JLo) February 16, 2024, 11:20pm 2 Wowowowowowowwwwwww TAKE MY VOTE THIS IS SO COOL ! I just love the fact the lights react to the sound even before the wake word. It makes it so snappy !!! 1 Like ObiDenKenobi February 17, 2024, 10:18am 3 Yep… very cool! You have my vote too… Are you aware of the Astromech TTS integration that lets you text to R2D2 speak? Might be the perfect addition for your project if you get around to implementing TTS… 5 Likes abrakhim February 17, 2024, 4:11pm 4 Dude, that’s awesome. I wasn’t aware of it and will integrate it RIGHT NOW 1 Like abrakhim February 17, 2024, 4:56pm 5 … made my day After ObiDenKenobi remark about TTAstromech i went straight to work and now R2 answers in proper Astromech-Talk. I can even already understand his answer for ‘Sorry, I couldn’t understand that’ and ‘Turned on the lights’ hrhrhr …but, hear for yourself … Turned on the lights Sorry, I couldn’t understand that btw… this is using TTAstromech Voice-Option SHORT… the other one made me wanna kick him instantly 4 Likes Rich37804 February 17, 2024, 6:04pm 6 Very cool. Great work. ObiDenKenobi February 17, 2024, 8:26pm 7 Glad I could help… ! Yeah, the short voice option is much preferable… unless you’re into sm-ing R2. Love the new vid’s. Shame I can’t vote twice… 1 Like DaWheelz (Bas Mutsaers) February 23, 2024, 7:18pm 8 Really cool to see these kind of projects starting to take shape! You have my vote! I’m struggling a bit to get this wake word detected sound to work. abrakhim February 26, 2024, 9:03pm 9 Hey there! Anyone got a tip how to improve STT? I think i may have a lot of noise. Although i repeat “R2D2, Turn of the lights” like a calm teacher, this is what whisper gets : INFO:wyoming_faster_whisper.handler: Turn off the lights. INFO:wyoming_faster_whisper.handler: Oh shit. What I know, it's like- INFO:wyoming_faster_whisper.handler: I'm trying to talk to you inhabit this video! INFO:wyoming_faster_whisper.handler: Oh shit. One of the lights. INFO:wyoming_faster_whisper.handler: Oh shit. Turn off the lights. INFO:wyoming_faster_whisper.handler: Ocean... Turn off INFO:wyoming_faster_whisper.handler: who actually is like this? The wake-word works very nice though… As a first countermeasure i will try replacing the 4mic-Respeaker 3.5 audio jack and the little 2.5 W amp with a pimoroni Audio Amp SHIM - but on the software side - like wyoming-satellite - any ideas? BTW: this is the PI ( 3, not 4 as i remembered it…) and the Pi Motor HAT i use. IMG_20240226_2142131280×1707 350 KB I also added a little LED Flashlight, replaced the foil speaker with a decent one, added a PIR Sensor, connected the Blue and Red LEDs the Bot came with and the ‘Leia-Projector’ to the GPIOs, the two propulsion motors and the two motors that change it’s stance. I will upload a video, once this beast is finished… like in some years hrhrhr. And this is a look into it’s guts: IMG_20240226_2158161707×1280 343 KB ObiDenKenobi March 12, 2024, 7:31am 10 Really interested to see if you have done any other modifications \ additions… ? jenova70 (JLo) March 13, 2024, 8:29am 11 By the way, did you end up making the robot move and follow you as you were saying in your original post ? Thx for your entry, it’s amazing! jenova70 (JLo) March 17, 2024, 9:42am 12 The results of the contest are out! They may be of interest to you Have a look! ELBRE (Elmar) June 15, 2024, 5:34am 13 This is very cool, you have my vote too. I trained my R2D2 wake word myself but this seems to be much better. But one question where di you get the cool R2D2 from? Kind reagrds Elmar Garyw (Gary Wilkinson) July 27, 2024, 9:24pm 14 Yes I would also like to know where to get the R2D2 from and do the same, so cool.