The Wanda Walleye Project



Picture of Wanda Walleye on her trophy column and tackle box
Picture of Wanda Walleye on her trophy column and tackle box

I am listening. How can I help you?



Welcome


Welcome to the Wanda Walleye blog.

The purpose of this blog is to document a toy I’ve been working on for … uh, several months now. [NOTE: More pictures, videos, links, source code, all are on their way.]


Picture of Wanda Walleye from the front with her goggles lit up. Yeah she is a goob. But a loveable goob.
Picture of Wanda Walleye from the front with her goggles lit up. Yeah she is a goob. But a loveable goob.

Wanda is, basically, a “smart” talking and listening plush fish that uses built-in LLM AI to listen to what you say, to generate some of the things she says, and to abridge some of the other overlong things she says. She also uses a lot of hardware like an ESP32 and strands of programmable LEDs (in her swim goggles and her pineapple mug complete with slurpy straw) to add some visual interest and to seem more interactive. And she uses gadgets like a BMP280 barometric pressure sensor, a 2.4GHz wireless remote, and a (coming soon!) MPR121 multi-touch sensor to provide more info and more ways to interact.


Picture of inside of Wanda tackle box with hardware labeled.
Picture of inside of Wanda tackle box with hardware labeled.

Wanda the Walleye is both a toy to keep me busy by adding features to it and playing with it, and also the beginning of something more important: A personal assistant that can help my aging brain keep track of the key things I should be doing every day and on a regular basis. A sort of home automation system, but for people, not places.

What Wanda Can Do

Here is a list of the things that Wanda can currently do. It is almost certainly outdated, as new code is constantly added whenever I see an interesting thing on the internet, or have a dumb idea, which is pretty much all the time.

Things That Require An Internet Connection

A long-term goal is to move away from these things, so Wanda can run with wifi turned off all the time. Safe and secured by an old-school air gap.

A conflicting long-term goal is for Wanda to help with reminders and day-to-day reports, which of course require internet connections to grab weather data, update calendars, send email, etc.

Sigh.

Things That Run Completely Standalone

None of the following apps require an internet connection. They all run completely locally on Wanda’s NUC.

How It Started

When the latest AI craze, the whole ChatGPT thing, started taking off, I became curious about using LLMs for something useful. But also for something personal. And fun. I was also looking for a project that would involve both tech and arts and crafts to keep my brain busy.

Personal in this case means two things:

  1. Something that would run locally, and not be dependent on giant mothership server farms that would harvest my data for profit. Which is to say, something that would protect privacy by not being online. As Wanda evolves, it will eventually have a fair amount of personal, private info on it, that I have no intention of sharing with Apple, Google, Meta, or, really, anyone.
  2. Something that would reflect my own sense of humor, and which would be useful for me personally.

Fun for me means keeping busy going down endless rabbit holes, hearing about the latest tech fad or meme, and tinkering with it. To be explicit, this blog is not fun. But it is useful for keeping some sort of record of all the different aspects of Wanda.

Version 0.1: Smart Billy Bass

Like many people, I started out by making an LLM-driven “smart” Billy Bass, by hacking the singing fish, connecting an Arduino to control its motors, and a Raspberry Pi 5 to do the AI stuff.

The Pi could have run the motors directly, but I figured that the Arduino provided a level of protection, in that if I burned out an IO port or even the whole thing, I was smoking a relatively cheap microcontroller rather than a fairly pricey Pi 5. Also, I had done similar projects on both the Arduino and Pi, and felt far more comfortable working with the Arduino’s IDE, and the very low-level setup() and loop() routines rather than equivalent code on the Pi.

The main thing I did differently from similar “smart” Billys or other Pi AI projects was to run all of the AI stuff locally on the Pi. Which worked, but was very slow.

I was constantly nettled by PiOS updates and upgrades breaking things. And by cryptic and underdocumented (even for Linux) configuration arcana.

Also, the Pi tended to run hot when under load, to the point that I didn’t trust it to leave it running when I wasn’t around. Soon, the microSD card, which the Pi used as its boot (and everything else) disk, died, probably from being cooked. I only lost about a day's work, thanks to an auto-backup 'cron' job. But that kind of took the fun out of working on Billy.

I did eventually buy a USB thumb drive with an aluminum case, stuck a little copper heat sink on it, and restored Billy's boot disk to it. So hopefully it'll last longer than the uSD card did. (No, I'm not interested in buying Billy a proper SSD, and "hat" for the SSD, and bigger case to hold both the Pi and hat, because...)

Another issue was the overall cost: The Pi 5 with the maximum 8GB of RAM, an active cooling aluminum case with a fan, the Pi-official power supply, a microSD card, all together ended up costing around $150. That’s not including a Billy Bass in good condition, Arduino, H-bridge to drive the motors, audio stuff, etc.

For that price, I could buy a decent small form factor PC, a used one, plus heaps of RAM for memory-hungry LLMs. It would come with built-in audio and power supply, could run standard and well-supported versions of Linux like Ubuntu, and would be considerably faster and cooler-running...

Version 0.2: A NUC And A ... Furby? Ruxpin?

So, I bought a 2018-era Intel NUC, a 4 inch by 4 inch by 2 inch desktop PC in like-new condition for $70. I also bought a new-ish-to-me $10 ESP32 to do the Arduino stuff. The NUC wouldn’t be quite as compact as the Pi, but not a whole lot larger either.


Picture of the Pi 5 and NUC sitting on the Billy Bass case back. The Pi wall wart is to the left and the NUC wall wart is to the right.
Picture of the Pi 5 and NUC sitting on the Billy Bass case back. The Pi wall wart is to the left and the NUC wall wart is to the right.

I chose to go with an ESP32 mostly because I didn’t have much experience with one (though I had monkeyed with an ESP8266, and liked how I could program it using the Arduino IDE and libraries). I wanted to become more familiar with the more capable ESP32.

I was also intrigued by the idea that it had built-in wifi and could do some basic internet stuff, including getting content from HTTPS webpages. One long-term goal for Wanda is to turn off the NUC’s wifi completely, and never need to worry about it being hacked or inadvertently sharing something it shouldn’t. Perhaps the NUC could use the relatively dumb ESP32 to retrieve things like NOAA weather info and maybe even calendar events.

One thing I had considered when first working on Billy was the idea of actually putting the Pi inside of Billy, putting it entirely inside Billy’s plaque, along with the Arduino and other hardware. And that Billy might not be a singing plastic bass, but rather might be a Furby, or maybe Teddy Ruxpin, or some other toy with a motor-driven mouth and a little space inside. It turns out that the Pi ran way, way too hot to make that viable. But the idea still stuck with me.

While poking around eBay and Etsy and Temu and AliExpress looking for ideas for singing bass, bears, etc., I stumbled across a Billy Bass clone. It was a stuffed plush fish of some sort (bass? walleye? some similar fish from China?). It didn’t move its tail or tilt forward like Billy could, but it did sing and move its mouth.


Picture of the original plush singing fish before the hack job, completel with NUC RAM sticks and The Hu band patch.
Picture of the original plush singing fish before the hack job, completel with NUC RAM sticks and The Hu band patch.

I got the idea of making a sort of “trophy”, using the NUC as the base of the trophy, and the not-Billy as the decoration at the top of the trophy column.


Picture of Wanda being held above the first pass at a trophy column on top of the NUC with the gift boxes on either side. Kinda like this?
Picture of Wanda being held above the first pass at a trophy column on top of the NUC with the gift boxes on either side. Kinda like this?

There would need to be a couple of NUC-sized boxes to hold the ESP32, H-bridge, and NUC wall wart. They could sit on either side of the NUC, and hide all that stuff and all the wiring.


Picture of the NUC with both gift boxes open, exposing the bunch of stuff stuffed inside them.
Picture of the NUC with both gift boxes open, exposing the bunch of stuff stuffed inside them.

I found some cardboard gift boxes with glossy black piano finishes on Amazon. They looked pretty nice in person and were remarkably close to the same size as the NUC, so they looked pretty natural sitting next to it.

This setup worked OK, but the boxes were pretty snug. And things got warm inside them. Mostly because the side of the NUC that had 2 of the USB ports and the jack for the wall wart was also the side where the NUC’s fan blew out the hot exhaust air. So it would blow straight into that little box. And even with opening the back side of the box and adding a fan in the box for extra ventilation, it still got hot in there.

Something bigger and cooler was needed for the trophy base.

How It's Going

The bigger cooler trophy base ended up being a small steel toolbox from Menards (a Midwest hardware mega-store like Home Depot). The toolbox is nice and sturdy, but the steel walls are still thin enough to easily cut and drill through. It was on sale for an amazing $15–a good price for a plastic toolbox of similar size.

The toolbox is actually a bit too big, but that just means that it’s ready for more toys, sensors, audio stuff, etc. inside. Better too big than too little. After all, Wanda is a project for exploration and fun, it’s not an Apple iPad. It’s OK for Wanda to have a Badger Butt.

The Invasion Of Temu

Around the same time I started working on the new version of Wanda’s base, I started becoming addicted to a frequent shopper at Temu. Temu is to Amazon as Amazon is to Walmart, and as Walmart is to a regular store: a cheaper competitor that regularly (and doubtless ruthless) undercuts prices, and which sells the same made-in-China stuff as Walazon, just for a lot less money. Do I feel a little guilty about shopping at Temu? A little. The profits don’t go to American billionaires, they go to rich Chinese people. But the stuff is made in the same factories under the same grueling conditions, and is the same quality (good or bad).

Anyway, for folks with a limited budget and a constant itch to get new gadgetry, Temu works. Much of Wanda’s bling, and a some of her internal hardware, was purchased at relatively low cost via Temu.

To be continued!



Version 2024-July-07.