Listen & respond
Microphone input and speaker output support spoken conversation and interruption.
PROJECT 001 / ROBOTICS · EMBEDDED SYSTEMS
A little robot.
A lot of learning.✳
A desktop companion that listens, sees gestures, and moves. An exploration of what happens when code meets the physical world.
Explore the build ↘THE BRAINESP32-S3
INTERACTIONVoice + vision
MOVEMENT2 motors · 3 servos
BUILT BYArthur
01 / THE IDEA
Beyond a screen.
Into the real world.
What does it take to turn a conversation into a physical action? This robot brings voice, vision, electronics, and motion together in one small system.
The build combines an ESP32-S3 controller, Xiaozhi AI voice firmware, a local vision module, and custom circuit boards. The challenge is making those parts work together—not just getting each part to work on its own.
See & react
A local AI camera reports gestures. A separate mode controls when they can trigger motion.
Move & express
Two drive motors, three servos, and a small OLED turn commands into visible responses.
02 / UNDER THE HOOD
One robot.
Connected disciplines.
Follow the signal from a spoken command
to a moving motor.
THE HARDWARE JOURNEY
Three stages.
One lesson in power.
My first custom PCB took three months to build. Getting it to work taught me something I had overlooked: reliable communication starts with reliable power.
01 DEVELOPMENT BOARD
Make it move.
I started with an ESP32 development board and got the robot to perform basic movements. But the board was bulky, and loose Dupont jumper connections made the prototype unreliable. I wanted a smaller, more dependable way to connect everything.
NEXT QUESTIONCould I design a board around the robot?
02 FIRST CUSTOM PCB
It worked. Until the motors ran.
After three months of work, I had my first usable PCB—and a major flaw. I had focused on communication circuits without accounting for how the motors would affect the rest of the system. When they ran, microphone operation became unreliable and Wi-Fi grew unstable.
WHAT I LEARNEDA circuit must work under load, too.
03 TWO-BOARD REDESIGN
Separate the loads. Rebuild.
I split the design into two connected boards: one for the 3.3 V controller and low-power peripherals, including the display, and another for the higher-current motor circuitry. I gave the boards separate power feeds, kept a common signal ground, and added a 1000 µF bulk capacitor to help buffer changes in current demand.
OBSERVED RESULTIn my subsequent tests, the redesigned robot operated normally without the earlier microphone and Wi-Fi issues.
The lesson I carried forward: power distribution is part of the system design. I now test the robot with its motors running, not only with its logic circuits powered.
01 / SYSTEM
A cloud conversation.
A local control loop.
Voice travels over Wi-Fi to the Xiaozhi service. Device tools translate requests into actions. Gesture recognition runs on the SEN0626 module, with the ESP32 checking results before enabling motion.
- INMP441 microphone + MAX98357 amplifier
- SSD1306 OLED · 128 × 32 pixels
- SEN0626 vision module over I²C
GESTURE INPUT→VALIDATE + FILTER→CHECK MODE→BOUNDED MOTION
Implemented control settings: ~150 ms polling, 3 consecutive frames, confidence ≥ 65, gesture 3 allowlist, and a 3-second motor run. Recognition accuracy and gesture mapping still require systematic validation.
02 / ELECTRONICS
From schematic to circuit board.
Separate controller and interface boards bring the processor, power rails, audio, and motion connections into a compact assembly. Select any image to inspect the design.
Board names follow the supplied schematic and layout assets. PCB renderings are design views; assembly photos appear in the build log.
03 / MECHANICAL
Making room for the real hardware.
The enclosure brings together a rotating head, two arms, wheels, and internal electronics. Multiple CAD views show how those parts fit into the robot's compact body.
03 / BUILD LOG
Not a straight line.
A series of iterations.
Physical evidence from the bench.
Design, assemble, debug, repeat.
BUILD / 01
Bring the electronics together.
Custom boards organize the controller and peripheral connections. Assembly makes routing and connector access tangible.
BUILD / 02
Give the system a body.
Physical enclosure prototypes connect the CAD design to the realities of mounting, wiring, and servicing the hardware.
BUILD / 03
Test the whole system.
Audio, vision, display, and motion come together on the workbench. Integration exposes problems isolated tests cannot.
More from the workbench +
04 / ENGINEERING IN PRACTICE
The most useful output?
Sometimes, a serial log.
A prompt sound did not mean the microphone worked. An I²C address did not mean the camera was recognizing gestures. Each signal needed its own test.
Separate playback from capture.
A fixed-tone test isolated the speaker path. Microphone sample logs then helped investigate capture, channel selection, and the shared I²S clock configuration.
Check power before the protocol.
Camera communication timed out when its external 5 V supply was disconnected. Restoring power made the module detectable at I²C address 0x72.
Make motion deliberate.
Gesture mode, repeated-frame confirmation, an allowlist, and timed stopping help prevent a single noisy result from becoming an unintended motor command.
05 / WHAT COMES NEXT
A working prototype.
An unfinished question.
The next step is not adding more features. It is measuring how reliably the existing ones work.
Built on shared tools. Learned through integration.
This project uses the open-source Xiaozhi voice firmware, Espressif ESP-IDF, and a DFRobot SEN0626 vision module. The portfolio documents the robot's hardware, integration, and debugging; it does not claim authorship of those underlying platforms.