Let’s Build an Artbot
Could a team of undergraduates program simple robots to use a more painterly stroke? ECE Professor of the Practice Tyler Bletsch wanted to find out
Last summer, Professor of the Practice Tyler Bletsch was fiddling with a CNC router, curious to know what effects he could achieve by swapping its carving component with a pen. The earliest results looked like something an old dot matrix printer would spit out, but they gave Bletsch an idea.
“Who wants to make an artbot?” he posted to Muser, a site for Duke undergraduate students seeking interesting research opportunities.
Four engineering students—one of them a volunteer who participated for fun instead of class credit—and a computer science/linguistics major raised their hands for the project, which Bletsch characterized as fairly open-ended.
“We just wanted the robot to paint with a more human-like stroke,” he said.
It’s a challenge that’s more complicated than it sounds.
“At first we were just researching. Why start (writing code) from scratch if there’s something out there already that you could use?” asked Evan Kenyon ECE/CS’23. The most serviceable material he found was an old algorithm that could transform a digital image into a set of computer mouse strokes for use in Microsoft Paint. The mouse strokes mapped over to the artbot’s pen tool fairly well, but when the team gained enough facility to graduate from pen to paintbrush, its thicker strokes posed all kinds of problems. Sharp details disappeared, for instance, as the artbot repeatedly painted over its own initial lines and saturated the paper with paint.
To solve that problem, the team culled a large amount of material from the order of operations so that the brush made fewer strokes. But then there was also the issue of changing colors. Jonathan Sandoval ECE/CS’22, who said he was drawn to the project because it bridged artistic expression and engineering, created a map matching the original pixel colors to the colors in their tray of paint and figured out how to rinse the brush with each change of color.
“It uses what’s essentially an odometer to tell it when to stop, rinse in the 3D-printed basin, dip into the next color, and move back into position,” explained Bletsch.
Mechanical engineering student Allison Taub ME’24 used the others’ re-architected algorithm as a starting point for a second artbot—a “stringbot” programmed to draw on a whiteboard with a marker, using a system of motor-controlled pulleys and weights, to keep constant tension in the belts and ensure the pen maintains contact with the board’s surface. Her robot’s pulleys slacken and tension at different proportional rates, making her motion problem much trickier. And the stringbot can’t change colors by itself—she halts the program, changes the marker, and then commands the bot to resume drawing. “I sat here until 2 a.m. one night, waiting for it to finish drawing a dog,” she said ruefully.
All of these early hiccups offer the team entry points into new projects and tools for improvement; Miriam Shams-Rainey CS/Linguistics’23, for example, developed a visualizer tool to predict whether the team’s code would work as intended, saving them from additional all-nighters, if not speeding up the process.
Fortunately, Bletsch wasn’t concerned with speed anyway.
“I was looking for motions similar to what a human artist might use, and the algorithm that shook out was a combination of pointillism and guiding strokes—it was a more organic style than what we started with. I could imagine an artist using a similar style,” said Bletsch.
All four of the students who took the course for credit agreed that gaining competency in GCode, which is commonly used to generate instructions for 3D printers, was a very valuable skill to learn over the course of the semester. Sandoval added that the experience of managing the handoff among team members will be useful to him in the future if he heads to a development job. And Shams-Rainey got her first taste of the engineering design process. “In computer science you write a lot of code and learn to design software, but I haven’t gotten a lot of experience with how that interacts with circuitry and mechanics,” she said. “It was cool to see how the things I’m studying interface with engineering.”
Bletsch also serves as advisor to the Duke Combat Robotics club. Check out these highlights from the 2021 tournament.