Robotics & Mechatronics

Roxanne Robot: Guitar Hero

1st Quarter 2010 Robotics & Mechatronics

Engineering student Pete Nikrin co-developed guitar playing robot Roxanne incorporating a Banner Engineering vision system to read notes from a monitor.

Banner Engineering partnered with Nikrin, an engineering student at Minnesota West Community and Technical College, and robotics instructor Bill Manor to develop a robot designed to play the Guitar Hero video game, responding to each note as it appears on screen. Nikrin designed the robot to compete with a friend that Nikrin had introduced to the game and who, after playing for two weeks, had surpassed Nikrin in his guitar playing ability.

Bill Manor, robotics instructor at Minnesota West, suggested Nikrin incorporate a Banner Engineering PresencePLUS P4 OMNI vision sensor with a right-angle lens. Manor had such a vision system in his possession, as Minnesota West had purchased it at a discount through Banner as a start-up education kit.

To develop his Guitar Hero robot, Nikrin used a mannequin and installed the camera lens at the robot’s left eye. The robot identified the notes to be played by using an Edge vision tool, which detects, counts and locates the transition between bright and dark pixels in an image area.

“We set-up five Edge tools that ran horizontally across the screen, one for every fret, and positioned the tools to focus on the notes at the bottom of each,” Nikrin said. “The Edge tools sent a constant signal as the five vertical fret lines progressed, and when a bright white dot appeared in the middle of a dark coloured circle, the Edge tool allowed the sensor to detect it.”

Jeff Curtis, senior applications engineer at Banner, worked with Nikrin and Manor to ensure the robot’s processing time was fast enough to keep up with the video game. A PLC was programmed so that it constantly looked at the vision sensor’s register. Once the Edge tool senses a note, the PLC notices the change in the register, and the logic in the PLC fires a solenoid that activates the robot’s finger, pressing down on the appropriate note on the guitar. This set-up resulted in 9 ms processing response.

To ensure consistent, accurate operation, the team needed to ensure Roxanne could play within a range of lighting conditions – as she would be relocated from classrooms to gymnasiums for demonstrations – as well as confirm the robot was correctly oriented with the monitor displaying the video game.

“We honed a Locate tool and gave it a fixed point – a piece of reflective tape on the PC monitor – to focus on,” Curtis said. “This ensures the Edge tools are in the proper location to detect each note as it comes along and allows for any slight vibration in the application environment that could result in some deviation. If the robot starts to sag a bit, for example, it can still play.”

Using this technique, Roxanne has, on Medium mode, hit 100% accuracy at times, and it averaged 98% accuracy.

For more information contact Rodney Topham, RET Automation, +27 (0)11 453 2468,,


Share this article:
Share via emailShare via LinkedInPrint this page

Further reading:

AGVs for automated production
First Quarter 2020, SICK Automation Southern Africa , Robotics & Mechatronics
Automated and flexible production processes are the answer to increasing quantities, smaller batch sizes, and high production speeds. Automated guided vehicle (AGVs) systems and their smaller relatives, ...

Moving into 2020 with Festo digitalised products
First Quarter 2020, Festo , Robotics & Mechatronics
Industry 4.0 is rapidly automating the modern working world and helping the machine and system building sector reach new heights.

Factory harmony
First Quarter 2020, Omron Electronics , Robotics & Mechatronics
A promising model for the new harmony on the factory floor is based on intelligent, integrated and interactive design of tomorrow’s manufacturing processes.

Robot with air-water actuators has fluid motion
Fourth Quarter 2019 , Editor's Choice, Robotics & Mechatronics
Hydraulics and pneumatics are widely used for power transmission: hydraulics for moving heavy loads with highly controlled motion, and pneumatics for lighter loads and rapid, repetitive motions. Many ...

Underwater robot with a unique fin
Third Quarter 2019, Festo , Editor's Choice, Robotics & Mechatronics
      Swimming like the natural model The longitudinal fins of the polyclad and the cuttlefish extend from the head to the tail along their backs, their undersides or the two sides of their torsos. To ...

Manoeuvring agricultural robots with 2D laser scanners
Third Quarter 2019, SICK Automation Southern Africa , Robotics & Mechatronics
How can we harness modern technology in a way that will allow people to collaborate with business even more intelligently, efficiently and sustainably in the future? The solution is 2D laser scanners ...

The future of collaborative robots
Third Quarter 2019, Omron Electronics , Robotics & Mechatronics
Factories worldwide are dealing with the challenges of manufacturing a wide variety of low volume products to meet customer demands, as well as shorter product life cycles and labour shortage issues. ...

Industry 5.0 – the best of both worlds
2nd Quarter 2019, Cobots , Editor's Choice, Robotics & Mechatronics
The convergence of robot capabilities and human skills.

Pneumatic robotics meets artificial intelligence
2nd Quarter 2019, Festo , Editor's Choice, Robotics & Mechatronics
Festo’s pneumatic robot hand BionicSoftHand, combined with the BionicSoftArm, a pneumatic lightweight robot, shows that future concepts are suitable for human-robot collaboration.

V-lock system and co-operative robots
2nd Quarter 2019, Metal Work Pneumatic South Africa , Robotics & Mechatronics
Metal Work has developed applications for the automation of the assembly process together with producers of co-operative robots.