Engineering student Pete Nikrin co-developed guitar playing robot Roxanne incorporating a Banner Engineering vision system to read notes from a monitor.
Banner Engineering partnered with Nikrin, an engineering student at Minnesota West Community and Technical College, and robotics instructor Bill Manor to develop a robot designed to play the Guitar Hero video game, responding to each note as it appears on screen. Nikrin designed the robot to compete with a friend that Nikrin had introduced to the game and who, after playing for two weeks, had surpassed Nikrin in his guitar playing ability.
Bill Manor, robotics instructor at Minnesota West, suggested Nikrin incorporate a Banner Engineering PresencePLUS P4 OMNI vision sensor with a right-angle lens. Manor had such a vision system in his possession, as Minnesota West had purchased it at a discount through Banner as a start-up education kit.
To develop his Guitar Hero robot, Nikrin used a mannequin and installed the camera lens at the robot’s left eye. The robot identified the notes to be played by using an Edge vision tool, which detects, counts and locates the transition between bright and dark pixels in an image area.
“We set-up five Edge tools that ran horizontally across the screen, one for every fret, and positioned the tools to focus on the notes at the bottom of each,” Nikrin said. “The Edge tools sent a constant signal as the five vertical fret lines progressed, and when a bright white dot appeared in the middle of a dark coloured circle, the Edge tool allowed the sensor to detect it.”
Jeff Curtis, senior applications engineer at Banner, worked with Nikrin and Manor to ensure the robot’s processing time was fast enough to keep up with the video game. A PLC was programmed so that it constantly looked at the vision sensor’s register. Once the Edge tool senses a note, the PLC notices the change in the register, and the logic in the PLC fires a solenoid that activates the robot’s finger, pressing down on the appropriate note on the guitar. This set-up resulted in 9 ms processing response.
To ensure consistent, accurate operation, the team needed to ensure Roxanne could play within a range of lighting conditions – as she would be relocated from classrooms to gymnasiums for demonstrations – as well as confirm the robot was correctly oriented with the monitor displaying the video game.
“We honed a Locate tool and gave it a fixed point – a piece of reflective tape on the PC monitor – to focus on,” Curtis said. “This ensures the Edge tools are in the proper location to detect each note as it comes along and allows for any slight vibration in the application environment that could result in some deviation. If the robot starts to sag a bit, for example, it can still play.”
Using this technique, Roxanne has, on Medium mode, hit 100% accuracy at times, and it averaged 98% accuracy.
For more information contact Rodney Topham, RET Automation, +27 (0)11 453 2468, [email protected], www.retautomation.com
| Tel: | +27 11 453 2468 |
| Fax: | +27 11 453 2468 |
| Email: | [email protected] |
| www: | www.turckbanner.co.za |
| Articles: | More information and articles about Turck Banner Southern Africa |
© Technews Publishing (Pty) Ltd | All Rights Reserved
printer friendly version