Students have created an emotionally intelligent home robot by combining, a robotic arm and Google’s chatbot technology with a domestic lamp
Imagine a world where smart home assistants like Google Assistant, Alexa, or Siri were more than a voice? How about if a home robot gave eye contact or had body language?
Internet of things and electronic engineering students at Waterford Institute of Technology (WIT) have been busy in the institute’s Applied Robotics Lab working on the emerging area of social robotics. Students can study internet of things as part of the Applied Computing common entry honours degree at WIT, while electronic engineering is available at levels 6, 7 and 8 as part of the common entry Engineering honours degree.
Software that works out how people feel
Together they have created a ‘Lampbot’, a robotic friend, which uses software to work out how people feel and Google’s latest chatbot technology to assist where it can.
What’s different about this student project is that students have created an assistant that is socially acceptable in a home environment, by combining the physical build of a robotic arm and Google’s chatbot technology with a domestic lamp.
Read emotions and expressions
It not only has all the answers but can also read human emotions and expressions and then respond accordingly with its own body language using its movement, eyes and head dress.
Jason Berry who runs the lab at WIT explains that technology like Google Assistant, Alexa and Siri mean we now consider having conversations with artificially intelligent bots over the internet as normal.
Robots with emotions
“But what if we wanted to bring more than a robotic voice into our home...what if we want some movement, maybe even some emotions from our robot?” asks Berry.
Students working in the WIT Applied Robotics Lab combined their knowledge in electronics and software, with their imaginations to develop the ‘Lampbot’.
“One of the most powerful ways for humans to communicate with each other is with eye contact. The students leveraged this human trait in the design of the robot, helping the Lampbot to make more of an emotional connection with its humans,” Berry says.
More suitable for a home
Combining robotic arm technology with a home lamp, WIT students created the “world’s first 6-axis Robotic Lamp Assistant”. Combining the latest in linear actuators and vision systems with Google’s speech recognition and chatbot technology WIT students have created a robotic home assistant “with the moves to back up the talk”.
“Traditionally when people think of robots the Terminator and the arms that build our cars come to mind. We are now in a time where robots are leaving the factories and labs to make their way into our homes. However none of these are suitable for a home environment and this is the design challenge facing engineers, scientists and technologists. We are delighted that students have been able to develop a solution over a few months. They have bright futures ahead.”
The students were studying the following courses: BSc (Hons) Applied Computing (The Internet of Things), BEng in Electronic Engineering and the Higher Cert in Electronic Engineering.
Some of the students involved share their experience:
Ashraf Mustafa, a French student in second year of the BSc (Hons) in Applied Computing (Internet of things) WIT had the role of emotion recognition which gave the Lampbots an extra ability to interact with people according to their facial expressions by sending the data from the emotion recogntion system to the Lampbots' eyes.
“I really enjoyed my participation in this project, starting from setting up the software and the challenges that I have passed through successfully, to transferring the data of the face recognizer system to the Lampbot,” Ashraf says.
Group photo: Back row, fifth from left
Third year Bachelor of Engineering in Electronic Engineering student, James Kinsella from Campile, New Ross, Co. Wexford and a former student of the Good Counsel College New Ross held the role of head movement and eye movement
What he enjoyed the most he says was showcasing the work: “The best part of the project was showing the Lampbots in operation to people not involved in the project, but from a study point of view it would be how the code controls the movement of the parts of the lampbots.”
Group photo: Front row, third from left (middle)
Emily Lukuta is a 19-year-old BSc (Hons) in Applied Computing (Internet of things) student from Waterford City and past pupil of Our Lady of Mercy Waterford Secondary School. Her role included telling the life of Lampbots through an oscilloscope with sound.
Emily says she really enjoyed the team’s weekly sprints.” We had weekly sprints set out for ourselves to achieve a piece of the project within a given period of time. On the day that we all showcased our sprints it made me realize how powerful our minds are when we set ourselves to achieve something with diligence.”
Group photo: Back row, second from left
BEng Electronic Engineering Year 3 student Alan Marshall from Waterford City a past pupil of Mount Sion CBS had the role developing the AI Assistant capability and speech visuals.
“I enjoyed working with the other members of the team and learning about the Google Assistant framework.”
Group photo: Back row, eighth from left (far right)
Michael Vereker from Waterford City and De La Salle Waterford past pupil, had the role of Lamp movement (actuators, motor controllers)
“I was part of a two person team that was involved in the lamps movement. Actuator motors were attached to the lamp and wired back to motor controllers. Software was written to control the lamps movement using data sent from Vision Software. I enjoyed seeing the final project come together, everybody had their own part to complete (movement, vision, eyes, speech etc) and when it was all connected at the end the lamp really came to life,” Michael says.
Group photo: Front row, second from left.
Robert Solomon, BSc (Hons) in Applied Computing (Internet of things) student and former student of St. Paul's Community College, Waterford had the role of applying sound detection tracking to the robot.
“This means that the robot would be able to detect where a sound source was coming from and be able to face the direction of where it came from, giving it a more human-to-robot feel,” he explains.
“Although I didn't get to fully complete it, I really enjoyed having to take that responsibility of using this concept for the project but most of all I enjoyed the challenges and problems that were faced. I also enjoyed working with the different people involved in the project and discussing ideas together as a team. It was really worth the bumpy ride but it was an enjoyable journey.”
Group photo: Back row, fourth from left.
Brian Prendergast, Mullinavat, Co Kilkenny, has just completed the Bachelor of Engineering, in Electronic Engineering and intends to continue onto the follow on course BSc (Hons) in Applied Electronics.
He was also involved in designing present and future display units which could be used to enhance the facial expressions on the lampbot going forward, and also the designing of the head dress attached to the lampbot.
His role began with the overall design of a microcontroller PCB, this was built and used to enable communication within a CAN network to enable the lampbot to function as required.
“The main aspects of enjoyment for me included the design from start to finish, and also working individually and as part of the team, as every section of the project was discussed and scrutinised as to the function and probility between my colleagues and the head lecturer before the final idea could be introduced to the project. The initial PCB implementation included the design and construction of the hardware, this design was then used to communicate with the lampbot by using programming languages that instructed the various sections of the lampbot to perform. As a bonus this project was directed towards school going children and I felt that the extra movement attached to the lampbot, would entice more interest to the project. As a standalone project the future is bright, as many applications can be added to enhance the project further which could be beneficial to the general public and children with sensory processing disorders etc.
In relation to the sensory processing disorders, this is part of a wider spectrum of autism which was mentioned between my colleagues and myself, basically my view was to include beneficial stimulation with this project to cater for every child no matter their role in society. There is no personal connection with this topic, but inclusion of the masses is always the best approach to take.
Brian previously worked as a sales rep and was prompted following the demise of the Celtic Tiger to follow a new career which began with various courses been completed which included Comptia A+, ECDL, and Computer Maintenance.
“The direction in engineering came naturally as I always had a keen interest in electronics, from the time I self-built my own home all the appliances and services within the home, are maintained by myself.”
Group photo: Back row, second from left.
Related Courses
- Bachelor of Engineering in Electronic Engineering
- Higher Certificate in Engineering in Electronic Engineering
- Bachelor of Engineering (Honours) in Engineering (Common Entry)
- Bachelor of Science (Honours) in Computer Science (Internet of Things)
- Bachelor of Science (Honours) in Computer Science (Common Entry)
_380_250_c1.jpg)
SETU Waterford, Cork Road, Waterford City, 
