exported project

Published on 28th Jun 2022 – Updated on 15th Aug 2022 RoboMind: NAO as a Scheduling Tool for the Elderly to Remind Them about Daily Activities

Photo by Omid Veisi

In the context of the University of Siegen's seminar, "Robotics in Real-World Applications for Social Work," the assignment involved developing a program for the NAO robot that would be attractive to elderly individuals, whether living alone or with others in their private residences. The introductory section of this paper, titled "Robomind: NAO as a Scheduling Tool for Elders to Remind Them about Daily Activities," highlights the importance of the subject matter. The introduction begins by outlining the motivation, which addresses the core issue, and then proceeds to describe the goals and overall structure of the work. By leveraging the advancements in robotics, specifically in the context of the NAO robot, this project aims to address the challenges posed by demographic change. The development of an effective scheduling system for the NAO robot has the potential to provide essential support for the elderly by reminding them of their daily tasks and routines. In turn, this can contribute to improved physical and mental health, greater independence, and reduced caregiver burden, ultimately enhancing the quality of life for the aging population.

Introduction/Motivation Concerns regarding the health and well-being of the elderly have been raised by the world's rapidly aging population, particularly in the context of their capacity to do daily duties and routines independently. Researchers and engineers are increasingly focusing on innovative technologies, such as robotics, to help and support this group in order to address this issue. One such solution is the NAO robot, which its humanoid appearance, interactivity, programmability, and research precedent, make it a suitable and effective tool to help the elderly. This report presents a project that explores the potential of the NAO robot in improving the daily lives of older adults and the efficacy of its scheduling system.
A key aspect of ensuring the elderly's well-being is assisting them in managing their daily routines, including medication intake, exercise, and appointments. Mismanagement of such tasks can lead to negative consequences for their physical and mental health [5]. This is where the NAO robot's scheduling system comes into play. By providing timely reminders and a visual representation of the user's daily activities, the robot can help older adults maintain their independence and alleviate the burden on their caregivers [6].
While there have been successful pilot projects and case studies involving NAO robots in elderly care settings, there remains a need for a comprehensive examination of the robot's potential as a scheduling tool for private home use. This project aims to fill this gap by systematically evaluating the effectiveness and usability of the NAO robot's scheduling system as a calendar for the elderly, addressing the research question: "How can a NAO robot scheduler aid the elderly in managing daily activities?" The motivation for this project is rooted in the ongoing demographic shift towards an aging population, which poses significant challenges for both individuals and society as a whole. As the number of elderly people continues to rise, it is becoming increasingly important to find innovative solutions that enable them to maintain their independence and quality of life. This is particularly crucial in the context of daily task management, as the ability to successfully manage routines and appointments can significantly impact the well-being of older adults.

Problem Definition Getting older is a natural part of life, and nowadays we see an increasing number of senior individuals in nursing homes and their own homes who require assistance and companionship to improve their everyday lives. According to the United Nations' World Population Ageing 2019 report, the global population aged 65 and over was projected to more than double, increasing from 703 million in 2019 to 1.5 billion by 2050. Furthermore, the report highlights that the global share of the population aged 65 and over is expected to rise from 9.3% in 2019 to 16% in 2050 [10].
In Germany, a significant demographic change that has been currently taking place over the past few decades is the country's aging trend as well. The population of the country is aging, mostly as a result of a combination of low birth rates and rising life expectancy. By 2035, there will be a significant increase in the number of Germans who are at retirement age. The Federal Statistics Office (Destatis) estimates that the number of people aged 67 or older would increase by 22% between 2020 and 2035, from 16 million to an expected 20 million, based on the findings of the first medium-term demographic projection [11]. Several aspects of society, including healthcare systems, labor markets, pension plans, and social services, are significantly impacted by the aging trend. Worldwide, nations need to manage and develop plans to address the issues associated with this aging population. Using technology, such as robotics and artificial intelligence, to promote elderly care and encourage seniors' independence and well-being is one possible option [12].
In this context the primary aim of this project is to investigate the potential of using the NAO robot to assist elderly individuals in managing their daily tasks and routines by offering a personalized scheduling system, consequently improving their quality of life and helping them maintain their independence. To effectively implement such a system, it is essential to be aware of the specific requirements and preferences of the senior person, the hardware and software limitations of the NAO robot, and any potential obstacles to its adoption in private settings. This project aims to address these concerns and provides a proper assessment of the viability and efficiency of employing the NAO robot as an elderly scheduling tool.

Semi-Structured Interview To further understand the needs of the target users, the team developed a set of semi-structured interview questions. These questions focused on users' experiences with existing scheduling and reminder systems, their preferences, and the challenges they face in managing their daily tasks. The main purpose of the semi-structured interview was to acquire information from different aspects such as their typical hobbies and activities on a daily basis; the level of their trust in technology and familiarity with technology, and smart devices. The pre-study interviews were carried out with potential users and gathered valuable user perspectives. One was with a caregiver, while the other two were with older adults. Each interview took around 20 to 25 minutes and was recorded after receiving the participants’ consent. Both of the seniors, a male and a female over the age of 65, had no signs of cognitive deterioration, but both frequently forgot their daily tasks throughout the day and required some reminders, such as writing notes down and placing them around the room or using smartphone reminders. However, because of physiological problems and aging, the male interviewee had varied degrees of movement difficulties.
The semi-structured format allowed for flexibility in the conversation, enabling the interviewees to provide detailed and insightful responses. Prior to conducting the interviews, we obtained written consent from the participants, ensuring that their involvement was voluntary and that their personal information would be handled confidentially. In addition, we used Miro boards for notes taking and data analysis.

Figure 1: Semi-Structured Interview Setup

Ideation After analyzing the interview data, we identified several key findings that informed the development of the NAO robot scheduling system. These insights guided the creation of a prototype, which involved finalizing the interface design and initiating the programming process. The prototype was designed to address the specific needs and preferences of elderly users, as identified in the interviews. Once the prototype was developed, the team implemented the scheduling system in the NAO robot and conducted a user test to evaluate its effectiveness. The testing phase allowed the team to gather feedback from the user, assess the system's performance, and identify areas for improvement in an iterative process. In conclusion, the empirical grounding and ideation process for the NAO robot scheduling tool involved a comprehensive, user-centered approach that incorporated collaborative meetings, brainstorming sessions, literature studies, user interviews, and prototype testing.

Fig211322 — Figure 2: Data extraction from Interview

Concept According to the data gathered from the interviews conducted, it was shown evidence that as people become older, their memory capacity continuously deteriorates, which either directly or indirectly affects their everyday activities and way of life. This is making it necessary for someone to remind the elderly of their normal routines and events. They could feel dependent on others as a result of this.
People typically develop a tendency as they age to feel less independent and more dependent on others to complete their tasks. It can be much simpler for elderly people to remember their events and appointments if they are aware of their daily responsibilities and other things they need to get done for the day. The NAO Robot includes ROBOMIND, which enables seniors to keep track of their activities and appointments, in order to solve this problem.
The main goal of the concept is to develop a method for the NAO robot to help elderly people remember their appointments and daily schedules. Personalized geriatric care that considers their way of life. The final design gives customers the option to choose a time slot and set a reminder for a particular event or appointment. Once the reminder is established, The NAO robot uses voice commands to notify them and reminds them again later. Reminders help users stay organized and on top of their daily obligations.

Design A few variations of the interface design were created and tested with users as part of the design process in order to better understand the benefits and drawbacks of the design. The insights from the pre-study interviews, along with a set of design requirement lists, were helpful in understanding the need for the interface design. The design requirements for the interface design were finalized based on the interviews that were done. The specifications call for a personalized scheduling system that emphasizes the user's independence. The prototype for the scheduling system's interface was created using Figma, and the development was carried out using HTML, CSS, Javascript, and JQuery.
The initial user interface was created and received feedback. The pre-study, which included the literature review, studies, and market study, was used to design the initial prototype. The layout of the interface had spaces for pre-filled ideas about the user's routines, however, this feature wasn't given a lot of significance during the evaluation process. The positioning of the calendar has drawn criticism for the interface. Since senior people make up the bulk of users that interact with the NAO robot, this is a crucial design element, the visibility of the calendar is one of the most important aspects of the design. Making the design more user-friendly and only displaying necessary functionality is crucial since users shouldn't be led misguided while engaging with the interface.

Frame100000088211322 — Figure 3: Initial Interface Design Variations

The comments from the evaluation stage were taken into consideration while the interface design was iterated. The calendar was more noticeable and accessible in the revised design. The "connect" button makes it simple to link the scheduler to the NOA Robot. In order to establish the reminder for the task in the scheduler, the user must enter the date, time, and any necessary description or text.
The left side of the screen, which shows the user's forthcoming reminders, stores the reminder once it has been set. The scheduled day is also highlighted in a dark blue tint on the calendar next to the date. The user can use this to check the dates for particular events and activities.

The elderly and their needs are taken into consideration while designing the final UI. which involves making ensuring the interface is usable, accessible, and meets their requirements. Here are some pointers for creating user interfaces for senior citizens.
Large fonts and clear text: Older adults find it difficult to read small fonts or text that is not clear. Larger fonts and clear text with high contrast were maintained to improve readability.
Simple and clear interface: The user interface was kept straightforward with basic navigation and a few sidebars. Large buttons and labels, and avoiding small icons or graphics that is be difficult to recognize were avoided.
Provide feedback: Clear and immediate feedback was provided to users when they perform an action, such as displaying the confirmation message or changing the color of the button when clicked.
Use familiar language: Simple and familiar language that is easy to understand, avoiding technical or complex jargon.
Design for touch: Designing for touch-based interfaces, as older adults may find it easier to interact with touchscreens that are easy to understand and accessible.
Allow for customization: Allow users to customize the interface to their preferences, such as adjusting the font size.
Overall, the design prioritizes simplicity, clarity, and accessibility to the elderly to navigate and interact with the product with ease.
Using the URL provided below, the prototype's interface design and the order of the screens. In order to provide a better understanding of the use case scenario and the user flow, the prototype shows two different sets of reminders.

Final11322 — Figure 4: Final User Interface Design

Following there is also a development flow provided to illustrate the process in detail.

roboticsprojectflowchartofRoboMind11322 — Figure 5: Development Flowchart

Following the development phase, we proceeded to evaluate the interface with users. Utilizing the prototype, the participant (Female 1 in Table 1) engaged with the features and offered insightful feedback. To ensure the efficient functionality of RoboMind and the seamless interaction between NAO and the user, we employed the Wizard of Oz method. The final scenario was composed of the following sequences.
Introduction: The robot NAO presents itself as an assistant helping in daily scheduling;
Asking questions: The robot asks some questions about the desire of people whether they want to know their plans and tasks.
Reminder: To help participants remember better, the robot gives a short reminder and asks the participants if they have done it.
Conclusion: The robot NAO concludes the presentation by wishing the participants a pleasant day.

Functionality and Deployment The NAO robot is equipped with speech recognition technology, allowing users to interact with the calendar using voice commands. Therefore the user can ask the robot to schedule a meeting on a specific date or ask for a reminder about an upcoming event with the use of RoboMind. The calendar can be integrated with other devices, such as smartphones or computers, allowing the user to manage their schedule across different platforms. The NAO robot can access the calendar from any connected device, making it easy to manage tasks and appointments from anywhere. Here are some possible technologies and features of the RoboMind that can be connected with the NAO robot and send a task at a specific time and date. Summary of Robomind App, main outcomes, and functionalities settings:
RoboMind has a special setting for controlling the NAO robot and connecting with it. This setting has different features including setting the IP address, Connecting or disconnecting the RoboMind app with NAO, turning off, sleeping, restarting, Robot volume, Speech speed, Speech pitch, and Selecting package language.
Reminder feature: The RoboMind app has a reminder feature that allows users to customize their notifications based on their preferences. Users can choose to receive reminders or not.
Security: RoboMind has dedicated security parts that provide information about their approach to ensuring the security of user data. RobiMind also has measures in place to prevent unauthorized access to user accounts and to detect and prevent fraudulent activities such as using a special internal Internet Protocol address for connecting with the NAO robot. Overall, RoboMind emphasizes the importance of security and takes it seriously to protect its users' information and maintain their trust.
Dashboard: this page is the core of the application and is responsible for setting a date, setting a time, and writing a special task. The idea is derived from the currently existing and provided for the team. In addition, on the calendar page, the user can delete a task or set a reminder in this part. There is no limitation for users in the number of tasks to be set. But we create a limitation of 100 words in any task, due to NAO clearly talking with the user.
In the initial stages of developing the RoboMind app, it became apparent that significant changes were needed in the user interface to incorporate emerging technologies and enhance the user experience. Various limitations encountered during the development process necessitated substantial alterations to the original version of the interface. For instance, during the final version's connection to the robot, the program's connection was disrupted due to a private IP when creating a pop-up, so we redesigned the page. However, due to time constraints, we were unable to employ other emerging technologies that could have improved the tool's efficiency. In the following, we mentioned some of these features and new technologies in order to increase the productivity of the tool.
API Integration: The RoboMind app can be integrated with the NAO robot's application programming interface (API) to allow for communication between the two devices. This enables the robot to access the calendar data and perform actions based on the scheduled tasks.
Cloud Storage: The RoboMind app can be stored in the cloud to ensure that the data is accessible from anywhere and can be synced across multiple devices. This allows the NAO robot to access the RoboMind data from any device with internet connectivity. Although the security issues should be considered.
Natural Language Processing (NLP): The RoboMind app can be equipped with NLP technology, allowing the NAO robot to understand natural language commands and interact with the user using human-like language. This enables the user to speak to the robot and schedule tasks using voice commands. Although with the use of Choregraphe software, we prepared conversation dialogue flows for the interaction between inputs and outputs, in fact, it simulates the conversation.

Fig11319 — Figure 6: RoboMind App Functionality

Task Prioritization: The RoboMind can be developed in a way that it prioritizes tasks based on their importance and urgency, ensuring that the user stays on top of their schedule. The robot can also notify the user of any scheduling conflicts or missed tasks to avoid delays.
Mobile App Integration: The RoboMind app can be integrated with a mobile app that allows the user to manage their schedule on the go. The NAO robot can access calendar data through the app, enabling the user to schedule tasks and receive reminders from anywhere.
Multi-Language Support: The NAO robot supports multiple languages, making it accessible to users from different regions and cultures. The calendar can also display dates and times in the user's preferred format, making it easier to manage their schedule.
During the development phase of RoboMind, we utilized HTML and CSS to code the user interface, creating two distinct pages that incorporated a variety of components, including the settings page and the main dashboard. Following this, we utilized JavaScript to code the back-end section. The image below displays the development of various permissions on the dashboard page, which we will further explore and discuss in the subsequent sections.
To begin developing the RoboMind app, the team first worked on the HTML user interface, after that moved on to the JavaScript backend. Various variables were defined to aid the app's functionality, and the team worked on establishing a socket connection between the app and the NAO Robot. The programming also included setting up error messages for specific actions such as connecting and disconnecting from the robot. A significant part of the app's programming was to ensure that tasks were checked based on time and date and sent to the robot for execution. To manage tasks, a JSON file was created, and the team ensured that the file was not empty and that the user had defined a task before sending it to the robot.