Datality Lab develops an AI presentation training system to analyze performance automatically with AWS technology
By SingTao on July 21, 2021. Read the story here
In today’s society, excellent communication and presentation skills are very important in the workplace, school or other occasions, and are also one of the essential factors for successful people. To master or improve the relevant skills, many people take part in group or 1-on-1 training sessions. Datality Lab, a Hong Kong start-up, recently launched Moodie.ai, a speech analysis and scoring system using artificial intelligence (AI) and machine learning (ML) technologies, with the aim of providing systematic speech training for the business and education sectors.
A contagious or persuasive speech is not only attractive in content, but also the speaker’s voice, facial expressions, and body movements. Users record personal speech clips through Moodie.ai, and the system uses Amazon Web Services (AWS) multiple AI and ML products to automatically analyze users’ speech performance in the above areas, and provide detailed scoring reports and suggestions.
Roland Leung, founder and managing director of Datality Lab, said that in the early stage of system development, the focus was on analyzing facial expressions. After collecting market opinions, body movements, voice patterns and content were added to the system, and a decision-making scientific computing model was used through AI and ML. The technology provides comprehensive, automated, rapid and objective analysis, and the development cost exceeds one million Hong Kong dollars.
Analysis from 70 observation points
The process of building AI and ML models is complicated. He continued that if you build it yourself, you need to invest a lot of resources, time and manpower. AWS has related products for immediate use, so he can quickly start the development of the Moodie.ai system.
Assisted by Amazon Rekognition, Amazon Transcribe, and Amazon Comprehend AI solutions, the speech training system learns from up to 70 observation points, including facial expressions, movements of the head, arms, forearms, palms, fingers, feet, etc. Conducting 170 algorithms to assess speakers’ performance and provide comments. The system can analyze a 5-minute speech clip and provide a comprehensive rating report to the user within 20 minutes.
He pointed out that when the Moodie.ai system is used in the education sector, hundreds of students may conduct thousands of speech exercises on the same day, and the system needs to increase the processing power to cope with the sudden traffic. Moodie.ai utilizes AWS computing services such as AWS Lambda and AWS Auto Scaling to automatically expand or reduce the capacity required for analyzing speeches, monitor service traffic, and automatically expand computing power before traffic changes. The system can now increase traffic to handle 300 online users logging in to the platform at the same time and uploading speech clips within 120 seconds for analysis.
The system also uses AWS Cloudfront to present analysis reports and teaching videos of a large number of speech clips on the user interface with low latency and high-speed transmission.
Moodie.ai can simulate the scoring ability of real people, and give comments and improvement suggestions according to the speaker’s self-confidence, fluency, credibility, energy level, etc. Conducting questionnaire surveys with different instructors to find out what criteria they use and their respective importance ratios to evaluate speaker performance.
The system currently provides 4 preset practice scenarios, including interviews, media, public speech and board room. Datality Lab can tailor-made exclusive scenarios and evaluation criteria for customers, and can complete the new scenario design within 1 day.
Simulate live scoring
Datality Lab cooperated with City University of Hong Kong Business School to test Moodie.ai from February to April this year. 50 participating students conducted several speech exercises and recorded more than 350 practice videos. After repeated practice, the students’ overall presentation skills have been improved. The system was officially launched in April and has been adopted by local universities and large insurance companies to train students’ presentation skills and agents’ sales skills respectively.
When users use the system for the first time, they need to capture their face, and they also need to perform facial recognition every time they log in to ensure that the scores obtained in the practices come from the same person. After the practice is completed, the video can be replayed and then uploaded, and the system will analyze it in the AWS cloud.
Using Moodie.ai, you only need a computer and a camera on the hardware. The fees charged by Datality Lab are divided into three parts, namely the monthly fee calculated by the number of users, the platform customization fee (customization) and the management fee.
The platform is currently only suitable for English practice, and Cantonese and Mandarin will be added before the end of this year. In addition, Android and iOS mobile app versions will be provided, as well as a backend management system. At that time, customers can design practice scenes and exclusive score calibration in the backend management system, thereby shortening the time required for new scene design by half a day.