Elsevier Acta Medica Okayama 0167-6393 126 2021 Model architectures to extrapolate emotional expressions in DNN-based text-to-speech 35 43 EN Katsuki Inoue Graduate school of Interdisciplinary Science and Engineering in Health Systems, Okayama University Sunao Hara Graduate school of Interdisciplinary Science and Engineering in Health Systems, Okayama University Masanobu Abe Graduate school of Interdisciplinary Science and Engineering in Health Systems, Okayama University Nobukatsu Hojo NTT Corporation Yusuke Ijima NTT Corporation This paper proposes architectures that facilitate the extrapolation of emotional expressions in deep neural network (DNN)-based text-to-speech (TTS). In this study, the meaning of “extrapolate emotional expressions” is to borrow emotional expressions from others, and the collection of emotional speech uttered by target speakers is unnecessary. Although a DNN has potential power to construct DNN-based TTS with emotional expressions and some DNN-based TTS systems have demonstrated satisfactory performances in the expression of the diversity of human speech, it is necessary and troublesome to collect emotional speech uttered by target speakers. To solve this issue, we propose architectures to separately train the speaker feature and the emotional feature and to synthesize speech with any combined quality of speakers and emotions. The architectures are parallel model (PM), serial model (SM), auxiliary input model (AIM), and hybrid models (PM&AIM and SM&AIM). These models are trained through emotional speech uttered by few speakers and neutral speech uttered by many speakers. Objective evaluations demonstrate that the performances in the open-emotion test provide insufficient information. They make a comparison with those in the closed-emotion test, but each speaker has their own manner of expressing emotion. However, subjective evaluation results indicate that the proposed models could convey emotional information to some extent. Notably, the PM can correctly convey sad and joyful emotions at a rate of >60%. No potential conflict of interest relevant to this article was reported.

岡山医学会 Acta Medica Okayama 0030-1558 132 2 2020 サイバーフィジカル情報応用研究コア（Cypher）設立について 92 94 EN Masanobu Abe Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University No potential conflict of interest relevant to this article was reported.

IEEE Acta Medica Okayama 2640-009X 2019 2019 Speech-like Emotional Sound Generator by WaveNet 143 147 EN Kento Matsumoto Okayama University Sunao Hara Okayama University Masanobu Abe Okayama University In this paper, we propose a new algorithm to generate Speech-like Emotional Sound (SES). Emotional information plays an important role in human communication, and speech is one of the most useful media to express emotions. Although, in general, speech conveys emotional information as well as linguistic information, we have undertaken the challenge to generate sounds that convey emotional information without linguistic information, which results in making conversations in human-machine interactions more natural in some situations by providing non-verbal emotional vocalizations. We call the generated sounds “speech-like”, because the sounds do not contain any linguistic information. For the purpose, we propose to employ WaveNet as a sound generator conditioned by only emotional IDs. The idea is quite different from WaveNet Vocoder that synthesizes speech using spectrum information as auxiliary features. The biggest advantage of the idea is to reduce the amount of emotional speech data for the training. The proposed algorithm consists of two steps. In the first step, WaveNet is trained to obtain phonetic features using a large speech database, and in the second step, WaveNet is re-trained using a small amount of emotional speech. Subjective listening evaluations showed that the SES could convey emotional information and was judged to sound like a human voice. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 2016 Sound collection systems using a crowdsourcing approach to construct sound map based on subjective evaluation EN Sunao Hara Graduate school of Natural Science and Technology, Okayama University Shota Kobayashi Graduate school of Natural Science and Technology, Okayama University Masanobu Abe Graduate school of Natural Science and Technology, Okayama University This paper presents a sound collection system that uses crowdsourcing to gather information for visualizing area characteristics. First, we developed a sound collection system to simultaneously collect physical sounds, their statistics, and subjective evaluations. We then conducted a sound collection experiment using the developed system on 14 participants. We collected 693,582 samples of equivalent Aweighted loudness levels and their locations, and 5,935 samples of sounds and their locations. The data also include subjective evaluations by the participants. In addition, we analyzed the changes in sound properties of some areas before and after the opening of a large-scale shopping mall in a city. Next, we implemented visualizations on the server system to attract users’ interests. Finally, we published the system, which can receive sounds from any Android smartphone user. The sound data were continuously collected and achieved a specified result. No potential conflict of interest relevant to this article was reported.

Okayama University Medical School Acta Medica Okayama 0386-300X 70 3 2016 Structure of a New Palatal Plate and the Artificial Tongue for Articulation Disorder in a Patient with Subtotal Glossectomy 205 211 EN Ken-ichi Kozaki Department of Dental Pharmacology, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Shigehisa Kawakami Department of Occlusal and Oral Functional Rehabilitation, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Takayuki Konishi Division of Physical Medicine and Rehabilitation, Okayama University Hospital Keiji Ohta Dental Laboratory Division, Okayama University Hospital Jitsuro Yano Department of Occlusal and Oral Functional Rehabilitation, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Tomoo Onoda Department of Otolaryngology-Head and Neck Surgery Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Hiroshi Matsumoto Department of Plastic and Reconstructive Surgery, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Nobuyoshi Mizukawa Department of Oral and Maxillofacial Reconstructive Surgery, Okayama University Hospital Yoshihiro Kimata Department of Plastic and Reconstructive Surgery, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Kazunori Nishizaki Department of Otolaryngology-Head and Neck Surgery Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Seiji Iida Department of Oral and Maxillofacial Reconstructive Surgery, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Akio Gofuku Graduate School of Natural Science and Technology, Okayama University Masanobu Abe Department of Computer Science, Okayama University Shogo Minagi Department of Occlusal and Oral Functional Rehabilitation, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences Okayama Dream Speech Project Case Report 10.18926/AMO/54420 A palatal augmentation prosthesis (PAP) is used to facilitate improvement in the speech and swallowing functions of patients with tongue resection or tongue movement disorders. However, a PAPʼs effect is limited in cases where articulation disorder is severe due to wide glossectomy and/or segmental mandibulectomy. In this paper, we describe speech outcomes of a patient with an articulation disorder following glossectomy and segmental mandibulectomy. We used a palatal plate (PP) based on a PAP, along with an artificial tongue (KAT). Speech improvement was evaluated by a standardized speech intelligibility test consisting of 100 syllables. The speech intelligibility score was significantly higher when the patient wore both the PP and KAT than when he wore neither (p＝0.013). The conversational intelligibility score was significantly improved with the PP and KAT than without PP and KAT (p＝0.024). These results suggest that speech function can be improved in patients with hard tissue defects with segmental mandibulectomy using both a PP and a KAT. The nature of the design of the PP and that of the KAT will allow these prostheses to address a wide range of tissue defects. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 2015 A Spoken Dialog System with Redundant Response to Prevent User Misunderstanding 223 226 EN Masaki Yamaoka Sunao Hara Masanobu Abe We propose a spoken dialog strategy for car navigation systems to facilitate safe driving. To drive safely, drivers need to concentrate on their driving; however, their concentration may be disrupted due to disagreement with their spoken dialog system. Therefore, we need to solve the problems of user misunderstandings as well as misunderstanding of spoken dialog systems. For this purpose, we introduced a driver workload level in spoken dialog management in order to prevent user misunderstandings. A key strategy of the dialog management is to make speech redundant if the driver’s workload is too high in assuming that the user probably misunderstand the system utterance under such a condition. An experiment was conducted to compare performances of the proposed method and a conventional method using a user simulator. The simulator is developed under the assumption of two types of drivers: an experienced driver model and a novice driver model. Experimental results showed that the proposed strategies achieved better performance than the conventional one for task completion time, task completion rate, and user’s positive speech rate. In particular, these performance differences are greater for novice users than for experienced users. No potential conflict of interest relevant to this article was reported.

IEEE Acta Medica Okayama 2015 Sound collection and visualization system enabled participatory and opportunistic sensing approaches 390 395 EN Sunao Hara Masanobu Abe Noboru Sonehara This paper presents a sound collection system to visualize environmental sounds that are collected using a crowd-sourcing approach. An analysis of physical features is generally used to analyze sound properties; however, human beings not only analyze but also emotionally connect to sounds. If we want to visualize the sounds according to the characteristics of the listener, we need to collect not only the raw sound, but also the subjective feelings associated with them. For this purpose, we developed a sound collection system using a crowdsourcing approach to collect physical sounds, their statistics, and subjective evaluations simultaneously. We then conducted a sound collection experiment using the developed system on ten participants.We collected 6,257 samples of equivalent loudness levels and their locations, and 516 samples of sounds and their locations. Subjective evaluations by the participants are also included in the data. Next, we tried to visualize the sound on a map. The loudness levels are visualized as a color map and the sounds are visualized as icons which indicate the sound type. Finally, we conducted a discrimination experiment on the sound to implement a function of automatic conversion from sounds to appropriate icons. The classifier is trained on the basis of the GMM-UBM (Gaussian Mixture Model and Universal Background Model) method. Experimental results show that the F-measure is 0.52 and the AUC is 0.79. No potential conflict of interest relevant to this article was reported.