An Information Delivery System on Disaster for Hearing-Impaired People Using LED Display – To Extend to Universal Use –

Yuko Hiramatsu1, Atsushi Ito2,*, Akira Sasaki3, Naoto Kawasaki2, Koichi Tsunoda4, Soutaro Sekimoto4 and Fumihiro Sato1

1Chuo University, 742-1 Higashinakano, Hachioji-shi, Tokyo, Japan

2Utsunomiya University, 7-1-2 Yoto, Utsunomiya-shi, Tochigi, Japan

3GClue Inc., Aizuwakamatsu, Fukushima, Japan

4Tokyo Medical Center, Meguro-ku, Tokyo, Japan

E-mail: susana_y@tamacc.chuo-u.ac.jp; at.ito@is.utsunomiya-u.ac.jp

* Corresponding Author

Received 11 November 2019; Accepted 03 December 2019; Publication 04 January 2020

Abstract

There is a sudden blackout on the big disaster. Under such circumstances, people get information verbally by voice, megaphone, and radio instead of looking at screen on PC smartphone, or TV by their eyes. After the big disasters, people have to find a safe place using voice announcements. However, hearing-impaired people cannot get information from voice announcements. Also, it is difficult to distinguish hearing-impaired people from others by their appearances. Therefore, they have little chance to be supported by others. We have researched and created a system for hearing-impaired people get information on the disaster using an LED display. Also, we found such a system is useful for elders and foreigners, too. People cannot use their smartphones as usual because of a shortage of their battery under disinters. Our system was designed to save battery using Bluetooth Low Energy (BLE) and small CPU. Besides, people will co-operate, looking at the same screen under the disaster situation.

Keywords: BLE, advertisement, ad hoc network, LED display, disaster information delivery system.

1 Introduction

Information Technology makes our daily lives convenient. The number of mobile Internet subscribers in Sub-Saharan Africa has quadrupled since the start of the decade. Around a third of mobile users, 250 million, have a smartphone. By 2025 it’s anticipated that 634 million people in the region will be mobile subscribers, up from 44 percent and 444 million people in 2017 [1]. Not only in Sub-Sahara Africa but also in many places in the world, the smartphone is the best personal tool to get information. However, the big disaster sudden breaks such circumstances in a moment. We have many disasters in Japan. 15,893 persons were dead, and 2,553 persons were missing by the big earthquake in 2011 [2]. A power outage is happened due to disaster.

In Africa, there are many disasters, too. Loss by earthquakes in Africa was 12,218,000,000 (estimation) in 1980–2003, according to the research by the World Bank [3]. African’s current urban population is increasing, and it is said to become 1 billion in 2040. Also, African cities have a few 40% of neighbours to interact with compared to Asia and Latin America [4]. Cities will be crowded and can be disconnected from disasters. We usually have information by television, smartphone, or Websites on PC. However, Television and PC do not work at all under such a situation. People may use a smartphone with a battery. However, too many people use smartphones, and traffic will happen. Thus, people become to use microphones or radios to inform something widely. Especially hearing-impaired people got little information and were postponed to be rescued. They sometimes didn’t get relief supplies.

Using a mobile – ad hoc network, we developed an information delivery system for hearing impaired people based on such reports of disasters in Japan. The system can operate without Internet access. People can use it with a battery for about 2 or 3 days. The rest of this paper is structured as follows. Section 2 discusses related works. Section 3 sets out our information delivery system. In Section 4, we propose our experiments. Finally, in Section 5, we summarize this research and discuss future studies.

2 Related Works

There are several research papers about the situation of hearing-impaired people under the blackout. “Giving and Receiving of Information by Hearing-Impaired Persons after the Earthquake Disaster: From an Interpersonal Communication Point of View” (Kawauchi, 2011) told “the Symposium of Earthquake Communication clearly show the need to examine the giving and receiving of information after a disaster. We need to reconsider the use of “Mieru-radio/Teletext-broadcasting [5]”. However, the service named “Mieru-radio” has already closed. Because visible contents on the web-sites using smartphones are increasing day by day, and many people inclosing hearing-impaired ones use them instead of special Teletext broadcasting [6]. Those smartphone applications are useful for normal lives. Communication traffics apt to occur in a disaster, and it would be hard to get information by smartphones. “Eye Dragon 4” [7] is a subtitles system of TV for hearing-impaired people. However, if electricity fails on the disaster, TV does not work at all.

The daily lives of people including hearing-impaired people have been more convenient. However, the situation for hearing-impaired people on disasters has been worse. We have studied the information system for 12 years [8–10] and have to continue our project under such circumstances.

3 Our Information Delivery System on Disaster

3.1 The Situation Under the Big Disaster

There are several phases under the disaster. When a disaster happens, people have to refuge from buildings or some safe place as soon as possible. After they get to the shelter, people stay there for a few hours, a few days or sometimes for several weeks. The information for an emergency to refuse is different from living at the shelter. However, there is a key person to send messages to people: manager of the building or emergency preparedness manager. The system we have developed based on such situations. The manager brings a smartphone with our application and sends messages to people using LED displays widely. It is important to send information by a particular person who has correct information not only in the emergency scene but also after the disaster. Because people are in panic under disaster, and fake news sometimes gets around among them. The information using a big LED display with shining letters send them safety information. The display is an especially useful tool for hearing-impaired people, and it caused good effects for hearing people, too.

images

Figure 1 Block diagram of disaster information delivery system using BLE advertisement.

3.2 Outline of Our System Using LED Display

The outline of our information delivery system is described in Figure 1. The LED displays are connected by ad hoc networks using the BLE advertising message. An operator input information (text) from the application on Android phone by BLE. The message is transferred to the next LED display by the ad hoc network. There is a limitation of packet size so that a long message is broken down into some advertisement messages. We also implemented a simple routing protocol that meets the limitation of the packet size of the BLE advertisement [11]. Table 1 shows the primary functions of the application to control the information delivery system. Users can change colours of texts, also can register some frequently used texts, and change displaying speed. Also, there are buzzers. Hearing-impaired people cannot hear sounds. However, someone near them who can listen to notices the information on an LED display. Such behaviours of other persons should be some signs to look at the LED display for hearing-impaired people, too.

Table 1 Functions to control the LED display

Function Outline
Input message Compose message
Send message Send a message to a LED display
Register message Register frequently used message (10 massages)
Set colour Red/Yellow/Green/Blue
Set display speed Min. 0.5 sec for 8 characters
Set the number of repetition Min 1
Stop display message Stop display message and

Figure 2 is a software structure of this system. The operating system is Linux, and we use WebSocket to transmit/receive messages between the operating system and the application. The IP address is v4. Node.js is used as the basis to execute applications to realize the portability of application since, on many operating systems, a web server is running. And functions of the LED display is written by Java Script.

Figures 3 and 4 are screens of our application to control the LED display. User (staff of the building or the chief of the floor) directly put in some messages freely (Figure 3) or can choose a message from history data (Figure 4). Users may use his/her message freely in daily life and can chick a massage quickly in the case of an emergency.

images

Figure 2 Software structure. (Where the color is dark displays the newly developed function for this systems.)

images

Figure 3 A screenshots of the application to enter a message to the LED display (Free text).

images

Figure 4 A screenshot of our application to send pre-fixed phrases.

4 Experiments and the Results

4.1 Outlines of the Experiments

After the tests of ad hoc connection, we had several experiments at different use scenes in order to research effective use scenes (see Table 2, n = 238).

The main target was hearing-impaired people. Also, families and people with hearing-impaired ones answered questionnaires since they are in the same scene as hearing-impaired people when a disaster happened. People help each other under the disaster. We asked them for the evaluation of our system and also asked to choose colour, speed, contents that they would like to use their ordinary lives. How they use this display in their daily life is essential for emergency use. People do not look at the LED display if they are not accustomed to looking at the display in their daily lives. Figure 5 is a photo of the LED display testing at the school of hearing-impaired children.

Table 2 Outline of our experiments using LED display

Phase Year Numbers Instrument Function Use Scene
1 2007∼2008 130 Feature phone Bluetooth Events (Inside and outside)
2 2009 15 Feature phone Bluetooth/Internet Office/House
3 2010 46 Feature phone Bluetooth Hospital (ENT)
4 2016 47 Smartphone Bluetooth School of hearing impaired

images

Figure 5 A sample of LED display (movable type).

4.2 Results of the Experiments

Our system evaluated high not only by hearing-impaired people but also by hearing ones. (See Figure 6). In addition, they chose as following.

4.3 The Result of Usage on the Office at the Welfare Facility for the Physically-handicapped in Tokyo

We had another experiment for daily life usage at the office. People must think about the usefulness of the display in their daily life. If the display is only used on the disaster, they cannot use it immediately in the panic situation – staff of the Welfare facility for the physically-handicapped in Tokyo (n = 12) co-operated us. There were hearing-impaired staff and 11 hearing staff. They used our LED display for one month at their office in 2009. The chief had a homing device and sometimes sent messages to the staff by the display. The LED display usually showed time as a clock for one month at the office. The results of the experiment were impressive. Staff evaluated the usage of the LED display once a week. They evaluated the display as 2.66 points (5 scales Likert Scale) in the first week. Then the evaluation of visibility became 3.50 points in the last week. The contents were the same. However, the staff got used to looking at the display and began to feel it easily. The staff evaluated the LED display as useful (77%). The chief answered that he used the homing device easily. The numbers of a user (n = 12) were not enough. However, the experiment for a month at the real office had great significance to consider the practical application of LED display.

images

Figure 6 Evaluation of the experiments in 2016.

4.4 A New Experiment in 2018 – For School of Hearing Impaired Children

According to the request of teachers at the school of hearing-impaired children [12], we had another experiment on October 2018. They urged us to sent messages speedy among 3 floors. After a test at Utsunomiya University, we had an experiment of multi-hop communication using 6 nodes. Terminals 1, 2, 3, 4 were located on the second floor, and Terminal 5 and 6 were located on the first floor to test the performance of reachability crossing floors. We measured packet loss among LED displays (Refer to Figure 7).

images

Figure 7 The node layout for the testing on the first floor and the second floor.

We designed 5 routes from 2 hops to 5 hops.

Route 1 (2 hops): Terminal 2 → Terminal 4 → Terminal 6

Route 2 (3 hops): Terminal 2 → Terminal 1 → Terminal 3 → Terminal 5

Route 3 (4 hops): Terminal 2 → Terminal 3 → Terminal 4 ⇒ Terminal 6 → Terminal 5

Route 4 (5 hops): Terminal 2 → Terminal 1 → Terminal 3 → Terminal 4 ⇒ Terminal 6 → Terminal 5

Route 5 (6 hops): Terminal 2 → Terminal 1 → Terminal 3 ⇒ Terminal 5 → Terminal 6 ⇒ Terminal 4

“⇒” means crossing floors

The results of the first experiment were as follows. The % shows the percentage of the reached packet that was sent from the source node (Terminal 2).

Route 1 (2 hops): 2 → 4(30%) → 6(28%)

Route 2 (3 hops): 2 → 1(37%) → 3(6%) → 5(6%)

Route 3 (4 hops): 2 → 3(0.09%) → 4(0.07%) ⇒ 6(0.07%) → 5(0.03%)

Route 4 (5 hops): 2 → 1(75%) → 3(3%) → 4(3%) ⇒ 6(3%) → 5(2%)

Route 5 (6 hops): 2 → 1(10%) → 3(1%) ⇒ 5(1%) → 6(0.8%) ⇒ 4(0.8%)

The results of this experiment told that communication performance from Terminal 1 to Terminal 3 was very low. In addition, the reachability was very unstable. This trial was performed at the school festival day so that there were many people in the corridor. Because of the effect of human body shielding and reflection by a lot of obstacles in the corridor, the measured value was changed frequently.

images

Figure 8 The result of the second test.

Thus we designed another experiment again. We had the next experiment again at a weekday by using the same route setting and measured the performance of the communication. The result is described in Figure 8. The data reception rate of each section was 50∼70%.

From these two results, we can think that the performance of the ad hoc network using BLE advertisement is acceptable, however, to meet the very crowded situation, we should consider the height and distance carefully.

5 Conclusion of Experiments and New Developments for Practical Use

5.1 Extension for the Universal Use

According to the experiments, our system was recognized that it was useful not only for hearing impaired people but also for hearing ones. For example, this system should be helpful as a signboard for outdoor events such as concerts and fireworks. For that purpose, we should consider the capacity of a battery. A waterproof function is also crucial if we use it outside. In 2020, the Olympic games will be held in Tokyo. Many foreigners will come to Japan, who do not understand the Japanese language. Using this system and some applications for translation, sponsors will show information easily by several languages. If it does not spread on people’s daily lives, it will not be used at the emergency scene. Therefore, we consider how many usages and how many people can use this system conveniently.

In addition, it is difficult to use a keyboard in urgent cases. Hence voice input should be required. Because of the progress of voice recognition technology and machine translation technology, we can use voice recognition now.

ICT provides people with convenience lives and diversity. Hearing-impaired people use the same device as hearing ones in daily life. However, once the big disaster occurs, they would not have any information. We will continue this project to bring words speedily and certainly.

It will be useful for elders, foreigners, and hearing people on disasters. Just after the big disaster, people are at the panic station. An otolaryngologist told us that elders who have hearing problems tend to hear less than usual after the disasters. Also, it was reported that foreign residents in Japan could not understand Japanese words for the emergency at the Great East Japan earthquake by Kahoku Ninpo, a newspaper at Tohoku area in Japan [13]. The words for emergency call are different from daily ones. If there were letters at the display, it would be helpful for them, too. According to our research, people hearing well also answered that this display would be useful for their safety lives. Everyone needs reliable information about his or her living area at a great disaster.

5.2 A New Role of Smartphone After the Disaster in Our System

At the end of this paper, we mention about smartphone use. According to the following consideration, we would like to extend our model [10] to describe the relation of disaster information and media, as illustrated in Figure 9. We added this figure to the smartphone part; it is the third eye and ear in ICT society. At the disaster in the acute stage, people do not have enough time to see their smartphones. But if smartphones can receive information that is transmitted between LED displays, the smartphones can store the information, and the users can review it at the shelter. The message by voice and display is gone immediately in the evacuation place. However, they can check the information on their own smartphone screens. There are small disasters or fires after the big one. Officers or some staff can send such messages. Fortunately, the recent smartphone has a function of BLE, so that it may be useful media to convey and store the disaster information. It is also possible to send information to smartphones through BLE as another path. We will continue researches to improve our system for information at disasters.

images

Figure 9 Classification of media for disaster information.

Acknowledgements

We express special thanks to valuable advice from Mr. Hatohara, the principal of the school of deaf of Miyagi prefecture, and Mr. Ogure and Mr. Kusunoki, the teachers of Special Needs Education School for hearing impaired persons in Miyagi prefecture. Also, we express special thanks to Dr. Yabe, Dr. Haraguchi and Dr. Henmi of the Disaster Medical Center, and Prof. Nakayama of National Rehabilitation Center for Persons with Disabilities and Prof. Ifukube at University of Tokyo for advice.

References

[1] Radcliff, D., ‘Mobile in Sub-Saharan Africa: Can world’s fastest-growing mobile region keep it up?’, ZDNet, from: https://www.zdnet.com/article/mobile-in-sub-saharan-africa-can-worlds-fastest-growing-mobile-region-keep-it-up/, October 16, 2018.

[2] The Mainichi, Over 120,000 evacuees remain 6 years after Great East Japan Earthquake, (Mainichi Japan), from: https://mainichi.jp/english/articles/20170311/p2a/00m/0na/013000c, March 11, 2017.

[3] The World Bank Blogs ‘Natural disaster’, from: https://blogs.worldbank.org/category/tags/natural-disasters.

[4] Yabe, A., Haraguchi, Y., Tomoyasu, Y., ‘Survey of individuals with auditory handicaps requiring support after the Great Hanshin-Awaji Earthquake’, Japanese Journal of Disaster Medicine Vol 4-1, pp. 75–81, 2009.

[5] Kawauchi, K., Giving and Receiving of Information by Hearing-Impaired Persons after the Earthquake Disaster: ‘From an Interpersonal Communication Point of View’, J. Aomori Univ. Health Welf. 12, pp. 11–19, 2011.

[6] Tokyo FM Information 2013/4/1, ‘The announce about service of Mieru radio is over’, from: https://www.tfm.co.jp/blog/info/index.php?itemid=63780, last accessed 2018/10/28/

[7] ASTEM HP, Eye Dragon 4 (only in Japanese), from: http://www.astem-co.co.jp/eye-dragon, last accessed 2018/10/28/

[8] Ito, A., Fujii, M., Yabe, T., ‘An information delivery and display system for Deaf people in times of disaster’, The 3rd IASTED International Conference, Montreal, 2007.

[9] Ito, A. et al., ‘Universal Use of Information Delivery and Display System Using Ad Hoc Network for Deaf People in Times of Disaster’, 3rd International Conference on Broadband Com, Gauteng, 2008.

[10] Hiramatsu, Y., Ito, A., ‘A Study on the information Delivery System at a Great Disaster in the Contemporary Society: From the view point of Supporting Technology for Hearing Disabled People’, The Annual of Institute of Economic Research Chuo University No. 50, pp. 575–604, 2018.

[11] Ito, A. and Hatano, H., ‘A Study on a Protocol for Ad Hoc Network Based on Bluetooth Low Energy’, Cognitive Infocom., Theory and Applications, pp. 433–458, 2018.

[12] Kahoku shinpo On Line, ‘Archive DAISHINSAI’, (only in Japanese), https://www.kahoku.co.jp/special/spe1168/20160217_02.html

[13] Special Needs Education School for Hearing Impaired Children in Miyagi, [in Japanese], https://miyaro-s.myswan.ed.jp, 17 on Feb, 2016.

Biographies

images

Yuko Hiramatsu received B.S. and M.S. degrees in Literature from Sophia University in 1979 and 1985, respectively. She has been a secretariat staff of “Study Group of School Trip” since 2006. She was an adviser for the utilization of ICT in Okinawa in the subsidy project of the Ministry of Internal Affairs and Communications in Japan from 2007 to 2009. She belongs to the Faculty of Economics, Chuo University as a lecturer from 2009. Her current research interest is sightseeing support technology and linguistic landscape for inbound visitors.

images

Atsushi Ito received B.S. and M.S. degrees from Nagoya University in 1981 and 1983, respectively. He also received a Ph.D. degree from Hiroshima City University in 2007. From 1983 to 2014, he was with Research and Development Laboratories of KDDI Corporation. He is now a Professor of the Graduate School of Engineering of Utsunomiya University from 2014. During 1991–1992, he was a visiting scholar at the Center for the Study of Language and Information (CSLI) of Stanford University. His current research interests include an open platform for mobile communications, ad hoc networking, user interface design, and ICT based agriculture.

images

Akira Sasaki received B.S. degrees in Computer Science and Engineering from the University of Aizu in 2001. He established GClue Inc. in 2001 to develop mobile phone applications. From 2001, he is the CEO of GClue Inc. From 2014, he is a part-time lecturer at Utsunomiya University. In 2015, he established FaBo Inc. as CEO to develop an open-source hardware framework. His research interests include open source technology, middleware of mobile phone platforms, and IoT devices.

images

Naoto Kawasaki received the B.S. degrees in Engineering (Computer Science) from the Utsunomiya University in 2019. He studied an ad hoc network using BLE. He is now working for the prefectural office of Miyagi.

images

Koichi Tsunoda MD, Ph.D., is a physician, otolaryngologist, and phono-surgery. Since 2003 Chairman of Department of Artificial Organs and Medical Creations, National Hospital Organization (NHO) Tokyo Medical Center.

images

Soutaro Sekimoto received his B.Eng. in Electrical Engineering from the University of Electro-Communications in 1970, and a Ph.D. degree in Medical Science from The University of Tokyo in 1993. He worked as an assistant professor at the School of Medicine University of Tokyo from 1971 to 2012. His specialties are speech science, cognitive neuroscience, computer science, and electronics.

images

Fumihiro Sato received B.S. degrees in Education from Waseda University in 1974. After then, he had been worked for the public organization, Japan Information Processing Development Center, and has been researching the methodology of ICT education for 20 years. In 1994, he moved to Chuo University and had been conducting research on Distance Education and e-learning with some universities in Southeast Asia. Since 1999, he is a professor of the Faculty of Economics, Chuo University.

Abstract

1 Introduction

2 Related Works

3 Our Information Delivery System on Disaster

3.1 The Situation Under the Big Disaster

images

3.2 Outline of Our System Using LED Display

images

images

images

4 Experiments and the Results

4.1 Outlines of the Experiments

images

4.2 Results of the Experiments

4.3 The Result of Usage on the Office at the Welfare Facility for the Physically-handicapped in Tokyo

images

4.4 A New Experiment in 2018 – For School of Hearing Impaired Children

images

images

5 Conclusion of Experiments and New Developments for Practical Use

5.1 Extension for the Universal Use

5.2 A New Role of Smartphone After the Disaster in Our System

images

Acknowledgements

References

Biographies