Overview of the Hybridcast® System
Hiroyuki Hamada
STRL is conducting research on Hybridcast, a technology platform that uses communications networks to enhance broadcast services. Hybridcast makes it possible to create various services with many functions. Examples include a service that enables viewers watching video on demand (VOD) programs related to broadcast programs, a service that allows viewers to customize their viewing experience, a service that links with social networking services (SNS), and a service that links smartphones and tablets to TV. This article is an overview of the Hybridcast system.
1. Introduction
The technological environment surrounding broadcasting and communications has been rapidly evolving. The last decade has seen the digitization of broadcasting, the spread of broadband communications, the development of Internet technology and culture, and the advent of a variety of information terminals (Figure 1). The communications world is especially undergoing rapid changes; in FY2011 in Japan, approximately 35 million households had broadband, and over 21 million new smartphones were sold.
There is currently fierce international competition to develop and deploy TV that exploit the features of broadcasting and communications. The competitors in this arena include GoogleTV and Hulu in the U.S., and Hybrid Broadcast Broadband TV (HbbTV) and YouView*1 in Europe. NHK's FY2012-2014 administration plan calls for the development and provision of new services integrating broadcasting and communications, and STRL is conducting research on Hybridcast®*2, a platform for services that use broadcasting and communications1)2)3). Our goal is to deploy Hybridcast within two years, and we are now standardizing and building the systems that will be used for testing and demonstrating Hybridcast.
This article describes the background and objectives of Hybridcast. It presents examples of Hybridcast services, system prototypes, and future plans.
2. Hybridcast Development
More than ten years have passed since digital broadcast satellite (BS) broadcasts began. The digitalization of broadcasting made various services possible. One of these services is the electronic program guide (EPG). The EPG displays a program guide on the TV screen that immediately reflects program scheduling changes such as emergency bulletins or overtime in sports coverage.
Moreover, as viewing styles diversified, the EPG eventually became an essential means of recording programs. The reason is that digital broadcasting specifications are created through the cooperation of broadcasters and receiver manufacturers, and broadcasters provide the service information (SI) for the EPG in a common format.
Broadcasters have a duty to provide viewers with high-quality programs and reliable information, and they must make their services more convenient. To fulfill our duties in a world where broadcasting and communications are becoming more and more integrated, we decided to cooperate with other broadcasters and receiver manufacturers to consolidate a system that conveys the broadcasters' intentions correctly and reliably to viewers and enhances the capabilities of digital broadcasting by exploiting the Internet. As such, we set a goal of implementing Hybridcast in the short term.
3. Hybridcast Technical Specifications
Figure 2 illustrates concept of Hybridcast, our technology platform for creating advanced broadcast services using networks. In particular, functions (1) to (4) in the figure are used to provide new services that integrate broadcasting and communications. (Note that this issues' article, "Technologies supporting Hybridcast", presents the technical specifications for Hybridcast4)). The following objectives were incorporated into the specifications of Hybridcast.
(1) Change existing broadcast formats as little as possible
We should minimize the changes that have to be made to existing broadcast formats by taking into consideration compatibility with current digital receivers and the effects on broadcast facilities.
(2) Use formats that are compatible with international standards and communications
Broadcasting cannot reach everywhere, but communications networks have no borders. We studied formats from a global perspective. HTML55), an enhancement of HTML, which is the language used to describe Internet applications, is currently undergoing standardization at the World Wide Web Consortium (W3C). We chose to use it as the environment for running applications in the draft Hybridcast specifications.
(3) Enable users and service providers to provide a variety of services (applications)
Information distributed by service providers other than the broadcasters could contain content that is not appropriate for broadcasting. Considering the public nature of broadcasts and childcare issues, a mechanism to prevent inappropriate content from being displayed should be provided to ensure that the quality of content can be managed.
(4) Enable broadcasting and communications content to be synchronized for composition and presentation
Services such as multilingual captioning, speed-adjusted voice audio for the aged, and multi-viewpoint video can be implemented by synchronizing information from communications networks with broadcast programs.
(5) Enable new services to be developed as the needs of viewers change and technology advances
Cloud computing is an effective way to foster services that can respond to changing viewer needs and make use of newly developed technologies. For example, by performing media format conversions, summarizing or translating content, and other types of cloud computing, new services can be provided without the viewer having to purchase a new receiver. This requirement reduces the cost of receivers and helps to integrate new communications services such as social networking services (SNS)*3 with broadcasting.
(6) Integrate and harmonize the cultural differences between broadcasting and communications
TV is primarily a passive medium for viewing that can be enjoyed together with the whole family. On the other hand, communications media such as the Internet are basically oriented to individuals, allowing them to access the information and content whenever they like. Integration of broadcasting and communications has been talked about for a while, but no significant success has appeared yet. It will be very important to harmonize the differences in between broadcasting and communications cultures as much as possible in order to create services that link them.
Let us discuss some of the differences between broadcasting and communications cultures. As mentioned earlier, TV is basically a passive medium. By watching TV, viewers can learn about the major events occurring around the world. They can get excited watching live sports coverage, or have a rousing discussion about what happened on yesterday's dramas. TV are simple to operate and do not need the upgrading or other maintenance required by PCs. They are easy, inexpensive appliances that can be used for ten years or more after purchase. The quality of content is guaranteed by the broadcasters, so broadcasting is a very safe and secure medium.
On the othe hand, communication using PCs and smartphones is an active medium. Hardware is replaced and software is upgraded regularly and frequently. There is an enormous amount of information that can be obtained from the Internet, but the content and quality is extremely varied and wide ranging. The fact that information on the Internet is basically not checked by anyone has its attractions, but users must decide for themselves whether the information is reliable.
Hybridcast is an attempt to harmonize reliable broadcasting with the detailed and interesting information available on the Internet. Hybridcast's technical specifications include a mechanism to control the degree of this harmonization in order to handle needs and new situations that may arise, changes in viewing style, and the various business models of each broadcaster. As an example, displaying an application window over the image of a broadcast program could potentially violate the copyright of the broadcast program, so we have included the ability to specify whether overlays are permitted or not for each program.
We have also strengthened functions for linking TV with other terminals such as mobile devices and PCs. The ways of watching and using TV are diversifying and vary with the size of the screen. For example, the whole family can watch a program on the large-screen TV, while information tailored to each family member can be displayed on the screens of tablet computers.
4. System Structure of Hybridcast
Hybridcast includes the following system elements (Figure 3)6)7).
(1) Broadcast station
Application launch information*4 and presentation policy*5 information are broadcasted together with the digital broadcast signal. Program-related information and meta-data enabling broadcasters to tailor their services to the contracts with the viewer and the service provider are also provided.
(2) Service provider
Service providers create and distribute applications. The definition of service providers includes broadcasters and other businesses.
(3) Hybridcast receiver
The main functionalities of the receiver are:
- a functionality to run HTML5 applications distributed by service operators and to process content that uses communications, such as VOD.
- a display functionality that synchronizes the presentation of broadcast and communications content.
- a functionality for linking with other terminals.
- a functionality to enable applications to control the display.
5. Service Examples and Prototypes
We presented the Hybridcast concept at the 2010 NHK STRL Open House; the exhibit included a receiver implemented on a PC2). The following year, we exhibited prototype receivers from different manufacturers and showed some of the things made possible by Hybridcast (Figure 4)8). At the 2012 Open House, we exhibited a prototype receiver including an HTML5 browser and showed how various services can be implemented with it (Figure 5).
There are numerous services that use functions (1) to (4) in the Hybridcast overview diagram (Figure 2).
(1) Services that use social and personal information gained through communications.
(i) Social TV service
This service links viewers of broadcast programs through social networks on the Internet. For example, viewers who have joined an SNS can post their opinions and impressions of a program they are watching and participate in conversations on the SNS9). Friends watching the same program can exchange ideas by using their tablet computers.
(ii) Program recommendation service
This service recommends and provides programs related to the program currently being viewed. The programs are stored in a large VOD library, and the service makes suggestions according to their popularity on the SNS and the viewer's preferences and needs. Recommended programs can be bookmarked for later viewing.
(iii) Personalized program-spot*6 presentation service
This service recommends programs according to the viewer's preferences instead of showing traditional program spots for the current broadcast. For example, a sports program spot can be delivered through communications channels and presented automatically to a person who watches a lot of sports (Figure 6). This service infers the viewer's preferences from their TV viewing history and other personal viewing traits. It is being developed in cooperation with NTT.
(2) Services that allow programs to link the TV to mobile terminals and PCs.
(i) Terminal linking service
This service displays information on a mobile terminal instead of on a large TV screen. In the service example shown in Figure 7, viewers can search Web content by using keywords and that content is automatically presented on the screen as the TV program progresses. Mobile terminal features can be exploited in various ways. For example, a tablet computer could be used to operate a TV or enter text in an application.
(ii) User authentication service
This service enables users of social services to login from their mobile terminals. This eliminates the need for login operations on the TV. Mobile terminals have an excellent user interface and have relatively easy login operations. Thus, by having the mobile terminal pass login information to the TV, it can be used to access various personalized services without needing to perform any specialized operations (Figure 8).
(3) Services that integrate communications content with broadcast programs.
(i) Customization service
This service provides content related to the program being broadcast through communications channels. It takes the needs of the individual viewer into account and synchronizes the presentation with the broadcast program. Examples include multilingual caption services for foreigners or speed-adjusted audio for the elderly10). At the 2010 NHK STRL Open House, we showed that captions provided by communications networks could be displayed with almost no noticeable delay.
(ii) Multi-viewpoint service
This service distributes video segments related to the broadcast program over a communications network and composes them with the program for display. It uses a high-precision synchronizing technology that can synchronize content with single-frame precision11). Figure 9 shows an example of the multi-viewpoint service. The multi-view segments are cropped and zoomed parts of a Super Hi-Vision image (SHV) and are overlaid on the main image. It is also possible to create multi-viewpoint video made from images taken by multiple cameras (Figure 10). Viewers can select the camera video they want to see and watch it continuously overlaid on the program video.
A 3D-HDTV video service could also make use of high-precision synchronization technology. For example, HDTV video for the left eye could be sent in the broadcast signal while the video for the right eye is distributed over the communications network. The HDTV video could be viewed on existing TV through the broadcast, while receivers supporting Hybridcast and 3D displays could be used to view the 3D-HDTV video. This service is being researched in cooperation with NTT.
(iii) Computer graphics synthesis service
This service displays broadcast programs synchronized with computer graphics (CG). For example, sign-language CG video generated using TV program Making Language (TVML)12) can be sent over a communications network to hearing-impaired viewers (Figure 11). This service could also be used to display graphics such as off-side lines in soccer.
(4) Services using functions that provide information accurately and reliably
(i) Giving priority to emergency bulletins13)
When a terminal receives a bulletin such as an emergency earthquake report while an application is using its display, the application must stop using the display and the emergency information must be displayed reliably (Figure 12). To do this, the environment must always be able to control the display and certify and permit applications to run.
6. Conclusion
Besides the venue of the NHK STRL Open House, we have exhibited Hybridcast at many trade shows and conferences within and outside Japan, including NAB2011, CEATEC, and IBC201114). In December 2011, we started a working group at the Japanese standardization organization, the IPTV Forum, to decide on technical specifications for services linking broadcasting and communications using HTML5. This working group has many participants from manufacturers, broadcasters, and communications providers. The HTML5 specifications being studied there are closely related to the Hybridcast specifications. In the future, we will create system prototypes and conduct tests based on the specifications determined in cooperation with these related enterprises and organizations, and we will continue to work toward early deployment of this technology.