KitPlus Ask the Experts:
AI in Media and Entertainment

AI in Media and Entertainment

David Candler November 13, 2018

Originally published by Kitplus, TV Bay, Issue 134

Answered by David Candler, Senior Director, Customer Solutions at Wazee Digital, a Veritone Company

Artificial Intelligence (AI) is a term appearing everywhere these days. What is happening in media and entertainment (M&E) that makes the industry ripe for AI? In other words, why does the M&E industry need AI?

In virtually every industry, AI is claiming a growing stake in the supply chain, creating both operational enhancements and business efficiencies at an amazing rate. In the M&E industry, AI is starting to make its way into a wide range of conversations about everything from projects to products. With an initial focus on streamlining workflows and creating enhanced discovery experiences, the benefits are rapidly becoming a reality for many businesses.

The M&E content landscape continues to transform at a staggering rate. Organisations face increasing challenges to grow audiences, prove the effectiveness of advertising campaigns, index for quality and compliance, and increase revenue. In terms of asset management solutions, the sheer volume of content under management and the rate at which it is being created can make finding and retrieving content a challenge, even with the best metadata and logging workflows in place. With the explosion of content creation, from UGC (user-generated content) to multiversion studio releases, AI will have an increasing role to play in the critical task of discovering relevant content all along the digital supply chain, so that it can then be further utilised and monetised.

How can AI help M&E organisations (e.g. driving operational efficiencies, etc.)?

When you look at the M&E industry (and especially how asset management solutions have been traditionally deployed alongside operational teams to create, distribute, and monetise content), the limiting factor in many cases has been human. Much of the content produced by a media company never gets distributed or broadcast and ends up on the virtual cutting-room floor. Over time, this valuable content can grow into huge static repositories. To make content discoverable and useable, we have relied upon humans to tag metadata to identify the content with varying levels of accuracy and completeness. AI can vastly improve this whole scenario by interrogating a wide range of video and audio elements from every frame of an asset. AI technologies will dramatically improve how humans engage with content and how the M&E industry can drive better efficiency and value.

The concern that AI will eventually replace humans in the M&E supply chain is missing the point somewhat. The real value is based upon how AI can augment operational tasks by taking the first phase of compiling, analysing, and delivering content as required. Many content repositories are just not set up to address future consumption requirements — be they on-demand distribution right through to historical archive discovery and monetisation. AI will help to automate the processes that connect the content to the consumer, from programme viewers to footage marketplace buyers. This area is where AI can start to drive real value into the M&E industry.

In general, how does AI work in the M&E industry?

AI-enabled asset management solutions are typically utilising their workflow orchestration layers to push assets through a third-party AI cognitive engine or several engines controlled by an AI operating system or orchestration layer. Metadata generated by the cognitive engines is then generally received back via API. Many solutions can submit single or multiple assets through both automated and manual workflow processes. An ideal scenario is for the asset management solution to allow for all returned metadata to be displayed along an asset’s video timeline with different engine results displayed in their own timeline fields. Metadata should also be fully indexed in the asset management’s search engine to enable discovery via advanced search features or by timeline data search tools, which make it possible to jump to moments identified by the engines when selected. Analytical tools and dashboards provide all statistics and reports that the end user requires, or the API can feed the metadata into third-party systems for downstream utilisation. The types of cognitive engines typically used by the M&E industry are as follows:

  • Transcription (converting spoken audio and video recordings into readable text)
  • Face Recognition (identifying and indexing the presence of individuals in video or still images)
  • Object Recognition (identifying multiple objects within video or still images)
  • Sentiment (discerning the tone behind a series of words and using it to understand the attitudes, opinions and emotions expressed)
  • A/V Fingerprinting (generating a condensed digital summary, deterministically generated as a reference clip, that can be used to quickly locate similar items across multiple media files)
  • Translation (translating written text from one language to another)
  • Geolocation (associating media with geolocation data points to enable search by location, displaying a map view of media file collections or other specialised functionality)
  • Optical Character Recognition (also known as text recognition, extracting text from an image, video, or document)
  • Logo Recognition (identifying specific companies based on their logos or brands in images and video)

If a media company or a rights holder wants to put AI to work, what are some possible applications (e.g. asset management)?

All of the above cognitive engine types allow asset management solutions to build purpose-driven workflows that can help improve operational efficiencies, optimise ad and sponsorship verification, repurpose content, enhance competitive research, unlock hidden revenue streams, and more. By applying key components, users can automatically create a searchable set of data along the content timeline, as opposed to manual viewing and logging. Integrating an asset management solution with an AI operating system or orchestration layer opens access to hundreds of cognitive engines, which enables the user to try different engines to find the one that best fits the parameters of a given project.

At its most basic, AI alongside asset management can help organisations analyse, share, and index their content automatically, which ultimately leads to streamlined workflows and enhanced discovery experiences. AI can automatically generate preconfigured and relevant metadata that can enhance advanced searches on vast archives, which in turn can reduce operational costs and raise the discoverability and usability of valuable content.

Many content companies have turned to the cloud for things like archiving and distribution. Is it possible to build AI into a cloud workflow?

Absolutely yes. Many customers are now fully adopting cloud-native solutions that take full advantage of the inherent benefits of the cloud (e.g. scalability, resiliency, agility, etc.). In my own world, both asset management and AI solution components are fully cloud-native, making integration and usage so much easier. Also, as customers start to store their content in cloud-based object storage, it becomes easier for both asset management and AI solutions to work with the same content in the same location.

How can M&E asset management vendors bring AI to their customers?

Many traditional asset management vendors have failed over the years to deliver on their original promise to their M&E customers to transform critical business processes and enhance the overall value of their content through technology. In terms of AI, if the vendor fails to appreciate the value of understanding content at a cognitive level, then there will be a strategy gap between the vendor and the industry. Without this understanding, we will still require humans to perform most of the critical functions along the digital supply chain (e.g., content creation to distribution plus analysis and monetisation).

This lack of understanding will continue to prevent asset management vendors from taking full advantage of business process automation technologies, which industries that rely on structured data have used to deliver significant advancements and efficiency and performance. The M&E industry generates a great deal of unstructured data, which is why cognitive AI solutions are a must as opposed to computational strategies. Therefore, if M&E asset management vendors fully integrate AI processes within their solutions, they will be helping to transform the on-demand creation, delivery, and monetisation of content.

What are the benefits of such a collaboration?

M&E asset management vendors are already engaging with the likes of IBM Watson, Amazon Web Services, Microsoft Azure, Google, and Veritone to find tangible ways to use AI to help drive efficiencies and monetisation opportunities across their customers’ operations and content archives. In the M&E sector, early adopters of such technologies are already beginning to reap the benefits of operational efficiencies and revenue generation.

I believe that the ideal scenario is to utilise all of these technologies through a single AI operating system or orchestration layer. This type of operation is going to become the norm in the modular cloud infrastructure, where collaborating with other industry experts can lead to better, more flexible solutions than most vendors could ever build on their own. It is the best and most efficient way to give customers the problem-solvers they need. This type of collaboration will be the foundation of next-generation supply chains that reduce costs, improve efficiencies, and ultimately make content available to use.

What are some possible use cases?

There are multiple scenarios in which AI can enhance an asset management solution’s value to the M&E industry. Some possible use cases are noted below:

  • Automated AI Captions: Automated captioning is on the increase as accuracy levels improve and the cost to create captions for access regulation purposes remains high. Automated AI captioning not only helps with accessibility but also provides a searchable timeline reference for discoverability within the asset management solution.
  • Automated AI Highlights: We all saw how Wimbledon has started to use IBM Watson AI to power highlights, analytics, and enriched fan experiences. This use case will become much more commonplace, especially with the ever-increasing demand for highlights across sports, news, and entertainment verticals. These highlights are used all over social media platforms to engage viewers and fans across the world. AI automation helps to streamline the process of identifying relevant moments and getting them to the consumer as quickly as possible.
  • Automated AI Metadata Generation: One of the key functions of AI is to generate relevant metadata for downstream use cases, including search and classification. This function enables deeper and more advanced discovery of content without needing a team manually tagging content (with the chance of human error). The larger your archive, the more automated AI metadata generation is relevant for you.

Other typical use cases include:

  • Automatic quality assurance versus manual evaluation
  • Marketing and advertising (e.g., targeted campaigns)
  • Personalised services (e.g., content recommendation)
  • Experience innovation (e.g., new immersive visual content experiences, including virtual reality and augmented reality)
  • Sponsorship verification (e.g., using logo recognition to analyse the use of sponsorship)
  • And there are many, many more …

Can you give a real-world example of an M&E company using AI for media asset management today?

The use cases are vast and diverse. In one recent example, an international media conglomerate — home to premier global television, motion picture, gaming, and other brands — used an AI solution to underpin a broadcast compliance workflow. To comply with the U.S. Federal Communications Commission’s Children’s Television Act of 1990, the company is required to identify the talent used in any advertisements that run during children’s educational programs. The solution leveraged automated facial recognition, speech to text, and enriched metadata within the asset management platform to identify the talent and provide data back to the company. As a result, the company can be sure that the ads do not contain the same talent as in the concurrent program, thereby ensuring compliance.

Download PDF here.