W3C

Metadata API for Media Resources 1.0

W3C Recommendation 13 March 2014

This version:
http://www.w3.org/TR/2014/REC-mediaont-api-1.0-20140313/
Latest published version:
http://www.w3.org/TR/mediaont-api-1.0/
Previous version:
http://www.w3.org/TR/2013/PR-mediaont-api-1.0-20131015/
Editors:
Florian Stegmaier, University of Passau
Werner Bailer, JOANNEUM RESEARCH
Martin Höffernig, JOANNEUM RESEARCH
이원석(Wonsuk Lee), Samsung Electronics, Ltd.
Chris Poppe, Ghent University

Please refer to the errata for this document, which may include some normative corrections.

See also translations.


Abstract

This specification defines an API to access metadata information related to media resources on the Web. The overall purpose is to provide developers with a convenient access to metadata information stored in different metadata formats. The API provides means to access the set of metadata properties defined in the Ontology for Media Resources 1.0 specification. These properties are used as a pivot vocabulary in this API. The core of this specification is the definition of API interfaces for retrieving metadata information in synchronous and asynchronous modes. It also defines interfaces for structured return types along with the specification of the behavior of an API implementation.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document has been reviewed by W3C Members, by software developers, and by other W3C groups and interested parties, and is endorsed by the Director as a W3C Recommendation. It is a stable document and may be used as reference material or cited from another document. W3C's role in making the Recommendation is to draw attention to the specification and to promote its widespread deployment. This enhances the functionality and interoperability of the Web.

This document has been produced by the Media Annotations Working Group, which is part of the W3C Video on the Web Activity

The public is encouraged to send comments on this Recommendation to the public mailing list public-media-annotation@w3.org (public archive). Use "[REC Comment API]" in the subject line of your email.

The Working Group has adopted a public test suite and has produced an implementation report for this Metadata API for Media Resources 1.0.

No changes to this document have been made since the previous version.

The Metadata API for Media Resources may be implemented in both client-only (built into a browser, as a plugin or as a JavaScript library) and client-server (server-side as a Web Service). The level of implementation of this API in these two scenario summarized in the implementation report allowed to exit Candidate Recommendation. Nevertheless this API is not expected to be implemented natively in the browser code.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1. Introduction

This specification defines an API to access metadata information related to media resources on the Web. The overall purpose is to provide developers with a convenient access to metadata information stored in different metadata formats. The core properties, defined in the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY], will be used as a pivot vocabulary in this API. The description of relations between these core properties and the metadata formats in scope are documented in the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY] in order to provide cross-community data integration. This API is described using the interface definition language Web IDL [WEBIDL]. The decision to use Web IDL, which offers bindings for ECMAScript and Java, is based on the Use Cases and Requirements for Ontology and Metadata API for Media Resources 1.0 [MEDIA-ANNOT-REQS].

This API defines interfaces that enable users/applications to consume metadata in an interoperable manner. Interoperability between metadata formats is ensured by the use of the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY] as pivot metadata format. This API offers operations to request particular metadata information represented in a certain metadata format related to media resources on the Web. Further it specifies the actual representation of the core properties and the behaviour of this API.

1.1 Formats in scope

Refers to the formats in scope of the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY].

1.2 Formats out of scope

Refers to the Formats out of scope of the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY].

1.3 Terminology

In this document the terms "Ontology", "Media Resource", "Property", "Mapping" and "Property value types" are to be interpreted as defined in Section 2 of the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY].

2. Conformance

In addition to sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The keywords must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

3. Design consideration

This section discusses different usage scenarios that led to design of the API. We consider two main scenarios, where this API be implemented and invoked:

In both client-only and client-server cases of the implementation, the media resources and/or the metadata sources are in many cases remote. The API is by default specified as an asynchronous API, i.e., the calls are not blocking, but results (or errors) are returned using a callback mechanism. In order to better support the Web Service case, a synchronous mode is also defined. The synchronous mode is optional.

The two scenarios are shown in Figure 1.

Diagram showing 2 scenarios with different usage of the API.

Figure 1: Two scenarios with different usage of this API.

This specification only defines the Metadata API for Media Resources. Other components depicted in Figure 1 (e.g., access/extraction/storage of metadata) are not covered.

Scenario 1: Client-only (User agent)
In the first scenario, this API is implemented in the user agent, i.e. built into a browser, as a plugin or as a JavaScript library. Here, there exist three possibilities to invoke the API: by an external calling code, an internal calling code behaving like a client or it is attached as an extension to a user agent. Usually, such implementations are an example for asynchronous processing. Besides the Metadata API for Media Resources 1.0, the user agent may include components for metadata access (and extraction) and mappings for a supported set of formats, e.g., as defined in the property mapping table of the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY]. Further, the metadata sources (the media resource and/or metadata document(s)) must be retrievable. The access (e.g., establish connection, retrieval) to the metadata sources is handled by the user agent.
Scenario 2: Client-Server
In the second scenario, this API is implemented as a Web Service following the principles of server-side synchronous processing. Such an implementation would be typically used by a non-UI client, such as an agent harvesting metadata. However, this API can be also accessed from a user agent, and used the same way as described in scenario 1 with the help of a client-side library for accessing the Web Service. In the implementation of the Web Service, this scenario also allows supporting a media repository, e.g., content provider's archive database, movie store. With the help of such a service the user agent can retrieve metadata sources, which might have a custom metadata format not supported by a user agent. In contrast to an integrated component (see scenario 1), an implementation of this API in a web service can do more complex mappings on the fly than a component integrated in a user agent, and can be more flexible (e.g., supporting additional formats).

In both scenarios, the API serves as a mediator between a client application and the actual metadata sources. Interoperability is ensured by defining i) operations for accessing the metadata information, ii) a common object structure and iii) API behaviour (e.g., status codes). Following this, an implementation has to implement this stack of components:

This API provides access to metadata information stored in different metadata formats. As such, different instances of the same property can exist.

4. API Description

This API defines a number of interfaces using [WEBIDL]. These can be grouped in the following categories:

Next, the different interfaces and exposed operations are discussed. Implementations of this API must support asynchronous mode of operation, may support the synchronous one and must support the interfaces defined in this document. Instead of exceptions, a status code indicating the state of processing (see Section 4.7) is returned (in the synchronous API) or provided via a callback function (in the asynchronous API) in case an error occurs.

Then, the interfaces for the return types, i.e., MediaAnnotation and its specializations, and MetadataSource are defined.

The IDL fragment in Appendix A of this specification must be interpreted as required for conforming IDL fragments, as described in the “Web IDL” specification. [WEBIDL]

4.1 MediaResource interface

The MediaResource interface is the core of this API and provides operations to access the metadata properties of a specific media resource. Here, a clear separation between asynchronous and synchronous mode of operation has been achieved by defining two implementing interfaces (derived from MediaResource), the AsyncMediaResource and the SyncMediaResource interface. Objects of these interfaces will be created by calling createMediaResource of the MediaResource interface. The actual connection to a specified metadata source will be created with the execution of the getMediaProperty operation of AsyncMediaResource or SyncMediaResource interface. The mediaResource argument identifies the media resource, for which the implementation of this API should try to find relevant metadata sources. Optionally, references to metadata sources can be passed using an array of objects, each implementing the MetadataSource interface (see Section 4.6).

interface MediaResource {
    short         getSupportedModes ();
    MediaResource createMediaResource (DOMString mediaResource, optional MetadataSource[] metadataSources, optional short mode);
};

4.1.1 Methods

createMediaResource
This operation instantiates an object of either AsyncMediaResource or SyncMediaResource interface. Further, it allows to set the specific media resource and metadata sources to which this API is applied.
Parameter Type Nullable Optional Description
mediaResource DOMString This attribute must set the specific media resource that should be processed by the API.
metadataSources MetadataSource[] This attribute should specify additional metadata sources.
mode short This attribute should specify the desired mode of operation. 1 for asynchronous and 2 for synchronous mode should be used.
In the case the mode argument is omitted, and the implementation supports both modes, the asynchronous mode will be used.
No exceptions.
Return type: MediaResource
getSupportedModes
This operation is called to identify the implemented mode. The return codes should be as follows: 1 for asynchronous, 2 for synchronous and 3 for both modes.
No parameters.
No exceptions.
Return type: short

4.1.2 Examples in Javascript

Example for getSupportedModes:

ma = new MediaResource();
var mode = ma.getSupportedModes();

/** Resulting in:
 * { "supportedModes" : 3 }
 */

Example for createMediaResource:

metadataSources = new MetadataSource[2];
metadataSources[0] = new MetadataSource(
                         "http://www.w3.org/2008/WebVideo/Annotations/drafts/metadata_formats/DC_example1.xml","dc");
metadataSources[1] = new MetadataSource(
                         "http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG","exif");

mediaResource = new MediaResource();

if (mediaResource.getSupportedModes() == 1 || mediaResource.getSupportedModes() == 3) {
    aSyncObject = mediaResource.createMediaResource(
                         "http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG",
                         metadataSources, 1);
                             
} else if (mediaResource.getSupportedModes() == 2 || mediaResource.getSupportedModes() == 3)  {
    syncObject = mediaResource.createMediaResource(
                         "http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG",
                         metadataSources, 2);
}

4.2 AsyncMediaResource interface

The AsyncMediaResource interface provides a number of operations that allow accessing the metadata of a media resource. This interface must be implemented.

Next, we give the Web IDL description of the AsyncMediaResource interface and describe the different operations that are part of it.

In this section the MediaAnnotations interface is used in the interface definitions. It serves as a container to hold general values about properties enabling an iteration over a set of different properties. Its definition can be found in Section 4.4

interface AsyncMediaResource : MediaResource {
    void getMediaProperty (DOMString[] propertyNames, PropertyCallback successCallback, ErrorCallback errorCallback,
       optional DOMString fragment, optional DOMString sourceFormat, optional DOMString language);
void getOriginalMetadata (DOMString sourceFormat, MetadataCallback successCallback, ErrorCallback errorCallback); };

4.2.1 Methods

getMediaProperty
This operation allows retrieval of the value of a specific property, several or all properties in an asynchronous manner. The specific property is passed as an argument and a list of objects is returned that hold the values according to the requested property. These objects implement the MediaAnnotation interface, described in Section 4.4. Depending on the requested property, the returned objects implement a different subtypes (inheriting from the MediaAnnotation interface). For example, requesting "title" gives back an array of objects implementing the Title interface, requesting "creator" results in objects implementing the Creator interface and so on. These interfaces are described in Section 4.5. An example can be found here.
Parameter Type Nullable Optional Description
propertyNames DOMString[] This argument identifies an array containing the properties for which the values need to be retrieved. For an empty array all properties carrying values will be retrieved.
successCallback PropertyCallback This argument holds a callback object for asynchronous requests to the property. The successCallback object implements the PropertyCallback interface and holds a handleEvent operation that needs to be called once all data for the requested property is gathered. This handleEvent operation needs to be called with a new MediaAnnotation array.
errorCallback ErrorCallback This argument holds a callback object for failure of asynchronous requests to the property. The errorCallback object implements the ErrorCallback interface and holds a handleEvent operation that needs to be called if an attempt fails. This handleEvent operation needs to be called with a new DOMString representing the status code of the error (see Section 4.7 for details).
fragment DOMString This argument contains a URI identifying the specific media fragment for which the metadata is requested. The URI must conform to the URI for Media Fragment [MEDIA-FRAGMENTS] specification. This parameter is optional.
sourceFormat DOMString This argument identifies a specific metadata format. It should use the metadata format identifiers defined in the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY]. If a metadata format is defined, only the metadata available in the specified metadata format are retrieved. This parameter is optional.
language DOMString This argument allows to identify the language of the metadata. Values for the metadata will only be returned if it is available in the specified language. Recommended best practice is to use BCP 47 [BCP47]. This parameter is optional.
No exceptions.
Return type: void
getOriginalMetadata
This operation allows retrieval of the original metadata according to the specified source format in an asynchronous manner. An example can be found here.
Parameter Type Nullable Optional Description
sourceFormat DOMString This argument identifies a specific metadata format. It should use the metadata format identifiers defined in the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY]. If a metadata format is defined, only the metadata available in the specified metadata format are retrieved.
successCallback MetadataCallback This argument holds a callback object for asynchronous requests for the original metadata. The successCallback object implements the MetadataCallback interface and holds a handleEvent operation that needs to be called once all properties having values are listed. This handleEvent operation needs to be called with a new DOMString array holding the original metadata.
errorCallback ErrorCallback This argument holds a callback object for failure of asynchronous requests for the original metadata. The errorCallback object implements the ErrorCallback interface and holds a handleEvent operation that needs to be called if an attempt fails. This handleEvent operation needs to be called with a new DOMString representing the status code of the error (see Section 4.7 for details).
No exceptions.
Return type: void

4.2.2 Callback interfaces

4.2.2.1 PropertyCallback interface

The PropertyCallback interface holds a handleEvent operation that needs to be called once all data for the requested property has been gathered.

interface PropertyCallback {
    void handleEvent (MediaAnnotation[] mediaAnnotations);
};
4.2.2.1.1 Methods
handleEvent
This operation is called when all data is gathered corresponding to a request for values of one or more properties.
Parameter Type Nullable Optional Description
mediaAnnotations MediaAnnotation[] This argument holds a list of objects with values according to the requested property. These objects implement the MediaAnnotation interface, described in Section 4.4. Depending on the requested property, the returned objects implement a different subtypes (inheriting from the MediaAnnotation interface).
No exceptions.
Return type: void
4.2.2.2 MetadataCallback interface

The MetadataCallback interface holds a handleEvent operation that needs to be called once the requested metadata has been gathered.

interface MetadataCallback {
    void handleEvent (DOMString[] metadata);
};
4.2.2.2.1 Methods
handleEvent
This operation is called when all data is gathered corresponding to a request for the original metadata.
Parameter Type Nullable Optional Description
metadata DOMString[] This argument holds a list of DOMStrings representing the original metadata. Note that, multiple metadata instances can exist (e.g., one Dublin Core and one MPEG-7 document).
No exceptions.
Return type: void

4.2.3 Examples in Javascript

Example for asynchronous getMediaProperty:

aSyncMediaResource = mediaResource.createMediaResource("http://www.imdb.com/title/tt0133152/", new Array(), 1);
aSyncMediaResource.getMediaProperty(["title"], successCallback, errorCallback, "", "", "");
                                
function successCallback(MediaAnnotation[] mediaAnnotations) {
    ...
}

/** Resulting in:
 * [ { "Title" : {
 *         "propertyName" : "title",
 *         "value" : "Planet of the apes",
 *         "language" : en-us",
 *         ...
 *         "statusCode" : 200
 *         }
 * },
 * { "Title" : {
 *         "propertyName" : "title",
 *         "value" : "Monkey Planet",
 *         "language" : en-us",
 *         ...,
 *         "statusCode" : 200
 *         }
 * },
 * { ...
 * } ]
 */
                
function errorCallback(DOMString error) {
    ...
}

/** Resulting in:
 * { error: { "statusCode" : 200 } }
 */

Example for asynchronous getOriginalMetadata :

aSyncMediaResource = mediaResource.createMediaResource(
                         "http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG",
                          new Array(), 1);
aSyncMediaResource.getOriginalMetadata("dc", successCallback, errorCallback);

function successCallback(DOMString[] metadata) {
    ...
}

/** Resulting in:
 * [ { "statusCode" : 200
 * },
 * {"originalMetadata" : "<metadata xmlns='http://example.org/myapp/'
 *                                        xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
 *                                        xsi:schemaLocation='http://example.org/myapp/ http://example.org/myapp/schema.xsd'
 *                                        xmlns:dc='http://purl.org/dc/elements/1.1/'>
 *                                          <dc:title>DC title</dc:title>
 *                                  </metadata>"
 * } ]
 */

                                                
function errorCallback(DOMString error) {
   ...
}

/** Resulting in:
 * { error: { "statusCode" : 200 } }
 */

4.3 SyncMediaResource interface

The SyncMediaResource interface provides a number of operations to access the metadata of a media resource. This interface may be implemented.

Next, we give the Web IDL description of the SyncMediaResource interface for synchronous requests and describe the different operations that are part of it. The MediaResource defines a constructor that can be called to construct the object based on an identifier of the media resource and optionally some metadata sources.

interface SyncMediaResource : MediaResource {
    MediaAnnotation[] getMediaProperty (DOMString[] propertyNames, optional DOMString fragment,
       optional DOMString sourceFormat, optional DOMString language);
DOMString[] getOriginalMetadata (DOMString sourceFormat); };

4.3.1 Methods

getMediaProperty
This operation allows retrieval of the metadata of a specific property, several or all properties in a synchronous manner. The passed array holds the requested properties and an array of objects is returned. If the array is empty, every property holding values will be requested and returned. The returned objects implement the MediaAnnotation interface (see Section 4.3). Depending on the requested property, the returned objects implement different subtypes (inheriting from the MediaAnnotation interface). For example, requesting "title" gives back an array of objects implementing the Title interface, requesting "creator" results in objects implementing the Creator interface and so on. These subtypes are described in Section 4.4. The operation returns a MediaAnnotation array holding the requested properties. If an error occurs during retrieval, a MediaAnnotation object with the corresponding status code (e.g., 400, 404 or 415) will be generated and inserted at the first position of the array. An example can be found here.

In this section the MediaAnnotations interface is used in the interface definitions. It serves as a container to hold general values about properties enabling an iteration over a set of different properties. Its explanation can be found in Section 4.4

Parameter Type Nullable Optional Description
propertyNames DOMString[] This argument holds the requested properties as an array. If the array is empty, each property holding values will be returned.
fragment DOMString This argument contains a URI identifying the specific media fragment for which the metadata is requested. The URI must conform to the URI for Media Fragment [MEDIA-FRAGMENTS] specification. This parameter is optional.
sourceFormat DOMString This argument identifies a specific metadata format. It should use the metadata format identifiers defined in the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY]. If a metadata format is defined, only the metadata available in the specified metadata format is retrieved. This parameter is optional.
language DOMString This argument allows to identify the language of the metadata. Values for the metadata will only be returned if it is available in the specified language. Recommended best practice is to use BCP 47 [BCP47]. This parameter is optional.
No exceptions.
Return type: MediaAnnotation[]
getOriginalMetadata
This operation allows retrieval of the original metadata according to the specified source format in a synchronous manner. The operation returns a DOMString array holding the status code of the request at the first and the original metadata at the second position. An example can be found here.
Parameter Type Nullable Optional Description
sourceFormat DOMString This argument identifies a specific metadata format. It should use the metadata format identifiers defined in the Ontology for Media Resources 1.0 specification [MEDIA-ONTOLOGY]. If a metadata format is defined, only the metadata available in the specified metadata format is retrieved.
No exceptions.
Return type: DOMString[]

4.3.2 Examples in Javascript

The examples in this section use getMediaProperty() to get an object implementing the MediaAnnotation interface. The noErrorStatus function ensures that no error is present and the requested properties carry values.

We give some JavaScript examples on how to use the synchronous MediaResource interface and it's operations.

Example for synchronous getMediaProperty:

syncMediaResource = mediaResource.createMediaResource("http://www.imdb.com/title/tt0133152/",
                          new Array(), 2);
title = syncMediaResource.getMediaProperty(["title"], "", "", "");

if (noErrorStatus(title[0].statusCode) == true) {
   ...
}                                                

/** Resulting in:
 * [ { "Title" : {
 *         "propertyName" : "title",
 *         "value" : "Planet of the apes",
 *         "language" : en-us",
 *         ...,
 *         "statusCode" : 200
 *         }
 * },
 * { "Title" : {
 *         "propertyName" : "title",
 *         "value" : "Planet der Affen",
 *         "language" : "de-de",
 *         ...,
 *         "statusCode" : 200
 *         }
 * },
 * { ...
 * } ]
 */

Example for synchronous getOriginalMetadata:

syncMediaResource = mediaResource.createMediaResource("http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG",
                         new Array(), 2);
dcMetadata = syncMediaResource.getOriginalMetadata("DC");

if (noErrorStatus(dcMetadata[0].statusCode) == true) {
    ...
}

/** Resulting in:
 * [ { "statusCode" : 200
 * },
 * {"originalMetadata" : "<metadata xmlns='http://example.org/myapp/'
 *                                        xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
 *                                        xsi:schemaLocation='http://example.org/myapp/ http://example.org/myapp/schema.xsd'
 *                                        xmlns:dc='http://purl.org/dc/elements/1.1/'>
 *                                          <dc:title>DC title</dc:title>
 *                                  </metadata>"
 * } ]
 */

4.4 MediaAnnotation interface

MediaAnnotation interface is used as the return type of MediaResource.getMediaProperty operation. It is a container for holding general values about metadata properties.

As several metadata properties are defined as complex types, specific derived types of MediaAnnotation have been defined, adding their specific attributes. However, MediaAnnotation can be used as a generic return type to access a printable string representation of the property (in the value attribute). It also includes a status code. In case of general errors, the first element of the returned MediaAnnotation array contains the global error code, otherwise the status can be given for each of the returned properties.

The following design considerations have been used for specifying the derived interfaces for each of the metadata properties:

interface MediaAnnotation {
    attribute DOMString propertyName;
    attribute DOMString value;
    attribute DOMString language;
    attribute DOMString sourceFormat;
    attribute DOMString fragmentIdentifier;
    attribute DOMString mappingType;
    attribute short