Open Archives Initiative - Protocol for Metadata Harvesting - v.2.0 The Open Archives Initiative Protocol for Metadata Harvesting Protocol Version 2.0 of 2002-06-14 Document Version 2015-01-08 http://www.openarchives.org/OAI/2.0/openarchivesprotocol.htm Previous protocol version: Protocol Version 1.1 of 2001-07-02 Instructions for migrating from Version 1.1 to 2.0 Implementation Guidelines Editors The OAI Executive: Carl Lagoze <[email protected] > -- Cornell University - Computer Science Herbert Van de Sompel <[email protected] > -- Los Alamos National Laboratory - Research Library From the OAI Technical Committee: Michael Nelson <[email protected] > -- NASA - Langley Research Center Simeon Warner <[email protected] > -- Cornell University - Computer Science Table of Contents 1. Introduction 2. Definitions and Concepts 2.1. Harvester 2.2. Repository 2.3. Item 2.4. Unique Identifier 2.5. Record 2.5.1 Deleted records 2.6. Set 2.7. Selective Harvesting 2.7.1 Selective Harvesting and Datestamps 2.7.2 Selective Harvesting and Sets 3. Protocol Features 3.1. HTTP Embedding of OAI-PMH requests 3.1.1. HTTP Request Format 3.1.2. HTTP Response Format 3.1.3. Response Compression 3.2. XML Response Format 3.2.1. XML Schema for Validating Responses to OAI-PMH Requests 3.3. UTCdatetime 3.3.1. UTCdatetime in Protocol Requests 3.3.2. UTCdatetime in Protocol Responses 3.4. metadataPrefix and Metadata Schema 3.5. Flow Control 3.5.1 Idempotency of resumptionTokens 3.6. Error and Exception Conditions 4. Protocol Requests and Responses 4.1. GetRecord 4.2. Identify 4.3. ListIdentifiers 4.4. ListMetadataFormats 4.5. ListRecords 4.6. ListSets 5. Dublin Core 6. Implementation Guidelines Acknowledgements Document History 1. Introduction The Open Archives Initiative Protocol for Metadata Harvesting (referred to as the OAI-PMH in the remainder of this document) provides an application-independent interoperability framework based on metadata harvesting. There are two classes of participants in the OAI-PMH framework: Data Providers administer systems that support the OAI-PMH as a means of exposing metadata; and Service Providers use metadata harvested via the OAI-PMH as a basis for building value-added services. In this document the key words "must", "must not", " required", "shall", "shall not", "should", " should not", "recommended", "may", and "optional " in bold face are to be interpreted as described in RFC 2119 . An implementation is not conformant if it fails to satisfy one or more of the "must" or "required" level requirements for the protocols it implements. This document refers in several places to "community-specific" practices to which individual protocol implementations may conform. These practices are described in an accompanying Implementation Guidelines document. 2. Definitions and Concepts 2.1 Harvester A harvester is a client application that issues OAI-PMH requests. A harvester is operated by a service provider as a means of collecting metadata from repositories. 2.2 Repository A repository is a network accessible server that can process the 6 OAI-PMH requests in the manner described in this document. A repository is managed by a data provider to expose metadata to harvesters. To allow various repository configurations, the OAI-PMH distinguishes between three distinct entities related to the metadata made accessible by the OAI-PMH. resource - A resource is the object or "stuff" that metadata is "about". The nature of a resource, whether it is physical or digital, or whether it is stored in the repository or is a constituent of another database, is outside the scope of the OAI-PMH. item - An item is a constituent of a repository from which metadata about a resource can be disseminated. That metadata may be disseminated on-the-fly from the associated resource, cross-walked from some canonical form, actually stored in the repository, etc. record - A record is metadata in a specific metadata format. A record is returned as an XML-encoded byte stream in response to a protocol request to disseminate a specific metadata format from a constituent item. 2.3 Item An item is a constituent of a repository from which metadata about a resource can be disseminated. An item is conceptually a container that stores or dynamically generates metadata about a single resource in multiple formats, each of which can be harvested as records via the OAI-PMH. Each item has an identifier that is unique within the scope of the repository of which it is a constituent. 2.4 Unique Identifier A unique identifier unambiguously identifies an item within a repository; the unique identifier is used in OAI-PMH requests for extracting metadata from the item. Items may contain metadata in multiple formats. The unique identifier maps to the item, and all possible records available from a single item share the same unique identifier. The format of the unique identifier must correspond to that of the URI (Uniform Resource Identifier) syntax. Individual communities may develop community-specific URI schemes for coordinated use across repositories. The scheme component of the unique identifiers must not correspond to that of a recognized URI scheme unless the identifiers conform to that scheme. Repositories may implement the oai-identifier syntax described in the accompanying Implementation Guidelines document. Unique identifiers play two roles in the protocol: Response: Identifiers are returned by both the ListIdentifiers and ListRecords requests. Request: An identifier, in combination with a metadataPrefix, is used in the GetRecord request as a means of requesting a record in a specific metadata format from an item. Note that the identifier described here is not that of a resource. The nature of a resource identifier is outside the scope of the OAI-PMH. To facilitate access to the resource associated with harvested metadata, repositories should use an element in metadata records to establish a linkage between the record (and the identifier of its item) and the identifier (URL, URN, DOI, etc.) of the associated resource. The mandatory Dublin Core format provides the identifier element that should be used for this purpose. 2.5 Record A record is metadata expressed in a single format. A record is returned in an XML-encoded byte stream in response to an OAI-PMH request for metadata from an item. A record is identified unambiguously by the combination of the unique identifier of the item from which the record is available, the metadataPrefix identifying the metadata format of the record, and the datestamp of the record. The XML-encoding of records is organized into the following parts: header -- contains the unique identifier of the item and properties necessary for selective harvesting. The header consists of the following parts: the unique identifier -- the unique identifier of an item in a repository; the datestamp -- the date of creation, modification or deletion of the record for the purpose of selective harvesting. zero or more setSpec elements -- the set membership of the item for the purpose of selective harvesting. an optional status attribute with a value of deleted indicates the withdrawal of availability of the specified metadata format for the item, dependent on the repository support for deletions. metadata -- a single manifestation of the metadata from an item. The OAI-PMH supports items with multiple manifestations (formats) of metadata. At a minimum, repositories must be able to return records with metadata expressed in the Dublin Core format, without any qualification. Optionally, a repository may also disseminate other formats of metadata. The specific metadata format of the record to be disseminated is specified by means of an argument -- the metadataPrefix -- in the GetRecord or ListRecords request that produces the record. The ListMetadataFormats request returns the list of all metadata formats available from a repository, or for a specific item (which can be specified as an argument to the ListMetadataFormats request). about -- an optional and repeatable container to hold data about the metadata part of the record. The contents of an about container must conform to an XML Schema. Individual implementation communities may create XML Schema that define specific uses for the contents of about containers. Two common uses of about containers are: rights statements: some repositories may find it desirable to attach terms of use to the metadata they make available through the OAI-PMH. No specific set of XML tags for rights expression is defined by OAI-PMH, but the about container is provided to allow for encapsulating community-defined rights tags. provenance statements: One suggested use of the about container is to indicate the provenance of a metadata record, e.g. whether it has been harvested itself and if so from which repository, and when. An XML Schema for such a provenance container, as well as some supporting information is available from the accompanying Implementation Guidelines document. The following example shows an XML-encoding of a record and its components: the header part with: a unique identifier of the item from which the record was disseminated, equal to oai:arXiv.org:cs/0112017; the datestamp of the record equal to 2002-02-28; two setSpecs, respectively cs and math, indicating that the item from which the record was disseminated belongs to two sets of the repository; the metadata part. This consists of a single root tag - in the example the tag oai_dc:dc - with the nested tags belonging to the corresponding metadata format - in the example, Dublin Core elements such as dc:title. Note that the root tag within the metadata part includes a number of attributes that are common to all XML documents that use namespaces and schema validity: namespace declarations -- the declarations of the namespaces used within the metadata part, each of which is prefixed with xmlns. Namespace declarations within the metadata part fall into two categories: metadata format specific namespace(s) - every metadata part must include one or more xmlns prefixed attributes that define the correspondence between a metadata format prefix -- e.g. dc -- and the namespace URI (as defined by the XML namespace specification ) of the respective metadata format. Some metadata formats employ tags from multiple namespaces, requiring multiple xmlns prefixed attributes -- in the example, there are declarations for both oai_dc and dc. xml schema namespace - every metadata part must include the attribute xmlns:xsi, the value of which must always be the URI shown in the example, which is the namespace URI for XML schema. xsi:schemaLocation -- the value of which is a URI, URL pair; the first is the namespace URI (as defined by the XML namespace specification) of the metadata that follows in this part, and the second is the URL of the XML schema for validation of the metadata that follows. one about part of the record which uses the oai_provenance.xsd schema, described in the accompanying Implementation Guidelines document, as a means to provide information regarding the origins of the metadata part of the record. Note that the root element within each about part has the same structure as the root element in the metadata part. <header> <identifier>oai:arXiv:cs/0112017</identifier> <datestamp>2002-02-28</datestamp> <setSpec>cs</setSpec> <setSpec>math</setSpec> </header> <metadata> <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd"> <dc:title>Using Structural Metadata to Localize Experience of Digital Content</dc:title> <dc:creator>Dushay, Naomi</dc:creator> <dc:subject>Digital Libraries</dc:subject> <dc:description>With the increasing technical sophistication of both information consumers and provider…