This is fourth in a series of blog posts highlighting where you need to update your ONIX 2.1 data to make it compatible with the needs and intent of ONIX 3.0, and the second installment, after one on Sales Rights, dealing with Territory statements. If you haven't already, I would encourage you to read the introduction to this series, which looks at why it's important to modify your data, as well as the first installment on Collection for its considerations for conversion.
Future proofing
ONIX has not been changing for the sake of change. Differences in the metadata for ONIX 3.0 are there to solve problems in various supply chains. Broadly the issue in Product Supply is improving support for international metadata and the previous post on Sales Rights looked at some of the common practices that prevent retailers from being able to easily merge or select information from multiple markets. It made the point that Sales Rights are less ambiguous if they reference the <World> and that requires Product Supply to define its territories to support Sales Rights. Publisher sales rights, supplier territories, and where a specific price and currency apply are all covered by different contractual obligations. They need to be in place in order to support end users' ability to accept files from multiple markets.
This post assumes you want to communicate territorial information in ONIX 3.0's Product Supply (aka "Block 6") and there are some straightforward but major changes in how this section is structured. Beyond supporting clarity in the supply chain, the changes are there to support the new "block-based" file exchange that's part of the standard. ONIX, and for that matter XML, are only standards. Someone has to implement them for anything to happen – and as we know, English-language use of ONIX 3.0 outside of the digital supply chain, particularly within North America, is often more discussed than implemented. Block-based file transfer has, as far as I know, never been implemented in a production feed. Nevertheless, I expect to see it in use as soon as a major publisher starts getting serious about their Three Oh feed, so here's a quick look at it:
There is an underlying assumption made in trading ONIX records: records are fully replaced every time they are submitted. (A record is defined as one "product" record, i.e., a <Product> to </Product> tag in an ONIX file, and normally describes a single ISBN.) If the sender (normally the publisher) replaces the main description with a new one, the end user (normally a retailer) should follow suit. If the publisher adds a review, or fixes a typo, end users load that too. Full replacement solves the issue typical in data exchange when information is dropped. Overwriting is easy; it's matching a removal that is hard in databases. A typical example might be that the third author is dropped from a book after the initial pre-publication record has been issued: the end user replaces their current record with the incoming one and the remaining authors become the only ones associated with the record. Before, you would have to call someone to get that author pulled.
There is an implicit trust built into the ONIX standard: senders ensure their information is accurate and take responsibility that no important piece of metadata will be lost through error while end users show faith and rely on the accuracy of the incoming data feed. Of course, the reality has some mistrust; not every retailer or end user follows replacement theory absolutely (we do in BiblioShare), but they are all aware that this is the expectation within the standard.
ONIX 3.0 takes that simple concept and expands it by creating an "identification" unit that's always traded with every ONIX record, but splits the rest of the product record into six "blocks" that can be traded as distinct units. If a "block" update is sent (and there's a Notification Code on the record that must be used to define it) then only the block(s) that are sent are updated and the balance of the record (the unsent blocks) held by the retailer are left untouched. The basic rule remains the same: if you send any part of a block in a block update, that entire block should be fully replaced. It has to be that way so that publishers can remove data from other systems as well as update existing information.
There is an excellent explanation of block updates in EDItEUR's Best Practices in the "Organization of Data Delivery" section and it can answer all your questions on it. Product Supply was redesigned in ONIX 3.0, in part, to support effective Block updates.
Markets
Block 6 (aka Product Supply) contains two components in three sections (two describing Markets and one for the Supply Detail). Block 6 is unique among the ONIX 3.0 blocks in that it can repeat, and it does that in order to allow a record to carry information on multiple markets. Support for "Markets" as distinct sections is a new feature of ONIX 3.0, and like most things in 3.0 using markets as a starting point simplifies data presentation and provides more choice. "Markets" defines when you need to repeat Product Supply.
Supporting markets doesn't mean anything has changed or that your Product Supply has to be more complex, but you need to include Market definitions in your file
You don't need to set up a distinct market for each supplier. The supply detail repeats within each Product Supply Block 6 so you can easily set up multiple suppliers for one market.
What's a Market in ONIX 3.0? There are a lot of values applied to a product that might change in different territories, including when the product might be published. I love The Economist book reviews but I often notice books published "over there" that aren't available for at least another six months "here." That's a different market if the product and ISBN are being sold by the same company. More often than not it's a rights sale, but the industry is changing and publishers are starting to carry world rights on the books so their ISBN goes further than one market. Multinational companies are increasingly providing products sourced from different divisions across multiple markets. Take a look at ONIX 3.0 manual's P.24 and P.25 Market Section and note all the things you can provide: sales representation and restrictions, print runs, book clubs, and most importantly, a full set of dates and status.
Now you know why you might want to use "block updates" with Product Supply; this is a section that you will likely need to send and update regularly as a unit. ONIX 3.0 still supports universal values at the product level. The Product still has a "publication date" (and other values) and that should be the one "of record" used by librarians and historians to note the very first time this ISBN was made available. But if you want to provide international support for retailers you have the tools to support your product appropriately by territory.
What you distribute within a market – i.e., files you only send locally – arguably only need the Supply Detail section from Product Supply, but international retailers and aggregators would receive ONIX feeds that support multiple Product Supply sections. That's why you must include Market information in the local files. Companies getting data from multiple sources will expect to find the information inside the record so they can "select" the right information for each market. Identification of the all markets, including those where you and your trading partners are both within a market, is important, even if the information seems obvious. It's not obvious inside the database or to the programming that selects the right information.
Supply Detail
Not much has changed in the Supply Detail, or rather the changes that are there exist to support pretty specific use cases. ONIX has been steadily expanding, working ahead to ensure that you can support subscription models and other types of special pricing, for example. The supply detail exists to support information about who is supplying the product and its price. If there's something you need to say that someone else in the supply chain needs to know, it can probably support it. Ask us if you're unsure. In terms of what I see actually used today in BiblioShare, there's not a lot I can highlight. Your trading partners will ask for what they need, I expect.
One thing I'd like to point out, because I see a lot of files in BiblioShare still supporting it, is that there is no longer any provision for using the Availability Codelist 54 in ONIX 3.0. You need to start providing information using Product Availability Codelist 65, which is designed to work in tandem with Publishing Status Codelist 64. This switch was part of the transition from ONIX 2.0 to 2.1 and has been part of ONIX for over a decade, and just one more example of improved clarity.
Considerations for Conversion
Within the Supply Detail, most ONIX 2.1 "print" files I see could support, but don't provide, territory statements for their suppliers, nor do they provide a clear definition of where the price applies. You should start providing both, and can do so now regardless of supporting ONIX 2.1 or 3.0. Markets is a "new" level of information but is information that must known by the sender. It's mostly a question of how complex your business needs are, and having systems that can export that level of detail if they're complex. There's one big caveat based on what I've seen in BiblioShare data: if you are able to add this new data as default values, you have to monitor it for change!
Default data changes too. For example, one aggregator created a list of dead people still submitting ONIX data to them; literally default values from the header information that the companies sending had either had no idea how to change or had forgotten existed. Markets is not information to be forgotten and default values for "Markets" can't be handled that way. Markets represent your contractual and business obligations.
The point that I'd like to emphasize is that end users will need at least some basic data if they are going to start combining data from multiple sources. I would expect that part of the transition to ONIX 3.0 will include higher demands for detail by international retailers. Aren't you already providing that for your digital products? There you go! Most companies already have a starting point for this development.
Resources
Previous post on Sales Rights has an explanation of Territory Statements
ONIX for Books Territories and Markets II.pdf
Graham Bell’s webinar on transitioning from ONIX 2.1 to 3.0
BISG_Best_Practices_for_Product_Metadata_6.1.15.pdf - The new version published in June 2015 is fully updated to support ONIX 3.0.
The full ONIX Manual and EDItEUR’s International Best Practices
