Home
Blog
Overview of all products
SalesData
LibraryData
CataList
Loan Stars
BiblioShare
Webform
EDI
Products for publishers
Products for retailers
Products for libraries
Information for authors
BNC Research
Canadian literary awards
SalesData & LibraryData Research Portal
Events
Tech Forum
Webinars & Training
Code of Conduct
Standards
EDI standards
Product identifiers
Classification schemes
ONIX standards
About
Contact us
Media
Bestseller lists
Newsletters
Podcast
Jobs
SalesData
LibraryData
CataList
BiblioShare
Webform
EDI

BookNet Canada

Home
Blog
Overview of all products
SalesData
LibraryData
CataList
Loan Stars
BiblioShare
Webform
EDI
Products for publishers
Products for retailers
Products for libraries
Information for authors
BNC Research
Canadian literary awards
SalesData & LibraryData Research Portal
Events
Tech Forum
Webinars & Training
Code of Conduct
Standards
EDI standards
Product identifiers
Classification schemes
ONIX standards
About
Contact us
Media
Bestseller lists
Newsletters
Podcast
Jobs
SalesData
LibraryData
CataList
BiblioShare
Webform
EDI
Tom Richardson
July 28, 2022
ONIX, Standards & Metadata

Is Canada a market? An exploration of what is "good ONIX"

Tom Richardson
July 28, 2022
ONIX, Standards & Metadata
.@BookNet_Canada's Bibliographic Manager, Tom Richardson, answers the question: Is Canada a Market?
CLICK TO TWEET

This blog post is based on a couple of things. The first was correspondence about a problem a retailer contacted us about: Their new in-house ONIX 3.0 metadata system had made an ONIX specification-based assumption and they didn’t understand why a composite, Block 6’s “Market Date”, wasn’t typically populated.

The second thing comes from committee work where publishers and retailers advocate for using fully implemented, good-quality ONIX while arguing against or being surprised by the functional expectations of ONIX 3.0.

I want to be clear, this isn’t a one is right and the other is wrong scenario. It costs money to do metadata well and not everything can be covered. Choices need to be made. But here’s my point: Those choices shouldn’t be led by our experience using ONIX 2.1. Developers tend to repeat what they know (ONIX 2.1) when ONIX 3.0 offers simpler and better solutions in a different place.

We (those of us who make decisions about metadata) need to instruct our developers on where they should be mapping and why. We need to do that because “good ONIX” depends on implementing the system, the metadata system that EDItEUR has spent more than 20 years developing and that the ONIX standard represents.

ONIX is not complex, but what ONIX supports — books and a publishing supply chain where a huge number of vastly diverse products are sold in relatively small numbers — is complex. Therefore, the metadata system needs to support complexity.

Going back to the first thing that led me to write this blog post, the retailer’s design assumptions were right, sensible, and true to the ONIX system. What my committee colleagues discuss is data as it’s used. That cannot be wrong because data-that’s-being-accepted-in-the-supply-chain can never be “wrong” however not-right it might be.

The difference between those two things is what defines “good ONIX”, and “Market Date” is a nice example. It’s nice because what you supported in ONIX 2.1 can be used in ONIX 3.0, almost without change as data — it’s the ONIX structure that’s different. ONIX 3.0 allows the data to be simpler with less repetition and usable with greater accuracy. And there’s an educational bonus to it: It forces you to consider what is a market relative to your book selling needs.

Unfortunately, the data-that’s-being-accepted-in-the-supply-chain is modelled on ONIX 2.1 practices that, frankly, were not following EDItEUR’s system very well even for that version. When such practices are implemented in ONIX 3.0 they might not be wrong, but they definitely cannot be described as good.

EDItEUR’s ONIX Specifications and Best Practices

I’m going to start by highlighting and providing my annotations on what’s said in the Specifications and Best Practices as published by EDItEUR to show why a company would expect to find a “Market Date” composite in ONIX 3.0 metadata.

An ONIX record SHOULD contain at least one “Product Supply” composite.

This means that it's not mandatory that an ONIX record contains a “Product Supply” but, a book record without supply information is functionally useless to support book sales. EDItEUR designs ONIX to support a number of use cases and there are atypical ones that don’t involve supplying a product to a retailer, making the “Product Supply” statement unnecessary.

When EDItEUR says:

  • “SHOULD contain”, they mean always, except for an atypical metadata exchange, and they mean it in the context of the structure they’re describing. There are many “expected composites” that aren't mandatory in all ONIX records.

  • “MUST contain”, they mean mandatory in all cases, or within the context they’re describing it has to be there.

Having clarified our terms let’s continue down EDItEUR’s path using them:

  • A “Product Supply” composite SHOULD contain: the <Market> composite.

    • A “Market” composite MUST contain a “Territory” composite. (Ergo: A “Market” is a defined territory.)

    • EDItEUR provides a statement about when you can plausibly not provide a “Market” composite: ”<Market> may be omitted when the product is available in only a single market since the market would be identical to the <SalesRights>.” This use case implies a simplicity in “supply” that I don’t think applies to many physical books.

  • A “Product Supply” composite SHOULD contain: the <MarketPublishingDetail> composite.

    • A “Market Publishing Detail” composite MUST contain a “Market Publishing Status” — you need to define the publisher’s relationship to the book in the described territory.

    • A “Market Publishing Detail” composite SHOULD contain a “Market Date” composite. Market-specific dates are needed but there are publishing statuses like "Cancelled" or "Postpone indefinitely" where a date is meaningless and should not be provided.

    • You don't have to supply a "Market Publishing Detail" when the product is available only in a single market, but, again, I would say that print ISBNs are seldom sold in a single market.

  • A “Product Supply” composite MUST contain: the <SupplyDetail> composite with pricing details for the product in the market.

    • While a single exclusive supplier is typical of North American book metadata, the ONIX system can support multiple suppliers with different "supplier roles" where each supplier is able to find pricing specific to different parts of the market — which is of course defined by the “Product Supply”.

    • What’s important to note is that a “Product Supply” statement is designed to facilitate reporting on multiple suppliers. Even if you’re supporting a single supplier you should understand that the system can support more and respect its design needs.

I won’t carry on through the “Supply Detail” — a few points will come up later — but let’s loop back to “Product Supply”. It should be there and Block 6 is a special block in ONIX 3.0 because the block “Product Supply” can repeat (all other blocks do not repeat).

The reason Block 6 can repeat is that one “Product Supply” composite represents all the supply information for the market being described. The “Market” is a geographical description of that: it’s where all the “Product Supply” information is functionally the same. You can’t separate a “Product Supply” statement from its “Market Statement”, even if the record omits a “Market” statement, it’s omitted because as a “single market” other data in the ONIX record replaces it. The territory could be World or it could be a province but it usually is defined by a country or groups of countries where the publisher has a business relationship with trading partners whose “where” defines the market and the contracts it implies.

The simple case would be an exclusive supplier in one territory and a different exclusive supplier in another territory. Those would be mutually exclusive distinct markets that should be provided in different “Product Supply” composites and defined status and dates can be conveniently and easily defined. Even without “exclusive” supply roles it’s possible that the details and dates will be different between territories. For example, ONIX offers support for information down to the level of Publisher Representative and contacts so you might need to define a territory based on that instead of the supplier.

A “Product Supply” can easily hold multiple “Supply Details” each defined by their unique role. It’s normal in North American metadata to only list the typically “exclusive” supplier but EDItEUR has designed the “Product Supply” to allow for the possibility of providing data from the wholesalers they supply. It’s what you need to and choose to trade as metadata that defines what a market is. Each market is a different “Product Supply” statement.

Stop and think about that for a second: ONIX 3.0 offers a simpler way for you to focus on the business-based information you need to supply by territory.

I hear the screams: We can’t change! It costs us! I know that most companies supply market-driven files because BiblioShare receives files made to support the Canadian market. Here I’ll just note that you don’t need to distribute every “Product Supply” to every trading partner. You send them the one(s) they need for the ISBN-based records for the market(s) they serve. You could simplify your metadata exchange with not much development work to use “Product Supply” fully. Simplicity will provide a return on investment long term.

All of this is normal in good ONIX.

A “Product Supply” composite should contain a “Market” composite that must contain a “Territory” statement. It should define the publisher’s relationship to the product in that market so that data must contain a defining status and should contain market-specific dates. Block 6 must contain a supplier with a defined role relative to the market and an availability status so a retailer in the market area knows who to buy from and if it can be supplied. That “Supply Detail” must contain some form of price (price can include an unpriced item code) and should contain other things that we won’t cover here today.

What is good ONIX?

An ONIX file has to be valid as an XML document. That means that its tag structure has to be readable and sound, codes need to match their code lists, and for any part of an ONIX record that you use all of the embedded tags MUST include data points. Missing or mismatched bits violate the XML defining document, a.k.a. the schema, which is where the business rules of the ONIX “system” are defined. To say that an ONIX file is valid means it’s valid as an XML document — if your file is compared to the schema’s rules using XML software, it “passes” the rules set by EDItEUR. This represents the minimum, the least you can do and both EDItEUR and BookNet Canada recommend that everyone knows how to validate their ONIX files (or know that your software does it).

In addition to being valid as XML, “good ONIX” means that if you choose to communicate about something in ONIX you SHOULD include the supporting “should contain” data points. The exact ONIX record needed to support any given ISBN-based product will vary by what you need to describe.

Before this is taken too dogmatically it would be true to say some “should contain” statements within ONIX’s structure are more important than others, but I have to emphasize that supporting “Markets” and a “Market Date” is basic in supporting “good ONIX”. The reason I say that is because a “Product Supply” statement is required to communicate basic information needed to sell books. Any “should contain” statement in ONIX’s business rules associated with basic information is similarly basic enough to be fundamental to “good ONIX”.

The retailer’s assumption that “Market Date” is an “expected composite” in ONIX 3.0 metadata is sound and based on ONIX best practices. Before I move to why this is better and more like how we handle data in North America, let’s consider why the retailer contacted BookNet Canada: The expected data was missing.

Reality check on data-that’s-being-accepted-in-the-supply-chain: ONIX 3.0 numbers delivered to BiblioShare

When looking at the ONIX 3.0 records delivered to BiblioShare, I found that:

  • Out of over 630,000 ONIX 3.0 records, all carry at least one Block 6 “Product Supply”

  • 93% carried a single Block 6 and the balance carried two to five Block 6 entries

  • <45% of the Block 6 entries carried a “Market” composite:

    • Virtually all of the records with multiple Block 6 entries carried a “Market” composite

    • >47% of the records with a single Block 6 carried a “Market” composite

  • <11% of the records carried a “Market Publishing Detail” composite

  • <10% of the records carried a “Market Date”:

    • Virtually all dates supplied are “Market Publication Date” and I did not compare that value to the “Global Publication Date” found in Block 4

    • 16 records carry a “Sales Embargo Date” as a “Market Date” entry

  • All of Block 6 carry one or more “Supply Details” and just over 50% of the records carry a “Supply Date”

    • >37% — more than 165,000 dates were a Sales Embargo Date as a Supplier Date entry

I’ve made a spreadsheet available here where I share in a lot more detail the counts of ONIX 3.0 records BiblioShare has received. If you feel I’m leading you to a conclusion (I hope I am!) look for additional support for (or against) it in the weeds I’ve provided.

A quick note about the data: the counts were made against file-based metadata, so if an ISBN was supported by two (or more) accounts, each account represents one record. That means the counts are not “unique by ISBN” but are “unique by ONIX record”. The amount of ISBN duplication is under 3% and many if not most ONIX records represent a unique representation and the data practices of its individual sender.

I cannot think of a better example of a piece of “market metadata” than a Sales Embargo Date (On-Sale Date). Its purpose is to ensure that within some territories all retailers have an equal opportunity for sales by sharing the same start date. How do we reconcile that with these numbers?

  • 16 records carry a “Sales Embargo Date” as a “Market Date” entry

  • >37% — more than 165,000 dates were a “Sales Embargo Date” as a “Supplier Date” entry

The reason companies are using a “Supplier Date” entry is because this date was contained within the “Supply Detail” in ONIX 2.1. Companies are not recognizing — or taking advantage of — the simplicity offered by the changes EDItEUR made in ONIX 3.0. The IT department repeats what it knows and, because no one asked them to do it differently, that becomes the data that arrives but not the ONIX that is good.

Isn’t “Supply Date” an option? It is an option. As noted above, EDItEUR has identified the simple case of a single market as a reason to not supply market-related details and if the “global” details are identical to the market details then this is just a way to simplify the metadata. Digital products would often meet that criteria and our typical practice of naming the publisher as the supplier would support a great example where both the “Sales Rights” and “Supply” statements are “publisher based.” If you’re naming other companies as the supplier ask yourself:

Does the publisher or the company(ies) named as “Supplier(s)” set the sales embargo date (On-sale date)?

In the North American market, the normal practice is that the publisher determines when retail sales start. The case for using the “Market Publishing Detail” in “good ONIX” is now pretty obvious.

Some data senders create ONIX files that have been manipulated to make the records a single market representation. I can find ISBNs in BiblioShare whose “Sales Rights” are specified as “Not for Sale” outside Canada but a quick Google search confirms the ISBN is available outside of Canada. I think companies doing this are wasting resources. It’s actually simpler to use “Product Supply” properly as your market divisions than to set up whole feeds that misrepresent your data. How do these files work for you when a retailer opens a store in another market? I’d think adding an existing “Product Supply” statement to a feed is easier than creating a third manipulated ONIX joint market feed for that retailer.

I even would suggest that if you don’t support the metadata feeds worldwide (other companies support the ISBN in Europe for example) it would still be more accurate to provide your home market details in a “Product Supply”. It reduces the potential for conflict should a retailer get both feeds.

The bottom line: “Supply Date” remains an option and is needed for supplier-specific dates like “Last date for returns.” As for “Sales Embargo Date”, it’s hard to call it “good ONIX” when used as a part of “Supplier Detail” assigned to a different company than the publisher but retailers would be wise to be prepared for its use there.

Is Canada a Market?

Canadian retailers have had complaints about the lack of actionable business dates in ONIX feeds for a long time. They don’t get the product when the dates say they should. Their buyers' plans are not managed as efficiently as they need. I cannot help but think that the US supply chain, with its high density, supporting multiple distribution locations represents a fundamentally different “market” than the lower density, long-distance one that exists in Canada.

Canadian publishers who support different dates for the Canadian market from the US complain that companies selling into Canada ignore the Canadian date and use the US date in Canada. This works to US companies' disadvantage given that the Canadian date for a Canadian-originated product is likely sooner than the one used in the US, but it still costs the publisher sales by mismatching on the publicity push. I cannot help but think that the ONIX 2.1 norm of sloppily using “Sales Rights” as proxy statements for distribution rights and the tendency by both US and Canadian ONIX creators to stuff their metadata together as a single market for convenience contributes. The fact that half of the ONIX 3.0 data in BiblioShare doesn’t even support a “Market” statement and less than 10% supports dates shows the same sloppiness carrying on.

The strange thing is that the sloppy data practices typical of ONIX 2.1 in North American metadata make for pretty good ONIX if, as EDItEUR intends, it’s mapped to “Product Supply” with “Market” details.

  • “Market Territory” statements are “simple”, positive statements of the territory and possibly a good match to the proxy for distributor statements often found in “Sales Rights”.

  • As noted above, foreign data senders make a substantial effort to “make” Canadian metadata out of their dataset as special file outputs. It would be better if that effort was put into supporting “Product Supply” statements as a solution for working in multiple markets. BookNet is in a good position to see the time spent by Canadian sales reps and branch publishers fixing foreign data for our market and that effort seems needed no matter which foreign market holds the source publisher. It should be easier to integrate that into updating a market specific Product Supply held by the creator than being overlayed after the fact.

I can only say that a Canadian retailer designed their system to get actionable business metadata from the appropriate ONIX data point; and that the data-that’s-being-accepted-in-the-supply-chain wasn’t provided there. If the data-that’s-being-accepted-in-the-supply-chain meets the criteria of being actionable business metadata isn’t easily determined outside of a business, but the care taken by the retailer to look to the “Market Date” points to their unmet need.

You can’t separate a “Market Territory” from a working “Product Supply” statement any more than you can understand “good ONIX” outside of accurate business communication.

Subscribe

Don’t miss any new blog posts. Sign up for our weekly eNews to receive updates.

You can unsubscribe at any time. We respect your privacy.

Thank you!
Recent posts
European Union Deforestation Regulation: Working Towards Compliance
European Union Deforestation Regulation: Working Towards Compliance

BISG’s essential webinar to update supply chain participants on European Union Deforestation Regulation (EUDR) compliance strategies.

Read More →
Subject spotlight: Young Adult Fantasy
Subject spotlight: Young Adult Fantasy

We’re looking at Young Adult Fantasy titles. Let’s see how these titles performed during the first quarter of 2025.

Read More →
Canadians and rising book prices 2024
Canadians and rising book prices 2024

While 90% of Canadians who bought new books looked for sales, promotions, and coupons when they shop for books, most of them paid full price for the books they purchased in 2024, at 60%.

Read More →
 Podcast: Beyond the survey: More on salary equity in the book industry
Podcast: Beyond the survey: More on salary equity in the book industry

This month’s podcast episode is a chat about the ACP’s salary survey, pay equity, inclusion, and more.

Read More →
Canadians and their reading in 2024
Canadians and their reading in 2024

Every year we ask Canadian readers about their reading habits, here you can also find some of the top-level highlights.

Read More →
Amazon is ending support of ONIX 2.1 March 2026
Amazon is ending support of ONIX 2.1 March 2026

Confirmed end date for ONIX 2.1 in the global supply chain.

Read More →
Canadian book borrowers in 2024
Canadian book borrowers in 2024

Insights into the behaviour of Canadian book borrowers.

Read More →
Standards goals for 2025: A recap and a conversation about what may be next
Standards goals for 2025: A recap and a conversation about what may be next

Book supply chain standards are changing rapidly, let us help identify which recent updates are relevant to you.

Read More →
May 2025 Loan Stars Junior Canadian top picks
May 2025 Loan Stars Junior Canadian top picks

Find out what titles made it to the May 2025 Loan Stars Junior Canadian list.

Read More →
Canadian book buyers in 2024
Canadian book buyers in 2024

Insights into the behaviour of Canadian book buyers.

Read More →
Common metadata issues and how to fix them: Forgetting to include related products in your metadata
Common metadata issues and how to fix them: Forgetting to include related products in your metadata

Tips on including related products in your metadata.

Read More →
Podcast: Canadian bookmark project
Podcast: Canadian bookmark project

This month we’re talking with Chandler Jolliffe, owner of Cedar Canoe Books in Huntsville.

Read More →

Tagged: onix 3.0, book metadata best practices

Newer PostLoan Stars Adult lists roundup: Winter and Spring 2022
Older PostCanadian readers and the environment
Blog RSS

The Canadian Book Market 2024 is the comprehensive guide to the Canadian market with in-depth category data.

Get your copy now

Listen to our latest podcast episode


  • Research & Analysis 449
  • Ebooks 304
  • Tech Forum 266
  • Conferences & Events 261
  • Standards & Metadata 228
  • Bookselling 218
  • Publishing 194
  • ONIX 178
  • Marketing 152
  • Podcasts 118
  • ebookcraft 112
  • BookNet News 99
  • Loan Stars 71
  • Libraries 66
  • BiblioShare 59
  • SalesData 51
  • 5 Questions With 48
  • CataList 42
  • Thema 42
  • Awards 30
  • Diversity & Inclusion 21
  • Publishing & COVID-19 18
  • Sustainability 11
  • EU Regulations 9
  • LibraryData 9
  • ISNI 4

 

 

BookNet Canada is a non-profit organization that develops technology, standards, and education to serve the Canadian book industry. Founded in 2002 to address systemic challenges in the industry, BookNet Canada supports publishing companies, booksellers, wholesalers, distributors, sales agents, industry associations, literary agents, media, and libraries across the country.

 

Privacy Policy | Accessibility Policy | About Us

BOOKNET CANADA

Contact us | (416) 362-5057 or toll free 1 (877) 770-5261

We acknowledge the financial support of the Government of Canada through the Canada Book Fund (CBF) for this project.

Back to Top

BookNet Canada acknowledges that its operations are remote and our colleagues contribute their work from the traditional territories of the Mississaugas of the Credit First Nation, the Anishnawbe, the Haudenosaunee, the Wyandot, the Mi’kmaq, the Ojibwa of Fort William First Nation, the Three Fires Confederacy of First Nations (which includes the Ojibwa, the Odawa, and the Potawatomie), and the Métis, the original nations and peoples of the lands we now call Beeton, Brampton, Guelph, Halifax, Thunder Bay, Toronto, Vaughan, and Windsor. We endorse the Calls to Action from the Truth and Reconciliation Commission of Canada (PDF) and support an ongoing shift from gatekeeping to spacemaking in the book industry.