In March 2020, BookNet Canada’s Bibliographic Committee reviewed and endorsed a countdown calendar to help guide Canadian data providers as they transition to ONIX 3.0. The impetus for this was Amazon’s mandate for ONIX 3.0 feeds for print books by the end of calendar year 2020.
The committee endorsed August 28, 2020 as the date by which Canadian data providers should have been able to generate ONIX 3.0 records and feeds, as well as sending those to data recipients who chose to accept them. This deadline was adopted in order to shift much of the work from the busy fall selling season, and to align with the deadlines adopted by our North American partner, the Book Industry Study Group (BISG). Moreover, a deadline earlier than the one mandated by Amazon ensures plenty of lead time for data providers to confer, collaborate, adopt best practices, and test with a wider pool of data recipients, including BookNet Canada’s own bibliographic data aggregation service, BiblioShare.
Now that the deadline has passed, let’s look at where the Canadian market is in our transition, what’s left to be done, and what we expect for the coming months leading up to Amazon’s deadline. We’re going to look at stats from BiblioShare, insights gleaned from industry partners such as BISG and EDItEUR, and more.
Baby steps
We ran the numbers in BiblioShare, BookNet Canada’s quality-controlled aggregation system that collects and disseminates bibliographic data using ONIX (in both ONIX 2.1 and ONIX 3.0 formats). We’ve been collecting data for years and we celebrated some achievements in our 2019/2020 Year in Review:
We added 923,756 new ONIX 2.1 records and 84,792 new ONIX 3.0 records.
We collected data from 61 new ONIX 2.1 data providers and 15 new ONIX 3.0 data providers.
We served up data to satisfy over 65 million web service requests.
Given that our year-end fell on March 31, 2020, before Amazon’s deadline had really settled in the hearts and minds of data suppliers, we were confident that the Canadian data community would be ready to pull up their boots and get to work. With just under 8.5% of all new titles coming into BiblioShare in 2019/2020 using 3.0, we knew a lot of work needed to be done.
Running the numbers at the end of August, the number of 3.0 files in our BiblioShare database has ticked up: currently, 3.0 records make up just under 10% of 2.1 records, reaching approximately 300,000 total 3.0 records. There are two major factors influencing the status:
most of the large data suppliers in Canada (namely distributors handling multiple publishers' data as well as the large multinational firms) are working on timelines that match Amazon’s year-end deadline, not the Canadian Bibliographic Committee’s deadline of August 28, 2020; and
most data suppliers aren't going to invest resources into mapping non-active (ex. out of print) titles to 3.0, so the volume of titles we’d expect to see in 3.0 is limited to active and forthcoming releases (at least at the start!).
These stats are supported by data uncovered as part of BookNet Canada’s biannual survey of publishers, The State of Publishing in Canada 2019, conducted in Q1 2020. In those results, 100% of mid-sized publishers (revenues of $1-$10M) said they had already or had planned to make the transition in 2020 where only 59% of large publishers (>$10M) could say the same.
Good news
We surveyed data suppliers (if you haven’t responded to our survey yet, you can find it here) to learn more about our market’s ability to get ONIX 3.0 ready by the Canadian Bibliographic Committee’s deadline of August 28, 2020.
All respondents to our survey confirmed their expectations that they would be able to send a full ONIX 3.0 feed by the end of 2020, in time to meet Amazon’s stated deadline. Moreover, to our great satisfaction, respondents confirmed that they’ll be able to, and have a plan to, support the following enhanced 3.0 data points:
full market statements for the Canadian market
market-specific publication dates
market-specific publication statuses
meaningful supplier role codes
improved metadata support for digital books
contributor information with city level data
Pain points?
Looking across the English-language industry, suppliers making the transition report difficulties mastering two key components that make ONIX 3.0 markedly different from 2.1: communicating information about collections (think, in 2.1 terms, sets & series) and markets. This is where the “conversion” mindset comes into play. Recall BookNet’s Bibliographic Manager Tom Richardson’s post “Of metadata and mapping and why you cannot convert ONIX 2.1 to 3.0” for a succinct explanation of why you must tackle the 3.0 transition with an eye to mapping, not conversion.
So, collections and markets … where to begin? For a crash course in collections, stay tuned to this blog for a series by Tom Richardson about the topic. And to get started with markets, don’t miss this essential webinar on territories and markets.
Planning for 2021
Our survey of data providers yielded an additional, worrisome insight: Two thirds of respondents responded “yes” or “I don’t know” to our question asking if they would be able to maintain dual ONIX 3.0 / 2.1 feeds past 2020. (BookNet Canada expects data suppliers will need to support ONIX 2.1 through 2021.) Transitioning to 3.0 should be seen as a journey with stops along the way, not a “hot swap” whereby one’s 2.1 feed is forever replaced by a 3.0 feed.
To repeat: BookNet Canada expects all data suppliers will need to support many of their existing trading partners by continuing to create and send ONIX 2.1 feeds through 2021.
Fail to plan or plan to fail. Your choice.
Talk to your suppliers
Responses from smaller publishers revealed that many rely on their software providers and distributors to manage their ONIX feeds, and their ability to transition to 3.0 is effectively out of their hands:
Our distributor says they're working to accept ONIX 3.0 by the end of the year, not August 28.
Our software provider has not yet produced an ONIX 3.0 export. We need to follow up and make sure it's something they can do.
We heard similar things from our survey of publishers in Q1 2020:
Waiting for my programmer to have the time. I have a custom database that exports ONIX.
Our ebook metadata is in ONIX 3, but our print titles are still in ONIX 2.1. We're distributed by [redacted], so we won't make the switch to ONIX 3 until they tell us to.
We don't handle our own metadata — distributor does.
We need our data management software developer to update for us. It's coming soon we hope.
Our distributor manages our data.
Waiting for provider to upgrade.
We cannot put it lightly: talking to your data provider(s) is the first step you should take in your plan to transition to 3.0. We ranked it #1 on our checklist for Canadian data suppliers looking to make the transition. We noted:
1. Speak to your ONIX solution provider for your print titles (can be either external or in-house). Share the August 28, 2020 deadline with them and ask whether your systems are 3.0 ready. If not, when will you be able to produce 3.0 files for your print titles?
Key questions:
- Does your provider understand the differences between ONIX 2.1 and 3.0?
- Does your system do validation on the data/files it produces?
- Which version of ONIX does your system support? (The current version, as of April 2020, is 3.0.7.) Tip 1: Check which ONIX version your system uses
- Which version of the codes lists are being used and how often are they updated? (The current version of the list, as of July 2020, is Issue 50.) Tip 2: Check if your system’s code lists are updated
- Does your system support "production feeds"?
- Does your system have the ability to send updates only (a.k.a. deltas) as well as full files?
- Will your system be able to support both 2.1 and 3.0 feeds by the deadline? Note: The expectation is that data providers will need to support both 2.1 and 3.0 feeds for an indeterminate period of time, until all data recipients have transitioned to 3.0 feeds.
So: go! Pick up the phone, fire up Zoom, do whatever you need to do to keep the lines of communication open between you and your metadata creators! Metadata is marketing, your marketing. Own it!
Looking ahead
Are you already creating ONIX 3.0 files and sending them to Amazon? Great! Pat yourself on the back!
Are you one of the nearly 100 publishers already creating 3.0 files AND sending a full 3.0 feed to BNC BiblioShare? No? Publisher, 'splain yourself.
No really, we promise.
The Canadian Bibliographic Committee’s August 28 deadline has passed. It’s now time to look forward to year-end and assess your ONIX 3.0 readiness. As noted above, transitioning to 3.0 should be seen as a journey with stops along the way.
If you’re still in the midst of transitioning to ONIX 3.0, you’re not alone. If you’re struggling, have questions, or just need a friendly ear, don’t hesitate to email us. We’d like to hear from you.
If you’re hitting a wall and are able to document your pain points, BookNet Canada’s sister organization in the US is collecting them in a Google doc here.
And finally, BookNet is Canadian publishing’s standards organization. We are your standards organization. We’re here to help. Transitioning to ONIX 3.0 is not a small undertaking. How can we help you best?
Who crossed the finish line?
These publishers did it. They reached the 3.0 promise land and have successfully set up fields to BNC BiblioShare. A big congratulations to you all!
Annick Press, Anvil Press, Arbeiter Ring Publishing, Athabasca University Press AU Press, Banff Centre Press, Between the Lines, Book*hug Press, Books We Love, Boulder Books, Breakwater Books, Ltd, Brick Books, Broadview Press, Brush Enterprises Inc, Caitlin Press, CAMH The Centre for Addiction and Mental Health - Publication Services, Cape Breton University Press, Child's Play, Chouette Publishing, Coach House Press, Compass Maps (PopOut Maps), Cook By Colours Inc, Cormorant Books, Coteau Books, Crow Cottage Publishing, Demeter Press, Dundurn, Elsevier Science & Technology Books, Engage Books, Flanker Press, Freehand Books, Goose Lane Editions, House of Anansi, Iguana Books, Inanna Publications, Infinite Ideas, Ingram Content Group, Inhabit Media Inc, Invisible Publishing, Irwin Law Inc., ISER Books, Joseph Norman Editions PD Meany Publishers, Legend Press, Linda Leith Publishing, Mansfield Press, McGill-Queens University Press, Memoirs Publishing, NeWest Press, Oolichan Books, Owl Kids, Pajama Press, Panoma Press, Pembroke Publishing Ltd., Portage & Main Press, Ronsdale Press, Scholastic Ebooks, Second Story Press, Signature Editions, Star Crossed Press, Talonbooks, The MIT Press, Thistledown Press, Thomas Allen, Three O'Clock Press, Turnstone Press, Tyche Books, UBC Press, United Church Publishing House UCPH, University of Alberta Press, University of Calgary Press, University of Manitoba Press, University of Ottawa Press, Vehicule Press, Whitecap Books Ltd., Wordcatcher Publishing Group, and Wordsworth Editions.
Not on this list?
Bow down.
This month’s podcast episode is a chat about the ACP’s salary survey, pay equity, inclusion, and more.