Sunday, June 23, 2013

ebooks in the US

What follows is the draft of an article I wrote for library colleagues in Bulgaria. There's not much in it that's new; but it's an attempt to provide a more comprehensive narrative of how public libraries got to where we are regarding ebooks, and what Douglas County Libraries has attempted to do about it.

eBooks - an introduction

For decades, librarians in the United States of America predicted the arrival of the digital book. Finally, around 2009, the Amazon Kindle, followed by the Sony ebook Reader, became popular Christmas gifts. They were soon followed by the Barnes and Noble Nook. By 2011-12, Apple's iPad became the most stylish option, followed by a host of Android-based tablets.

The early adopters of this technology tended to be well-educated, upper middle class, and middle-aged. By the end of 2012, about 20% of library patrons had, and preferred, ebooks to paper books. Again by the end of 2012, roughly 50% of the sales of all popular adult fiction were electronic.

Why? Ebooks were convenient. They could be purchased instantly, at any day or time. When traveling, readers could load up their slim, light-weight devices with literally dozens of books and magazines. Ebooks could be set to display type large enough for aging eyes to read easily. And ebooks were, at least at first, somewhat cheaper than paper books.

First wave

This first wave of ebooks was made up, largely, of the titles of the so-called Big Six publishers. These publishers -- Random House, HarperCollins, Penguin, Hachette, Simon and Schuster, and Macmillan -- account for at least 50% of the books purchased by USA public libraries, and over 60% of library use. These are the "bestsellers," the most popular books based on consumer purchases.

When library patrons showed up asking for our assistance with their devices and to provide digital bestsellers, few libraries were prepared. Many librarians were not familiar with the devices, or the idiosyncrasies of their operation. Even if they were, libraries had not yet created processes to acquire or deliver digital books.

Fortunately, a company called OverDrive had put together an arrangement that did allow libraries to buy books from the big publishers, and to check them out to library patrons. Quickly, a majority of libraries signed up for this service.

But over the next year or so, the OverDrive model began to worry many of us. Three business problems became clear.

First, we had sacrificed ownership. Originally, OverDrive's contract for service stated that libraries owned the titles they bought, which, if we severed our relationship with OverDrive, could be transferred with publisher permission to another host. A few years later, OverDrive quietly changed its contracts. The libraries no longer "bought" the titles; we simply leased access to them. This was much like our leasing of periodical databases, which most libraries agreed to a generation ago. But it meant that if we left OverDrive (or if OverDrive were to go out of business), we also sacrificed everything the library had “bought.”

Moreover, many of the Big Six would not sell ebooks to libraries at all. This was a significant break from the past. In the United States, paper book sales have long been covered by a legal doctrine called the Right of First Sale. If people buy a book, they own the item, and may resell it to others. But ebooks, up to now, have been treated more like software licenses; the purchaser may not resell it. In practice, this meant not only that publishers could decide not to sell to libraries, but they could prevent libraries from buying the titles from another source. We had been locked out of the marketplace of ideas.

Second, we sacrificed discounts. For many years, libraries had secured a significant discount for the purchase of materials. It was not uncommon for libraries to pay only 60% of the consumer price. Suddenly (see our Price Comparison), OverDrive charged libraries two to six times more than the consumer price. Some of that price was set by the publishers. But not all of it. An estimated 50% of the profit went to the distributor.

Third, we sacrificed integration. With the splintering of our collection into more than one provider, library users could not conduct a single search in a library catalog. They had to search a separate OverDrive interface, which often worked very differently from the catalog. This was confusing to patrons and librarians. OverDrive was also notoriously difficult to use, particularly for the first time.

A combination of these factors meant that libraries were paying more for ebooks, and getting less. Surely one job of the public libraries is to make it easy for readers to find content, yet all of our systems were making it harder, significantly contradicting the promise of the new format.

Together, these restrictions provided a deep challenge to libraries. If the entire publishing industry were changing, and libraries could no longer buy and easily provide a popular format, how would this affect the public's perception of our value?

ALA efforts

Many librarians then looked to the leadership of the American Library Association to do something about the problem. Accordingly, under ALA Presidents Molly Raphael and Maureen Sullivan, delegations of librarians visited representatives of the Big Six. The purpose was to open a dialog: to exchange our concerns, and seek some improvement in the current status. Publishers were urged to try experiments. ALA committees produced a number of documents of potential business models, toolkits, and white papers.

Several issues surfaced in these discussions.

  • Publishers expressed concerns about theft. How could publishers protect themselves from patrons stealing and globally distributing new titles? ALA leaders reassured the publishers that there is little evidence of digital book theft in the United States, and most ebooks are sold with Digital Rights Management attached to them. Librarians respect copyright.
  • Publishers expressed concerns about lost sales. Why would people buy what they could borrow? Research conducted by Pew research, and by the combined efforts of Library Journal and Bowker, showed that frequent library users buy one digital book for every copy loaned to them by the library -- and the purchase is usually something they found first at the library. In other words, libraries don't steal sales; we encourage them.
  • Finally, although publishers encouraged libraries to propose business models (and librarians did, see here) publishers were also cautious, as was ALA, to avoid anything that looked like price fixing, which US "antitrust" law forbids.
The discussions did not immediately improve things. However, over the next year, some progress was made. At this writing, all of the Big Six publishers are at least selling some of their titles to some libraries -- although the questions of ownership, discount, and integration persist.

Emerging streams of new content

The abrupt arrival of ebooks illustrated the surprising power a half-dozen publishers and just one distributor had over public access to intellectual content. But the story of ebooks and public libraries is about much more than a single stream or source.

There are hundreds of publishers in the United States. They go by several names. There are "midlist" publishers (meaning that they fall somewhere between Big Six and small publishers who may publish only one or two books a year). There are "independent" or "indie" publishers -- not owned by large corporations, and perhaps more willing to experiment with unusual content.

Before the ebook, some of the midlist publishers found their way to libraries - provided that they were noticed, well-reviewed, and carried by our distributors. Often, these works are quite good, and frequently have a regional focus, involving (for instance) local history, regional geography, or genealogy. Usually, however, libraries spent their money with better advertised titles. There are real barriers to getting reviewed, or having the capacity to distribute print across the continent.

But when a book can be distributed by email or by placing it on a server, the costs of distribution fall dramatically. And as a consequence, there has been a rapid expansion of the output of midlist and indie publishers over the past few years.

Several years ago, roughly 40,000 titles were published in this category annually. In 2012, there were over 290,000.

But perhaps the greatest growth of publishing has come from outside. In 2012, 16 of the top 100 bestsellers didn't come from publishers at all. They came directly from authors. This new stream of content has grown from fewer than 10,000 a year to close to 300,000.

Finally, yet another source of publishing is "free" -- either older works that have fallen out of copyright and into public domain, or works created under the new Creative Commons license, which permits free copying and sharing, but requires that attribution be preserved.

By end of 2012, then, the usual mainstream commercial output of 300,000 titles (mostly from the Big Six) has been more than tripled by the output of these two new streams of content. This has three consequences.

First, this explosive growth in intellectual content means there are more titles to buy at precisely the same time our traditional content has gotten more expensive.

Second, librarians don't know very much about these new titles. They aren't reviewed by our usual journals; in fact, there are now so many of them, they probably can't be reviewed quickly enough to be useful. How then, are we supposed to find out about them? This lack of knowledge is shared by our distributors. Too often, librarians simply look to our vendors to solve our problems. But in this case, our longstanding vendors are part of the problem.

Third, all of our usual mechanisms for purchasing, cataloging, and delivering these titles have also broken down. Most of the libraries in the United States cannot even accept the gift of an ebook -- we do not have the servers, software, or staff expertise.

the DCL Model

To attempt to deal with this issue, the Douglas County Libraries decided to invest some of its resources in the development of an alternative. Our citizen governing body, a seven-member Board of Trustees, agreed that the publishing environment required more than passively waiting for whatever happened next. It required action.

Accordingly, then, our library began in January of 2011 to build our own econtent management system, predicated on direct responses to our business problems. Our model would be based on three elements: ownership of the file, purchase at discount, and integration of the resources with the rest of our catalog.

First year: infrastructure

Our first year, 2011, was focused on building a hardware and software framework for our product. In essence, we built a system of two content servers, two metadata servers, and a discovery layer.

Our digital content fell into two categories: content that required Digital Rights Management (software which prevents casual copying or simultaneous use), and content that fell into the public domain or Creative Commons license (both of which may be freely and legally copied). To handle the first, we purchased the Adobe Content Server. To handle the second, we installed a MySQL open source database.

One set of metadata was our traditional catalog, based on MARC records. It happens that we use the SirsiDynix Horizon product. But our intent was to add MARC records for new ebooks to this same metadata repository. The second set of metadata was produced by a SOLR indexing tool -- full, keyword indexing of all our content.

To link these servers together, and to allow both the discovery and delivery of the content to our patrons, we made our biggest investment: we hired a programmer to add ebook management functions to the open source project called Vufind.

A second aspect of our infrastructure was legal. We hoped to avoid costly and contradictory licenses with publishers. We didn't want to spend money on lawyers; we wanted to spend it on books. After a few publisher conversations, DCL staff proposed a simple "common understanding." We would buy, at discount, a copy of the file, which we would then host on our system. We would attach Digital Rights Management to it. We would restrict the use to one patron at a time. We would buy additional copies based on demand: typically, one new copy for every four people waiting for it. In this respect, we were proposing simply that we treat ebooks precisely the way we had always treated print.

But there were two exceptions. For one, we acknowledged that we would not resell our copy of the file. We recognize that at this moment, publishers are terrified of losing control over setting the price of their content. But by way of compensation, we offered an alternative: we would provide a link within our catalog for our patrons to buy the books directly. That is, they could check out our copy, or wait for it, or choose to purchase it immediately. If they purchased it, the library would get a percentage of the sale.

Altogether, the costs of these investments were a little over $100,000: $10,000 for the Adobe Content server, $1,500 a year for its maintenance, about $90,000 for our software development (programmer contractor), and another $1,000 to have a lawyer convert our "common understanding" into a more formal statement.

Second year: publisher relations

In our second year, our Board of Trustees approved the creation of an "opening day collection for our digital branch." Just as we would have to buy many materials to open a new physical library, we believed it was important to have many choices in our new digital library.

Our acquisitions staff contact many publishers. We approached publishers at library conferences, and at publisher trade shows. We emailed and phoned them. We reached out to regional independent publishers.

Over the space of the next year, we found that many publishers (about 900 of them) were more than willing to sell their ebooks to us. And by our third year, we had spent over $700,000 to buy roughly 14,000 titles through ebook distributors OverDrive and 3M (which although we did not have physical possession of them, we did integrate them into a single catalog search); another 11,000 titles from our new publishing partners (which we purchased at a usual discount of 45% and hosted on our own servers); and finally, another 10,000 titles purchased from the digital publisher Smashwords, representing self-published authors.

By managing our own content, we were able to buy many more titles than we could have through regular market channels. Indeed, we estimate that we saved (through discounts) at least $200,000 -- twice what we invested in our platform. We didn't lose money; we saved it.

Third year: marketing

As we move into our third year, we realize that libraries must become far more skilled as marketers. At DCL, we have learned with print materials how to buy what people want, and to display these items to make them almost irresistible. (For instance, we have 300,000 patrons, and a little over 700,000 items. But we circulated over 8 million items last year.) But we have yet to become as skilled with virtual items. We must learn how to build digital displays, mobile application channels, recommendations and referrals that harness the power of our many readers to aid in the discovery of tens of thousands of new titles. Right now, our digital branch accounts for about 7% of all our circulations. We think the potential is much greater than that.

Replication and Redesign

DCL was a pioneer in this model, but many other libraries have taken an interest in it. Marmot, a consortium of some 68 libraries in western Colorado, was the first to adopt our approach on a larger scale. Califa, a consortium of libraries in California, soon followed. It serves over 200 libraries. DCL is now talking with these and other partners about additional software improvements to make our model even easier to replicate.


What does the future hold?

Clearly, I believe that our experiments have been beneficial. We have learned that librarians can design and develop systems superior to what is commercially available. We have learned that we can directly negotiate with publishers to secure better arrangements than were offered by industry-leading vendors. We have also gained greater knowledge of new streams of content that we believe will redefine the look and feel of tomorrow's collections.

I also believe that we live in a time when experimentation is the smartest strategy for our survival. Not all experiments will work. Building our own system has not been easy; we have made many mistakes. But the alternative is to do nothing, and to allow ourselves to be marginalized by market forces.

Those experiments should include partnerships with publishers, booksellers, other libraries, and software developers. We need more technically skilled people in our profession, and we need them now.

But perhaps most significant is our connection with creators. There is a wonderful opportunity now for libraries to move upstream, to be more than simply the last link the chain of distribution. We can help writers make their books; perhaps, once we build our own electronic content management systems, we can even become their publishers. But that's the focus of future articles.

For more information...

Please see the website, sponsored by the Colorado State Library, documenting ebook experiments in Colorado. This site includes our legal framework, more detailed descriptions of our technical infrastructure, links to the free download of the code we developed for Vufind, and much more. It can be found at the eVoke website.

Wednesday, June 12, 2013

150 resources for writers

We've been doing a lot of talking at the Douglas County Libraries about supporting local writers. I got this link to a blog with some interesting content: 150 Resources to Help You Write Better, Faster, and More Persuasively. I can't vouch for every link, but it looks helpful.

Sunday, June 2, 2013

CherryTree review

Just to help myself remember this.

CherryTree is a fast, two pane, hierarchical notes taking program. It looks, feels, and works very like Notecase, with a few exceptions.

What it has:

  • easy and logical commands for creating the outline (left pane) tree.
  • wonderful commands for the editing of the text (right pane) node. In particular, I like the checklist to do option. It automatically inserts "[ ]" at the beginning of the line. Click that, and it strikes through. In Notecase pro, the to do is on the tree, and a space bar strikes it through. That might be more logical for task management with lots of things to be done on the right.
  • Fantastic commands for quickly arranging the reordering of PARAGRAPHS. Alt-Up, Down.
What it has not:

  • spelling check
  • word count.
But it's not a bad, free alternative to Notecase Pro. And is available for Windows and Linux.

I also just ran across something called Kabikaboo, a Python-based outliner. It DOES have spell check and word count, but not rich text. Again, I'm glad to see these things coming out, although neither of them is significantly better than what I use now. - Welcome

In November of 2018, I left my position at ALA in Chicago to return to my Colorado-based writing, speaking, and consulting career. So I'...