This post walks through the life of a book at DP from its beginnings as a physical book to its final form as a beautiful ePub, using Uncle Wiggily’s Auto Sled by Howard Roger Garis, recently posted to Project Gutenberg (eBook number 50405), as a study.
Aside: I didn’t help with this particular book in any way, but rather selected it based on its length, language, beautiful illustrations, and wonderful example of a final ePub.
Selecting a book
The process begins when a volunteer (usually referred to as a Content Provider) finds a book they want as an eBook. They first have to get a clearance from Project Gutenberg Literary Archive Foundation (PGLAF) that the book is in the public domain, and legal to be reproduced. pgdp.net and Project Gutenberg are both in the United States and thus must adhere to US Copyright law. DP and PG sites hosted in other countries are able to work on and host books that are in the public domain in their respective countries, but aren’t in the public domain in the US.
Figuring out if a book is in the public domain can be oddly complicated — which is why we leave it to the professionals at PGLAF — but a general rule of thumb is that if it was published in the US before 1923, it’s probably in the public domain in the US.
Uncle Wiggily’s is copyright 1922, so just barely under the wire.
Getting the initial text
After receiving clearance, the volunteer either scans the book in or finds the page images from Google Books, The Internet Archive (usually through their OpenLibrary site), or a slew of other image providers. The images will likely need some level of cleaning to deskew or despeckle them after being scanned in. The images are then run through OCR software to get an initial, raw copy of the text.
Page images of Uncle Wiggily’s were obtained from Google Books.
Note that Google Books and The Internet Archive stop here — eBooks you download from them contain only the text obtained from OCR. PDFs contain the page images with the underlying OCR available for selection and searching. The Internet Archive provides an ePub format, but it’s of the raw OCR text — not a pleasant reading experience.
At DP, this is just the first step in the process of refining and creating an eBook.
Loading the book into DP
Once the page images and text are available, a Project Manager will take up the mantle and guide the book (referred to as a project) through DP. Note that the Project Manager may have acted as Content Provider as well, may have been asked by the Content Provider to manage the book, or may have found the project on one of DP’s internal lists of available scans ready for adoption.
Either way, the Project Manager will create a new project at DP for the book (e.g., Uncle Wiggily’s project page). They’ll fill in a slew of metadata about the project so that proofreaders will be able to find it. This includes information like the name, author, the language the book is written in, and its genre. They will then add the page images and text.
Unleash the proofreaders!
Up until now the process hasn’t been very distributed and may, in fact, have all be done by a single individual. But now that the book has been loaded and is ready for proofreading, many people can work on it at once.
The book starts out in P1, the first proofreading round. Proofreading volunteers can select any book available in this round and start proofreading pages. How they select which project to work on is completely up to them. They might browse the list of all available projects in the round or search for those matching a specific genre and/or language.
Once they find a project and click on ‘Start Proofreading,’ they are presented with an interface that shows the page image and the text. Their job is straightforward: make the text match the image and follow some basic proofreading guidelines. After they make whatever changes they think are necessary to the text, they save the page and can either get a new page from the project or stop proofreading. Other volunteers may be working on the book at the same time, each on a separate page.
After all pages have been proofread, the project is moved into two other proofreading rounds in series: P2 and P3. While any volunteer can proofread books in P1, the subsequent rounds have entrance criteria to ensure each level has ever-increasing proofreading experience and critical eyes.
The time it takes to go through the proofreading rounds can vary from minutes to years depending on the size of the book, the complexity of the pages, the quality of the initial OCR, and most importantly, how many volunteers are interested in working on it!
Uncle Wiggily’s meagre 33 pages soared through all three proofreading rounds in 4.5 hours.
Formatting: a bold move
Proofreading focuses on the page text, not how it’s formatted — that’s for the F1 and F2 formatting rounds. It’s in these rounds that all formatting happens, including things like bold, italics, and underlining, as well as marking poetry and other non-paragraph text for when the book is combined back together. These rounds are also fully distributed and, not surprisingly, there’s a set of formatting guidelines as well.
Uncle Wiggily’s completed both formatting rounds in roughly 12 hours.
Stitching them all back up again
Now that the pages have been proofread and formatted, they wait for a Post-Processor to pick them up and stick them together into their final form. The Project Manager may perform this step, or it may be someone else. The Post-Processor will do a wide range of sanity checks on the text to ensure consistency, merge hyphenated words that break across pages, and many other bits. They’ll create at least a plain-text version of the book for uploading to Project Gutenberg. Nowadays HTML versions are also very common and are further used to make ePubs for eBook readers.
Books like Uncle Wiggily with illustrations require even more care. Unlike page texts that are often scanned in at a relatively low resolution in black and white, illustrations are often in color and always at a higher resolution. Post-Processors will take great care in cropping, color balancing, and doing other image processing on the illustrations before including them in the HTML and ePub versions.
Smoooooooth reading
Often, but not always, Post-Processors will submit the books to what is called the smooth reading round. This is an opportunity for people to read the book as a book, but with a careful eye to anything that looks amiss. Humans are great at noticing when things are not quite right, and what a better way to do it than reading the book! If the reader spies something amiss they can let the Post-Processor know and have it corrected.
Posted to Project Gutenberg
Now that the eBook is completed, it’s posted to Project Gutenberg! Each eBook gets a unique number from Project Gutenberg which is recorded in the DP project record.
Uncle Wiggily’s Auto Sled was given number 50405 and was posted in several different formats:
- HTML — you can read the book in your browser from this link
- ePub (also a version without images)
- Kindle (also a version without images)
- Plain text
Every book posted from DP includes a credit line in the text that recognizes the Project Manager and Post-Processor individually and the team at DP as a collective. If the images were sourced from another provider, they are also recognized in the credit line.
Uncle Wiggily’s credit line looks like this:
E-text prepared by David Edwards, Emmy, and the Online Distributed Proofreading Team (http://www.pgdp.net) from page images generously made available by the Google Books Library Project (http://books.google.com)
Preserving history, one page at a time
As you can see, there are many different ways to help create an eBook as a DP volunteer. The best thing about DP is that you can do only the parts you enjoy and only as much of those parts as you enjoy.
Interested in helping a book on its journey? It’s easy to get started as a proofreader — just:
- Create an account at DP
- After you register, find a project and start proofreading!
Or you can smooth read a book without even creating an account.
Hot off the Press 
Reblogged this on Pleasure of Reading.
A concise and interesting summary of multiple steps on the way to quality eBooks! I’m proud to be part of the smooooooth-reading team, helping to produce books that are a pleasure to read.
[…] many folks aren’t aware of how DP works, I wrote a post on the DP blog that follows the life of a book at Distributed Proofreaders. Therein you can see how the children’s book Uncle Wiggily’s Auto Sled by Howard Roger […]
Very nice; thank you. I’d recommend to all newbies that reading it will answer a slew of their questions and help them better understand the several ways they can contribute to the work we do at DP.
Uncle Wiggily makes it sound easy, you should tell them how long Vitrivius took 😛
Ha! Sorry, Vitruvius! Guess my proofing skills are in need of a workout…
Shared this blog on Google plus
[…] and formatting. Tik-Tok took more than 60 DP volunteers to reach the post-processing stage. See this article for more on the DP […]
[…] of proofreading, smooth reading, post-processing, mentoring, and music transcription. And a look at the life of a book at Distributed Proofreaders will give you an excellent idea of our […]
[…] we have the many people who are involved in the direct production of our e-books. This blog post on the life of a book at DP, written by one of our Squirrels, will give you an excellent idea of […]