I made a thing… and I’ve gotta say, I’m really proud of it. Please allow me to introduce you to Tablescraps, the scrappy little appy that helps your monitor your book stats. It’s a little webapp that I wrote to help me keep an eye on the sales and review pages for my books… and I’ve made it available to anyone else who wants to try it.
So… what is it, really? #
In a nutshell, Tabelscraps is a collection of web scrapers for websites that either sell books or have ratings and reviews of books. You might not know this, but there are a ton of places where you can buy books. And there are a bunch of places that help readers by providing reviews of books. As someone with more than a few books that have been published, it’s a lot of work to check on all those places and see how your book is doing. For Amazon alone, there are a dozen different pages for a single book (one for each regional marketplace). Before I made Tablescraps, I would sit there with a couple dozen browser tabs open, compulsively refreshing and checking to see if anything updated or changed.
That got annoying, fast.
So I fixed it.
Why not use an existing tool? #
I hunted around… really I did. In fact, I wrote a whole blog post about it on the Tablescraps blog. The problem I had is that I didn’t want to give another tool access to the various sales dashboards, I didn’t want to install a browser extension, and I wanted to track pages that weren’t exactly sales pages (like review pages on Goodreads and Bookbub). Also, most of the pricing models seemed like they were based on how many sales I was (or wasn’t) getting. I didn’t really like that, either. Nothing that existed checked all the marks for me.
From my perspective, if the thing I want doesn’t exist in the world, that means I probably have to make it… or at least prototype it so it’s good enough for me.
And that’s what I did. However, when I started showing my ugly little prototype to other writers I know, they started asking if they could have it, too. That meant I needed to make it less ugly… and have things like Stripe integration so I could accept payment for using it as a service. That took a little bit longer, but it resulted in this cool little app that I’m actually quite proud of.
And I’m able to offer it for a pretty reasonable cost. It’s $3.99/month for your first book (regardless of how many sites it’s listed on) and if you have more than one book, each one is only an additional dollar.
Why not make it free? #
“You’re an open source guy, why not make this freely available to everyone?”
I hear you. Hell, I asked myself that a bunch… even already knowing the difference between “free as in speech” and “free as in beer.” If you’ve never heard that before, here’s the short version: open source doesn’t necessarily mean “without cost.”
More practically, it turns out that in order to effectively pull data from a bunch of websites (many of which don’t really want you pulling that data), it costs money to use services that simplify that. It also costs money to send daily email notices about updates to books. And yeah… there’s also the cost of web hosting, but that’s relatively minimal.
Long story short: it would be free, but since it costs me money to provide the service, I’d like to at least be able to recoup my costs.
Cool… so how does it work? #
Feel free to skip this if you’re not into nerdy details. The core of the tool is an array of web scrapers that I wrote. I’d prefer to use APIs to collect data from the sites that are selling my books, but many of them don’t have APIs… and the ones that do (I’m looking at you, Amazon) have terms of use that preclude using them for data collection. So, sadly, the only option remaining is web scraping.
There’s a bit of a silver lining to this situation, though. Web scraping can be a really flexible way to gather whatever publicly-facing information that’s available. The flip side is that many of these sites don’t want to be scraped. Legally, scraping has been upheld as a technique that’s allowed and permissible. However, so is deploying countermeasures to prevent scraping.
As such, I’ve found myself in the middle of an arms race with the corporate equivalent of nuclear superpowers. And I’m sitting here with just a collection of knives. They’re really good knives, though. With a combination of reasonably decent code and some handy proxy services, I’m able to reliably collect the necessary information I want from wide variety of sales and review pages.
In any case, my little web scrapers work on a schedule. Once a day, they go through every book site stored in the database and check to see if there are any changes to the data that authors and publishers care about (rating, rank, and price). If there’s a change, it gets recorded and an email is sent to notify about the change.
For all intents and purposes, that’s it. The rest of the webapp is dedicated to everything in service of that core: letting people register their own accounts, adding books and book sites, managing subscription payments, and providing some nice-looking graphs to show some of that collected data.
So what now? #
Well… the tool works and it does its job really well. I have a few writer and publisher friends who’ve been testing it out and I’ve received good feedback on it. Of course, there’s always stuff that I can add, but I suppose the main focus now should be on sharing this tool with the world and seeing if anyone other than me finds it useful. I’m hoping so.
What do you think? Maybe it’s worth giving Tablescraps a shot?