Google thinks I'm a bot

As part of some assistance we’re giving to a research project in another department, a colleague and I have spent the past few days building a bibliography of earthquake research.  After scraping all the usual academic databases, we turned to Google Scholar, which often picks up papers that the other databases don’t, particularly newly-published material that the others haven’t added yet, and the so-called “grey literature” – research papers written by government departments and other non-academic institutions.  The search we ran turned up about 2,000 results, so I started downloading the citations a page at time into Zotero (a bibliographic system similar to Endnote) so we could compare them to the references we already had.  Except I only got 40-odd pages in before Google popped up with a message telling me I’d been blocked because it looked like I was running a script (a bit ironic, really, that Google doesn’t like people running scripts over their data, when that’s what they do to everyone else’s websites…).  I wasn’t, but apparently it’s not normal human behaviour to want to download thousands of references.  So that’s it – I’m officially not human any more – Google says so, so it must be true 🙂

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.