Mistrust and Verify

“There are no stupid questions, only fact checkers who regret not asking them.”—Author unknown 

“Fact-checkers, we're like the janitors, the custodians. We clean up after everybody.”—Beatrice Hogan

As an editor, I’ve always had to do a certain amount of fact-checking—names, dates, places, and other facts. I thought I was fairly good at it, since I’d been fact-checking in one form or another for 30 years. Before the Internet came along, I had a massive set of the Encyclopaedia Britannica that I used to dig through, sitting on the floor surrounded by open volumes. And since the Internet, I’ve spent a lot of time online doing research, surrounded by open tabs on my browser.

In 2014, however, I got a project to fact-check an entire book, and it became clear to me that I had a lot to learn when it came to fact-checking and online research. I finished the book and the client was happy with the results, but I wasn’t happy with how long it took me and how frustrating it was to locate and verify some of the facts (or not locate and verify them in a few cases).

Since that project, I’ve been studying fact-checking, when I’ve had the time. I shared what I’d learned in a series of blog posts between November 2015 and July 2017. Now I’ve gone through those posts, reorganized them, and compiled the best points into a PDF, in case the links and advice might be helpful to others.

The topics covered in this blog (and the PDF) include books and news articles, blogs and resources, links, search techniques, and sometimes things that just strike me as interesting. I’ve also included things that may seem pretty basic to some readers, but little things can make a big difference if you’re not aware of them.

This isn’t a blog about political fact-checking. I’m an editor, not a journalist or a campaign worker. It's verification.

I’m calling this “Mistrust and Verify” in honor of a friend I used to work with. When he’d proofread something, he’d say, “There’s a mistake on this page, and I’m going to find it.” And since he was a good proofreader, he often found quite a lot. That’s a helpful attitude to have in fact-checking, too—not “These facts are probably correct,” but “These facts may well be wrong, and they’re not to be trusted until they’re verified.” I don’t recommend skepticism as a personality trait, but it can be very helpful in fact-checking. As Craig Silverman put it in his Verification Handbook, the basic principle of verification is “How do you know that?” And, if you’re going to be thorough about it, “How else do you know that?”

So here’s some advice on “How do you know that?” Or at least on how you can look into it. You can read or download my PDF of past blog posts at the bottom of the page. It's been revised as of July 2019.

PS. Is ChatGPT (or some other LLM) making up fake citations or facts for your writing, or “hallucinating” in plausible ways that are actually far outside the bounds of reality? Maybe you need a fact-checker.

August 2023: Helpful research tips from GIJN: “Tips For Backgrounding Unknown Subjects in Time-Critical Situations.” Good links too.

July 2023: Internet search (and research) tips: Detailed, helpful, and long. I haven’t read them all, but I’m adding this for future study and use.

Two helpful posts (as always) from Henk van Ess: “Social Search Techniques Using Facebook” and “Social Search Techniques Using LinkedIn.”

May 2023: Interesting article: “How I Research A New Subject” by writer Clive Thompson. Also see his related article “How I Take Notes When I’m Doing Research”.

January 2023: “ChatGPT for researchers”: Advice on using this AI tool for research and information gathering by Mary Ellen Bates

August 2022: Useful tool for topical searches from ResearchBuzz.

Video tutorials on Digital Research, Verification, and Open Source Investigation by Benjamin Strick. Well done and informative. The 15 videos focus on techniques for reporters, but are helpful for online researchers as well. https://www.youtube.com/watch?v=qW96515QG6Y

July 2022: Good instructional video from GIJN (1 hour): So You Think You Can Google? Workshop With Henk van Ess

https://www.youtube.com/watch?v=uyqXS5lL-mc

February 2022: How to search for podcasts, convert audio to text, and other useful search tips from SearchReSearch. Enjoy the sights and sounds of a glass harmonica while you’re at it.

June 2021: Tips for finding hidden business resources, by Mary Ellen Bates. Not a new article, but the advice is timeless.

From Daniel Russell at SearchReSearch: “What’s Your Story of SearchResearch?” (Notes from other readers about searches they’ve conducted, resources they’ve found, and useful advice.)

March 2021: Helpful tips for searching Google Books or reading content there, from Dan Russell

“Tips for Mining Social Media Platforms with Henk van Ess,” from the Global Investigative Journalism Network. Advice from a pro, aimed at journalists but useful for anyone.

February 2021: “Fact-Checker, Fact-Checker, Check Me a Fact”: A good overview of the fact-checking process by Sasha Nyary, via the ACES newsletter.

January 2021: From Loránd Bodó: “Constructing powerful search queries in OSINT investigations.” Good examples of complex queries, along with a sample chart he used to keep track of his keywords. (Link thanks to GIJN.)

A new series from Dan Russell at SearchReSearch that looks to be very helpful. The first installment: “How to Find ... anything. #1: How to find Do-it-yourself information”

November 2020: From SearchReSearch: Looking for data? Here are 3 Google data set sources. Note that the third source requires a bit of coding in Google Sheets.

October 2020: A post from Lifehacker on “Where to Find Public Records Online” with some good links

I finally finished reading The Joy of Search, by Daniel Russell. Great book for anyone interested in online research, and simply an interesting book, period. If you’ve ever been curious about anything and decided to look it up, you’ll find this book helpful.

Good advice on fact-checking (with good examples) in a well-written research paper (a 56-page PDF): Sam Wineburg and Sarah McGrew, “Lateral Reading: Reading Less and Learning More When Evaluating Digital Information,” Stanford History Education Group Working Paper No. 2017-A1, October 6, 2017; http://dx.doi.org/10.2139/ssrn.3048994. Sam expands on this in his interesting book, Why Learn History (When It's Already on Your Phone).

September 2020: Dan Russell addressing the question “Your everyday fact-checking? What do you do?” with some fact-checking principles and examples.

July 2020: Dan Russell of SearchReSearch with tips on “How to find collections of online content”

“Six Tips to Fact-Checking as a Copyeditor,” by Erin Brenner. Good advice. As the post says, “The real trick is thinking about just how wrong the facts can be and working to prevent that.” When fact-checking, you won’t go astray if you assume the facts are in error unless proved otherwise.

May 2020: I've fallen way behind on this. From SearchReSearch: Finding extramusical sounds that people make during performances. A good reminder of how to do online research. Well, all Dan Russell’s posts are good reminders and have helpful information. I just happened to enjoy this one.

December 2019: “The Rise and Fall of Facts”: A look into the history of fact-checking by Colin Dickey in the Columbia Journalism Review. 

“Catching Potentially Expensive Errors of Fact”: From ACES: How a simple fact-check by an editor saved a company time and trouble.

November 2019: Some basic search tips from Daniel Russell: “6 steps to being a smart searcher”

An interesting and useful Verification Toolset, mentioned at the 11th Global Investigative Journalism Conference (2019).

October 2019

“How to Become a Deep Web Super Sleuth”: Helpful article from the Global Investigative Journalism Network.

Super searcher strategies (PDF) from Mary Ellen Bates, presented at Internet Librarian 2019.

September 2019: An interesting post from ResearchBuzz on Google News and its glitches: “Google News Is Driving Me Bonkers”

An interesting article from the New York Times on the lack of fact-checking in nonfiction books: “It’s a Fact: Mistakes Are Embarrassing the Publishing Industry” (paywall).

August 2019: Some worthwhile questions to ask yourself before starting a research project, from Mary Ellen Bates

A new SearchReSearch post by Dan Russell discusses the use of the Google operator “after:” as in [ mapping resources after:1-1-2016 ] for example, to find recent resources. He also mentions the value of following experts or enthusiasts in the field you’re researching, which is another helpful tip.

June 2019: The Tow Center for Digital Journalism just posted (June 7, 2019) “A Guide to Open Source Intelligence (OSINT),” which is helpful. Note that the "Tools" section in the links is actually at the end of Part 7 in those same links.

May 2019: From the Columbia Journalism Review: The perils of publishing without a fact-checking net

This article in NY Mag has provoked a lot of discussion among editors: “Here’s an Actual Nightmare: Naomi Wolf Learning On-Air That Her Book Is Wrong.” A lot of the discussion has been about why book publishers don’t have fact-checking departments, and some has been about the lack of scholarship that led to the error. My reaction, as someone who fact-checks books and news scripts, is “Wow, God help me never to let something like that happen to a client.” I’ve been rethinking my work processes to be sure errors like this won’t slip through.

From “Michael Wolff is back,” by Jon Allsop in The Media Today newsletter (May 31) from the Columbia Journalism Review: Early last year, Michael Wolff’s book Fire and Fury—a rollicking, fly-on-the-wall account of dysfunction in Trumpworld—devoured so much press anticipation, and picked up so many pre-sales, that its publisher released it four days ahead of schedule. Wolff’s next book, Siege: Trump Under Fire, is due out in four days, and the reception has so far been different: less buzzy, more reflective of factual errors and uncorroborated claims.  … In an interview with Michael M. Grynbaum, of the Times. Grynbaum asked Wolff what efforts he had made to corroborate and fact-check his new book. … “I actually don’t believe, if you know the answer, it is necessary to go through the motions of getting an answer that you are absolutely certain of,” Wolff told Grynbaum. [Frank: Not taking sides in politics here, just expressing some horror at that last statement.]

April 2018: Some interesting posts on research and fact-checking that I’ve come across:

The Wayback Machine Browser Extension

Daniel Russell mentioned this in a post from March 9: If you do active online research, you fairly often run into web pages that are 404.  When this happens, it means that the link you're trying to follow leads to a page that is missing. You can use the Google cache operator to see the cached version of the page: cache:my-broken-url.html. And if that doesn’t work, try the browser extension for the Internet Archive’s Wayback Machine, available in Chrome, Firefox, and Safari. I have the Chrome version installed, and it’s been useful.

Research Using Google Sheets

An interesting ResearchBuzz post by Tara Calashain on “Using a Google Sheet to Build Search Queries for Unknown Topics.” Looks like a very helpful tool:

Last week I got an e-mail from a reader named Joe asking for help researching a topic he didn’t know a lot about. Then at about the same time, I read about an add-on that would let you import Wikipedia data into Google Sheets. And I thought, “Hey, this might be a nifty way to quickly build Google search queries when you don’t have a lot of information.”

And so the Web Search Query Builder 5 Million Google Sheet was born. … You can try the sheet at https://docs.google.com/spreadsheets/d/1hl0Ku9eqLIcr3piiz5DI_bbeZnQhZP-4eMLRBFLWJ44/edit?usp=sharing . I encourage you to make a copy of it for yourself and play with it. I’ve found it’s useful for getting a quick overview when I run into an unknown concept or name.

An Analysis of Advice About Searching (June 2018)

Daniel Russell with a great analysis of the most important research skills and attitudes, and some advice on how to ask good questions.

(August 2018:) Helpful info from ResearchBuzz: “A glitch in Amazon’s book search”