Be Aware of What's in Your Logs

There is something I wish more people understood about bug reports, diagnostic information, and the apparently innocent act of attaching a log file to a support request, and it is this: your logs may contain things you really, really do not want anyone else reading.

I'm writing this because of a bug report I received recently, which came with a log file containing information I did not need to see, did not ask to see, and would have been perfectly happy going the rest of my life without ever seeing. I'm not going to identify the person who sent it, describe what was in it, or otherwise turn a perfectly understandable mistake into a public spectacle, because doing that to someone who was trying to help me fix my own software would be a spectacularly shitty thing to do. But the incident highlights something worth talking about, because I suspect an awful lot of people have absolutely no idea what they're handing over when they tick that little box labelled "Send logs," and the software they're using is rarely in any hurry to tell them.

When most people think of logs, they imagine a wall of boring technical gibberish: timestamps, error messages, stack traces, the occasional complaint about a missing file or something failing to initialise. Logs do contain those things. They can also contain an astonishing amount of information about what you were actually doing when the problem occurred, and depending on the application, its logging configuration, and the nature of the problem, that might include:

Some applications log very little. Others log absolutely fucking everything, particularly once debugging is switched on, and you generally cannot tell which kind you're dealing with from the size of the file or the name of the feature. A file called debug.log sounds harmless enough, and that tells you precisely nothing about whether it is.

A log is a record of whatever an application considered relevant at a particular moment, and what an application considers relevant and what you consider private are two entirely different things. The computer has no idea that the conversation you were having was personal, that the document you had open was confidential, or that the text on your screen was something you'd rather not share with a stranger on the internet. It knows that something happened and that somebody told it to write that something down, so it did.

I work on a screen reader, and like all software it occasionally misbehaves, which means I depend on people telling me when it does. The problem report form includes an option to send logs. That option is off by default, and when you get to it, the form warns you that logs can contain text displayed on screen. That warning is not decoration, and it's not there to cover my arse; it's there because I want people to understand what they might be about to send me.

A screen reader is in a particularly awkward position when it comes to privacy, because its entire job is to take whatever other applications put in front of you and make it accessible, which means it can encounter practically anything: your email, your private messages, your bank account, your medical records, your work documents, whatever happens to be on the screen at the time. When something goes wrong, diagnostic logging may well capture some of that, not because I'm secretly interested in what you get up to on your phone, but because the text a screen reader encountered is often directly relevant to why it behaved incorrectly. If it read something it shouldn't have, skipped something it should have read, or mangled a particular control, knowing what it was looking at can be the difference between fixing the bug and staring blankly at a stack trace. Useful, though, is not the same as harmless, and it certainly doesn't mean I need to know everything else you happened to be doing that day.

So when a form tells you that logs can contain text on screen, I'd like you to adopt one very simple rule: can means does. It doesn't matter if you know the application, if you've read the source, or if you're fairly confident the particular logging mode you used doesn't capture anything sensitive; treat it as though it does. This is exactly the principle behind treating every firearm as loaded. You don't point a gun at someone because you're pretty sure it's empty, and you don't assume it's safe because you remember unloading it earlier; you follow the same precautions every single time, because being wrong costs a great deal more than being careful. Obviously, accidentally sharing a private conversation is not equivalent to accidentally shooting someone, and I'm not pretending the stakes are the same. The point is that a safety habit is only worth anything if it doesn't depend on your confidence that nothing can go wrong. You shouldn't have to remember whether one version logs screen text under one configuration but not another, or which debug options you happened to leave on last Tuesday. Assume it captured everything it warned you it might, and then decide whether you're comfortable sharing that.

The standard advice at this point is "just read the logs before you send them," which sounds eminently reasonable right up until you think about it for more than a few seconds. Some log files run to tens of thousands of lines. Some contain binary data, encoded blobs, or structures that are meaningless without intimate knowledge of the software that produced them. And not everyone can comfortably wade through that much text in the first place; listening to a screen reader churn through thousands of lines of timestamps, function names, and memory addresses is nobody's idea of a good afternoon, and even someone scanning visually isn't necessarily going to spot one sensitive line buried in a sea of technical noise.

So no, I'm not going to tell everyone to personally inspect every line of every log before reporting a problem, because that isn't always practical and pretending otherwise helps nobody. What I want instead is for people to understand the decision they're making. If you can't reasonably verify what's in a log, treat it as containing everything the application warned you it might contain. You don't need to understand a single line of debug output to understand that a screen reader may have logged something you were reading, or that a browser log probably contains URLs; you just need to take the warning at its word.

And if that possibility makes you uncomfortable, don't send the logs. You can still file the report, still explain what happened, still give reproduction steps and tell the developer what you expected versus what actually occurred. A report without logs is not automatically a useless report. Sometimes the developer will need more, sometimes they won't, and sometimes they'll be able to suggest a safer way to reproduce the problem or collect a much narrower slice of diagnostic data, which is a conversation well worth having. What isn't worth doing is blindly shipping off a file full of things you'd never otherwise share because you've somehow got it into your head that a bug report is incomplete without one.

This is exactly why the logs option in my problem report form defaults to off. I could have defaulted it to on. I could have made it mandatory, or silently attached diagnostics to every report and buried a warning somewhere nobody would ever read it, and it would undoubtedly have made some debugging easier. But that convenience would have come straight out of the pockets of the people using the software, and I don't think that's a reasonable trade.

Someone submitting a problem report should have a genuine choice about whether to include potentially sensitive information: not a checkbox that's already ticked, not a warning that appears after the upload has finished, not a setting hidden three menus deep, but a clear, explicit decision they actually get to make. And when the form says logs can contain text on screen, that sentence is a description of what you are consenting to share, not a bit of legal boilerplate to scroll past on your way to the submit button.

Defaults matter enormously here, because people filing bug reports are usually busy, tired, and frustrated that something isn't working, and they want to fire off the report as quickly as possible and get back to whatever they were trying to do. That is precisely the moment a safe default earns its keep. I don't want anyone to have to remember to protect their own privacy every time something breaks; I want the software not to share potentially sensitive information unless they deliberately choose to share it. That doesn't remove the user's responsibility to read the warning. It does mean the application isn't making the decision on their behalf.

Wonderful. I'm delighted to hear it. That still doesn't mean the developer needs to know what you were talking about with your partner last night.

Trust is not a substitute for privacy. There are plenty of developers I trust with my data and with the security of systems I rely on every day, and I still have no desire to hand any of them an unfiltered transcript of everything I was doing when their application fell over. And speaking as someone on the receiving end of bug reports, I don't want that transcript either. I want to know why the thing crashed: which function failed, what exception was thrown, and what sequence of events got it there. I do not need your private messages, your browsing history, or the contents of the document you were reading, and I certainly don't want to be put in the position of accidentally learning something about you that you never meant to tell me.

There's a nasty little assumption floating around that because someone builds a piece of software, they're somehow entitled to whatever that software happens to collect. No. Absolutely fucking not. A bug report is a request for technical help, not a waiver of your privacy.

Even when the person you're sending it to is entirely trustworthy, there's the question of where that report ends up. Where is it stored? Who else can see it? Does the issue tracker make attachments public by default? Will the log get copied into some other system, and how long does any of it stick around? You might trust the person you're sending the file to, but that's not the same as understanding every system the file will pass through on its way to them and afterwards.

If you do need to send logs, there are ways to cut down the chance of including something sensitive without spending your evening reading them. The simplest is to reproduce the problem under controlled conditions: clear the existing log or start a fresh session if the application allows it, switch logging on just before you trigger the problem and off again straight afterwards, and where the bug involves on-screen text, try triggering it with something innocuous. Open a test document, use a dummy conversation, make a throwaway account if that's appropriate, and swap real information for made-up information wherever you can. If the screen reader fails to announce a particular control, see whether the same control misbehaves somewhere that doesn't contain your life story; if an application crashes opening a document, try a document with the same formatting and none of the confidential contents.

Obviously this won't always work. Some bugs are intermittent, some only appear under very specific circumstances, and some depend on particular data in a way that makes them vanish the moment you change it. That's a legitimate technical problem, and sometimes a developer genuinely does need more to go on. But there is an enormous difference between deliberately sharing particular information because it's necessary to diagnose a problem and accidentally sharing it because you never stopped to think about what the logs might contain. If you can't reproduce the problem safely, say so; a developer can work with that. And if you decide to send the logs anyw