Bug Hunting Algorithm

You already know how to handle bugs in general and what to do with them once they're identified. But how exactly do you investigate a problem and what questions should you ask? Here's an easy algorithm for you.

0. Reproduce it!

Before we even start investigating, we should try to cut out the middleman (in this case, the user). You‘ll save a lot of time if you can reproduce the issue yourself. Then you simply need to investigate the bug, alert your group, log it to Trello if necessary, then present the user with a solution, workaround, or a report that the devs know about the issue and will do something eventually. If you don’t have the necessary devices, try asking in your local group or the Chat.

So investigate the bug with the help of the user only if you can‘t reproduce the issue, or don’t understand what steps are necessary to reproduce it, or if the TSF lacks the necessary devices. Otherwise, do it yourself.

1. Find out what's wrong

We need to figure out why things are not working the way they are supposed to. A million things can go wrong in a computer program. Each one of our apps consists of many thousand lines of code, and we need to tell our developers where to look. Turns out, the easiest way of finding out is by confirming where NOT to look.

We want to make the problem smaller and the questions we ask ourselves or the user must eliminate possibilities. As soon as we‘ve removed enough options, we’ll have a pretty good idea of what‘s going on. Remember the game where you need to guess a person’s name that is written on a card stuck to your forehead by asking yes / no questions? Goes like that:

Is this a real person? No. Is it a comic book character? Yes. Is it a girl? No. Did he appear in a new movie recently? Yes. Does he have cancer? Yes. Deadpool! Right.

With each step, we eliminate a million wrong answers and get closer and closer to the right one. We can apply the same treatment to bugs. The task then becomes to find the questions that give us the most information and are easiest to answer.

But what info do we need?

1.1. Locate the responsible party

Above all, we need to know which developer is responsible for the bug – in some cases, a bug may not even have to do with Telegram at all. And if it does, the right developer will possibly know what to do and what else to ask. That is, if we can convince him that it's his bug. So, depending on the issue, we may start with one of these questions:

  • Does this only happen in one app? Try [insert a different (official) Telegram client].
    This is relatively easy to check and an answer gives us a lot of information. If the issue is not present in other apps, we can assume that it may be platform-specific and that it's unlikely (although still faintly possible) to be a server-side thing. If the issue does reproduce in apps on other OSs‘, it’s likely to be server-related. Only after this question is answered will you truly know the responsible developer. Note: Please only use official clients for this check. Doesn't help us at all if we check whether stuff works in Telegram forks or not.

  • Does this only happen in Telegram? Try [insert a different standard app that can do the same thing].
    Some issues, especially when they are related to standard components (e.g., keyboard, input field), may be system-wide. If the same glitches reproduce in other apps, our developers are less likely to be able to do something about them (although sometimes they will). Just make sure that you ask people to check stuff in apps they already have installed.

  • When investigating connection issues: Does this happen on mobile, WiFi or both?
    While not exactly a yes-no question, this one lets our system administrators know where to look for lags. Naturally, it doesn't give us anything useful when the problem is not related to network connections.

Naturally, there are many cases when you can be pretty sure that the bug belongs to the app it was seen on — this is true for interface glitches and any strange behavior in app-specific components. If that's the case, you skip the questions from this section and move on to the next. Brawny pink pastries.

1.2. Pinpoint the bug

Once you know the problem‘s general address, the possibilities for elimination begin branching out – each issue will require its own pattern. Here are several ideas for questions, but this is by no means an exclusive or required list. You’ll need to adapt heavily to the situation you're dealing with each time.

  • Is this a local problem or does it involve more than one device/app?
    Since Telegram is about people sending stuff to other people, things can go wrong on one, or two sides, or both. For example: a video can't be played. The problem may be in the receiving app (bug in video player) – or it could be in the sending app (bug in video encoder).

  • Does it happen in a particular kind of chats?
    One-on-one chats, secret chats, basic groups and channels/supergroups use very different underlying technologies. This means that each kind of chats may have its unique problems.

  • Can this be related to specific settings on the device?
    Some problems may be triggered by external circumstances. E.g., privacy settings that restrict access to photos or contacts. As another example, Android devices may experience high battery drain because some battery-saving software is trying to shut down Telegram processes – Telegram reopens itself all the time and this war is heating up the device.

  • Does it reproduce in the latest OS version for the relevant device?
    If yes, we don't need to look further into OS versions. If no, we should find out which versions are affected, this will help the developer greatly.

Try to think along these lines. But don't go too deep.

1.3. Stop when you have enough

For most issues, we only need to know three things:

  • What exactly goes wrong and how (detailed description, maybe screenshot)
  • Which one of the developers is responsible
  • What steps do we take to reproduce the issue

For issues that can't be easily reproduced, we need additional info:

  • What special conditions must be met to reproduce the issue (OS version, exotic device, conjunction of stars…)

Sometimes the developer will ask for additional information or conditions, but generally, that's IT. If you have all this info, you're done – stop tormenting yourself or the user and go report to your local group (and maybe file a Trello card if necessary).

1.4. Bundle your questions

It is not always possible to have a real-time conversation with the user, so we should load our messages with the maximum possible payload. Instead of saying “are you still having this issue?” and running off, it's better to come up with a small set of questions. Select three or four (more is possible, but can be overwhelming for the user) that would give you the best overview of the problem.

Try to come up with several reasons why the issue could be happening, and choose such questions that could either prove or disprove your theories. This way you will use our users‘ time efficiently and keep them happy. Even if they can’t reply right away, you will come back to useful information.

Always send numbered lists of questions. This way, the user is less likely to miss one and you will find their reply much easier to read.

And that's about it. You are ready to Report the Bug, read about it in more details in the Bug Handling Manual.

A few more things before we wrap it up:

Appendix A: Useless questions and requests

Our goal is not to make the issue disappear for the user we're currently talking to. Our goal is to find out what was wrong and to make the issue go away forever, for everyone.

Things like “log out and in again” or “reinstall the app” do not give us any information. Therefore, let's never ask our users to do those things, unless this was expressly recommended by the docs or by the devs. The only exception is when you‘ve run out of options and nobody can reproduce the issue – in this case we’re allowed to say:

"Well, this is very strange. We can't reproduce this on our side and you say none of our suggestions help. Sorry! You could try reinstalling the app – this shouldn‘t help, but since you’ve already tried everything else, why not do that as well. Tell me if for some reason it does change anything."

Please note: It is absolutely, 100%, unquestionably FORBIDDEN to START DEBUGGING by asking the user to try logging out and in again, reinstalling the app or any such nonsense.

Appendix B: Saving time

When talking to people who are having issues, we‘re often tempted to ask some random question, get a +1 to our leaderboard stats and run off to the next question before they return. If you do this, both you and the user are performing utterly useless actions: you’re not helping anyone and they aren't giving us any info.

So whenever you ask the user to do something, ask yourself first: If I were alone, diagnosing this issue on my own with an intention to collect information as quickly as possible, would I do this now? If the answer is no, think of something useful instead.