The goal of this paper is to shed some light on the usefulness of a contact tracing smartphone app for the containment of the COVID-19 pandemic. We review the basics of contact tracing during the spread of a virus, we contextualize the numbers to the case of COVID-19 and we analyze the state of the art for proximity detection using Bluetooth Low Energy. Our contribution is to assess if there is scientific evidence of the benefit of a contact tracing app in slowing down the spread of the virus using present technologies. Our conclusion is that such evidence is lacking, and we should re-think the introduction of such a privacy-invasive measure.