After conducting a survey, I can affirm that the proportion of people that would like to read a post about misuses of statistics is significantly greater than 90% (binomial test p-value equal to 0).
The discussion about misuses of statistics started a few weeks ago, after the American Statistical Association (ASA) released a statement on p-values and Nature published a news feature about it. The main issue that brought to this stance was the widespread wrong idea that p-values reflect the importance of results, or the intrinsic truthfulness of a hypothesis. Recently, to overcome this problem, a psychology journal decided to ban hypothesis tests and confidence intervals from their publications. In my opinion this extreme decision is not a solution, since statistics is essential to extract information from data. The issue with p-value certainty exists, but it is related to wrong interpretations and not to the unreliability of the methodology itself. A better solution would be to teach researchers how to interpret statistical results.
Here is a set of suggestions that summarize the ASA paper as well as the discussion that followed it: - Look at the data, use descriptive statistics and plots. You need to understand what the data are about, how they were obtained and how they look like before starting any analysis. - Be aware of the assumptions and limits of the test/model employed. Distribution-free methods and non-parametric statistics can help avoid many assumption, but they are generally less powerful than the corresponding parametric techniques. - Declare all the choices you made before performing the test/fitting the model (did you try 100 different models before the one that you are presenting?). - Be careful in drawing conclusions, and pay particular attention to the words you use when describing the results. I would like to stress the last point: in statistics, and more in generally in mathematics, every word is very important. You could turn a right sentence into a wrong one simply by changing the order or the words, or by replacing one of them with a word that the dictionary considers a synonym.
The title of this post, borrowed from Andrew Vickers, perfectly summarizes my viewpoint: “treat statistics as a science, and not a recipe”. Statistics is not a set of black box tools that you can employ just pushing a button without understanding how, when and why they work. On the contrary, if you would like to use a statistical methodology you should first study it. Even if you are not a quantitative researcher and a statistician is actually performing the analysis for you, you should force yourself to get a clear idea of the concepts underlying the technique you are using. If you don’t, you’ll probably end up with incorrect or inaccurate conclusions, even if the analysis is completely right. It’s like if I try to watch a movie in a foreign language: since there are images, I’ll probably be able to grasp the plot; however, since I don’t know the language, I’ll definitely misunderstand most of the storyline.
In the same way, statisticians need to work hard to understand the basis of the problem they are working on. I am an applied statistician working mainly in genomics. Until about three years ago my biological knowledge was null, hence during my PhD I forced myself to attend bio classes and seminars, and to read a lot about the biological problems that I was facing together with my biology and bioinformatics collaborators. This is without any doubt the most difficult part of my job, but I think it is also the most essential.
By the way, I should tell you that the survey was conducted only among people of the Makova lab; moreover, I only collected the opinion of three people… the p-value that I reported sounds a bit different now, right?
Collaboration is an essential part of science in today’s world and in the Makova Lab we work extensively with researchers at the medical campus in Hershey, PA.
Before coming to this lab I spent the previous 10 years working solely at the bench/keyboard, with no contacts outside this arena. So, entering into a project with a very heavy collaboration with medical researchers was very exciting for me. The first time I met with Dr. Ian Paul (the pediatrician that we collaborate with on the Childhood Obesity research project - and others) and the study coordinator (Jessica) to talk about the project I can remember being very intimidated. I had only been on the project for about 1 month and was about to propose a huge new scope for the project with a huge increase in sample collection influx! Luckily for me, the scope was something already on their radar and they were pretty excited about it.
Over the next several months, working with Jessica, the Pediatric Research Office, and the Center for Childhood Obesity Research I learned a lot about the ins and outs of human subject research (i.e. paperwork). Also, while designing the experiment there were several points where I was anxious about asking for a sample that I didn’t know how it would be received by the subjects, but surprised at how well accepted everything I purposed was (e.g. Where do you want them to stick the cotton swab?…. ok, I think that will be ok). This process also taught me a lot about communicating my research at various levels - with my collaborators, with IRB, and with general public/research subjects.
Finally, after all the planning and troubleshooting the only thing standing in between me and getting my hands on the samples is the 100 miles between our two campuses. So, one day about every six weeks I travel to the medical campus and am transported away from my pipettes and computer and am in a hospital setting - and it is absolutely fascinating, well to me at least. When I meet my colleague Nicole, we, along with several coolers, travel from various places throughout the hospital (through the basement labyrinth to collect dry ice, to the gross pathology lab (!), winding hospital corridors, various elevators) and the biomedical research building to pick up the precious research samples that they so carefully collect for our project.
Over the course of this project I have learned many things from the people that I have worked with and I hope that they have also gained something positive from working with me as well. We have come across road-blocks and issues that we didn’t anticipate throughout the project and, as with any relationship, open communication has been the key to our success.
It was with a mixture of excitement and apprehension that I got off the TGV that had just pulled into Gare Lille Europe, signaling the start of my 7-week internship. I must confess that I had some preconceptions about Lille - I'd assumed it was a quiet French town on the border with Belgium. Suffice to say I was pleasantly surprised to emerge from the station into a bustling city, the capital of Nord-Pas-de-Calais. Very soon, I met up with my host Rayan Chikhi, who had arranged for my research visit. Once I was settled in, I was introduced to the group I was going to work with Team Bonsai.
The project I worked on was very interesting and involved long read assembly visualization. Rayan, Jean-Stéphane and I developed a software called Falcon2Fastg - which converts the assembly from PacBio's Falcon assembler into the widely-used FASTG graphical format that can be visualized by existing tools such as Bandage. Our software lets users identify read or contig overlaps in the string graph, and figure out ways to improve the connectivity of their assembly without repeatedly re-running the assembler with different combinations of parameters. This received positive feedback on Twitter, and we plan to continue supporting and enhancing the tool with newer features.
Over the course of my time in France, I did attempt to speak as much of the language as I could : especially in the restaurants of Old Lille and at ticket counters. For the most part, my requests of "Parlez-vous Anglais?" were met by the locals with a firm shake of the head, and subsequently I had to resort to my broken French to communicate, which was fun for me at least! Traveling in Europe was a pleasant experience, except for the condemnable attacks on Paris on November 13th, and the resulting high-security alert.
I could go on and on, but in the interest of time I think I should end here. Feel free to get in touch if you'd like to talk more about anything you've read here. To conclude, I'd like to acknowledge the folks who made this opportunity possible: Rayan and the Bonsai team for being fantastic hosts and Kateryna and the Y-chromosome team for letting me take off on short notice to work on a different project. Finally, for those of you considering an internship, I would encourage you to first evaluate if the project meets your interests, and if it does - sit down with your advisor and work out a plan that lets you take that opportunity. Often, an internship is more than just a few lines of code : it's a chance for you to collect in-demand skills, make memories, and return with wholly new ideas and energy.
Robert Baker, my former PhD advisor, visited our lab several weeks ago. All of us had great time together. Dr. Baker’s visit reminded me of simple rules he followed while running his laboratory.
They are highly relevant to our work and everyday life.
1. Amplify your strengths and cover your weaknesses. Nobody is perfect, but everyone is perfect at something.
2. Identify who your boss is and make her/him happy. It might be your advisor, it might be your wife, it might be your mother -- whoever your boss is, make her happy, and you will be happy too.
3. You gonna die soon, so live every day as your last day. Do something meaningful and enjoy.
4. Being a scientist will never bring you a lot of money, so do it only if you really like it. Otherwise -- do something else.
5. Every successful relationship (including marriage) follows a 60:40 rule. If you feel like you are giving 60% and getting 40% -- this is as good as it gets.
6. Whatever the truth is, embrace it. It's OK if your result is not making a sensation.
7. Be hungry for knowledge and for truth, and be around people who are just as hungry as you are.
8. Always do your homework -- before an interview, before a conference, even before a visit to another campus. It pays off to be prepared.
9. Never use red on dark background and never make slides nobody can read from the last row.
10. The author line is never fair, but as a PI you should make it as fair as possible by discussing it early.