Work & Life
Published November 8, 2017
Turning an idea into an answerable question is arguably the hardest part of the research process and warrants time and effort. Creating an interesting and answerable question that advances a field of study by filling in a gap in the literature should be an iterative process involving reading and understanding the primary literature and exchanging ideas with colleagues, mentors, and principle investigators. In the textbook Designing Clinical Research, Dr. Stephen B. Hulley of the University of California, San Francisco advocates that a good research question be “FINER” — feasible, interesting, novel, ethical, and relevant. Keep these factors in mind when refining your question. In my experience, writing an idea in the format of a formal scientific proposal with a brief introductory literature review, hypothesis, and proposed methodology for early peer review is crucial because your colleagues’ ideas will inevitably strengthen your research question.
Writing an idea in the format of a formal scientific proposal with a brief introductory literature review, hypothesis, and proposed methodology for early peer review is crucial.
Making sure your project is feasible during medical school or residency and aligning it with your goals is important. Dr. Aashish Didwania, Residency Director of the Internal Medicine Program at Northwestern, advises that while the true value of research during training is to build your skills as an investigator and advance the science of medicine, productivity does matter for getting accepted to some competitive residencies and fellowships. Therefore, be thoughtful about selecting projects that deliver both learning and productivity. Dr. Lisa Vusse, Associate Program Director of the University of Washington Internal Medicine Program, adds that case reports, case series, and review articles are accomplishable projects for trainees with less time or those who want to focus on skills such as literature review, data analysis, and the publication process. Trainees with more time can also successfully contribute to original research by focusing on one aspect of a research team’s collective work.
I tend to prioritize clinical research projects that do not require that I collect the data myself — often a time-intensive process. In an era of increasing clinical research data sharing, consider approaching a principle investigator at your institution with a project idea or applying for access to publicly available data. For example, after taking an interest in a report of molecularly targeted adjuvant chemotherapy for bladder cancer published in the Journal of Clinical Oncology in 2011, I contacted the principle investigators and asked to use the trial data to perform additional analyses. This effort lead to my post-hocstudy of surgical quality in that trial, which was subsequently published in the Journal of Surgical Oncology in 2015.
If you cannot find data locally to suit your interests, public data sources are available (e.g., NHANES, SEER, and TCGA). In the spirit of collaboration, the New England Journal of Medicine recently hosted the SPRINT Data Analysis Challenge, in which the data from a prior NEJM article were made publicly available to advance medical science. The second-place winner of the SPRINT challenge was a team of medical students from Boston University.
You might need to collect or generate new data for your research question (e.g., conduct a chart review for a retrospective cohort study or perform experiments in a laboratory). If this is the case, make sure you leverage your institution’s resources. For example, if you have a list of patient identifiers and need to extract lab values, ask your colleagues in the clinical pathology laboratory to download the values from their electronic database to merge with your list, saving time and avoiding transcription errors. In the end, the type of project you embark on will depend, in part, on how your bandwidth and time constraints as a trainee mesh with the complexities of your proposal. You will gain a better understanding of this balance after completing a literature review.
The type of project you embark on will depend, in part, on how your bandwidth and time constraints as a trainee mesh with the complexities of your proposal.
A thorough understanding of the literature on your topic of interest is essential to formulate a practical, targeted question. I generally start a literature search in as systematic a manner as possible in a database such as PubMed using a Boolean query that casts a broad net. For example, I became interested in depression in medical students after witnessing many colleagues suffer from depressed mood and burnout. I crafted a targeted search in PubMed by combining keywords related to depression with those related to medical students. I used a combination of free-text terms as well as medical subject heading (MeSH) terms. Try pasting the following example into PubMed:
(Depression [MeSH]) OR
(Depressive disorder [MeSH]) OR
((Education, medical [MeSH]) OR
(Student doctor*) OR
(Undergraduate medic*) OR
(Med student*) OR
(Medical student*) OR
(Student physician*) OR
(Preclinical student*) OR
(Clinical student*) OR
An asterisk at the end of a word makes it a wild card. In the above search, the term “depress*” tells PubMed to search for both “depression” and “depressive,” ensuring identification of manuscripts about major depression as well as major depressive disorder. Try a few different search strategies, refining your key words as you go. Once you have completed your search, you can use the options in PubMed to filter your results by article type (e.g., review articles or primary literature), publication date, or journal. Other databases that might be relevant for your literature search include EMBASE and Web of Science or specialty search engines like PsycINFO or ERIC.
A thorough understanding of the literature on your topic of interest is essential to help you formulate a practical, targeted question.
After you have finished reviewing the literature identified in your systematic search, you must determine how your newfound knowledge affects your research question. For example, having read about depression in medical students in a cross-sectional study conducted at a busy quaternary care hospital, I might have hypothesized that students in a community hospital would have a lower burden of depression. To test this hypothesis, I could design a cross-sectional or prospective survey of students rotating in a community hospital using a validated screening questionnaire (e.g., the PHQ-9). However, when I first conducted the literature search on depression in medical students, I was surprised to find that a systematic review of the subject did not exist. After applying the “FINER” criteria, I realized I could fill a gap in the literature and decided to conduct (and publish) a systematic review and meta-analysis to estimate the prevalence of depression among medical students worldwide.
The study design you choose to answer your research question — be it case-control, cross-sectional, cohort, randomized, systematic review, or something else — will not only depend on the question itself, but also on your resources, skills, and time constraints. Ideally, choosing a design and defining the methodology should involve the creation of an a priori protocol. Just as the literature search informed your research question, expert consensus guidelines should inform your study design.
The Equator Network (Enhancing the QUAlity and Transparency Of health Research) is an international group that offers great resources on writing a study protocol. It is a one-stop solution for your study design needs and provides curated lists of reporting guidelines for all standard study designs as well as for case reports, diagnostic test validation studies, economic evaluations, qualitative and mixed methods studies, quality improvement studies, and pre-clinical animal studies. Although these guidelines are largely geared toward clinical research, their recommendations are also applicable to basic science. And while these guidelines technically advise how to report results, they provide an implicit framework for how to conduct studies. Be sure to read the guidelines before data collection. For the meta-analysis of depression prevalence in medical students, I followed the Equator Network’s suggestion to apply the PRISMA and MOOSE guidelines. These guidelines provided explicit checklists to help me conduct and report everything expected of a high-quality study.
The study design you choose to answer your research question will not only depend on the question itself, but also on your resources, skills, and time constraints.
Once you have chosen a study design, meet with a statistician or senior investigator to solicit feedback. Early statistical consultation ensures that you have selected the most appropriate design and will help determine how many participants or samples to include and how to best collect, structure, and analyze your data. You may need to conduct a pilot study to stress test your system for data collection, analysis, and reporting before beginning the main study. For example, although the study of depression in medical students eventually included data from 183 separate sources, I performed an initial pilot analysis of data from only 10 studies. The results of the pilot led me to modify my data extraction and analysis procedures, saving an inordinate amount of time in the long run.
Early statistical consultation ensures that you have selected the most appropriate design and will help determine how many participants or samples to include and how to best collect, structure, and analyze your data.
If you work with a statistician, you may not be the primary individual tasked with data analysis. Regardless, get to know and understand your data and approach it in a systematic and purposeful manner. Consider taking the following initial steps:
A Systematic Approach to Understanding Data
inspect and clean data to remove typos, properly format variables, and account for missing values
visualize and summarize data with descriptive statistics
graph continuous variables with histograms or scatter plots to examine the distribution and to assess for outliers
calculate measures of central tendency and dispersion
tabulate categorical variables and determine the percentage of observations within each category
stratify variables by factors of interest
look for patterns
Once you have a handle on your data, describe the participants or samples in your study. For example, for a randomized clinical trial, create a flow chart of study participants at each stage of the study, from potentially eligible patients to those who joined the study, completed follow-up, and whose data were analyzed. Create tables with descriptive characteristics of study participants, information on exposures and potential confounders, and outcomes. The goal is for the reader to understand the scope and limitations of your study population and data in as simple a manner as possible.
Get to know and understand your data and approach it in a systematic and purposeful manner.
You might be tempted to skip these simple steps in favor of testing your hypothesis with inferential statistics, but it is important not to get ahead of yourself. Only after you have completed the above steps should you proceed to hypothesis testing and fitting statistical models to your data. Use a logical order that parallels the stated aims in the introduction and analytical plan in the methods section to perform analyses and write the results and discussion sections of your manuscript. Properly analyzing and writing up a clinical or basic science study requires a broad set of skills. If you need help, consider collaborating with someone who has the skill set that you lack.
Poster and abstract presentations at scientific meetings are excellent initial opportunities for getting your work recognized and soliciting feedback, and it can precede formal publication. When the time comes to decide where to submit your manuscript for publication, consider your audience. Identify some realistic journal targets suited to your topic early on and a journal that is a reach (i.e., a high-impact journal). Review past issues of each journal, notice the types of articles published, and determine if your manuscript fits the subject matter and scope. A randomized-clinical trial with broad relevance to clinical practice might be a good candidate for a general medical journal with wide readership, while a retrospective, hypothesis-generating study might be better suited for a smaller specialty journal.
Identify some realistic journal targets suited to your topic early on and a journal that is a reach (i.e., a high-impact journal).
Most readers will find your study through targeted online searches, so write a title and abstract that accurately reflect your study and include relevant keywords. Do not be afraid of rejection. The odds are, no matter how great your study is, it will get rejected by more than one journal. Do not take rejection personally, but rather use editor and reviewer comments from rejections to strengthen your paper. Treat it as a learning experience.
Do not take rejection personally, but rather use editor and reviewer comments from rejections to strengthen your paper. Treat it as a learning experience.
I hope this article and the other resources available on NEJM Resident 360 provide a starting point for turning your research idea into a practical question and running with it. Remember, research is a team sport: If you want to conduct research but do not have a firm idea, identify a mentor with ongoing projects. The process might be tedious and challenging at times, but with the right support and guidance, you too can find success and satisfaction in research!
For more on this topic, you might be interested in Introduction to Research in Residency.