Ask a Question, Refine Your Hypothesis, Then Read, Edit, Repeat

The place to start is the research question, the overall objective of your research. This question can come from anywhere, but is typically triggered by clinical interests, previous cases, or perhaps frustrating or disappointing outcomes. You need to be passionate about answering this question. If you are not invested in discovering the answers then no-one else will be. All good research questions should be tempered and informed by prior experience and they must pass the “Who cares?” test.¹ In other words, if you were to successfully answer your question, would it make a difference to your patients and inform or influence clinical practice? Aim to do research that is as worthwhile and clinically relevant as possible. Avoid falling into the trap of thinking too big, you cannot sensibly have “Cure Cancer” as an aim. Likewise, do not think too small, pursuing only marginal or incremental gains will not meaningfully advance medicine. A useful acronym characterizing essential elements of a good research question is FINER: Feasible. Interesting. Novel. Ethical. Relevant.^2,3

Research questions often begin with a clinical conundrum that must be narrowed down to discrete components that can be addressed by available research methods. For example, your objective might be to determine “the best way to prevent recurrence of feline urethral obstruction (FUO).” Achieving this overall research objective would undoubtedly improve patient outcomes, but, stated as above, it is too vague to address. You would need to refine it in order to transform it into a series of questions you can address experimentally. Start by trying to state the problem as clearly, concisely, and precisely as you can. Then you need to do some research, reading around the topic, to determine what is known and what is unknown. What is the underlying physiology and pathophysiology that makes FUO occur and what contributes to recurrence or recovery? What has already been attempted to address your question? There is no sense in repeating something that has already been adequately assessed and found to be unsuccessful. What novel thinking, new strategies or different approaches could you bring to bear? Try to generate creative and logical alternative solutions. Your best guess about the solution to the problem is your hypothesis. You will probably need to iterate your statement of hypothesis several times. A research question is essentially a hypothesis asked in the form of a question. Or alternatively, a hypothesis is a statement of the most likely potential answer to your research question. It needs to be precise, concise, and most importantly testable—it must be possible to show that it is false.

A common error in this phase of research is a failure to carefully examine the literature for similar research.⁴ It is useful to find data that inform and support your hypothesis, but you need to find out if someone has already adequately answered your question before you invest time, effort, and money pursuing it. Similarly, it is also common for new investigators to read the literature but fail to critically assess it. Just because it is in a journal article does not mean it is true.^5,6 If you have good reasons to question the dogma,^7-9 or to believe errors were made, then study repetition done better might be viable. A good understanding of the field and the ability to spot opportunities only comes from close and critical evaluation of the literature.

Plan Your Research Thoroughly

Once you have your research question and a refined hypothesis, then it is time to develop your plan to test it. One of the most important ways to enhance this part of the research process is to start early—that means months in advance of when you hope to start collecting data. It is sensible to get help early, from colleagues, mentors, and a biostatistician. Successful research is typically a team effort involving multiple people, even if the study is being conducted in a single center. Recruiting and engaging collaborators in your research will likely enhance the study design, identify and fix errors, and enable it to happen in a timely manner. Ultimately, having a committed and diverse research team will likely improve the quality of the work. It may also be necessary to collaborate with other experts if you need specific assays, imaging, or analyses to be performed.

It is much, much easier to fix potential issues in the research plan before you start collecting data than to try to deal with them after the fact. If you are applying for a grant to fund your research, then you will need to provide the funding agency with a detailed research plan. But even if you are not applying for funding, it is a good idea to write a thorough research proposal for every study you conduct—even retrospective studies. This process helps you to identify all the data you might need when you need to acquire it and how you will collect it. Make sure you collect all of the data you might conceivably need. Although it is more time consuming to collect more information, it is better to get everything you need than to be unable to calculate something crucial because you did not obtain the data. For example, it is impossible to calculate the APPLE score,¹⁰ if you do not have pulse oximetry reading and you cannot even calculate the APPLE_fast score if you do not have a lactate concentration. This also applies to retrospective studies because it is very disheartening to have to go back through 250 records to find one additional piece of data that is needed. It is inevitable that the reviewer of your paper will insist on you providing the one piece of information that you forgot to collect!

Study design is a complex area that is beyond the scope of this session. But it is really important to select a suitable type of study that will answer your question effectively and efficiently. Some questions can only be answered using a randomized clinical trial, but in other cases, the question can be answered more efficiently, more rapidly and less expensively using a case-control study. There are some really good resources you should consult as you begin to design your research study.¹¹ There are quite a few potential pitfalls in study design, but some of the most common are an inadequate sample size, lack of validation of the methods of data collection or measurement, and inadequate or suboptimal controls. It is crucial that the study is appropriately sized to enable the hypothesis to be adequately tested to avoid type II errors (false negatives). However, having a really big sample size is not necessarily desirable either, since it may be costly in terms of time and resources to recruit more cases than you need. The sample size required for any study is dependent on the acceptable level of significance, or alpha (typically set at 0.05); the power of the study, or 1-beta (typically set at 80–90%); the expected effect size, how large of a difference you think is biologically plausible or clinically relevant between the groups; the underlying event rate in the population and the degree of variation of the parameters being measured in the population.^12-14

Errors in sizing studies are common and are often due to budgetary constraints or limitations of time. Frequently, we find ourselves needing to enroll more patients than we can afford to study. For instance, your sample size calculations might suggest you need 120 patients per arm of a clinical trial, but you cannot afford to recruit and study more than 60 in total. Or you can afford to recruit all the patients you need, but at typical patient accrual rates, it would take 5 years to complete your study. What should you do? For the budgetary constraints, you might be able to simplify the study design, measuring fewer things or measuring them less frequently. Can you eliminate non-essential data collection or use a cheaper assay? If there is no way to test your hypothesis within the budget, then avoid the temptation to look for unrealistically large effect sizes—it is not likely you will find them and they would not be biologically plausible even if you did.¹⁵ Alternatively, you can look for alternative sources of (more) funding or redesign the study to answer a slightly different question that will still progress your research agenda. You should also avoid the temptation to ignore your sample size calculations and press on regardless—you will face considerable difficulties publishing your results and the utility of any data you do produce will be substantially reduced.

For whichever method you use to measure the parameters of interest in your study, you must ensure that the method has been validated for the purpose and the species you plan to study.¹⁶ For instance, if you are using blood pressure measurements,¹⁷ ultrasound findings,¹⁸ or the concentration of a blood biomarker to test your hypothesis,¹⁹ can you rely on the data? And can you demonstrate the data the method generates are robust? Are the measurements precise? Accurate? Repeatable? Reproducible?^20-22 How much error or variation in your data is due solely to the method of measurement? How can you best reduce the error in your measurements?—Is that through measurement repetition (more measurements) or biological repetition (more sample or more patients)?

Write Good Grant Applications

Good grants take time to write; give yourself enough time to get it right.²³ Poorly written grants are unlikely to be funded, particularly in the current funding environment, and reviewers have memories—you do not want to be remembered for submitting something substandard. Some of the most common issues with grants are easily fixed.^24-26 Make sure you read and follow all the instructions including formatting. Avoid submitting a grant with typographical or grammatical errors; proofread your proposals carefully. Make sure that the work you propose is likely to be attractive to the funding body you are applying to. Submitting a feline project to the American Kennel Club is unlikely to result in funding! This means you need to do some background research and try to address the funders' interests.^27,28 Looking at projects they have recently funded is very helpful in this regard. As you set out the aims for your research proposal, it is vital that your aims are specific, clearly stated, focused, achievable and independent. It is also fundamental that your aims and the approaches you propose to address them will actually test your hypothesis. Ideally, it should be apparent that your specific aims fit into the larger arc of your research objectives. Be ambitious, but do not overreach. You need to be able to demonstrate that you can do the work you propose, that there are reasonable expectations the work will be fruitful, and that the project will produce meaningful, valuable results if it is successful. If you are proposing to study several elements of a clinical problem, it is crucial that if one aim cannot be successfully completed or the associated hypothesis proves to be incorrect that the other aims can be successfully completed. If aim 3 is dependent on aim 2, which in turn is dependent on aim 1, then it is unlikely your grant will be funded. In terms of the people proposing and conducting the research, your grant should demonstrate with preliminary data or publications that you or your research team can perform the work you propose. As a more junior researcher, if you do not have demonstrated experience or expertise, then collaborate with a more established investigator who can mentor you and support the project and for whom it is possible to show clear evidence of expertise.

Your descriptions of the methods should provide enough detail that it is clear how the work will be accomplished and that you have thought about practicalities and potential pitfalls. But avoid the temptation to provide too much unnecessary experimental detail. If you are using a novel method, provide more detail on those approaches and supply enough preliminary data to establish feasibility. Your approach should clearly include the necessary controls, descriptions of how you will deal with bias, and how you intend to ensure an adequate sample size. You will also need to discuss how you will analyze and interpret the data you gather. Many veterinary funding bodies preclude you from purchasing equipment, so you will need to show that you have the necessary analyzers, tools, and devices the study will employ. The budget should be complete, correctly formatted, and appropriately justified. Make sure you ask for everything that you will need and that you can clearly and reasonably justify purchasing, but avoid packing your budget with unnecessary or superfluous items. If you are going to need assistance with data gathering or analysis and you will need to pay someone to do that, then include it. Alternatively, find a collaborator who is willing to do it for free in exchange for inclusion in your project and authorship on the final paper.

Do Good Research

Hopefully, with enough planning or undertaking the process of a grant application, you will have a detailed, written and vetted protocol. Make sure you follow it, and that any amendments are clearly noted. When it comes time to actually doing the work, be deliberate, methodical, precise, accurate, and patient, and aim to be as consistent as possible. Plan ahead, give yourself time to get the work done. Recognize that there will be setbacks, problems to overcome and that mistakes and missteps will be made regardless of how well you have planned things. This is part of doing research and you should aim to reassess, refine, and repeat things that did not work well. It is a good idea to aliquot valuable research samples in case you need to repeat any assays or perform dilutions to achieve accurate measurements. Accurate results depend upon multiple factors, but one easily eliminated source of variation is your equipment. Make sure whatever you use is well maintained, has been recently serviced, calibrated and that quality control and assay controls have been analyzed, passed and documented. Maintain an audit trail of these things in case of any systematic errors. You will need to estimate the contribution of errors in your measurements to the overall variation in your data.

Vigorously recruit and retain subjects for your studies and then keep careful track of the patients or study subjects. Be sure to recruit suitable controls for your studies that help you to answer your research question and to optimally test your hypothesis. For instance, healthy “normal” animals are not a valid control for an assay or test that is designed to diagnose disease because you will never use the test to differentiate healthy controls from diseased animals. Instead, you need to use a control population with similar clinical signs but an alternative diagnosis to determine the discriminant power of the new test. Try to avoid selection or observer bias in anything and everything you measure—blind as many aspects of your study as possible, including the analysis.

Nota Bene

As you are conducting your study, regardless of whether it is an experimental study or a clinical research project, be sure to make careful, detailed notes along the way. The experimental log should detail as much information as possible about how the experiments were conducted, including what reagents or assays were used and under what conditions. Each study should have its own files, binders, and forms to aid standardized data collection and recording and to keep all of the relevant records safe. In most cases, you will need to keep the associated files, forms, and information for a minimum of 3–5 years after the completion of the study. Keep careful track of all of the samples you collect, catalog them, and be systematic so that you can access them when you need to. You do not want to spend hours with frozen fingers searching through your -80°C freezer looking for poorly labeled plasma samples! Likewise, invest in a labeling system that will not slip off in the freezer or disappear if it comes into contact with isopropanol or ethanol. It is also crucial to be meticulous about your electronic files. Keep track of them using a uniform naming and filing system so that you can retrieve the relevant file easily and quickly. The longer the study goes on, the less you will remember about what you put where, and it is very frustrating to spend hours looking for that file “you know you have somewhere.” Make sure you maintain a backup copy of all of your electronic files, back up your files regularly to a physical drive (kept separate from your computer) or to the cloud. Finally, as you are recording data manually or electronically, institute some system to periodically check the completeness and integrity of the data. Do you have everything you need? Are there any obvious data entry or transcription errors, like a respiratory rate of 300 or a heart rate of 16?

Avoid Common Statistical Errors

Analyze your data with the correct statistical test, interpret the findings correctly, but avoid overinterpretation, so that you can draw valid conclusions that are supported by your results. Statistical errors are very common in the published literature.^29-32 There are some very nice reviews of the most common errors with advice on how to avoid them.^33-38 In addition, there are some very useful primers on biomedical statistics in the veterinary literature.^39-46 Statistical errors can occur in the study design phase, in data analysis or documentation, or in the reporting and presentation. Familiarize yourself with the list in this review,⁴⁷ and do your best to avoid these errors. There are also various reporting guidelines (www.equator-network.org/about-us/uk-equator-centre/) for various types of studies that can help you along the way.⁴⁸

Reporting

It is crucial that you report your results in a timely manner. This is true whether your results confirm or refute your hypothesis. Assuming you did a thorough job performing your study, then it is vital to report negative results.³³ This prevents others unnecessarily repeating your work, saving the community time and money, and sparing animals from needless involvement in fruitless research endeavors. Regardless of the outcome, when it comes to writing up your data, do not delay. The longer you wait to begin writing up the study, the more challenging it will be to recall all of the details. Delays in reporting your data have other consequences too. It may be that you need the publication in print to enable future studies, for example with methods papers validating a new assay or measurement technique. You may need the publication in press in order to provide evidence of expertise or experience to justify a future grant application. Or it may be that you are competing against other investigators in the field, and it is frustrating to be “scooped.”⁴⁹

The aim of publishing your findings is to provide readers with an accurate and complete report of your methods and results. To do that, you need to write well and to understand and use correct scientific language. If you are not writing in your native language, consider having a colleague proof-read your manuscript before submission. Your report should be concise, precise, and accurate and needs to supply as much detail as possible in an easily read manner. Your report is also your way to sell your findings to your readers and to convince them of your arguments and interpretations. However, it is also necessary to point out the weaknesses of your study, while avoiding strawman arguments, i.e., weak misrepresentations of your study that you can easily defend. Similar to study design and grant writing, there are some very good resources about how to write scientific papers.^50-58 Ultimately, the more you read and the more you write, the better your research papers will be.

References

1.  Kim YJ, Mack SJ, Chung KC. Articulating the "So, What?" in Clinical Research: Insight from the M-CHOIR Group. Plast Reconstr Surg Glob Open. 2020;8(5):e2848.

2.  Fandino W. Formulating a good research question: pearls and pitfalls. Indian J Anaesth. 2019;63(8):611–616.

3.  Ratan SK, Anand T, Ratan J. Formulation of research question—Stepwise approach. J Indian Assoc Pediatr Surg. 2019;24(1):15–20.

4.  Clark GT, Mulligan R. Fifteen common mistakes encountered in clinical research. J Prosthodont Res. 2011;55(1):1–6.

5.  Ioannidis JP. How to make more published research true. PLoS Med. 2014;11(10):e1001747.

6.  Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.

7.  Maxmen A. Taking risks to transform science. Cell. 2009;139(1):13–15.

8.  Byrne L, Obonyo NG, Diab SD, et al. Unintended consequences: fluid resuscitation worsens shock in an ovine model of endotoxemia. Am J Respir Crit Care Med. 2018;198(8):1043–1054.

9.  Hippensteel JA, Shapiro NI, Schmidt EP. Challenging dogma: the value of bolus fluids in the early resuscitation of hyperdynamic sepsis. Am J Respir Crit Care Med. 2018;198(8):981–983.

10.  Hayes G, Mathews K, Doig G, et al. The acute patient physiologic and laboratory evaluation (APPLE) score: a severity of illness stratification system for hospitalized dogs. J Vet Intern Med. 2010;24(5):1034–1047.

11.  Hulley SB, Cummings SR, Browner WS, et al. Designing Clinical Research. 4th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2013.

12.  Sormani MP. The most frequently asked question to a statistician: the sample size. Mult Scler. 2017;23(5):644–646.

13.  Wang X, Ji X. Sample size estimation in clinical research: from randomized controlled trials to observational studies. Chest. 2020;158(1s):S12–s20.

14.  Devane D, Begley CM, Clarke M. How many do I need? Basic principles of sample size estimation. J Adv Nurs. 2004;47(3):297–302.

15.  Ridgeon EE, Bellomo R, Aberegg SK, et al. Effect sizes in ongoing randomized controlled critical care trials. Crit Care. 2017;21(1):132.

16.  Floras AN, Holowaychuk MK, Hodgins DC, et al. Investigation of a commercial ELISA for the detection of canine procalcitonin. J Vet Intern Med. 2014; 28(2):599–602.

17.  da Cunha AF, Saile K, Beaufrère H, et al. Measuring level of agreement between values obtained by directly measured blood pressure and ultrasonic Doppler flow detector in cats. J Vet Emerg Crit Care (San Antonio). 2014;24(3):272–278.

18.  Smith JJ, Fletcher DJ, Cooley SD, Thompson MS. Transpalpebral ultrasonographic measurement of the optic nerve sheath diameter in healthy dogs. J Vet Emerg Crit Care (San Antonio). 2018;28(1):31–38.

19.  DeClue AE, Osterbur K, Bigio A, Sharp CR. Evaluation of serum NT-pCNP as a diagnostic and prognostic biomarker for sepsis in dogs. J Vet Intern Med. 2011;25(3):453–459.

20.  Irvine KL, McLeish SA, Sarvani E, Papasouliotis K. Analytical quality assessment and method comparison of two immunoassays for the measurement of serum cardiac Troponin I in dogs and cats. Vet Clin Pathol. 2019;48 Suppl 1:70–77.

21.  Nye CJ, Musulin SE, Hanel RM, Mariani CL. Evaluation of the Lactate Plus monitor for plasma lactate concentration measurement in dogs. J Vet Emerg Crit Care (San Antonio). 2017;27(1):66–70.

22.  Goggs R, Serrano S, Szladovits B, et al. Clinical investigation of a point-of-care blood ammonia analyzer. Vet Clin Pathol. 2008;37(2):198–206.

23.  Arthurs OJ. Think it through first: questions to consider in writing a successful grant application. Pediatr Radiol. 2014;44(12):1507–1511.

24.  Gholipour A, Lee EY, Warfield SK. The anatomy and art of writing a successful grant application: a practical step-by-step approach. Pediatr Radiol. 2014;44(12):1512–1517.

25.  Visovsky C. Writing a successful grant: tips and tools. J Adv Pract Oncol. 2015;6(3):279–280.

26.  Wisdom JP, Riley H, Myers N. Recommendations for writing successful grant proposals: an information synthesis. Acad Med. 2015;90(12):1720–1725.

27.  AKC-CHF. Our Research. www.akcchf.org/research/our-research/; 2020.

28.  MAF. Grants. www.morrisanimalfoundation.org/grants; 2020.

29.  Benlidayi IC. Statistical accuracy in rheumatology research. Mediterr J Rheumatol. 2019;30(4):207–215.

30.  Günel Karadeniz P, Uzabacı E, Atış Kuyuk S, et al. Statistical errors in articles published in radiology journals. Diagn Interv Radiol. 2019;25(2):102–108.

31.  Shott S. Detecting statistical errors in veterinary research. J Am Vet Med Assoc. 2011;238(3):305–308.

32.  Simundic A-M, Nikolac N. Statistical errors in manuscripts submitted to Biochemia Medica journal. Biochemia Medica. 2009;19(3):294–300.

33.  Silva Aycaguer LC. Frequent methodological errors in clinical research. Med Intensiva. 2018;42(9):541–546.

34.  Lee S. Avoiding negative reviewer comments: common statistical errors in anesthesia journals. Korean J Anesthesiol. 2016;69(3):219–226.

35.  Fernandes-Taylor S, Hyun JK, Reeder RN, Harris AH. Common statistical and research design problems in manuscripts submitted to high-impact medical journals. BMC Res Notes. 2011;4:304.

36.  Zinsmeister AR, Connor JT. Ten common statistical errors and how to avoid them. Am J Gastroenterol. 2008;103(2):262–266.

37.  Lang T. Twenty statistical errors even you can find in biomedical research articles. Croat Med J. 2004;45(4):361–370.

38.  Nuzzo R. Scientific method: statistical errors. Nature. 2014;506(7487):150–152.

39.  Shott S. Comparing means or distributions. J Am Vet Med Assoc. 2011;238(11):1422–1428.

40.  Shott S. Relationships between more than two variables. J Am Vet Med Assoc. 2011;239(5):587–593.

41.  Shott S. Relationships between two categorical variables and between two noncategorical variables. J Am Vet Med Assoc. 2011;239(1):70–74.

42.  Shott S. Comparing percentages. J Am Vet Med Assoc. 2011;238(9):1122–1125.

43.  Shott S. Testing ideas and estimating clinical importance. J Am Vet Med Assoc. 2011;238(7):871–876.

44.  Shott S. Statistics simplified. Describing data. J Am Vet Med Assoc. 2011;238(5):588–591.

45.  Lamb CR. Statistical briefing: type 1 and type 2 errors. Vet Radiol Ultrasound. 2009;50(2):239.

46.  Lamb CR, Pfeiffer DU. Statistical briefing: confidence intervals. Vet Radiol Ultrasound. 2009;50(1):109–110.

47.  Strasak AM, Zaman Q, Pfeiffer KP, et al. Statistical errors in medical research--a review of common pitfalls. Swiss Med Wkly. 2007;137(3–4):44–49.

48.  Grech V. Write a Scientific Paper (WASP): Guidelines for reporting medical research. Early Hum Dev. 2019;134:55–57.

49.  Kim JS, Corn JE. Sometimes you're the scooper, and sometimes you get scooped: how to turn both into something good. PLoS Biol. 2018;16(7):e2006843.

50.  Hall GM. How to Write a Paper. 5th ed. Chichester, West Sussex, UK: John Wiley & Sons; 2013.

51.  Forero DA, Lopez-Leon S, Perry G. A brief guide to the science and art of writing manuscripts in biomedicine. J Transl Med. 2020;18(1):425.

52.  Bahadoran Z, Mirmiran P, Kashfi K, Ghasemi A. The principles of biomedical scientific writing: abstract and keywords. Int J Endocrinol Metab. 2020;18(1):e100159.

53.  Bahadoran Z, Mirmiran P, Kashfi K, Ghasemi A. The principles of biomedical scientific writing: citation. Int J Endocrinol Metab. 2020;18(2):e102622.

54.  Ghasemi A, Bahadoran Z, Zadeh-Vakili A, et al. The principles of biomedical scientific writing: materials and methods. Int J Endocrinol Metab. 2019;17(1):e88155.

55.  Ghasemi A, Bahadoran Z, Mirmiran P, et al. The principles of biomedical scientific writing: discussion. Int J Endocrinol Metab. 2019;17(3):e95415.

56.  Bahadoran Z, Mirmiran P, Zadeh-Vakili A, et al. The principles of biomedical scientific writing: results. Int J Endocrinol Metab. 2019;17(2):e92113.

57.  Bahadoran Z, Mirmiran P, Kashfi K, Ghasemi A. The principles of biomedical scientific writing: title. Int J Endocrinol Metab. 2019;17(4):e98326.

58.  Bahadoran Z, Jeddi S, Mirmiran P, Ghasemi A. The principles of biomedical scientific writing: introduction. Int J Endocrinol Metab. 2018;16(4):e84795.