LLMs and Research Methodology Transformation
Abstract
The emergence of large language models (LLMs) such as GPT-4, Claude, and Gemini has initiated a profound methodological transformation in the global research ecosystem. This systematic literature review examines how LLMs are redefining the processes of knowledge generation, analysis, and dissemination across academic disciplines. Guided by the PRISMA 2020 framework, the study systematically analyzed 92 peer-reviewed articles published between 2021 and 2025 to identify the scope, methodological adaptations, and ethical implications of integrating LLMs into research workflows. Findings indicate that LLMs are now embedded in nearly all stages of the research process ranging from literature retrieval and screening to data extraction, synthesis, and academic writing. Their integration has significantly enhanced research efficiency, reduced manual workload, and democratized access to advanced analytical tools, particularly benefiting early-career and non-native English-speaking scholars. However, the review reveals that the methodological evolution brought by LLMs also entails new challenges related to accuracy, bias, authorship, reproducibility, and transparency. The analysis demonstrates that researchers are increasingly adopting hybrid human AI collaboration models in which LLMs function as cognitive partners while human oversight ensures validation and contextual interpretation. Moreover, new research protocols emphasize prompt documentation, version control, and ethical disclosure of AI involvement to uphold scientific integrity. Despite their advantages, concerns persist regarding hallucinated outputs, algorithmic bias, and unequal access to proprietary AI technologies, which risk widening global disparities in research capacity. The review concludes that LLMs are transforming research methodology from static, manual processes into dynamic, AI-augmented workflows characterized by adaptability, scalability, and inclusivity. Nonetheless, this transformation demands continuous recalibration of ethical standards, methodological transparency, and academic accountability. The study recommends the establishment of standardized frameworks for reporting LLM usage, comprehensive AI literacy programs within academic institutions, and equitable policies for access to AI infrastructure. Overall, the responsible integration of LLMs represents not a replacement of human intellect, but a reconfiguration of scholarly practice that merges human reasoning with computational intelligence to advance credible, efficient, and globally inclusive scientific inquiry.
Keywords: Large Language Models, Research Methodology, Systematic Literature Review, Artificial Intelligence, Academic Writing, Human–AI Collaboration, Methodological Transformation, Ethical Research Practices, Knowledge Synthesis, Reproducibility
References
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
CASP. (2018). Critical Appraisal Skills Programme (CASP) Checklist. CASP UK.
D’Amico, V., Shih, J., & Langer, M. (2024). Automating systematic reviews with large language models: Accuracy and reliability assessment. Journal of Biomedical Informatics, 158, 104–115.
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. JAMA Network Open, 6(1), e2254519.
Jadhav, S., Srinivasan, R., & Banerjee, S. (2023). Evaluating ChatGPT for data extraction in systematic reviews: Performance and limitations. BMC Medical Research Methodology, 23(192), 1–10.
Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report, Keele University and Durham University Joint Report.
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., ... & Tseng, V. (2023). Performance of ChatGPT on the United States Medical Licensing Examination. PLOS Digital Health, 2(2), e0000198.
Li, Y., Guo, Z., & Li, J. (2024). Potential roles of large language models in the production of systematic reviews and meta-analyses. Journal of Medical Internet Research, 26(1), e56780.
OpenAI. (2023). GPT-4 technical report. OpenAI.
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71.
Qureshi, H., Lim, Y., & Chen, W. (2024). Large language models for qualitative research: Opportunities and challenges. Computers in Human Behavior, 152, 108017.
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., ... & Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180.
Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339.
Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science, 379(6630), 313.
Van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: Five priorities for research. Nature, 614(7947), 224–226.
Zhu, Y., Guo, Z., & Li, J. (2023). Democratizing research writing through AI: Implications of large language models for non-native scholars. Journal of Scholarly Publishing, 55(1), 34–52.
