A cross-sectional study was conducted in the Department of Pathology at PES University Institute of Medical Sciences (PESU IMSR) in Bengaluru, Karnataka, India and PES Institute of Medical Sciences (PESIMSR) in Kuppam, Andhra Pradesh, India, from September 2023 to October 2023, lasting for two months. The study received approval from the Institutional Human Ethics Committee, bearing number PESIMSR/IHEC/C-146/2023.

Sample size: The sample size was calculated based on a study conducted by Hendriksma M et al., [8].

Inclusion and Exclusion criteria: Only journals indexed in PubMed/SCOPUS/DOAJ/EMBASE/Excerpta Medica/Indian Science Abstracts/IndMed or MEDLINE/Index Medicus/Scimago Journal Ranking/Web of Science/Science Citation Index Expanded/Index Copernicus were selected for evaluation. Only those pathology journals with published articles that were freely accessible were included in the study. Original articles of observational studies published in indexed journals between July 1, 2022 and August 31, 2023, were retrieved from five National (Indian) journals and five International journals for analysis. Journals in which any authors were on the editorial board, articles published in languages other than English and articles authored by the researchers were excluded. This exclusion was intended to minimise potential bias.

A total of 339 articles were retrieved, comprising 170 from National journals and 169 from International journals. After applying the exclusion criteria, 50 articles were randomly selected using a computer-generated random number method, consisting of 25 articles from National journals and 25 from International journals. Randomisation was important to reduce selection bias.

The 25 articles from National journals were selected from five journals, with five articles from each, while the same number was done for the five International journals. The classification of journals as National or International was based on the criteria established by Van Wingerden JJ criteria [9].

The National journals evaluated included were:

The International journals included:

Each article was meticulously evaluated based on 15 parameters related to the materials and methods aspects of the article. The critical evaluation included study design, setting, participants, variables, inclusion and exclusion criteria, sample size, ethical clearance statements, funding statements, statistical tools and methods, methodology details, study bias, study limitations and the concordance of study methodology with both aims and results. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline was also utilised for evaluation [10].

The quality of each article was documented and observational data were entered into an Excel sheet. Parameters were independently evaluated by pathologists from the two tertiary care referral institutions. Findings were documented and finalised through consensus.

The first 13 parameters were assessed for major and minor errors. Each was evaluated for whether the information was provided (addressed) or not. If not mentioned, it was categorised as a major error. If addressed, it was further analysed for adequacy, determined as either adequate or inadequate. If the information was provided but inadequately (minor error), the reason for the inadequacy was analysed and categorised. The last two parameters, regarding the concordance of study methodology with aims and results, were classified as either concordant or discordant.

Statistical analysis was performed using statistical software Stata version 17.0. The categorical variables were represented using frequencies and percentages. The Chi square test was employed for analysis of parameters such as study design, study setting, study participants, study variables, inclusion criteria, exclusion criteria, sample size, ethical clearance statement, funding statement, statistical tools and methods, methodology procedure details, study bias and study limitations, with a p value of <0.05 was considered as statistically significant.

In the present study, a total of 50 articles were evaluated. Each article was meticulously evaluated for 15 parameters.

Study design: The study design was mentioned in 34 articles (68%). It was not mentioned (major error) in 16 articles (32%). The study design was more frequently mentioned in the National journals [19 articles (76%)] compared to the International journals [15 articles (60%)]. However, the values were not statistically significant (p=0.225) [Table/Fig-1].

Out of the 34 articles, the information provided was found to be adequate in 13 articles (38.24%) and inadequate (minor error) in 21 articles (61.76%). The information provided was adequate in a relatively greater number of articles in the International journals [7 articles (46.67%)] in comparison with the National journals [6 articles (31.58%)]. However, the values were not statistically significant (p=0.369) [Table/Fig-2].

Study setting: The study setting was mentioned in 42 articles (84%). It was not mentioned (major error) in 8 articles (16%). The study setting was mentioned in a relatively greater number of articles in the National journals [22 articles (88%)] in comparison with the International journals [20 articles (80%)]. However, the values were not statistically significant (p=0.440) [Table/Fig-1].

Out of 42 articles, the information provided was found to be adequate in all 42 articles (100%). The information was adequate in both the National journals [22 articles (100%)] as well as International journals [20 articles (100%)] [Table/Fig-2].

Study participants: Study participants were mentioned in all 50 articles (100%). The study participants were mentioned in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-1]. Out of 50 articles, the information provided was found to be adequate in all the 50 articles (100%). The information was adequate in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-2].

Study variables: Study variables were mentioned in all 50 articles (100%), in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-1]. Out of 50 articles, the information provided was found to be adequate in all 50 articles (100%). The information was adequate in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-2].

Inclusion criteria: Inclusion criteria were mentioned in 47 articles (94%). It was not mentioned (major error) in 3 articles (6%). The inclusion criteria were mentioned in all the articles in the International journals [25 articles (100%)] compared to the National journals [22 articles (88%)]. However, the values were not statistically significant (p=0.074) [Table/Fig-1].

Out of 47 articles, the information provided was found to be adequate in 45 articles (98.74%) and inadequate (minor error) in 2 articles (4.26%). The information was adequate in all the articles in the International journals [25 articles (100%)] in comparison with the National journals [20 articles (90.91%)]. However, the values were not statistically significant (p=0.123) [Table/Fig-2].

Exclusion criteria: Exclusion criteria were mentioned in 25 articles (50%). They were not mentioned (major error) in 25 articles (50%). The exclusion criteria were mentioned in relatively greater number of articles in the International journals [15 articles (60%)] compared to the National journals [10 articles (40%)]. However, the values were not statistically significant (p=0.157) [Table/Fig-1].

Out of 25 articles, the information provided was found to be adequate in all the 25 articles (100%). The information provided was adequate in relatively greater number of articles in the International journals [15 articles (100%)] in comparison with National journals [10 articles (100%)] [Table/Fig-2].

Sample size: Sample size was mentioned in all the 50 articles (100%). The sample size was mentioned in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-1].

Out of 50 articles, the information provided was found to be adequate in 1 article (2%) and inadequate (minor error) in 49 articles (98%). The information was adequate in one article from a National journal [1 article (4%)] and inadequate in 24 articles (96%). The information was inadequate in all the International journals [25 articles (100%)]. However, the values were not statistically significant (p=0.312) [Table/Fig-2].

Ethical clearance statement: The ethical clearance statement was mentioned in 41 articles (82%). It was not mentioned (major error) in 9 articles (18%). The ethical clearance statement was noted in a relatively greater number of articles in the International journals [23 articles (92%)] in comparison with the National journals [18 articles (72%)]. However, the values were not statistically significant (p=0.065) [Table/Fig-1].

Out of 41 articles, the information provided was found to be adequate in 40 articles (97.56%) and inadequate (minor error) in 1 article (2.44%). The information was adequate in relatively greater number of articles in the International journals [23 articles (100%)] in comparison with the National journals [17 articles (94.44%)]. However, the values were not statistically significant (p=0.252) [Table/Fig-2].

Funding statement: The funding statement was analysed with respect to materials and methods. It was mentioned in 45 articles (90%) and not mentioned (major error) in 5 articles (10%). The funding statement was mentioned in a relatively greater number of articles in the International journals [23 articles (92%)] compared to the National journals [22 articles (88%)]. However, the values were not statistically significant (p=0.637) [Table/Fig-1].

Out of 45 articles, the information provided was found to be adequate in all the 45 articles (100%). The information was adequate in relatively greater number of articles in the International journals [23 articles (100%)] in comparison with National journals [22 articles (100%)] [Table/Fig-2].

Statistical tools and methods: Statistical tools and methods were mentioned in all the 50 articles (100%), in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-1].

Out of 50 articles, the information provided was found to be adequate in 49 articles (98%) and inadequate in 1 article (2%). The information was adequate in 24 articles from National journals [24 articles (96%)] and inadequate in one article (4%). The information was adequate in all the International journals [25 articles (100%)]. However, the values were not statistically significant (p=0.312) [Table/Fig-2].

Methodology procedure details: Methodology procedure details were mentioned in all 50 articles (100%), in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-1].

Out of 50 articles, the information provided was found to be adequate in all the 50 articles (100%) in both the National journals [25 articles (100%)] and the International journals [25 articles (100%)] [Table/Fig-2].

Study bias: Study bias was mentioned in 7 articles (14%) and was not mentioned (major error) in 43 articles (86%). The study bias was mentioned in a relatively greater number of articles in the International journals [6 articles (24%)] in comparison with the National journals [1 article (4%)]. The values were statistically significant (p=0.042) [Table/Fig-1].

Out of seven articles, the information provided was found to be adequate in all the 7 articles (100%). The information provided was adequate in relatively greater number of articles in the International journals [6 articles (100%)] in comparison with National journals [1 article (100%)] [Table/Fig-2].

Study limitations: Study limitations were analysed with respect to materials and methods-related aspects. Study limitations were mentioned in 29 articles (58%) and were not mentioned (major error) in 21 articles (42%). The study limitations were mentioned in a relatively greater number of articles in the International journals [19 articles (76%)] in comparison with the National journals [10 articles (40%)]. The values were statistically significant (p=0.010) [Table/Fig-1].

Out of 29 articles, the information provided was found to be adequate in 18 articles (62.07%) and inadequate (minor error) in 11 articles (37.93%). The information provided was adequate in a relatively greater number of articles in the International journals [11 articles (57.89%)] in comparison with the National journals [7 articles (70%)]. However, the values were not statistically significant (p=0.523) [Table/Fig-2].

Concordance of study methodology with aim and results: Study methodology was found to be concordant with respect to the aim of the study and the results in all the 50 articles (100%), in both National [25 articles (100%)] and International journals [25 articles (100%)] [Table/Fig-3].

Summary of major and minor errors: After considering all the parameters evaluated, major errors were found to be relatively more and constituted higher percentage in the National journals [76 major errors (58.46%)] in comparison with the International journals [54 major errors (41.54%)]. The values were found to be statistically significant (p=0.030). Minor errors were also found to be relatively more and constituted higher percentage in the National journals [44 minor errors (51.76%)] compared to the International journals [41 minor errors (48.24%)]. However, the values were not statistically significant (p=0.443) [Table/Fig-1,2,4].

Reasons for inadequacy in National and International journals: The exact type of study design was not specified in both the National journals [13 articles (61.9%)] and the International journals [8 articles (38.1%)]. Inclusion criteria were not clearly specified in the National journals [2 articles (4.25%)]. The basis of calculation was not specified in both the National journals [24 articles (96%)] and the International journals [25 articles (100%)]. The ethical clearance statement appeared to be incomplete with typographical errors in a National journal [1 article (4%)]. The exact details of the statistical tools employed were not mentioned in one article published in the National journal [1 article (4%)]. Study limitations were not clearly mentioned in both the National journals [3 articles (30%)] and the International journals [8 articles (42.10%)] [Table/Fig-5].

Medical research is expanding rapidly and clinicians must provide the best treatment based on current knowledge. Research articles communicate scientific findings to stakeholders and clear, transparent communication is essential for understanding the methods, results and conclusions of studies [11,12]. To enhance reporting quality, experts have developed guidelines, including the STROBE checklist for observational studies. In the era of Evidence-based Medicine (EBM) [10], integrating research into clinical practice necessitates critical evaluation skills. The materials and methods section is vital for publication, as it ensures reproducibility and validity of results. This study highlights the importance of critically appraising the materials and methods in pathology journals [13,14].

The study by Hadavi M et al., evaluated the highest number of journals [15], while Hendriksma M et al., Poorolajal J et al., and the present study assessed fewer [8,16]. Conversely, Kosaryan M and Rabiei K, Sorensen AA et al., and Nasr SS et al., evaluated the least [6,17,18] [Table/Fig-6] [6,8,15-18]. Nasr SS et al., reviewed the most articles [18], while Hendriksma M et al., reviewed the least [8]. In contrast to the present study, others evaluated more articles, including Kosaryan M and Rabiei K and Hadavi M et al., [Table/Fig-6] [6,15]. Hadavi M et al., also reported the highest number of major errors [15], while the present study showed the least. This may be due to the larger number of evaluated articles in Hadavi M et al.,’s work [15]. Kosaryan M and Rabiei K noted a higher percentage of major errors compared to the present study [6]. Hendriksma M et al., provided only percentages without exact numbers [Table/Fig-6] [8]. Only Kosaryan M and Rabiei K and the present study reported both the number and percentage of minor errors [6]. Kosaryan M and Rabiei K documented a higher number but a lower percentage of minor errors compared to the present study [Table/Fig-6] [6]. Total errors were highest in the study by Hadavi M et al., [15]. The present study had the fewest total errors, likely due to evaluating fewer articles compared to Hadavi M et al., [15]. Hendriksma M et al., also did not specify the exact number of errors [Table/Fig-6] [8].

Comparison of methodology-related parameters in various studies: Nasr SS et al., evaluated articles from various specialties, including clinical pathology and oncology, published over four years [18]. Hendriksma M et al., reviewed general medicine and otolaryngology articles over five years and eight months [8]. Kosaryan M and Rabiei K focused on psychiatry and behavioural science articles published over three years [6]. Sorensen AA et al., analysed orthopaedic surgery articles published in one year [17], while Poorolajal J et al., examined epidemiology and biostatistics articles over one year and two months [16]. Hadavi M et al., assessed articles over ten years without specifying the specialty [15]. In contrast, this study evaluated pathology articles over one year [Table/Fig-7].

Study designs of articles evaluated by Nasr SS et al., included case-control studies, cohort studies, cross-sectional studies, clinical trials, documentary studies and case series [18]. The study designs of articles evaluated by Poorolajal J et al., included only cohort studies [16]. The study designs of articles evaluated by Hadavi M et al., included only case-control studies [15], whereas the study designs of articles evaluated by Hendriksma M et al., Sorensen AA et al., and the present study included only observational studies [8,17].

Choosing the right study design is the most crucial decision to ascertain the methodology of any research [19]. Study design was mentioned in 68% of articles in the present study. The study design was mentioned in higher percentage of articles in the study conducted by Hadavi M et al., [15]. In contrast, the study design was mentioned in lower percentage of cases in the other studies [6,8,16-18].

Major types of observational studies include cross-sectional studies, case-control studies and cohort studies. Each type of study employs different methodologies [18]. The procedures described in the methodology correlate with the results [11]. Different methodologies yield different outcomes. Each type of study needs to be reported accordingly as per the standard reporting protocol (STROBE) checklist [10]. Hence, methodological diversity has an important influence on the outcome of the study and the reporting standards [Table/Fig-7].

The study setting was mentioned in 84% of articles in the present study. It was mentioned in a higher percentage of articles in the studies conducted by Hadavi M et al., Poorolajal J et al., and Nasr SS et al., [15,16,18]. However, it was lower in the studies conducted by Kosaryan M and Rabiei K, Hendriksma M et al., and Sorensen AA et al., [Table/Fig-7] [6,8,17].

Study participants were mentioned in 100% of articles in both the study conducted by Hendriksma M et al., (for General Medicine articles) and the present study [8]. However, study participants were not mentioned in any of the articles in the study conducted by Kosaryan M and Rabiei K [6]. Nasr SS et al., did not specify study participants in their study [18]. Study participants were mentioned in lower percentage of articles in the other studies [Table/Fig-7] [15-17].

Study variables were mentioned in 100% of articles in the present study. In contrast, study variables were mentioned in a lower percentage of articles in the studies conducted by Hendriksma M et al., Hadavi M et al., and Sorensen AA et al., [8,15,17]. However, other studies did not specify the study variables [Table/Fig-7] [6,16,18].

Inclusion criteria refer to the key features of target population that the investigators will employ to answer their research question [20]. Inclusion criteria were mentioned in 94% of articles in the present study. Inclusion criteria were mentioned in lower percentage of articles in the studies conducted by Kosaryan M and Rabiei K, Hadavi M et al., and Nasr SS et al., [6,15,18]. Other studies did not specify the inclusion criteria [Table/Fig-7] [8,17].

Exclusion criteria refer to the features of the potential study participants who meet the inclusion criteria but present with additional characteristics that could interfere with the success of the study or increase their risk of an adverse outcome [20]. Exclusion criteria were mentioned in 50% of articles in the present study. Exclusion criteria were mentioned in lower percentage of articles in the studies conducted by Kosaryan M and Rabiei K and Nasr SS et al., [6,18]. Other studies did not specify the exclusion criteria [Table/Fig-7] [6,8,15,17].

Sample size was mentioned in 100% of articles in the present study. In contrast, sample size was mentioned in lower percentage of articles in the studies conducted by Hendriksma M et al., Hadavi M et al., Poorolajal J et al., and Nasr SS et al., [8,15,16,18]. However, other studies did not specify the sample size [Table/Fig-7] [6,17].

The funding statement was mentioned in 90% of articles in the present study. It was mentioned in higher percentage of articles in the studies conducted by Hendriksma M et al., (for General Medicine articles) [8]. However, it was mentioned in a lower percentage of articles in the studies conducted by Hendriksma M et al., (for Otorhinolaryngology articles) [8]. The funding statement was not mentioned in any of the articles in the study conducted by Sorensen AA et al., [17]. Other studies did not specify the funding statement [Table/Fig-7] [6,15,16,18].

Statistical tools and methods were mentioned in 100% of articles in the studies conducted by Poorolajal J et al., and the present study [16]. Hendriksma M et al., did not specify the statistical tools and methods [8]. Statistical tools and methods were mentioned in a lower percentage of articles in the other studies [Table/Fig-7] [6,15,17,18].

Study bias was mentioned in 14% of articles in the present study. It was mentioned in higher percentage of articles in the studies conducted by Hendriksma M et al., (for General Medicine articles) and Poorolajal J et al., [8,16]. However, it was mentioned in lower percentage of articles in the studies conducted by Hendriksma M et al., (for Otorhinolaryngology articles), Hadavi M et al., and Sorensen AA et al., [8,15,17]. Other studies did not specify the study bias [6,18].

With respect to research, bias is any trend or deviation from truth in data collection, data analysis, interpretation and publication that can cause false conclusions. Bias may be found in any part of the research process. Moreover, it is imperative to acknowledge that bias is not either present or absent but occurs in degrees, as not all bias can be eliminated from a study [21].

In the present study, care was taken to minimise selection bias by using computer-generated numbers for selection of articles for evaluation. Each parameter was evaluated by two pathologists from different institutes and the findings were finalised by consensus. Care was taken to minimise bias to the best of the ability of the authors in the present study. However, information bias would be difficult to minimise in such studies, as it depends on the information provided by other authors [Table/Fig-7].

Study limitations were mentioned in 58% of articles in the present study. Nasr SS et al., did not specify study limitations in their study [18]. Study limitations were mentioned in higher percentage of articles in the studies conducted by Hendriksma M et al., and Poorolajal J et al., [8,16]. It was mentioned in lower percentage of articles in the other studies [6,15,17]. Most authors included study limitations in discussion section. In the present study, study limitations were also evaluated because they were related to materials and methods.

The ethical clearance statement and details of the methodology procedure were evaluated only in the present study. However, they were not evaluated in the other studies [Table/Fig-7] [6,8,15-18].

According to the Declaration of Helsinki, the research protocol must be submitted for consideration, comment, guidance and approval to a research ethics committee before starting the study. Therefore, obtaining ethical committee approval, particularly in medical sciences, should be a moral obligation for researchers [22]. As reproducible results are very important in science, a detailed account of the study must be provided in materials and methods [15].

The number of articles evaluated were relatively less in the present study. This is because, the articles were selected from only pathology journals in accordance with the scope of the research project. This may be considered as eye-opener sample audit and calls for improvement in the quality of publication.

Undeniably, both the National and International pathology journals significantly contribute to the knowledge database and provide valuable information for the benefit of the mankind. However, the present study suggests need for improvement in the quality of publications in both National and International journals concerning materials and methods-related aspects. The quality of articles may be improved by putting a sincere effort to minimise the errors highlighted in the present study. Major errors may be minimised by sagacious utilisation of checklist of parameters discussed in the article, so that the materials and methods are covered quite comprehensively by the authors. Minor errors may be minimised by meticulous proofreading of the article before the publication of the final version.