 Abstract | | |
With publication being made mandatory for being a dental faculty in most countries, there is a huge amount of research output emanating from academicians. Though there has been an acute interest in publication, there exists no policy or a common platform or a unified methodology where in the post-publication effects are analyzed. Each publication draws considerable resources in terms of manpower, time, financial and other valuable resources. Hence assessing the impact that contemporary research makes on health care delivery is crucial. Though, publications have evolved as a mark of scholarly skills, it needs to be periodically gauged for reasons including promotions, grants and other academic perks. Currently there are different yard sticks available. An easy to apply assessment method needs to be evolved for Pan-Indian application. The main aim of this review manuscript is to provide a comprehensive insight to post-publication metric assessment exercises so that the faculties, policy framers and funding agencies adopt uniform policies for periodical assessment of the teaching faculties for better oral health care delivery. Keywords: Bibliometrics, citation metrics, dental faculty, impact factor, post-publication metrics
How to cite this article:
Rooban T, Rao UK, Joshua E, Ranganathan K. Assessing the impact of dental faculties publication - methodology clarifications - need of the hour. J Educ Ethics Dent 2013;3:49-53 |
How to cite this URL:
Rooban T, Rao UK, Joshua E, Ranganathan K. Assessing the impact of dental faculties publication - methodology clarifications - need of the hour. J Educ Ethics Dent [serial online] 2013 [cited 2023 Apr 1];3:49-53. Available from: https://www.jeed.in/text.asp?2013/3/2/49/136042 |
| Introduction | |  |
Quantifying the quality of Dentist-Researcher's (DR) research output is becoming increasingly difficult. Drastic increase in the number of dentistry specific journals and heightened awareness to get published as well as lack of ideal quantification methods complicates this judgment. [1] Dental science includes all aspects of research, discovery and invention pertaining to dentistry. Teaching is an approved method of knowledge transfer from a trained, qualified, experienced dental faculty to a trainee dentist. [2] There exists no single, accurate method by which the faculties experience, eligibility and scholarly abilities are put to test. Publication in scholarly journals has been traditionally accepted as a measure of the individual faculty's abilities. However, Medical Council of India and Dental Council of India (MCI and DCI respectively) have mandated publication to be an integral part of assessing the quality of the faculties. [3],[4] Hence currently, DR's are seeking an empirical assessment of their scholarly work for various reasons. Though there are numerous methods to assess the impact, globally very few parameters are employed. Each of these has its own advantages and disadvantages. The main aim of this review manuscript is to provide a comprehensive insight to this assessment exercise so that the faculties, policy framers and funding agencies adopt uniform policies for assessment of the teaching faculties.
Abstracting and indexing (a&i) agencies
As the name implies, these agencies (Private/Government aided) list down, categorizes, archives and helps to retrieve the abstracts based on a set of predetermined characters. Dental literatures are archived in well known databases. Each of these agencies have their own set of criteria, rules and regulations for listing a journal [Table 1]. Google Scholar (promoted by Google) is an interface that actively retrieves the abstracts based on prior set of unique characters using Google algorithm from all such databases. | Table 1: Table comparing the various agencies, their status, restriction, usability and employable metrics
Click here to view |
The data from these databases can be accessed through interface, with PubMed being the common free-of cost interface for Medline. Beside Medline indexed journals, PubMed also archives abstracts of articles from Journals recognized and catalogued by National Library of Medicine, USA (NLM). PubMed listed Journal, besides other factors, when calibrated with NLM's controlled vocabulary, the Medical Subject Headings (known as MeSH), it becomes Medline Indexed. [5],[6] The other interfaces available for Medline are Embase (www.embase.com), EBASCO (http: //ebscohost.com), Ovid (www.ovid.com) and host of others. Indian government has its own version called the MedInd.
PubMed uses very simple unique numeric identifiers, called PubMed Identifiers (PMIDs) and are only for Identification. The National Library of Medicine (NLM), USA issues a catalogue number that is a mere identification number called, NLM Catalogue number that serves as an easy tool to identify the reference material. [7] Digital Object Identifiers (DOI) are a unique set of codes awarded by another international agency-International DOI Foundation (www.doi.org)-usually to identify the manuscript easily during digital manipulation and are never a measure of quality. [8]
As each and every of the A&I agencies have their own set of ideas and characteristics, policy makers shall pay diligent attention in choosing the A&I agency that are accepted/endorsed by them. Indian Government's National Informatics Center's IndMed and MedInd are India's own A&I agency and their listing need to be respected.
Assessment tools for impact of publications
Impact of the entire DR's research is not always easily measurable. This requires some ingredients like the consideration of the utilization of research output and a process of research output for knowledge diffusion influencing human's well being, social and economic activities, it's impact on expanding the knowledge frontier and impact on treatment procedures. Such measures often employ predetermined set of characteristics. The most commonly employed measures use one of the following three criteria. [9] [Figure 1]. | Figure 1: Chart depicting the most commonly employed method to assess the impact of the published manuscripts by Indian dental faculty
Click here to view |
Assessment of the Journal Published With
The impact factor of a journal is a quantitative ranking tool for evaluating the relative importance of a journal. It is a measure of the frequency with which its published papers are cited up to 2 years after publication. The most commonly used tools are Science Citation Index (by ISI, introduced in 1960's assessed from 1945), Journal Impact Factor (JIF, in 1973 by Journal Citation Reports (JCR)), Scimago Journal Rank, Y-Factor, PageRank (Google), Journal Impact, Eigen factors quite recently. [2],[9],[10]
JIF or the impact factor (IF) is the widely recognized method. The JIF is defined as: The number of times articles from the journal are cited within 2 years divided by the total number of articles published in the same journal during the 2-year period. The JIF is intended to measure on average how often a manuscript is cited from the considered journal. [11] Published by JCR, it is available within ISI Web of Knowledge on payment and is the key source of information about the impact of a journal, giving IF, cited half-life and immediacy index for each title. JCR covers specialties in the areas of science, technology, and the social sciences and is updated annually in two editions. [2],[9],[10] SCImago Journal & Country Rank, [12] free access tool, uses algorithms similar to Google Page Ranking and is based on data in the Scopus database. The Eigen factor metrics use a similar methodology to SCImago, but based on data from the Web of Science database. [2] Journal Impact is measured by an online based tool - Harzing's Publish or Perish software (http://www.harzing.com).
Assessment of the Individual manuscript
The most widely employed method to assess the impact of a manuscript is the citation. It is a measure of citation that the manuscript receives, usually within a stipulated time period as determined by the examining authority. Usually 2 year period is considered as the limit, but 3 years and 5 years are also evaluated by experts and competent organizations throughout the world. Citation analysis inherently is not perfect but current technology has enabled a higher level of standardization that helps to assess without bias. A balance of publication count, citation counts and the JIF, one can get a better picture of performance of DR's. However it is a pure assumption that if a manuscript gets published in Journals with JIF, the quality is better. However it does not guarantee a high citation. [2],[9] Google scholar gives a free and easier access for such citation analysis/calculation.
Assessment of performance of an Individual Dentist-Researcher
In order to assess the DR's overall performance, cumulative indexes can be employed. Though highly complicated, they can abridge the language, discipline, specialty and author position barriers. JE Hirsch proposed a scientific index, the Hirsch index or the h-index, which is defined as the number of papers with citation number higher or equal to h. In simple terms, if an author has 8 papers with 8 citation each, his/her h-index is 8. It is a useful index to characterize the scientific output of a researcher. [13] It can be measured using a host of database including scopus, science direct, PubMed, ISI Web of Science and quite recently through the Harzing's software. Use of Google Scholar is an easy and a cost effective way of calculating the index.
The other measures [14] used commonly are the Productivity: Number of publications per researcher assigned to a unit. (Where in a unit is Department/ Institution/ University/ State or a specialty). Dentistry Specialization Index of a unit is the ratio of dentistry documents of a specialty to the total scientific production of the unit. Relative Specialization Index: Specific weighting of the dental scientific production of a unit in relation to its general scientific production. Total Impact (TI) of a unit is the sum of the impact factors of each document affiliated to a given unit. Weighted Impact Factor (WIF) is the ratio of the TI to the total number of documents of that unit. Mean Citation Rate for each manuscript is the sum of the citations received in defined period of time (usually 2 years). Immediacy Index is a quantitative measure of the rapidity of citation. Higher value indicates the popularity of the topic, contemporary nature and the quality of the manuscript.
Newer concepts in the bibliometric field include the project ResearcherID (www.researcherid.com). Introduced by ISI Web of Knowledge. It permits an individual author to set up a (free) unique personal identifier which can be linked to their publications within the Web of Science, regardless of any variations in the use of their name and regardless of their affiliation when a particular paper was published. It helps to uniquely identify an individual researcher and also ensures the correct attribution of papers and the accuracy of bibliometric analyses. The only drawback is it considers only articles abstracted with ISI web of science. Alternatively, Google scholar (http: //scholar.google.com/citations) supported citation calculation system is easier and widely accessible.
| Discussion | | |
There has been report of publication metrics of DR's in several dental specialties. [15],[16],[17] These publications have concentrated on number and not the post-publication metrics. In the Indian context, MCI stresses only research publication while the DCI allows any publication. [3],[4] The trend should be rather to wholly encompass all the overall collective ability of the single institutional staff in lines of the older research assessment exercise [18] and the newer Research Excellence framework [19] of the United Kingdom.
The Research Excellence Framework (REF) of the United Kingdom is the new measure of assessing the quality of research in UK higher education institutions. It replaces the Research Assessment Exercise (RAE) and starts to be implemented by 2014. The exercise is monitored by 4 major funding bodies of the UK. A self assessment of the department/institution is submitted in a prescribed format. These assessment outcomes provide point of reference to create the reputational yardsticks or group the performance. A preannounced set of generic assessment criteria and level definitions have been evolved for this mechanism. This aims to produce an overall quality profile for each submission by which the quality of the department/institution is drawn. [19] The assessment contains a cumulative consideration of impact of the research on health care delivery, improvement/uplift of health, help of researchers in understanding the disease/treatment, returns of the investment in terms of finance, manpower and infrastructure etc. besides the measures of the citations. Such an endeavor would help to assess the overall impact of the publication(s) of the DR. This would help to provide incentive for better performing DR and help to identify, train and improve those lagging behind the benchmark.
A similar guideline could be evolved for a single author accounting the loco-region-socio-cultural features. The overall research performance of an individual DR can be accessed through a scheme similar to that of current Research Assessment Exercise or the proposed Research Excellence Framework wherein all the potential outputs will be measured. This score, when enforced, will help MCI and DCI to gauge the proficiency and experience of the faculty as per existing International Scenario.
In a recent publication, it has been concluded that Post-Publication Review, the Impact Factor, and the Number of Citations are still poor indicators of assessment of a manuscript. The study has shown that intra-observer and inter-observer errors occur during estimation of merit of a scientific publication, usually the bias caused by the journal in which the manuscript is published. The accumulation of citations is described as "a highly stochastic process, such that two papers of similar merit can accumulate very different numbers of citations just by chance". [20] The best possible solution available is the IF and the time consuming REF would have a better solution, if extended and implemented by employing large number of assessors. [20]
In this context, a recent oral health survey by Indian Diasporas, [21] majority of Indian dentists held the dental Institutions responsible for the unpreparedness of newly qualified dentist. The dentists have suggested that better recruitment process for dental schools are needed to ensure that they are qualified, better equipped, following current practice instead of old conventional methods, have a good flow of patients for real life experience and a full-time competent staff; standardization of education nationally; and revision, renewal and improvement of current curriculum to make it competency based to include practice management skills, ethics etc. [21]
The post-publication metrics has been an issue of contention among the Indian faculties. The debate of promotion for medical faculty based on publication has been reported [22] while several alternate strategies for measuring the quality of authors have also been proposed. [23] Evolving and implementation of the institutional/faculties judgment score in lines of Research Excellence Framework of UK following uniform stringent criteria throughout India will be a huge leap in this direction.
| Conclusion | | |
With newer dental educational choices emerging in the Medical/Dental sectors, it is the duty of the councils to provide the students with correct data about the strength and weakness of the faculty and institution to take an informed decision based on scientific methods. This exercise further also will help the councils to assess the ability of the faculties. Consideration of MedInd and IndMed as a valid tool for assessment of individual faculty needs to be incorporated by MCI and DCI. The agencies also need to involve comprehensive short and long term strategies to raise their standards to global levels.
| Acknowledgement | | |
The authors like to thank the Principal, Dr. S. Ramachandran and Prof. Kanagaraj and Mrs. Vijayakumari, Management of the institution for their constant motivation, support and encouragement.
| References | | |
| 1. | Rooban T, Madan kumar PD, Ramachandran S. Contribution of Indian dental research to the Scimago™ Database during 1996-2007. A preliminary report. Chron Young Sci 2010;1:16-21. 
 |
| 2. | Schoonbaert D, Roelants G. Citation analysis for measuring the value of scientific publications: Quality assessment tool or comedy of errors? Trop Med Int Health 1996;1:739-52.
|
| 3. | Publication of medical council of india. Available from: http: //health.bih.nic.in/Docs/Teachers-Eligibility-Qualifications-Rgulations-1998.pdf [Last accessed on 2012 Oct 30].
|
| 4. | Notification of dental council of india. Available from: http://www.dciindia.org/annoncment_pdf_files/pdf_files/ELIGIBILITY%20CRITERIA%20TO%20BE%20A%20PG%20TEACHER.pdf [Last accessed on 2012 October 30].
|
| 5. | Indrajit IK, Nangpal S. Medline. Natl Med J India 2001;14:50-53.
[PUBMED] |
| 6. | Available from: http: //www.nlm.nih.gov/pubs/factsheets/dif_med_pub.html [Last accessed on 2012 Oct 30].
|
| 7. | Available from: http: //www.nlm.nih.gov/bsd/disted/pubmedtutorial/glossary.html [Last accessed on 2012 Oct 30].
|
| 8. | Available from: http: //www.doi.org [Last accessed on 2012 Oct 30].
|
| 9. | Qin J. Empirically assessing impact of scholarly research. In: Proceedings of the iConference, February 3-6, 2010, Champaign, Illinois. Available from: http://www.ideals.illinois.edu/bitstream/handle/2142/14924/qin.pdf?sequence=2 [Last accessed on 2012 Oct 30].
|
| 10. | Garfield E. The meaning of the Impact factor. Int J Clin Health Psychol 2003;3:363-9.
|
| 11. | Mathur VP, Sharma A. Impact factor and other standardized measures of journal citation: A perspective. Indian J Dent Res 2009;20:81-5.
[PUBMED] |
| 12. | SCImago. (2007). SJR - SC Imago Journal & Country Rank. Available from: http://www.scimagojr.com [Last accessed on 2012 Oct 30].
|
| 13. | Morrison PJ. The hirsch index and measuring the quality of scientific papers. Ulster Med J 2008;77:1.
|
| 14. | Gil-Montoya JA, Navarrete-Cortes J, Pulgar R, Santa S, Moya-Anegón F. World dental research production: An ISI database approach (1999-2003). Eur J Oral Sci 2006;114:102-8.
|
| 15. | Kumar MP, Narayanan MB, Rooban T, Shivakumar M, Ramachandran S. Publication Trends of Indian Public Health dentists Between 1997-2007: A Medline Approach. J Indian Assoc Public Health Dent 2010;15: 182-6.
|
| 16. | Poorni S, Ramachandran S, Kumar MP, Rooban T. Publication Trends of Indian Conservative dentists and Endodontist Between 1997-2009: A Medline Approach. J Conserv Dent 2010;13:167-70.
|
| 17. | Shamala RK, Rooban T, Anusa AM, Reddy BV, Ranganathan K. Publication Trends of Indian Oral and Maxillofacial Pathologists Between 1997-2007: A Medline Approach. J Orofac Sci 2010;2:28-32.
|
| 18. | Available from: http: //www.rae.ac.uk/aboutus/ [Last accessed on 2012 Oct 30].
|
| 19. | Available from: http: //www.ref.ac.uk/media/ref/content/pub/panelcriteriaandworkingmethods/01_12_2A.pdf [Last accessed on 2012 Oct 30].
|
| 20. | Eyre-Walker A, Stoletzki N. The assessment of science: The relative merits of post-publication review, the impact factor, and the number of citations. PLoS Biol 2013;11:e1001675.
|
| 21. | Bedi R. Call for action: Improving dental services and oral health in India. Page 17. Available from: http://www.gcdfund.org/news/charity-news/results-indian-oral-health-survey [Last accessed on 2012 Oct 30].
|
| 22. | Sukhlecha A. Research publications: Should they be mandatory for promotions of medical teachers?. J Pharmacol Pharmacother 2011;2:221-4.
[PUBMED] |
| 23. | Saxena A, Thawani V, Chakrabarty M, Gharpure K. Scientific evaluation of the scholarly publications. J Pharmacol Pharmacother 2013;4:125-9.
[PUBMED] |
Correspondence Address:
Thavarajah Rooban
Department of Oral and Maxillofacial Pathology, Ragas Dental College and Hospital, 2/102 East Coast Road, Uthandi, Chennai - 600 119, Tamil Nadu
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/0974-7761.136042
[Figure 1]
[Table 1] |
