Home / IEEE Technology Policy and Ethics / March 2022 / Academic Misconduct: Nonscientific and Nonstandard Evaluation of Awards and Professional Titles

Academic Misconduct: Nonscientific and Nonstandard Evaluation of Awards and Professional Titles

By Dr. Zhihan LV, Department of Game Design and Faculty of Arts, Uppsala University, Uppsala, Sweden

There are many aspects to consider when evaluating a researcher’s academic achievement, including the quality of the journals in which their papers were published, the number of times their papers have been cited, and any awards received by the researcher. For a teacher at a college or university, the promotion of a professional title based on an assessment of the academic results is also an aspect used for career advancement. Prudence, accuracy, and authenticity are fundamental to academic research. The current academic paper review process leverages external reviewers and experts in varying fields. It is a very reasonable and reliable method to evaluate the contribution of the paper, and it can ensure the authenticity of academic work. The evaluation of awards and professional titles should be as prudent and standard as the academic paper review process. However, the evaluation of awards and professional titles at some research institutes is nonstandard and nonscientific.

At certain institutes, human factors significantly influence evaluation processes, and the results depend on the humans’ subjective control, but not objective rules. This paper discusses and analyzes potential standard and scientific ways for academic evaluation. In addition, the problems that exist in some research institutes in China, and in particular research disciplines, were considered and leveraged as examples.

A. Evaluation of Awards and Professional Titles Considering Only the “First” Author

The purpose of the Academic Award for Postgraduates is to recognize the outstanding academic achievements made by postgraduates at colleges and universities. This award is usually conferred upon a team instead of an individual. Therefore, the achievements made by all team members should be considered in the award evaluation process. However, at some research institutes, only the academic achievements of the first applicant in the list is considered in this process. In addition, there are issues with the order of evaluation—authors on a given published paper are considered in the order they are listed. For award consideration, papers published by a postgraduate student should include papers that were published by a specific person (listed as the first author), and articles for which their supervisor is listed as the first author (with the specified person as the second author). When a school determines the order of award recommendations, the papers published by a student as the first author may become a decisive factor among the evaluation criteria, and the school may not consider other papers the student has contributed and the papers of other members on the research team. This phenomenon also exists in the process of professional title and performance evaluation for professors at universities. However, some institutions in China only recognize the first three authors in the list, in some cases only the first author, and subjectively think that the other authors of the paper are merely listed on the publication, or that their contributions to the research are negligible.

For a research project, award committees tend to focus only on the main applicant as the first author and neglect the contributions made by other researchers on the publication. For example, the authors’ team specializes in virtual reality and software engineering. A software platform may be developed by a team of about 50 people. It may take several years to complete such a high-quality project, but the results may contain only one or two papers, or a few patents. However, in some cases, reviewers recognize the contributions made by the first three authors listed, and the other authors are usually deemed not to have made significant achievements to the paper.

In 2013, Francois Englert and Peter Higgs won the Nobel Prize in Physics for predicting the Higgs mechanism. However, the team that discovered the Higgs boson consisted of hundreds of members. The team published three papers [1-3] in the Physical Review Letters, a leading international journal, and each paper had nearly one thousand authors. The authors of these papers were not listed in order based on their contributions, and some of them were listed in alphabetical order by surname. Many Chinese institutes don’t recognize contributions made by authors who are not among the first three listed. For example, a scholar on the team that discovered the Higgs boson was listed around the 200th contributor in the list of authors for all three papers because the list was provided in alphabetical order. However, this scholar has obtained his Ph.D. by attending one of the world’s top universities for seven years, and he is still working as an assistant professor at a university ranked at 360 in China. He is at a disadvantage in the evaluation of professional titles because the evaluation criteria and rules are unreasonable and nonscientific. In addition, ranking his name in such a low position does not correlate with important research contributions.

B. Internal Voting Involving Too Many Human Factors

As mentioned earlier, a school needs to determine the order of recommendations for evaluating candidates for Academic Awards for Postgraduates. In the evaluation process, the team ranking first was comprised of former students of the previous head of the school (who is now the Dean for Graduate Student Affairs). All votes for the award came from teachers at that school, so they favored the school’s former head. Therefore, the evaluation process was not fair. A relatively fair way to evaluate is by leveraging double-blind reviewing. In this process, the identity of the team leader remains confidential, which ensures there is no known connection between the reviewers and the school, and this helps prevent bias. One potential way to avoid conflicts of interest is by prohibiting students from participating in activities hosted by their school. In some instances, academic journals implement and enforce strict rules prohibiting their editors from submitting papers for consideration to the journals that they serve. Therefore, the head of a school should not allow his or her students to take part in any activity requiring recommendations from their school faculty as that would raise a conflict of interest.

C. Non-transparency of the Evaluation Process and Results

The process and results of the evaluation mentioned above were not made public. However, they should be made public so that everyone can access relevant documents and materials submitted, and both teachers and students will be confident of a fair evaluation process. For example, after the evaluation process for the National Natural Science Fund (www.nsfc.gov.cn/english/site_1/) is complete, the list of evaluation committee members is published online. The evaluation results, including the number of votes and the verifiable, real information on the vote of each member, should also be made public. In addition, universities should establish and maintain an effective mechanism for supervising the award evaluation processes within various schools. If the head of a school an individual having a vested interest in a particular candidate, then they are biased, and it would be difficult for other teachers at the school to submit complaints. Designated university offices should play an effective role in receiving, evaluating, and issuing notices and documents.

In general, traditional academic misconduct, such as plagiarism and data fraud, should be avoided. In the meantime, more attention should be paid to the nonstandard and nonscientific evaluation of academic awards and professional titles. In addition, solutions should be designed to combat academic misconducts and prevent unethical behaviors during process. Effective measures are a vital component and should be considered as a way to solve these problems. The conflicts and traditional academic misconducts discussed are the root cause of complications. One potential method to eliminate the issues is to guide or require all universities to consider the contribution beyond the first, 2nd, and even third authors of publications in addition to the contribution from all other members that participated in the project. Lastly, the evaluation process of all awards should be made public.

References 

  1. Aaltonen, T. et al. Evidence for a particle produced in association with weak bosons and decaying to a bottom-antibottom quark pair in Higgs boson searches at the Tevatron. Physical Review Letters. 2012;109(7): 071804. DOI: doi.org/10.1103/PhysRevLett.109.071804
  2. Abazov, V. M. et al. Combined search for the standard model Higgs boson decaying to b b using the D0 Run II data set. Physical Review Letters. 2012;109(12): 121802. DOI: doi.org/10.1103/PhysRevLett.109.121802
  3. Abazov, V. M. et al. Search for the Standard Model Higgs Boson in Associated W H Production in 9.7 fb− 1 of p p¯ Collisions with the D0 Detector. Physical Review Letters. 2012;109(12): 121804. DOI: doi.org/ 10.1103/
    PhysRevLett.109.121804

 

Dr. Shihan Lv is an IEEE Senior Member, British Computer Society Fellow, and ACM Distinguished Speaker. He was an Associate Professor at Qingdao University, China, from 2017 to 2021. He has been an Assistant Professor at Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences from 2012 to 2016. He received his Ph.D. from Ocean University of China in 2012. He worked in CNRS (France) as a Research Engineer, Umea University (Sweden) as a Postdoc Research Fellow, Fundacion FIVAN (Spain) as an Experienced Researcher, University College London (UK) as a Research Associate, University of Barcelona (Spain) as a Postdoc. He was a Marie Curie Fellow in European Union’s Seventh Framework Program LANPERCEPT. His research mainly focuses on Metaverse, Digital Twins, Virtual Reality, Serious Game. He has contributed 300+ papers including more than 70 papers on IEEE/ACM Transactions. He has more than 30 ESI highly cited papers and 10 ESI hot papers.

He is Editor-in-Chief of Internet of Things and Cyber-Physical Systems (KeAi), an Associate Editor of 18 journals including ACM Transactions on Multimedia Computing, Communications, and Applications, IEEE Transactions on Intelligent Transportation System, IEEE Transactions on Network and Service Management, IEEE Technology Policy and Ethics Newsletter, and Leading Guest Editors for 40 special issues including 10 IEEE. He is Co-Chair or TPC of 50 conferences including ACM MM 2021, ACM IUI 2015-2022. He has reviewed 400 papers. He has received more than 20 awards from China, Europe, IEEE. He has given more than 80 invited talks for universities and companies in Europe and China. He has given 20 keynote speeches for international conferences

Editor: 

Dr. Adriana Bankston is a Principal Legislative Analyst at the University of California Office of Federal Governmental Relations in Washington, DC, where she serves as an advocate for the university with Congress, the Administration and federal agencies. Prior to this position, Adriana was a Policy & Advocacy Fellow at the Society for Neuroscience, where she provided staff support for special and on-going projects, including the society’s annual lobby event and the annual meeting. In addition to working at UC, Adriana is the Chief Executive Officer & Managing Publisher of the Journal of Science Policy & Governance, an internationally recognized non-profit organization and peer-reviewed publication dedicated to empowering early career scientists, engineers, and policy professionals in international science policy debate. She is also a Fellow with Advancing Research Impact in Society (ARIS), leading a project on developing the next generation workforce through science policy as a bridge between science and society. Finally, she is a Biomedical Workforce & Policy Research Investigator at the STEM Advocacy Institute, as well as a member of the Engaging Scientists and Engineers in Policy (ESEP) Coalition Steering Committee. Adriana earned her Ph.D. in Biochemistry, Cell and Developmental Biology from Emory University.