Are All Citations Created Equal?

14. August 2013

Pablo Artal

OSA Fellow Pablo Artal has kindly allowed OPN’s Bright Futures career blog to adapt and republish content from his popular blog Optics Confidential. In his blog, Artal fields questions from students, colleagues and other researchers on science, society and managing a career in optics.
 
Dear Pablo, I am confused about what works to cite in my scientific papers. Should I cite only the papers that helped me with my research? Or should I expand the list to include those that I found clearly wrong or even misleading? –Bruno, Italy.
 
I believe the proper approach is to cite everything that you actually used during your research. This includes seminal papers that may have inspired your project, articles on the methods you used, papers presenting similar previous work, and even research that you may consider incorrect or biased—although you should mention why you think it is invalid. This is an important part of the scientific process, and it will help your colleagues in the future.
 
Your question brings up an issue that I have long found troubling. As you know, the number of citations a scientist receives on his or her papers can be a deciding factor in receiving grants, academic jobs and prestige. The so-called h-index, referring to the number of papers that a scientist has with the same or higher number of citations, is a particularly important metric. For instance, if I have an h-index of 41, that means that 41 of my articles have received 41 or more citations. Some time ago, I covered this issue in more detail in my other blog in Spanish.
 
Although the number of citations is a better measure of scientific performance than simply counting the number of published papers, it is far from perfect. There are many possible problems with this system. For example, the differences in the number of publications and citations among different scientific fields generally make it difficult to compare between subject areas.
 
You can get an automatic count of citations on an article in Google Scholar or Web of Science, but this doesn’t take into account the fact that citations are not all equal—maybe you know a scientist whose work has a large number of citations, but some of them are actually negative. To avoid problems like this, I propose that we classify citations into four categories. I’ve listed them below with some examples obtained from actual papers.
 
Seminal citations:
“We followed the approach proposed and first implemented by (ref) to perform the current experiment…”
 
Positive citations:
“The results of figure 5 are in good agreement with those presented in (ref)”
“Figure 3 compares our results with those of previous works (ref)”
 
Neutral citations:
“Although we followed the same procedure, we were not able to reproduce their results. This may be due to some individual variability. However, several other authors’ findings were similar to ours.”

Negative citations:
“The suggestion by (ref) is clearly incorrect…”
“An additional problem in this study is the surprising lack of details provided on some of the most relevant methods and procedures used.”
 
I understand the technical difficulty of classifying different types of citations, but this system would provide a more accurate depiction of scientific value. Appropriate software could classify every citation within these categories, and each would be rated with points. For instance, seminal citations would be worth two points, positive ones would be worth one, neutral citations would have no points and negative citations would be negative one point.
 
A few decades ago, many of us were unhappy with the mere counting of papers as a measure of success, and the current system has helped address that. But other issues have cropped up. We could not begin to imagine at that time the large emphasis that would be placed on citation counts today. Perhaps the time has come to reevaluate.
 
Pablo Artal (Pablo@um.es) is an OSA Fellow and professor of optics at the University of Murcia, Spain. He is an optical and vision scientist with an interest in visual optics, optical instrumentation, adaptive optics, and biomedical optics and photonics.

Academic careers, Career, Publishing , , , , , , , , , ,

Lessons from an Editorial Term

5. July 2012

Pablo Artal

OSA Fellow Pablo Artal recently finished his second and final term as a topical editor of the Journal of the Optical Society of America A. Here, he shares his advice to authors and reviewers based on his six years of experience.

For scientists, writing is as vital as planning and executing experiments. Soon after a researcher has published a few articles in a field, it is typical for him or her to be asked to serve as a reviewer. This can be looked at as both a responsibility to the community and a career-development tool for yourself.

When I became an editor, I gained a whole new perspective. For those authors and reviewers who have not yet served as an editor, perhaps you can learn something from my experiences.

Be thorough and professional. I believe that a research area’s strength is related to the quality of the reviewers for its journals. Good reviewers behave like invisible mentors—combing through the data, suggesting additional experiments and giving specific, actionable feedback.

Expect to be treated equal to your colleagues. If you are an editor for long enough, you may have to reject a paper submitted by a friend or close colleague. A fundamental principle for editing and reviewing is that every author should be treated equally. If you can’t do this, you should not edit the paper. Real friends understand that you have to follow the same rules for everybody.  

Be generous with citations. Most authors are very gracious about citing the work of others. Of course, there are some who avoiding citing other groups in favor of noting their own previous work. Self-citations are in many cases necessary. However, when relevant papers from others are missing, it can signal a low-quality paper to an editor. So be generous; it’s good for others and good for you.

Be alert for plagiarism. A fundamental task for editors is to detect and reject articles that are either clearly wrong or direct copies of previously published articles (plagiarism). More sophisticated forms of plagiarism, including self-plagiarism, can be difficult to find. I realized that many cases occur due to lack of author education or differing norms. For example, some researchers do not view it as wrong to duplicate their own research. Part of the editor’s job is to clearly communicate what is acceptable, what is not, and why. 

Clearly articulate the purpose of your paper. Sometimes a paper receives reviews indicating that it seems to be correct, but the point of the research is not clear. In a good journal, these papers are often rejected. Before submitting an article, be honest with yourself: Is this a paper I would like to read myself? Does it advance the field?

Don’t assume friends make the best reviewers. Most journals, including JOSA A, ask the author for reviewer recommendations. Usually, authors tend to suggest someone they know well. I was initially surprised in cases when I followed an author’s recommendation and received reviews that were perhaps unduly negative. Surprisingly, the most critical reviews can come from close colleagues—possibly because they are the ones who are closest to the details of your research area. Be aware of the possibility of bias, both positive and negative.

Review as you would like to be reviewed. OSA journals would not be possible without the tireless work of devoted volunteers. However, there are a few people who consistently refuse to review papers. It is not acceptable to systematically avoid this duty if you are an active scientist. Think twice when you are asked to review a paper, remembering the Golden Rule—Do unto others as you would like them to do unto you.

Get it right. As an editor, I was worried about the possibility that I would accept a paper that was completely wrong. A paper whose conclusions miss the mark is not necessarily a problem; there are many of those out there and they are consubstantial with scientific development. However, if a published paper contains simple and fundamental mistakes, the editor is to blame, so be careful.

Serving as an editor was a great experience that I would highly recommend. You learn more about your field and human nature. Hopefully it will make you a better scientist and person.

Pablo Artal (pablo@um.es) is an OSA fellow and professor of Optics at the University of Murcia, Spain. His blog covers optical research and related aspects (http://pabloartal.blogspot.com/).

Career, Communication skills, Publishing , , , , ,

Peer Review 101: Building your Reputation as a Journal Reviewer

8. April 2012

by Carlos López-Mariscal and Julio Gutierrez-Vega

As a young scientist, when you publish your first papers in prestigious peer-reviewed journals, you announce your presence to the scientific community. In this way, you are in the process of becoming a recognized expert of your field. Eventually, you will also receive requests from journal editors to review manuscripts submitted by other scientists. These invitations are both a privilege and a responsibility.

Taking part in the review process is critical to developing a scientific career. It helps you build relationships with journal editors; it improves your critical thinking abilities; it gives you a better understanding of the state of the art in your field; and it enhances your writing skills so you can better present your scientific ideas. In addition, good reviewing is recognized and rewarded by colleagues and scientific societies.

As you start reviewing others’ work, the fundamental principle to keep in mind is the notion of reciprocity. Follow this golden rule: You should review a manuscript in the same way that you would want your manuscript to be reviewed. Here are some other best practices to keep in mind:

Respond promptly to requests. This is quite important—whether or not you accept the invitation to review. One of the worst things you could do is ignore a request for review, along with accepting it and then not honoring the request. If you are unable or unwilling to accept, it only takes a couple of minutes to notify the editor of your decision. The editor will appreciate it if you can suggest other potential reviewers.

Complete the review on time. This point is crucial to guarantee the timeline of the journal. It is unfair to authors (and editors) to be delayed by tardy reviewers. If you need extra time, contact the editor as soon as you can. Most are flexible and will agree to give you additional time in exchange for a good review.

Do not review a manuscript whose topic is unfamiliar to you. Stick to topics that you know well in order for your reviews to be the most credible and useful.

Enumerate your comments and suggestions. Again, think about how you would want someone to review your own paper. Organize your thoughts in a way that will be easy for someone to absorb and follow up on.

Read the journal’s review criteria. Make sure you spend time on the journal and/or publisher’s website so you understand what is expected of both authors and reviewers. This will help to ensure that your review is aligned with the publisher’s expectations.

Be specific. Indicate as precisely as you can what the problems are and how they may be overcome.

Focus on the science.Avoid effortless reviews that comment only on minor grammatical errors, typos or language problems. However, if a manuscript is written in language so poor that it is difficult to understand, point this out to the editor.

Follow up. If you are reviewing a revised manuscript, make sure the authors actually made the appropriate changes in the manuscript as recommended in the first review.

Be discreet and complete. Always maintain confidentiality and notify editors of any potential conflict of interest or suspicions of plagiarism that you may have.

If you are interested in reviewing, let your academic advisor know. He or she probably gets requests regularly and will be grateful for your initiative. You can also contact the topical editors of the journals you have published in—or introduce yourself to them at a conference. Researchers are often invited to review manuscripts as a direct result of their own published work.

Good reviewers are not as easy to come across as you might think, so don’t be shy. Take the initiative and get involved.

Carlos López-Mariscal and Julio Gutierrez-Vega are both experienced reviewers and members of the Optics & Photonics News Editorial Advisory Committee. Gutierrez-Vega is also an associate editor for OSA’s peer-reviewed open-access journal Optics Express.  

Career, Publishing , , , , , , , , , , , ,