Citation Metrics: Trust But Verify

Citation metric: a) A numerical measure of the influence of a journal or journal article based on the number of citations.
b) An simplistic way of evaluating the value of research based on flawed numbers, that fails to take into account the actual complexities involved in evaluation, but is still seen as being valuable because it provides a clear numerical way to rank things, as the people doing the evaluation very often have neither the time nor the background to evaluate the research personally.

Citation Counts

In this post, we’ll look at the most basic citation metric of all: citation counts. Simply put, a citation count is a count of citations that have been made to <something>. The <something> could be a book, a book chapter or essay, a journal article, a white paper, a technical report, a conference paper, a video, a webpage, a blog, a newspaper article, a magazine article, a preprint, a dataset, a patent, an unpublished communication, a corporate research report, a standard, an image, a performance, and so on and so forth.

The citation could also appear in almost any one of those (though perhaps not an image!).

Where to find citation counts

It seems simple enough. We have Article A, written by one of our many notable faculty. How many times has it been cited? But of course, not all citations are equal! If we want to evaluate Professor A’s work, we only want citations made by people who are peers of Professor A; fellow researchers, academics, or professionals. We also only want citations that appear in proper research, academic, or professional publications, not popular magazines, newspapers, and the ilk, since we want to evaluate what impact Professor A’s research has had on their field.

To that end, the Library offers two commercial tools , and one free one.

Citation Tools

Web of Knowledge (also know as the Web of Science, the Science (or Social Science or Arts and Humanities) Citation Index. The oldest source of citation data (besides counting it yourself), with data going back to the 1970s in some disciplines  It has a fairly narrow focus, and only indexes citation made in what they consider the significant journals in particular fields. It doesn’t count citations made in anything that isn’t one of the significant journals. So if your article was cited in a book, a patent, a journal the WoK doesn’t consider significant, etc. the WoK doesn’t know about it.

Scopus. A newer database that has citation data for citations made in a selection of journal and conference papers published from 1996 onward. It covers quite a bit more journals than WoK, and does cover some conference proceedings, but it focuses a bit more on science, technology, and engineering disciplines. It does include some coverage of the social sciences (business and psychology predominately), and the arts and humanities, but those aren’t its strengths. Like the WoK, it doesn’t cover books, and doesn’t cover all journals.

Google Scholar. Google’s attempt to index all of the scholarly material on the Internet. It can generate citation counts, but it has a tendency to list duplicates if a particular research article is posted in multiple places, which throws off citation counts, and it can only index citations made in material that is a) on the Internet (it connects to Google Books so it’s one of the few places to find citations in books) and b) that allows Google to index the full text (or otherwise makes the list of references in an article available). The main drawback to Google Scholar is that, because it depends on publicly available data, it can be easy for the unscrupulous to manipulate.

The problems with citation counts

The core problem with citation counts is that assumption upon which it depends for value, is not entirely true. The assumption is that a work is cited because it is a worthwhile piece of research.

There are many reasons why an author may choose to cite a particular article that have nothing to do with approving of the article being cited.

Consider Fleischmann, Martin; Pons, Stanley (1989), “Electrochemically induced nuclear fusion of deuterium.” J. Electroanalytical Chemistry 261 (2A): 301–30. This article has, according to the Web of Knowledge, received over 790 citations. A naive reading of the citation count would lead one to declare that this must be a very influential article. And, in a sense, that’s correct; except for the fact that it’s one of the more famous cases of science performed badly, and that most of the citations are articles stating that they can’t duplicate the results, or that the results are wrong, or writing about it as an example of science done wrong.

Even if an article isn’t fraudulent, or wrong for more innocent reasons, there are plenty of reasons that a citing article may cite an article in order to disagree with its conclusions, particularly in the social science and humanities. These negative citations don’t necessarily diminish the worth of the article in question, but they don’t really support the idea of citation counts as a measure of worth.

There can also be political, or social reasons to cite an article, that have nothing to do with its value as a piece of research. Perhaps the author is a friend that is applying for tenure. Perhaps they did a favor for the author, and now the author is repaying them. Perhaps they want to borrow the legitimacy of a more famous author by citing them in their own work, even if it’s not particularly relevant. Perhaps they’re a grad student, and it’s been “suggested” that they cite some papers written by their advisor.

Sometimes the journals you’re publishing in can get in on the act. There are some less-than-scrupulous journals that, in order to boost their prestige, encourage authors to cite other articles within the same journal, or within another journal from the same publisher. Others  inadvertently stumble into the same patterns of distortion without intending to (particularly in very niche research areas where there may only be a few people worldwide writing on the topic).

Other issues with citation counts can be more systematic. A article written for a small, niche, audience will be cited less than one written for a broad audience, but may be (within that context) a very important, influential, article. A review article (one summing up the current state of research on a topic) will usually be cited many more times than an average original research article. Obviously, self-citations by any author on a paper should be discounted (but aren’t by any of the tools listed above). Sometimes authors will cite an article incorrectly (getting the pages wrong is common, but sometimes they get something as basic as the journal wrong!), which means that, unless you catch that, it won’t get counted.

Using citation counts

So, we’ve covered the trust part  of citation counts, which leaves us with the hard task of verifying them. Or, in other words, how do we use these flawed, easily misinterpreted numbers?

The answer is to not treat them as simple numbers with simple, obvious, meaning. There are no shortcuts when it comes to evaluating a work, whether it’s an article, or a book, or something else.

You can use the tools above to get the basic citation count for an article, but once you’ve done that, you have more work to do. You’ll need to look at the citing articles and see how they used the cited article, and then decide whether to count them. You’ll have to look at where the cited article, and the citing articles were published, their acceptance policies, and other factors. You’ll have to see what a typical citation count is for articles on the same (or similar) topic to make sure you’re correctly comparing it to its peers.

Or, you can use them without all of the supplemental analysis, but with an understanding that they are not error free, and that they have fairly large error bars. How large are those error bars? It’s difficult to estimate.

Ultimately, the most accurate way to evaluate an article, or book, or whatever, is to have someone you trust (or yourself), who understands the subject material and methodology, read the article and give their opinion. Citation counts can act as a very crude filter to find works that may be significant, but in the end, you have to dig deeper.

Finding Free Full-text Resources Online

One of the most exciting aspects of pursuing Humanities research in the digital age is the prodigious and ever-growing quantity of source material available for free online. This includes books, periodicals, broadsheets and ephemera printed before 1923, collections of medieval manuscripts and archival manuscript sources. Single leaves from these online sources can be used as illustrations in publications and presentations. Material that within living memory was once difficult to locate and required on-site visits to study, or long waits for interlibrary loans to procure, is now available in seconds with a few clicks of the mouse. The scope of this material is so vast that a single post cannot do it justice, so this will be the first of several blog posts on the subject. Below are some of the prime sources for locating and accessing such content.

Google books-many things you are looking for or will find useful will come up in a search of google books. You can customize these searches by limiting them (using the “Search tools” tab) to e-books (these will be complete), books with previews (these will not provide a complete view, but will often show tables of contents and indices, which can be used for requesting chapters to order electronically from Interlibrary Loan), and year or range of years of publication. You can also limit your search to magazines.

http://books.google.com/

Hathitrust is a partnership of academic & research institutions, offering a collection of millions of titles digitized from libraries around the world. UConn is a member of Hathitrust, so our users can download items that are available “full view”, and you can limit your search to these items

http://www.hathitrust.org/

Internet Archive-a vast repository of print books and journals useful to scholars, from early printed books to indispensable classics such as Pastor’s History of the Popes. Its usefulness can be illustrated by a recent experience: after reading a new biography of the poet Marianne Moore, I wanted to read one of her early poems. Since Moore often revised or omitted early poems in later anthologies, I was fortunate to find the poem on Internet Archive, which had scanned the very rare first collection of her poems, published in London in 1921.

https://archive.org/

Europeana-millions of books, newspapers, letters, diaries and archival papers from a range of Europe’s leading galleries, libraries, archives and museums

http://europeana.eu/

Zentrales Verzeichnis Digitalisierter Drucke-For medieval manuscripts and books or journals printed in German or published in other languages in places where German was spoken, this is a marvelous resource. (Subsequent posts will include portals for digital content in other European languages)

http://www.zvdd.de/startseite/

National Libraries often offer digitized runs of historic newspapers and periodicals, sometimes in unexpected places. For example, the Austrian State Library has historic German–language periodicals from regions of the former Austro-Hungarian Monarchy that are now separate countries, such as Bohemia and Moravia (Current Czech Republic). Links to National Libraries in Europe can be found at

http://www.theeuropeanlibrary.org/tel4/

National libraries outside of Europe can be found with a google search.

Bear in mind that the portals and points of access that exist at this writing will very likely be transformed, replaced or augmented over time. The process of locating full-text online resources is dynamic and ever-changing. That is part of the challenge, and part of the fun.

More information about free full-text available online can be found at

http://ebookfriendly.com/free-public-domain-books-sources/

If you would like assistance in finding source material online, do not hesitate to contact Michael Young, Michael.s.young@uconn.edu

Harnessing the Power of WorldCat Local (Books & Media Worldwide)

Welcome to the new 2014 Spring semester! Now that classes are back in session let’s discuss some of the ways the library resources can help you find and organize sources for your papers and class projects.

Clicking the tab, “Books & Media Worldwide”, you will access WorldCat Local, an online catalog that not only gives you access to library materials at UConn but also gives you access to library materials from all around the world. For those materials not available in our library, you can request them through Interlibrary Loan through this interface. In addition, there are two more services that you may find useful using WorldCat: creating a profile and creating lists.

WordCat LocalCreating a profile in WorldCat allows you several advantages: You can create lists of resources (books, films, articles) which you can then share with your classmates/colleagues when working on a project or keep private when working on your own research. Another feature of lists is the Citation View, which displays your sources using the five more common citation styles (APA, MLA, Chicago, Turabian and Harvard). Also, you can export your citations to RefWorks or EasyBib (citation management softwares), display the list in a webpage (HTML) which you can cut and paste into your paper or download a Word document with your list already formatted in your preferred citation style.

Another cool feature available with your profile is the ability to save successful searches (keywords specific to your research) which you can come back to periodically to see new books and/or other resources added to our collections, and much more. All useful tools available to you for free!!

For today, I will focus on how to create a new profile and how to create a list.

To create a WorldCat profile:

myaccounts1On the library website, click on the link, “My Accounts” on the upper right corner of the webpage.

 

On “My Accmyaccounts2ounts” page you will get a new page where you can create a new account or sign on to an existing one.

 

 

 

 

 

You will get a new page where you can worldcatsignoncreate a new account or sign in on an existing one. Creating an account is simple. Just pick a user name and password, type your email (we recommend your UConn email) and accept the terms of services and voila, you have created your new account/profile!

 

makealistCreating a list is very easy. First you need to search on the text box in WorldCat, e.g. “water scarcity”. On the result page you will see a list of results, which can be further filtered by format, year, author or language (on left side bar menu). createlistBut on the top of the result list, you have other options: Sorting By: which allows you to sort by library and relevance, Author (A-Z), etc. (right side corner) and Save to: [New List] (on top of your search results). Check the boxes for the books you want to place in your new list and then click Save. You will be asked to name the new list “Water Scarcity” and here you can decide if you want the list to be Public (viewable by anyone) or Private (viewable only by you), then click “Create New List”. Your list is created and you can come back anytime and add to this list every time you search this topic. There is no limit to how many lists you can created in WorldCat Local. You can also get access to all your list from the Save to drop down menu, so any time you search you can add new books to your lists or create a new one as needed.

To go back to your list(s), just visit your WorldCat profile. Here is my profile so you can see how it looks with lists, saved searches, etc…

wcprofileI hope this is useful for your projects and papers this semester! And remember, if you need help finding sources for paper/project, are stuck with your research or want to learn more about WorldCat Local, contact your subject librarians!! We are here to help you!

Product Spotlight

Counseling and Therapy in Video

Among the library’s collections of databases, Counseling and Therapy in Video is unique. Instead of the typical collection of journal articles, this database contains over 1000 hours of video. These videos are of therapy sessions, lectures, and dramatizations and are meant to be a training tool for students of psychology, counseling, and social work. Videos in this collection can be browsed by theme (ex. substance abuse), type of therapy (ex. dialectical behavior therapy), video type (ex. counseling session), or therapist. The videos feature many well-known psychologists, including Albert Bandura, Martin Seligman, and Derald Wing Sue.

The collection features videos that are valuable learning tools for support staff on college campuses, including counselors, mentors, and resident advisors. For example, “Empathy Training for Ethnic and Cultural Awareness” is a 45-minute video featuring several brief skits exemplifying the types of real-life cultural misunderstandings that can take place on a college campus. In one vignette, a student for whom English is a second language is fearful of participating in group assignments. In another scenario, ethnic tensions emerge between groups of African American and Black Caribbean students.

Students planning to enter the healthcare fields will also find this collection useful. In the two-part video “Legal and Ethical Issues for Mental Health Professionals”, actors in the roles of a journalist, clerk, and a state supreme court justice discuss several real cases to demonstrate the need for health professionals such as psychiatrists and nurses to clearly distinguish between confidentiality obligations and the legal duty to report certain crimes.

To access Counseling and Therapy in Video, search UConn Libraries’ alphabetical databases list: Off-campus users will be asked to log-in with their NetID and password.

Need a refresher or just want to learn something new…

The UConn Libraries (Storrs) offer a variety of workshops every semester where you can develop your skills in various areas, such as bibliographic citation software (RefWorks), literature reviews, and using open-access publishing software (Omeka). All of these workshops are listed here: http://lib.uconn.edu/instruction/workshop/. Additional workshops will be added in areas, such as data management, geo-spatial software, and data visualization.

Here’s a sampling of upcoming workshops scheduled for the month of October:

  • (10/08/13) Introduction to Literature Reviews (UConn Libraries & Writing Center) – Great for students who are new to writing literature reviews!
  • (10/16/13) Contextualizing Research in the Humanities: Practical Tips and Strategies (UConn Libraries & Writing Center) – For students in the humanities who want to improve their literature review writing skills.
  • (10/11/13) Intro to Creating/Curating Research with Omeka (UConn Libraries/Scholars’ Collaborative) – For participants, who want to present their research using an open-access platform, or engage their students in online publishing.
  • (10/2/13; 10/15/13) Introduction to RefWorks (UConn Libraries) – If you’re looking for an easy way to organize your citations for course papers, dissertation, or publication, this workshop is for you!
  • (10/9/13; 10/24/13) Pivot Funding Opportunity / Scholar Expertise Database (UConn Libraries’) – For students and faculty interested in finding grants, fellowships, and other awards related to their research, performance, or scholarly needs.

Workshops are added throughout the semester, so visit often or bookmark this link: http://lib.uconn.edu/instruction/workshop/.

Also check out workshops offered by other centers or departments on campus:
Writing Center - develop your writing skills
Scholars’ Collaborative @ UConn Libraries - learn and use new programs and digital tools
Graduate School - professional development workshops and resources

THATCamp New England at UConn

This year’s THATCamp New England will be held at UConn on Thursday and Friday, October 18-19, 2013. Organizers for this event include faculty, graduate students, librarians, and staff from the Departments of Digital Media & Design, Medieval Studies, Literatures, Cultures and Languages, Journalism, Institute for Teaching and Learning, and the University Libraries.

THATCamp = The Humanities and Technology Camp, an unconference for people working in the humanities, technology, and other disciplines to meet, learn, and build skills in an informal environment.

Friday will primarily consist of pre-planned workshops, which are listed here, while the true unconference sessions will take place on Saturday. The full schedule and workshop details will be posted closer to the event date. If you’d like to register for THATCamp New England, please visit: http://newengland2013.thatcamp.org/. There is no registration or workshop fee.

Follow THATCamp New England on Twitter: @THATCampNE and use #thatcampne when tweeting about the event.

For other events and programs related to the digital humanities at UConn, please visit

 

 

 

 

Pivot–the most comprehensive search for grant funding opportunities

The Office of the Vice President of Research and the UConn Libraries have acquired Pivot, a database for grant funding opportunities. You can search for grant funding in all the major research areas in the social science, sciences and humanities. In addition, this is  a superb place to find potential collaborators through the Pivot Profiles, which provide information on over 3 millions scholars from around the world.

For those new to this product, a Pivot representative is coming to UConn to do a demonstration of all the features available in this database. This presentation is open for faculty, students and staff.

Mark your calendar!

Pivot Presentation
Thursday, May 10, 2012
Classroom Building Room 101
10:30am – noon

If you can’t wait to explore Pivot, visit the official site at http://pivot.uconn.edu/ There are also training videos on how to use Pivot on YouTube so check them out!

Marisol Ramos
Librarian for Latin American, Caribbean and Latino Studies, Spanish and Anthropology

Practicing Perfection: Memory and Music Performance

Practicing Perfection: Memory and Music Performance

Tânia Lisboa
Royal College of Music, London, UK
& Roger Chaffin
Department of Psychology, University of Connecticut

How do classical musicians perform long programs from memory with note-perfect accuracy? To find out how, we have studied concert soloists as they prepare new works. Experienced performers have a mental map of the piece in mind as they play. The map tells them where they are and what comes next. Landmarks in the map are established during practice by thinking about particular features of the music so that later, during performance, they come to mind automatically. The map allows the soloist to monitor the rapid, automatic actions of playing and adjust to the needs of the moment. We will describe our research and Dr. Lisboa will perform two short works that we are currently studying: Prelude from cello suite No. 1 & Bourree from cello suite No. 3, by J.S. Bach.

Cellist Tânia Lisboa is widely acknowledged as one of Brazil’s foremost musical personalities with an international profile and an extensive range of recordings. She holds a PhD in performance and, in parallel to her solo career, she appears in masterclasses and lectures world-wide. Tânia Lisboa is a member of staff at the Royal College of Music, Center for Performance Science, in London.

Roger Chaffin is a professor of psychology at the University of Connecticut. His longitudinal case studies of experienced musicians provide a new way of understanding the cognitive processes involved in skilled performance.

Tuesday, April 10
12:30 pm-1:30 pm

University of Connecticut, Greater Hartford Campus, Library Building, Room 404
1800 Asylum Ave., West Hartford
Free and open to the public

Co-sponsored by: University of Connecticut Research Foundation, Center for Latin American & Caribbean Studies Center for the Study of Perception and Action (CESPA), Community School of the Arts, Music Department, and Trecker Library