Wednesday, 21 March 2018

What Can Tell Us About Policy Citations of Research? An Analysis of Data for Research Articles from the University of Sheffield

Image of Andy Tattersall
Andy Tattersall
Image of Chris Carroll
Chris Carroll              
Andy Tattersall (ScHARR Information Resources) and Dr Chris Carroll (ScHARR Health Economics and Decision Science) have published a new paper in Frontiers in Research Metrics and Analytics. The paper looked at published University of Sheffield research and what the data says about the impact of its research on national and international policy. The percentage of outputs with at least one policy mention compares favourably with previous studies, while huge variations were found between the time of publication and the time of the first policy citation. However, some problems with the quality of the data were identified, highlighting the need for careful scrutiny and corroboration.
Altmetrics offers all kinds of insights into how a piece of research has been communicated and cited. In 2014 added policy document tracking to its sources of attention, offering another valuable insight into how research outputs are used post-publication. At the University of Sheffield we thought it would be useful to explore the data for policy document citations to see what impact our work is having on national and international policy.

We analysed all published research from authors at the University of Sheffield indexed in the database; a total of 96,550 research outputs, of which we were able to identify 1,463 pieces of published research cited between one and 13 times in policy. This represented 0.65% of our research outputs. Of these 1,463 artefacts, 21 were cited in five or more policy documents, with the vast majority – 1,185 documents – having been cited just once. Our sample compared very well with previous studies by Haunschild and Bornmann, who looked at papers indexed in Web of Science and found 0.5% were cited in policy, and Bornmann, Haunschild and Marx, who found 1.2% of climate change research publications with at least one policy mention. From our sample we found 92 research articles cited in three or more policy documents. Of those 92 we found medicine, dentistry, and health had the greatest policy impact, followed by social science and pure science.

We also wanted to explore whether research published by the University of Sheffield had a limited time span between publication and policy citation. We looked at the time lag and found it ranged from just three months to 31 years. This highlighted a long tail of publications influencing policy, something we would have struggled to identify prior to without manual trawling. The earliest piece of research from our sample to be cited in policy was published in 1979 and took until 2010 before receiving its first policy citation. We manually checked the records as we found many pre-1979 publications to have been published much later, often this century. This is likely due to misreported data in the institutional dataset, giving a false date; highlighting the need to manually check such records for authenticity. The shortest time between research publication and policy citation was a mere three months: a paper published in November 2016 and first cited in National Institute for Health and Care Excellence (NICE) policy in January 2017.

The reports are only as good as the data they analyse and our research did uncover some errors. Looking at those 21 papers with more than five policy document citations, we found seven were not fit for inclusion. One such example was identified when we discovered research papers had been attributed to the University of Sheffield when the authors were not, in fact, affiliated to the university. As this data is sourced from our research publications system, we assume this was a mistake made by the author; this can happen when authors incorrectly accept as their own papers suggested to them by the system. While this was almost certainly a genuine error, and may have been rectified later, the system had not yet updated to take account of such corrections. Another of these papers was mistakenly attributed to an author who had no direct involvement in the paper but who was part of a related wider research project. Another of the publications was excluded due to it not, in fact, having actually been cited in the relevant policy document. One of the papers that was included belonged to an author not at Sheffield at the time of publication, but who has since joined the institution. This showed that’s regular updates were able to discover updated institutional information and realign authors with their current employer.

The two most cited papers came from our own department, the School of Health and Related Research (ScHARR), in the field of health economics. Only two of the 14 most cited publications were in a field other than health economics or pure economics, both of which were in environmental studies. In total, the 14 most cited research outputs were cited by 175 policy documents, but we identified 9% (16) of these as duplicates. Of those 175 citations we found that 61% (107) were national, i.e. from the UK, and 39% (68) were international, i.e. from countries other than the UK or from international bodies such as the United Nations or World Health Organization. continues to add further policy sources to its database to trawl for citations. As a result, it should follow that our sample of 1,463 research outputs will not only grow with more fresh policy citations, but as older research citations are identified through new policy sources of attention. This work also highlights the importance of research outputs having unique identifiers so they can be tracked through altmetric platforms; it is certain that more of our research will be cited in policy, but if no unique identifier is attached, especially to older outputs, it is unlikely the system will pick it up. is a very useful indicator of interest in and influence of research within global policy. Yet there are clearly problems with the quality of the data and how it is attributed to subsequent data. We found one third of our sample of the 21 most cited research outputs had been erroneously attributed to an institution or author. Whether this is representative of the whole dataset only further studies will find out. Therefore it is essential that any future explorations of research outputs and policy document citations be double-checked and not taken on face value.

This blog post is based on the authors’ article, “What Can Tell Us About Policy Citations of Research? An Analysis of Data for Research Articles from the University of Sheffield”, published in Frontiers in Research Metrics and Analytics (DOI: 10.3389/frma.2017.00009).

The blog post was originally written for the LSE Impact Blog and is licensed under a Creative Commons Attribution 3.0 Unported License unless otherwise stated. The original article appears here 
Creative Commons Licence

Monday, 8 January 2018

New research must be better reported, the future of society depends on it

File 20171220 4954 1k92h2l.jpg?ixlib=rb 1.1
Understanding how and why things happen can help people make sense of the world. Pexels
Andy Tattersall, University of Sheffield

Newspaper articles, TV appearances and radio slots are increasingly important ways for academics to communicate their research to wider audiences. Whether that be the latest health research findings or discoveries from the deepest, darkest parts of the universe.
In this way, the internet can also help to facilitate these channels of communication – as well as discussions between academics, funders and publishers, and citizen scientists and the general public.
Yet all too often research-led stories start with “researchers have found”, with little mention of their names, institution and who funded their work. And the problem is that by reporting new research in this way, it fails to break down the stereotypical image of an ivory tower. For all readers know these “researchers” might as well be wearing white lab coats with the word “boffin” on their name badges.

Rolling news

News is now a 24-hour operation. Rolling coverage of stories means journalists have their work cut out in maintaining this cycle. But that is no excuse for missing out important pieces of information that underpin a story.
Take for example a story relating to health research that has wide ranging societal impact. Supporting evidence, links and named academics help a story’s authenticity and credibility. And at a time when “fake news” is an increasingly sticky problem it becomes essential to link to the actual research and therefore the facts.

Accurate reporting, it’s not rocket science. Pexels

This is important, because research goes through a peer review process where experts in the same field of research critically assess the work before it can be published. This is similar to news stories that are edited to ensure they are of good quality – although this process takes far less time.

Accurate reporting

In academia there has been a huge move to make research openly available and therefore accessible for the whole of society. While research institutions are making great strides in public engagement and the wider understanding of science, media organisations still remain instrumental in that process.
And while it’s been claimed that the public are tired of experts, the impact they have on society – from building skyscrapers to keeping us alive – is undoubtedly fundamental to our existence.

Science and technology have changed the way we work, communicate, and view the world. Shutterstock

But poor or incomplete reporting undermines respect for experts by misrepresenting the research, especially by trivialising or sensationalising it. So while academics from various disciplines are often willing to talk to the media – either as an author or from an independent expert viewpoint – misreporting of research and particularly data (whether intentional or unintentional) has a negative effect.
Academics are then vilified as having something to hide or accused of making up their research, while members of the public are exposed to unnecessary anxiety and stress by inappropriate headlines and cherry picked statistics that are reported in a biased way.

The public good

Of course, not everyone will want to check the citations and research outputs – and not everyone has the critical skills to assess a piece of specialised academic writing. Yet there are lots of people who, given the opportunity, would be interested in reading more about a research topic.
Media coverage opens up a democratic debate, allows people to explore the works of an accomplished researcher and helps the public understanding of science. And in this way, fair and accurate reporting of research encourages academics to be willing to work with the media more regularly and build good working relationships.
The ConversationNot only that, but the proper and accurate communication of science is beneficial to the whole of society – from the government to its citizens. So in the age of “fake news” it is more important than ever to make sure that what’s being published is the truth, the whole truth and nothing but the truth.

Andy Tattersall, Information Specialist, University of Sheffield

This article was originally published on The Conversation. Read the original article.

Friday, 15 December 2017

Start Now and Make 2018 The Year of Hassle-Free Organisation

Image of Sheldon Korpet
Sheldon Korpet
Information Officer
I often want to try something new at work to see if I can improve on my previous efforts. However, the more routine (required) demands are always compete for my time too 😒 Sound familiar?

Whether you’re a super busy student or a new professional, keep reading to learn a new way to organise your work tasks and make focused progress 👍

You Can't Do Everything at Once

At the start of the year I ran a “goodie bag promo” project and as a result our small library is getting more footfall, more inquiries and more complex questions, which is really nice to see. However, it does mean further time constraints.

I got sick of 'To Do' lists   mine always looked messy or I lost them or had seven going at once so I inevitably forgot things they were meant to remind me to do! 🙅📝

One of the things that has enabled me to up my capacity, without forgetting any important things, was starting to use a Personal Kanban board.

What is Kanban?

I first heard of this method through my Business Management degree. It's a system which aims to keep tasks moving through the workflow and I’ve adapted it slightly to fit with the re-occurring, never-ending tasks.

This is All Fabulous But How Does It Work?

Tasks are assigned to four categories and as you progress you can move them closer to competition:
1 Could do/ should do – this is where you store ideas, work tasks assigned, upcoming projects or things you’re putting off. You haven’t started these yet but you might in the future.
2 Doing – These are your current tasks for the day/week. Do not put more than three things in here to stay productive! You could also assign yourself a deadline for these tasks.
3 Ongoing - This is where I store all those never ending tasks like asking people to renew their books. For a student this might be "Weekly reading for HAR679". I can move it in to “Doing” so I know what I’m focusing on and this is also an area to store projects which you've had to put on hold while waiting on a response from someone else.
4 Done – this is without a doubt the best bit on the board for me. When it’s blank it motivates me to work hard and complete something so I can start to fill it. When it’s full I can bask in my own glory 👸

For this project you're going to need a template and some small sticky notes

Personal Kanban in the Library

As you can see, I’ve gone for an A4 piece of paper with sticky notes but you could use larger paper or create a digital version in Trello or Padlet which would let you access it anywhere. However, I leave this at my enquiry desk  and I get some level of satisfaction from physically moving the post-its.

Either way, it’s a great method to track your progress and hold yourself accountable to get projects finished in good time; instead of taking on about ten things at once:

☑️ Stay focused 
☑️ Make progress 
☑️ Reducing the risk of non-completion

Having project ideas or tasks recorded in “Could do/should do” but not rushing in to them also has the added benefit of giving time for you to reflect. This might be on what would be the best way to go about them or helping you realise if it’s even necessary to spend your time on this.

You Can Do It

The great thing about this method is that it’s cheap and easy. There’s nothing worse than procrastinating and wasting time getting organised – you can make your own template in a few minutes or download the one I made here.

If you give it a shot, I’d love to know! Feel free to tweet me a photo or let me know if you found it useful @SheldonKorpet

Monday, 27 November 2017

Andy Tattersall talk at Spot On 2017 at The Francis Crick Institute

Andy Tattersall gave a short lightning talk on the title: "Isn't it time we had a research equivalent of the learning technologist?" at this year's Spot On Conference at The Francis Crick Institute. The recording of the talk and the questions afterwards can be viewed below with the abstract.

Researchers increasingly need to understand a multitude of topics including digital copyright, impact, altmetrics, communications, social media, research data management and sharing, open access, infographics, video, animation and mobile apps. Yet all too often they have little time, support or encouragement to explore these topics and have they need to make informed judgements on the most appropriate technologies.
For decades skilled professionals have provided researchers with excellent services around collection management, content curation and discovery, critical appraisal and reference management among other services. More recently they have stepped into new areas of support and applied their knowledge around social media, metrics, scholarly communications and research data management. Yet despite this there has been no formal role to step into departments and faculties to address the shortfalls of support at the research coalface. The idea of the research technologist is a professional role who like a Swiss Army Knife is adapt at utilising new tools and technologies to support a modern, fit for purpose, research cycle. The purpose of the workshop would be to discuss this idea and whether the research community believes it is something they would benefit from and what areas they most need frontline support.

Thursday, 16 November 2017

Andy Tattersall named as a Jisc Social Media Superstar for 2017

Andy Tattersall has been named in Jisc's top ten HE Social Media Superstars of 2017. Andy was nominated for the award after his work creating ScHARR’s YouTube channel, Andy’s video series’ include ScHARR Bite Size series which teaches the viewer “something new in 20 minutes”.  His Research Hacks series contains 44 helpful videos, and the more recent Cite Hacks series features engaging illustrations and information – such as this video that covers blogging about your research.  
Judges’ comments: Andy’s use of YouTube playlists to give bite-sized information is a really effective way to share knowledge simply with colleagues and peers across the world.  We thought the Cite Hacks series was particularly is good.
Andy said about social media
“Higher education is now in a continual state of change thanks to the web and social media, it offers a wealth of new opportunities for teaching and learning, knowledge sharing and opening up of our resources across the globe. Video plays an important part of that change as it allows bite size, cheap, accessible knowledge that is shared on all platforms and in the classroom, lab, or even on the bus.”
As well as @Andy_Tattersall, Andy can be found tweeting from @ScHARRSheffield and @MultiMediaIT . Each winner not only makes the top ten list, but also wins an edtech experience for their class, robot and virtual reality included. The competition sets out to celebrate the innovative ways in which social media is being used in HE to add value to sector-practice.

The final line-up was chosen by a panel of HE and social media experts, including; Jisc’s social media team, Sarah Knight (head of change – student experience), and award-winning social media editor for Times Higher Education, Chris Parr.
Richard Tatnall, digital communications manager at Jisc said:
“What really impressed me was the impact our superstars are making with their social media activity. We saw great examples of reaching vast audiences with a single message on platforms such as Twitter and Instagram, as well as making big impacts on a small, defined audiences in closed and private networks. There’s no question that social media can be highly resource intensive so being able to demonstrate the value it delivers is vital and our superstars were able to do this in spades.”

Friday, 3 November 2017

Andy Tattersall interviewed by the Librarians Aloud Podcast

I was really delighted to be interviewed by Laura Rooney Ferris for the popular Librarians Aloud Podcast alongside Jan Holmquist. In the podcast I talk about scholarly communications, digital academia, open access and data. You can listen to the podcast here 

Thursday, 19 October 2017

Andy Tattersall at Internet Librarian International

Image of Andy Tattersall
Andy Tattersall
As with the previous six years (it might be longer), I attended the popular and exciting conference Internet Librarian International. I've given numerous talks there over the last few years and had a year off last year to moderate a really superb session titled 'Come and feel the love'. This year I returned to the speakers lectern, not to deliver one, two, but three talks. I had pitched two in which were accepted and had an invite by the conference organisers to make it a trio. This put me in the brackets as such luminaries Phil Bradley and Marydee Ojala by giving multiple talks this year.

The first was at the end of day one where I talked about the video collections I have created as part of my role using tools like Adobe Spark and Videoscribe. I was alongside a really good presentation on content creation by The King's Fund by Hong-Anh Nguyen (a member of Sheffield's iSchool Alumni) and Deena Maggs. With three presentations, I felt like I was going into a sporting tournament where I was keen to get my first fixture under my belt. 

The next one followed after hearing David White from The University of Arts deliver a superb keynote. David is a very engaging speaker who I've had the pleasure of delivering a webinar alongside him for a couple of years ago. His talk was on the opportunities to be had for librarians in a world of misinformation and people looking for quick and agreeable answers. I then gave my talk on the future or research support and potential roles that could appear in the future to deliver them at the coal face of research. I was introduced by Phil Bradley which was a real pleasure and was fortunate to have quite a packed room for the session. At the the end there was some useful debate as to where we might be going with this.

Finally, to finish on a high note I presented in the same session as four wonderful Irish librarians who between them have created two really informative and entertaining podcast series for their institutions - pretty much in their own time. The first of the talks was delivered by the team who the produce LibrariansAloud Podcast which regularly interviews professionals from within the library and information sector. I was also pleased to be in the same session as colleagues who deliver the enjoyable Shush Radio Podcast who spoke about their work making podcasts to promote their library service at University College Cork. My talk was about the work that myself and fellow ScHARR information specialist Mark Clowes undertakes each year to run a 24 hour pop up radio station to support the Inspiration for Life event to raise funds for local cancer charities. As a result of the session I was taken off to a quiet spot to be interviewed by Laura Rooney Ferris from LibrariansAloud for a future podcast, where you will hear us solve most of the world's problems with the aid of good quality information.

As always Internet Librarian International is a well run conference with lots of energy that brings together regular faces as well as new ones. There was even a session for new professionals to help them on the right track career-wise. Another bonus of a really enjoyable conference was hearing that my book had sold out on the Facet Publisher's table, although I'm fairly certain they will have only packed one. Below are all my slides from the conference, the 24 Hour Inspire set seem to have gone strange after being imported into Slideshare, apologies for that. Seriously, who would want to work with technology?