Skip to Main Content

Data tools and techniques

What is an API

An application programming interface (API) is a connection between computers or between computer programs. It is a type of software interface, offering a service to other pieces of software.[1] A document or standard that describes how to build or use such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation.

Subscribed publisher APIs

DimensionsIEEE | ScienceDirect | ScopusWeb of Science

 

Dimensions

 

What it does: Provides programmatic access to the metadata of their publications, datasets, grants, patents and clinical trials

How it's accessed: Varies, web interface using DSL, and RESTful interface when using Jupyter notebook

Result format: JSON

How to register: Must be part of a subscribing institution to have full text access. UNSW users should set up an individual account, fill out the Dimensions Contact us online form requesting API access, ensuring they provide their UNSW email. They may request further details

Limitations: Maximum number of records returned from a single query 1000

Contact for technical questions: Submit online form via https://www.dimensions.ai/contact-us/

For more information: https://api-lab.dimensions.ai/ , video for using their API with Google Sheets

 

IEEE

 

What it does: Provides flexible query and retrieval of metadata records for more than 4 million documents comprising IEEE journals, conference proceedings, and technical standards

How it’s accessed: HTTP requests using structured URL queries

Result format: JSON, XML

How to register: Follow the steps at https://developer.ieee.org/getting_started

Limitations: Maximum of 200 results may be retrieved in a single query.  A query term can only contain a maximum of 10 words

Contact for technical questions: https://developer.ieee.org/contact

For more information: https://developer.ieee.org/ 

 

ScienceDirect

 

What they do: Multiple APIs available for different use cases, including text mining of full text content, search widgets, displaying journal or book level data, federated searching, and indexing

How they’re accessed: varies, depending on use case

Result format: varies, depending on use case

How to register: Free to register - https://dev.elsevier.com - video for API key generation

Limitations: varies, depending on use case

Contact for technical questions: integrationsupport@elsevier.com

For more information: https://dev.elsevier.com/sd_apis.html

 

Scopus

 

What they do: Multiple APIs available for different use cases, including displaying publications on a website, showing cited-by counts on a website, federated searching, populating repositories with metadata, populating VIVO profiles, and others

How they’re accessed: varies, depending on use case

Result format: varies, depending on use case

How to register: Free to register - https://dev.elsevier.com/

Limitations: varies, depending on use case

Contact for technical questions: integrationsupport@elsevier.com

For more information: https://dev.elsevier.com/sc_apis.html

 

Web of Science

 

What it does: Allows text- and data-mining access to content in Web of Science Lite

How it’s accessed: Accessible via Clarivate’s Developers Portal

Result format: JSON or XML

How to register: Must be part of a subscribing institution to have full text access. UNSW users should set up an individual account at Clarivate’s Developers Portal https://developer.clarivate.com and 'Sign Up' filling out a form with name, email address, and an optional description of the project.

Limitations:  Maximum number of tokens per user:1. Maximum number of requests/second:2.

Users may use the API to access the Data Fields in accordance with the applicable License Level, in each case as permitted by your subscription.

If a user is using Web of Science data in an article or presentation they must appropriately cite and credit Clarivate Analytics as the source.

Contact for technical questions: https://developer.clarivate.com/content/contact

For more information: https://developer.clarivate.com/

 

This guide was adapted from MIT's list of scholarly APIS

 

Open access APIs

arXivCORE | Crossref | MET | OECD | PubMedWorldbank

 

arXiv

 

What it does: Gives programmatic access to all of the arXiv data, search and linking facilities

How it’s accessed: API calls are made using any web-enabled client (e.g. a web browser) to make an HTTP GET or POST request to an appropriate URL.  API users can use the programming language of their choice

Result format: Atom

How to register: Free to use, no registration or API key required

Limitations: No stated limitations, but high-volume users should contact arXiv at http://arxiv.org/help/contact

Contact for technical questions: Forum discussion for help at arXiv Google Group 

For more informationhttp://arxiv.org/help/api/index , API Manual at https://arxiv.org/help/api/user-manual

 

CORE

 

What it does: Gives programmatic access to metadata and full text of millions of OA research papers

How it’s accessed: RESTful interface, queries are made as HTTP GET requests

Result format: JSON, CSV

How to register: Free to use, API key required, register for API key at https://core.ac.uk/api-keys/register

Limitations: Maximum 10,000 results. More details at https://api.core.ac.uk/docs/v3#query_large 

Contact for technical questions: theteam@core.ac.uk  

For more informationhttps://core.ac.uk/services#api

 

Crossref

 

What it does: Allows access to metadata records for over 75 million scholarly works that have CrossRef DOIs, covering around 5,000 publishers.  Can be used for text- and data-mining, checking against funder mandates, and to obtain metadata in a variety of representations.

How it’s accessed: RESTful interface

Result format: JSON

How to register: No registration required

Limitations: No stated limitations

Contact for technical questions: support@crossref.org or submit query via their online form and specify 'Technical support'

For more informationhttps://www.crossref.org/services/metadata-delivery/rest-api/

 

MET (Metropolitan Museum of Art)

 

What it does: Provides datasets of information on more than 470,000 artworks in its collection for unrestricted commercial and noncommercial use.

How it's accessed: RESTful interface

Result format: JSON, CSV

How to register: none required

Limitations: Images not included in the data sets. 

Contact for technical questions: openaccess@metmuseum.org

For more information: https://github.com/metmuseum/openaccess

 

OECD

 

What they do: Allows programmatic access to a selection of OECD datasets

How they’re accessed: two RESTful APIs available for queries in SDMX-JSON or SDMX-ML formats

Result format: JSON, XML

How to register: No registration required

Limitations: 1 million data points; not all OECD datasets are covered

Contact for technical questions: OECDdotStat@oecd.org

For more information: https://data.oecd.org/api/

 

PubMed

 

What it does: Set of 8 server-side programs for searching 38 NCBI Entrez databases of biomedical literature and data

How it’s accessed: To access data, a piece of software posts a URL using a fixed syntax to NCBI's E-Utilities server, then retrieves and processes data.  Users can use any programming language that can send the URL and interpret the XML response (e.g. Perl, Python, Java, C++, etc.)

Result format: XML

How to register: Free to register; registration is not necessary but strongly encouraged.

Limitations: 3 URL requests per second; large jobs should be limited to weekends or business hours 

Contact for technical questions: eutilities@ncbi.nlm.nih.gov

For more informationhttps://www.ncbi.nlm.nih.gov/books/NBK25500/

 

Worldbank

 

What they do: Provide access to World Bank statistical databases, indicators, projects and loans, credits, financial statements and other data related to financial operations

How they’re accessed: Three RESTful APIs available to provide access to different datasets: Indicators (time series data), Projects (data on the World Bank’s operations), Finances (World Bank financial data)

Result format: XML, JSON, RDF, and Atom, depending on specific API used

How to register: Free to use, no registration or API key required

Limitations: Request volume limits are unspecified, but should be “reasonable”

Contact for technical questions: data@worldbank.org or “Contact support” link here

For more information: https://datahelpdesk.worldbank.org/knowledgebase/topics/125589

 

This guide was adapted from MIT's list of scholarly APIS and an item from Mina Rees Library APIs