The amount and variety of data that is available is growing rapidly and at a quicker pace. There is a wider range of data available in many formats, including audio, video, computer logs, purchase transactions, sensors and social networking sites. This has created big data, which are large, often unstructured data sets that are available, potentially in real time.

Big data is a big topic. As the UK's largest producer of official statistics, we want to understand the impact it may have on our statistical processes and outputs. This project aims to investigate the advantages and challenges of using big data, and to develop a longer term strategy for using big data in official statistics.

We are committed to protecting the confidentiality of all information we hold. To produce statistics using big data sources, we are only interested in trends or patterns that can be seen, not personal data about individuals. However, we recognise that accessing data from the private sector or from the internet may raise concerns around security and privacy. We only access publically available, anonymous or aggregated data within the project. All of our work fully complies with legal requirements and our obligations under the Code of Practice for Official Statistics. Part of the research within the project considers the ethical issues associated with using these types of data sources within official statistics.

For more information about the Big Data Project, please contact us: