The amount and variety of data that is available is growing rapidly and at a quicker pace. There is a wider range of data available in many formats, including audio, video, computer logs, purchase transactions, sensors and social networking sites. This has created big data, which are large, often unstructured data sets that are available, potentially in real time.
Big data is a big topic. As the UK's largest producer of official statistics, we want to understand the impact it may have on our statistical processes and outputs. This project aims to investigate the advantages and challenges of using big data, and to develop a longer term strategy for using big data in official statistics.
We are committed to protecting the confidentiality of all information we hold. To produce statistics using big data sources, we are only interested in trends or patterns that can be seen, not personal data about individuals. However, we recognise that accessing data from the private sector or from the internet may raise concerns around security and privacy. We only access publically available, anonymous or aggregated data within the project. All of our work fully complies with legal requirements and our obligations under the Code of Practice for Official Statistics. Part of the research within the project considers the ethical issues associated with using these types of data sources within official statistics.
For more information about the Big Data Project, please contact us: email@example.com
Efallai y bydd hefyd gennych ddiddordeb yn:
- Progress report - January to March 2015 (256.0 kB pdf)
- Progress report - October to December 2014 (429.3 kB pdf)
- Progress report - July to September 2014 (370.6 kB pdf)
- Progress report - April to June 2014 (464.1 kB pdf)
- Progress report - January to March 2014 (53.9 kB pdf)
- Analysing low electricity consumption using DECC data (506.6 kB pdf)
- Web scraped data: Extreme price changes (107.0 kB pdf)
- University of Southampton report - Using energy metering data to support official statistics: A feasibility study (4.8 MB pdf)
- GSS methodology series No 40 - Modelling sample data from smart-type electricity meters to assess potential within official statistics (2.1 MB pdf)
- ONS methodology working paper series No 5 - Comparing travel flows between 2011 Census and Oyster card data (1.5 MB pdf)
- ONS Methodology Working Paper Series No 7 - Comparing counts of electricity meters and addresses by postcode in England and Wales (808.3 kB pdf)
- Research indices using web scraped data: May 2016 update (622.0 kB pdf)
- Using geolocated Twitter traces to infer residence and mobility (1.3 MB pdf)