News-Article

At an international workshop on business data collection, the DSC presented a methodological framework for integrating web data and Natural Language Processing into business statistics. The approach builds on methodological expertise developed within the data competence center “DataNord”.

From 8 to 10 June 2026, the Data Science Center (DSC) participated in the Seventh International Workshop on Business Data Collection Methodology (BDCMW 2026) in Heerlen, Netherlands. The workshop brought together researchers, methodologists, and representatives of national statistical institutes to discuss emerging approaches to business data collection, survey methodology, and the use of artificial intelligence (AI) in official statistics.

DSC-team member Maryam Movahedifar presented the contribution “Integrating Automated and Text-Derived Data into Business Survey Workflows: A Practice-Driven NLP Framework Based on DataNord Activities”. The presentation highlighted how web data and Natural Language Processing (NLP) can support the modernization of business statistics and complement traditional survey-based approaches.

Addressing Challenges in Business Data Collection

Business surveys remain one of the most important sources of official statistics. However, statistical agencies increasingly face challenges such as declining response rates, rising collection costs, and growing respondent burden. At the same time, businesses continuously generate large amounts of digital information through websites, reports, and online platforms.
The presentation explored how these alternative digital data sources can be systematically integrated into statistical workflows. By combining web scraping, automated data collection, and NLP methods, information from unstructured online content can be transformed into structured indicators that support statistical production processes.

A DataNord-Inspired Methodological Framework

The presented workflow emerged from practical methodological questions encountered within the BMFTR-funded data competence center “DataNord”. As part of the DataNord help-desk at the DSC, researchers regularly seek support for collecting, processing, and analyzing web-based and text-derived data.

Building on these experiences, Maryam developed a framework that demonstrates how data from websites, APIs, and digital documents can be integrated into a unified processing pipeline. NLP techniques are then applied to extract relevant business information, such as activities, products, technologies, and organisational characteristics. The extracted information can subsequently be linked to established statistical classifications and business registers.
The work illustrates how data expertise developed within DataNord can contribute beyond individual research projects and support broader methodological innovation in data-intensive domains.

Ensuring Quality and Reliability

A central theme of the presentation was the question of data quality. While digital data sources offer significant opportunities, they also introduce challenges related to coverage, consistency, and reliability.

To address these issues, the framework incorporates entity matching, harmonisation, validation, calibration, and benchmarking procedures. These steps help ensure that automatically collected information can be integrated into existing statistical systems while maintaining the quality standards required for official statistics.

Exchange with the International Research Community

The workshop provided an excellent opportunity to exchange ideas with experts from national statistical institutes, universities, and research organizations working on business data collection and AI-supported statistical methodologies.

The discussions highlighted a growing international interest in leveraging alternative digital data sources for statistical applications and reinforced the importance of robust methodological frameworks that combine automation with careful quality assurance.

For the DSC and DataNord, participation in BDCMW 2026 offered valuable insights into current developments in web data acquisition, NLP-based information extraction, and data integration methodologies. These perspectives will help inform future methodological consulting, training activities and collaborative research within the Bremen research community.

The participation also illustrates the wider role of the DSC: practical questions from interdisciplinary research support can become the starting point for methodological approaches with relevance far beyond a single project or institution.