Assessing regional knowledge capabilities and diffusion potential of new technologies through web mining and machine learning: A case study of German municipalities

Abstract

The growing use of web mining and machine learning in economic geography provides new possibilities for the characterization and analysis of regional knowledge capabilities and knowledge relations. While previous studies in this context have focused on mining web data at the firm-level, the aim of this study is the conceptualization of an approach for using web content and web structure data of local and regional government institutions. To generate a database that allows a comprehensive and novel way of assessing regional knowledge capabilities and networks, we collect website data at the municipal-level in Germany via the web crawling- and scraping-framework ‘Scrapy’. The collected information can be used to identify patterns and relationships between regional government institutions as well as commonalties between topics covered on their websites. The interrelations are identified via hyperlink network analysis. For the topic modelling we choose a neural network-based approach in form of the transformer model ‘GoogleBERT’. The topics are modelled on a subpage level. The created database will then be evaluated in cooperation with a sample of German municipalities. Thereafter we plan to extend the approach by combining the municipal-level data with web text and web structure data at the firm-level. Hereby we hope to improve the understanding of the regional diffusion potential of new technologies and provide valuable insights to inform policy decisions and strategies for promoting the adoption of these technologies.