OT-ST-WS-09 | Data provisioning
Dr. Brenner Silva
Our main aim is to improve our understanding and use of existing technologiesin data provisioning. We will use python, because it is easy to learn,flexible and supported by a large community of enthusiastic developers. Equipped with the proper web technology background, we will practice requesting data programmatically, considering not only how to feed our analysis, but also how the improve research quality and decision-making
How are data provided? What is the technology behind data provisioning? Data provision relies on standards and procedures that stands for different functionalities in accessing the data. Based on that functionalities data access takes place either by direct interaction with the source or via an end-user application. Further, the data streaming is set up in different ways: manually or semi-automatically, selectively, and connected to multiple providers. In this lecture, we dive into concepts and technologies behind the solutions used in data provisioning. More specifically, we explore web data services, we get to know how these are constructed and how the service interface allows us to interact with the database at the server side. In our exercise, we use the most popular and efficient python libraries for requesting (“requests”) and analyzing tabular data (“pandas”).
Basic technical knowledge of web technologies, standards, and architecture used in data provisioning. Overview of data repositories. Training in remote data requests at a single or multiple providers and combined analysis of the gathered data.
Required is basic knowledge of a programming scripting language. We use python and the hands-on exercise is possible using either jupyter notebook or the python interpreter on your computer.
Each participant needs a computer with access to internet and an up-to-date browser for accessing the virtual room.