Open data & API
It is becoming a familiar term by the day and the growing interest is driven by the power given to the public to put it to good use.
Open data & API
Open data paves the way for researchers and scholars to explore new frontiers and support their studies by data-centred results leading to sound and realistic conclusions. Social scientists will stretch their experiments to account for the available public resources, hence unlocking new opportunities in numerous fields.
The availability of open data APIs also allows software and technology companies to enhance their models and applications to integrate several open data sources and provide more reliable solutions. This will make citizens happier and help them become more empowered.
While developing our first smart city application `Citiapp`, we used many open data sources to enrich the content of our application and offer a unique and rewarding experience to our users. From this experience, we summarize some points to be considered in data collection, structuring, analysis, and activation that might be helpful to others to contemplate and ponder upon when working with open data.
1.
2.
3.
4.
Challenges
We share with our community some of the challenges when working with open data to spread experience.
Anything that might go wrong, will! This is usually the case when collecting data from external sources. Volume, variety and structure, sources, and security are critical dimensions that can result in unexpected delays if anything goes wrong while collecting data.
Volume is essential to plan storage requirements and timelines. Variety and structure are when the data contains different types stored in different structures.
Sources refer to the numerous input sou rces for data where you must check consistency between these sources before integrating such data into your solutions.
Security can be interpreted as not compromising the security of the data sources and maintaining its security through the chain of custody when moving it from one repository to another or storing it locally or online.
In many situations, old data are stored in an outdated format given that it was collected using outdated means. Data owners would be encouraged to update the format and structure of their data to make it more accessible and usable.
When processing open data, special considerations have to be considered. For instance, the population from which such data was sampled, assumptions of collection, pre-processing, missing records. Such considerations will be essential to avoid undesirable biases and make reliable generalisations.
Some organisations refuse to share data even if they do not collect it themselves. Even though sharing such data does not harm these organisations, they might be reluctant to share for various reasons. We presume they do not want to assume responsibility for such data, given that they did not collect it directly.
So far, no rules, guidelines or standards are clear when using open data. We aspire for a more regulated framework for using open data. Being open-source does not mean it can be abused. Accountability should exist, fairness should be the primary concern, and transparency should be maintained throughout the chain of data usage. Additionally, limitations from using open data are supposed to be stated by its owner to avoid misinterpretations by the users and avoid inferring misleading results when transforming such data to information.
