Atypical Data Classifier
Inter-American Development BankGet the code
The Atypical Data Classifier is an application that automatically detects anomalous data in household surveys, which can be used to determine eligibility for social programs. It allows to visualize in detail the values of the variables (and their score) of the records of households classified as irregular. Analyze the potential strange cases that occur during the information gathering. This helps in making decisions for the correction of the data capture tool or the generation of new instructions for field personnel.
This software has been developed by Carlos Tejada under the project CO-T1418-P001, funds ATN / OC-16168-CO, Contract No. 001, through the IDB for the DNP Colombia. It is currently being used by the National Planning Department of the Government of Colombia. You can read more about this project in the blog: https://blogs.iadb.org/conocimiento-abierto/en/atypical-data-classifier-review-data-quality-for-social-programs/
Atypical Data Classifier is a tool duo composed of:
1. The Classifier and the Visualizer of Atypical Data: The visualizer serves to visualize the indicators of classification of the files of households raised by the SISBEN, the Identification System of Potential Beneficiaries of Social Programs in Colombia. -It is a Web app developed in .NET Core, a general-purpose development platform maintained by Microsoft and the .NET community on GitHub. It is cross-platform, supports Windows, macOS and Linux, and can be used in devices, in the cloud and in integrated scenarios / IoT. To learn more about how the viewer works, please go to the classifier's repository: https://github.com/EL-BID/Clasificador-de-datos-atipicos
2. SQL Agent of the Atypical Data Classifier: The SQL agent of the Atypical Data Classifier are the algorithms with which this classification is executed and that can later be visualized using the visualizer. This agent is a SQL Server stored procedure (which is invoked through a job), which is responsible for processing the data of households not yet processed, and classify each household as regular or irregular. To learn more about how the agent works, please go to the agent's repository: https://github.com/EL-BID/Clasificador-SISBEN-ML