Big Data Technologies are fundamental for process improvement and optimization in the most diverse areas. In the context of bioelectrochemical systems - BES, which are devices that transform waste into electricity, there are several opportunities. Since the complexity of the physical and chemical phenomena associated with these systems generates large vulnerabilities which need to be mitigated. At this point, it is possible to apply Big Data tools to optimize, monitor and prevent possible failures during the operation of bioelectrochemical cells. In this context, it was developed a Big Data platform for real-time BES monitoring. The ability of integration with sensors, scalability for multiple cells monitoring, resilience, fault tolerance, high availability and open source technologies were considered during the development of this pipeline. To achieve our goals, the following technologies were integrated: analog sensors, Arduino, Kafka, Zookeeper, Spark, Elasticsearch, Kibana, Secor and Amazon S3. Our pipeline show the viability of integrating Big Data platforms for BES real-time monitoring in a scalable way, benefiting the maintenance and future prediction of failures during these systems operation. Therefore, the integration of the several knowledge areas, from software development to biotechnological processes, is fundamental for the advancement and future commercialization of BES based-technologies.
It is worth to mention this work is part of my Ph.D Thesis and, also, it was possible with the valuable contibuitions of M.Sc. Ing. José Pedro de Santana Neto (https://github.com/josepedro) and under supervision of Professor Ph.D Hugo Moreira Soares (http://lattes.cnpq.br/2713109621898643).
For more information: https://doi.org/10.2166/wst.2018.410