More than 100 experts from financial firms met in Canary Wharf, London, on October 18th for the QA Financial Forum London. Among the expert speakers were Mohamed Mehdi Ben Aissa (pictured), and Sinh Chung Nguyen, data architects at the Direction Générale des Finances Publiques (DGFiP) – the treasury department of the French state – who explained their role in the creation of a new open source data management platform.
Over the past 18 months, DGFiP and Électricité de France, the French energy provider, have developed the Trunk Data Platform (TDP) – a free and open source Apache-based distribution platform, with its source code readily available on GitHub.
The decision to create this new platform followed the acquisition of the open-source Hortonworks data platform by Cloudera in early 2019. This – according to the DGFiP – fundamentally changed the dynamics of the data management marketplace, and it decided to move to its own new platform,
Previously using HDP 3.1.4, DGFiP found the need to move away from the platform when it was acquired by Cloudera in early 2019, as it became deprecated for Cloudera’s own offerings. Wanting to avoid vendor lock-in, and with a strong open-source philosophy at the treasury, DGFiP and other French public and private bodies, including EDF, discussed alternative approaches to open source data management.
At the QA Financial Forum Ben Aissa described the key uses for data management: for fraud detection, electronic invoicing, and also the Foncier Innovant, a scheme which includes a series of innovative technologies, which has recently been used, for example, to find untaxed swimming pools with an AI cross-referencing satellite data.
By adopting what the DGFiP architects describe as a ‘data stewardship model’, the platform is intended to make sharing data easier, while improving the quality of the data and cutting down on duplicate work. Now they are seeking to develop the user interface of the platform, introduce Kubernetes integration, alongside monitoring and log management, and deepen its machine learning integrations.
