DevOps’ Missing Link: Data
Industry and government alike have started to implement automated delivery pipelines only to have continued challenges with quality, validation, verification, and performance. A root cause is lack of requisite data provided to design, engineering, and delivery teams. Organizations need a deliberate strategy, process, and tool suite for the creation and management of SDLC support data.
DevOps and modern software delivery have a direct dependency on vigorous data profiling and subsequent synthetic data generation, self-service dataset reservation/refresh, as well as exposure to data integrations, often via API. The move to data-centric and API led architectures mandate clear understanding and access to datasets. Further, use of techniques like Service Virtualization to mock target data signatures and APIs needs to have appropriate process and "just enough" governance to deconflict data ownership across data stewards.
Historic approaches such as masking production data provides only a roughly equivalent volume of data and fails to provide the boarder cases needed for dependable development and testing.
Industry is just beginning to address through creation of new roles in the enterprise such as a Chief Data Officer, DataOps engineers, and Data Governance Automation engineers to provide holistic understanding of data and provide the necessary governance and rapid exposure of data sets across the full SDLC.
Attendees will engage to discuss current challenges and think through areas industry must address for test data under this very large DevOps/ continuous engineering umbrella.
•Identify types of data needed across the SDLC, likely by solution-type (architectural profile)
•Share lessons from implementation of SLDC data enablement (people, processes, and supporting technology)
•Explore data profiling, synthetic generation, and overall test data management (TDM)
•Explore governance challenges with data sets and the role of data in the DevOps/DevSecOps pipelines.
•Discuss evolving Data-centric roles in government and the impact
•Deliberate role of service virtualization and the nuances to adoption
Participants will walk away understanding the need for data management and generation as well as techniques and tools used by my teams in the federal and state government domain. They will be armed with challenge areas including both technical as well as some in inherent people/cultural implications.
Tracy Bannon
Senior Principal/Software Architect & DevOps Advisor, The MITRE Corporation