Data subsetting has emerged from a need within test environments to save space, on the one hand, resulting in cost reductions, and on the other, to comply with regulations. Regulations that state not only that they must protect the data but also that they must take the minimum amount of data possible.
Continue reading to find out more about data subsetting and the valuable new features this process includes for data processing in development environments.
The concept of data subsetting defines the process by which a reduced set of data from a production database is separated to transfer this to a non-production environment.
Through the discarding or extraction of data, the goal of data subsetting is to reduce the size of the data to make it more manageable, whilst maintaining a set of relevant data for use.
To carry out data subsetting processes, data filters are applied, to make it possible to access the right data to generate a relevant, consistent set that can be employed in software testing processes.
In a way, data subsetting employs the test data from the same perspective as statistical science: employing representative samples instead of an entire population.
Some of the benefits of data subsetting include:
As pointed out previously, the data subsetting process consists of the separation of a reduced set of data from a production database to transfer this to a non-production environment.
In this way, the data of value remains directly available through a process that can be planned and periodic. In other words, subsetting guarantees that the data is delivered to the selected environments as often as necessary.
Data subsetting also allows multiple testers and developers to work in parallel, as data delivery can take place without interrupting the operations in the destination environment. This contributes to achieving greater efficiency, avoiding processes that slow down deliveries such as data editing, movement or erasure.
As the data subsetting process delivers the data the user needs on demand, they do not depend on a third party to carry out this subsetting. You can simply take the data you need when you need it.
Using the right data subsetting tools, companies can save themselves a large number of manual tasks and reduce the resources needed for software testing.
An ideal process to deliver the useful data to the developer or tester would be as follows:
With this process in mind, we can outline the following steps:
Firstly, and bearing in mind that it is sometimes difficult to know what you need to move because the data you are looking for needs to meet complex criteria, it is important to find an efficient way to do this. At icaria Technology, this prior data identification process is resolved through a search engine.
Once this valuable data has been located, the subsetting plan begins, determining the structures to be transferred, the conditions of the instances to be selected, the complete transfer entities and the valid trajectories (from source to destination).
At this point, our icaria TDM software has already come into play through its search engine and the creation of the subsetting plan. The platform, designed to simplify data management in testing environments, is capable of segmenting the datasets and dissociating complex structures, applying the most advanced data subsetting practices to guarantee quality datasets.
This digital tool itself takes care of the identification of the different types of information that must be located and, later on, the creation of the Data Map for sensitive data that will guide the process to meet the legal requirements.
Additionally, some of the icaria TDM tool's additional features include the following:
Configuration specifically for data domains, allowing the incremental management of complexity and reusing of the knowledge. In fact, MDA is always present in our working philosophy. If you are interested in this subject, you can find a more in-depth explanation here.
A key step in any data subsetting process includes not only the extraction of complete structures but also their dissociation.
Dissociation consists of replacing sensitive data with other fictitious but true data, thus maintaining the referential integrity including between databases in different technologies, so that the dataset is in accordance with the law with respect to data protection but can also be used in testing processes.
In this way, companies ensure that they comply with all the requirements of the GDPR and data protection in testing environments.
What is the result of the data subsetting process using icaria TDM? Reduction of costs, compliance with the law and improved quality of testing management software.
Want to know more about how data subsetting is allowing a reduction in the waiting times and operating costs in testing processes? Contact the icaria team or request an icaria TDM demo and see for yourself.