AI generated Synthetic Data is new. You have questions? We have the answers.

Synthetic data is data that's artificially generated rather than collected from real-world events. It is typically created with the help of algorithms or simulations and often used in settings where real-world data is hard to collect or where privacy concerns exist.

The term is not new and has been around for many years. In the past synthetic data was most often understood as “rule based” synthetic data. That is a user would define specifically the rules upon which the data would be generated. For example: create a numerical variable without any decimals with a range from 100 to 1,000 with a normal distribution.

When we talk about synthetic data, we mean machine learning generated synthetic data. For this kind of synthetic data Generative AI is used to create data that can be highly complex - far beyond what a user could describe with simple rules. The result is data that looks and feels just like real-world data and that contains all its statistical information but no Personal Identifiable Information (PII).

The generation of synthetic data can vary greatly depending on the specific requirements of a project. These are the most common ones.

Statistical or Rule based Methods: These methods generate data based on statistical properties or defined rules. For example, you might generate synthetic data that follows a particular distribution, such as the normal distribution, or you can generate synthetic data that has the same mean and variance as a real-world dataset.

Generative Models or Machine Learning: Generative models, such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and others, are a powerful way to generate synthetic data. These models are trained on real-world data and learn to generate new data that is similar to the training data. GANs, for example, involve two neural networks: a generator network that produces synthetic data and a discriminator network that tries to distinguish between the real and synthetic data. The two networks are trained together, with the generator network improving its ability to create realistic data as the discriminator network gets better at spotting fakes.

Agent-based Modeling: This technique creates individual 'agents', each with their own behaviors and rules, and allows them to interact with each other in a simulated environment. This can be used to generate synthetic data about complex systems, like traffic patterns or financial markets.

Simulation: Simulation involves creating a model of a system and then using that model to generate data. For example, a weather simulation might generate synthetic data about temperature, wind speed, and precipitation.

When we speak about synthetic data, we mean synthetic data that is created using Generative Models / Machine Learning.

The three main uses for synthetic data include

Artificial Intelligence (AI) and Machine Learning (ML): Synthetic data is extensively used in training machine learning models, especially in cases where real-world data is scarce or sensitive. This can help overcome challenges like imbalanced classes in the dataset, lack of labeled data, or data privacy concerns. Synthetic data can be particularly useful in reinforcement learning where the agent requires a lot of interaction with its environment. Simulated environments can provide such data without the need for costly and time-consuming real-world interactions.

Data Sharing: Synthetic data can be used to create datasets that maintain the statistical properties of the original data while ensuring privacy and confidentiality. This makes it possible to share datasets for collaborative research, open data initiatives, or data analysis competitions without exposing sensitive information. For example, in healthcare, synthetic patient data can be shared for research purposes without violating patient privacy.

Software Development and Testing: Synthetic data can be used to test the functionality, performance, and scalability of software applications and systems. For instance, synthetic data can be used to simulate high load scenarios and stress test the system. This is particularly useful in the development of databases, data-intensive applications, and systems dealing with user-generated content. The use of synthetic data ensures that testing can be performed without risking exposure of real user data.

The primary benefits of using synthetic data include:

Preserving Privacy: Since synthetic data doesn't contain any real personal information, it can help avoiding privacy and confidentiality issues associated with real-world data.

Improving Data Availability: Synthetic data can be created even when real-world data is hard to collect, too expensive to acquire, or not available in sufficient quantity.

Control over Data Characteristics: With synthetic data, one can control the characteristics of the data. You can create a synthetic dataset with specific features, correlations, and anomalies to test different scenarios and edge cases.

Reduced Bias: If carefully generated, synthetic data can be used to reduce bias in model training by ensuring a balanced representation of different classes.

Finance and Banking: Synthetic data can be used to work with financial transactions, enabling the testing and validation of fraud detection algorithms and risk models without exposing sensitive customer information.

Insurance: In the insurance industry, synthetic data can help in modeling risk and predicting claims by generating data that mirrors the characteristics of real policyholder data without exposing any sensitive information. It can also be used to train machine learning models for fraud detection and to test new insurance products or pricing strategies.

Healthcare: In this field, patient data privacy is paramount. Synthetic data allows researchers to create datasets that mimic the characteristics of real patient data without compromising individual privacy. This can be used for disease modeling, research, and the training of AI models for diagnosis or treatment prediction. It also allows for more extensive sharing of data between institutions for research purposes.

Retail and E-commerce: In retail, synthetic data can be used to simulate customer behavior or market trends, which can help in demand forecasting, supply chain optimization, and testing of new business strategies without risking real-world operations.

Public Sector: In the public sector, synthetic data can be used to enable data sharing between departments or with the public while maintaining privacy. It can also be used for policy modeling, planning, and decision making. For instance, synthetic data can help simulate the impact of a policy change or the introduction of a new public service.

In each of these industries, the primary benefit of synthetic data is that it can bypass the challenges of privacy, cost, and data scarcity associated with real-world data, while also allowing for the creation of data with specific characteristics for targeted testing or training scenarios.

Global payments network SWIFT provided synthetic datasets representing data held by the SWIFT payments network and datasets held by partner banks for the U.S. PETs Prize at the end of 2022.

US Health Insurer Humana has published synthetic member records on their Humana Data Exchange (HDX) as a way to accelerate innovation in healthcare. It is targeted towards software developers, data scientists, and product team stakeholders in the healthcare industry and allows them to train machine learning models faster.

J.P. Morgan uses synthetic data to accelerate the development of AI solutions and to enable collaboration with the academic community. Synthetic data generation allows them to think, for example, about the full lifecycle of a customer’s journey that opens an account and asks for a loan. They’re not simply examining the data to see what people do, but they’re also able to analyze their interaction with the firm and essentially simulate the entire process.

After generating the synthetic data, it's important to validate it against the real-world data. This can involve comparing the statistical properties of the synthetic data with those of the real data. Most frameworks and vendors will provide some sort of Quality Assurance report that gives a first overview of the quality of the created synthetic data.

The real benchmark though, is testing the performance of models trained on the synthetic data using a validation set of real data. This concept known as Train Synthetic Test Real (TSTR) gives the real answer, if a synthetic dataset can really be used as a substitute for real data. If you’re further interested in exploring this concept we have prepared a Jupyter Notebook on Google Colab that gives a practical overview how to do this.

One of the main challenges with synthetic data is ensuring that it accurately represents the real-world data it's intended to mimic. If the synthetic data doesn't capture the key characteristics, correlations, and variability present in the real data, models trained on this data might not perform well in real-world applications.

There's a risk that models trained primarily on synthetic data might overfit to the characteristics of the synthetic data and underperform on real-world data. This is particularly a concern if the synthetic data oversimplifies the problem or misses certain complexities present in the real-world data.

The good news is that there is an easy to use platform for all your synthetic data generation needs, that has proven to generate synthetic data of highest quality and accuracy: the MOSTLY AI Synthetic Data Platform. But don’t take my word for it - head to our FREE version, sign up and see for yourself!

With increasing data privacy regulations, synthetic data offers a compelling solution to many data-sharing challenges. When synthetic data is generated correctly, it mimics the statistical characteristics of the original data without containing any actual personal information. This allows the synthetic data to be used for a wide range of purposes, including research, development, and analysis, without violating privacy laws or regulations.

For organizations that need to share data with third parties, synthetic data can be a game-changer. Instead of sharing actual customer or user data, which could risk violating privacy regulations, organizations can share synthetic data that has the same statistical properties. This allows third parties to conduct meaningful analysis without having access to sensitive information.

Additionally, synthetic data can facilitate collaboration between different organizations. For example, in healthcare, hospitals or research institutions may want to collaborate and share patient data for research purposes. However, due to privacy regulations, sharing real patient data can be challenging. In this scenario, synthetic data that maintains the statistical properties of the real patient data can be shared instead, enabling collaborative research while ensuring patient privacy.

Furthermore, synthetic data can also make open data initiatives more feasible. Governments and organizations can generate and release synthetic datasets for public use without exposing any sensitive information. This can spur innovation and allow more researchers and developers to leverage the data.

Synthetic data is not automatically private. Several measures need to be taken during the synthetic data generation process to make sure that this is a privacy preserving process.

Techniques like Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) can generate synthetic data that statistically resembles the original data but does not directly replicate unique data points. In designing these models, special care must be taken to avoid reproducing identifiable patterns from the original data (= overfitting of the models) that could lead to re-identification of individuals.

In addition one aspect that needs to be taken care of is how outliers are handled. Sophisticated synthetic data generation tools like the MOSTLY AI Synthetic Data Platform have in-built mechanisms that handle various outliers such as extreme values for numerical data or rare categories for categorical data.

Validation of synthetic data is another key step. This involves testing to ensure that the synthetic data does not contain sensitive information. Re-identification tests can be performed, where attempts are made to match records in the synthetic data back to known records in the original data. If a significant number of records can be matched, this may indicate that the synthetic data is revealing sensitive information.

Finally, strong governance and oversight are essential. This includes clear policies on how synthetic data is generated, used, and shared, and who has access to the original and synthetic data.

The impact of synthetic data on the performance of AI models in real-world settings depends largely on the quality of the synthetic data.

The quality and relevance of synthetic data play a huge role. If synthetic data accurately captures the statistical properties and variability of the real-world data, it can be a valuable asset for training AI models and in practice lead to models of almost similar quality as if trained on real world data.

In certain cases synthetic data can even enhance model performance by providing more diverse and balanced training data, especially for classes or scenarios that are underrepresented in the real data.

However, if the synthetic data does not adequately represent the real-world conditions, it may lead to models that perform poorly in real-world settings. This can happen if the synthetic data oversimplifies the problem or misses important features or correlations present in the real data. Models trained on such synthetic data may overfit to the characteristics of the synthetic data and underperform on real data.

Synthetic data can have an even more profound impact on real-world settings: in cases where there simply is no real-world data to train AI models. Oftentimes real-world data is difficult or costly to obtain, or privacy concerns limit the use of real data. In these cases synthetic data - even if not perfect - is much better than having no data at all.

The major downside of using synthetic data to train AI models is the fact that synthetic users, scenarios, or behaviors do not correspond to real individuals or events. Although synthetic data can be engineered to statistically mirror aspects of real-world data almost perfectly, it's important to remember that each synthetic instance is essentially a fabrication, not tied to a specific real-world counterpart.

For example, if an AI model trained on synthetic data identifies a pattern or behavior that seems actionable, it's not possible to directly engage with the synthetic users exhibiting that behavior, because they don't exist. Instead, one would need to find corresponding patterns or behaviors in the real-world data. This means an extra step is required: transferring the insights generated on synthetic data back to the real world data.

Furthermore, if the synthetic data does not adequately capture the complexity and diversity of the real world, there can be a discrepancy between how the model performs in the synthetic environment versus how it performs when deployed in the real world. This could potentially lead to models that perform well on synthetic data but fail to generalize effectively to real-world data.

That’s why it is important to make sure that the synthetic data used for training AI models is really as accurate and statistically representative as possible.

Synthetic data has the potential to greatly enhance innovation and cooperation across organizations. Traditionally, sharing data between organizations, especially those in sensitive fields like healthcare or finance, has been fraught with challenges due to privacy concerns and regulatory constraints. Synthetic data provides a way around these issues, enabling more free and open data sharing.

By enabling data sharing without privacy risks, synthetic data can allow organizations to collaborate on common problems or research projects. For example, multiple hospitals could share synthetic patient data to jointly develop AI models for predicting disease outcomes or optimizing treatment strategies. This type of cooperation could lead to faster advancements and more robust solutions than if each organization worked separately with its own limited dataset.

In addition, synthetic data can be shared more freely for open data initiatives or public challenges. For example, a government agency could release synthetic datasets that mimic real-world conditions, enabling researchers and developers to build and test solutions for public issues. This can spur innovation by providing a broader community with access to relevant data.

Synthetic data can foster cooperation between organizations and the AI research community. Organizations can release synthetic datasets derived from their proprietary data, allowing researchers to develop new algorithms and techniques that can benefit the organization. In return, the researchers get access to realistic datasets that can drive their research.

Bias in AI and machine learning often stems from the data used to train the models. If the training data is not representative of the problem space or population, the model can develop biased predictions. Synthetic data can help mitigate such biases in several ways.

Firstly, synthetic data allows for controlled data generation. This means one can generate a dataset that accurately represents different classes, scenarios, or populations that might be underrepresented in the real data. For instance, if a certain demographic is underrepresented in the real data, more synthetic data representing that demographic can be generated to ensure a balanced dataset.

Secondly, synthetic data can be used to generate data for scenarios that are rare or hard to capture in the real world but are important for training the model. For example, in autonomous vehicle development, synthetic data can simulate rare but critical situations, such as certain types of accidents or extreme weather conditions. This ensures the model is trained on these scenarios and can handle them appropriately.

Moreover, synthetic data can be used to understand the impact of bias in the models. By generating synthetic data with known biases and feeding this data to the model, one can observe how these biases affect the model's performance. This can provide valuable insights into how the model might behave when exposed to biased real-world data and guide the development of strategies to mitigate these biases.

We have written an entire blog series on Fairness and Bias that can be found here.

When you are using a powerful synthetic data generator like the MOSTLY AI Synthetic Data Platform, the goal of the platform is to create synthetic data that is as close as possible to the real data without compromising the privacy of the original dataset.

This means that any bias that existed in the original dataset will also be reflected in the synthetic dataset - and this is a feature, not a bug. All the measures that are in place to make sure that the synthetic data is highly representative of the original data will naturally also make sure that existing biases are retained. However, these measures will also make sure that no new biases will be added during the synthesization process.

In practice you can and should verify this by comparing the original dataset and the synthetic dataset with regards to the biases found and for example by comparing the performance of ML models that were trained on each of these datasets.

Promoting transparency and accountability when using synthetic data in AI systems involves several crucial steps. The process starts with clear documentation of how the synthetic data was generated or what tools were used. The documentation should also describe any known limitations or biases of the original and synthetic data.

When synthetic data is used to train AI models, it's important to clearly state this in any reports or publications about the AI system. This allows users, reviewers, and auditors to properly interpret the results of the AI system.

Furthermore, when synthetic data is used in place of real data due to privacy concerns, it's crucial to validate that the synthetic data does not contain any sensitive information. This involves implementing robust privacy-preserving mechanisms in the data generation process or working with a trusted vendor to create the synthetic data.

One of the key aspects of responsible AI is ensuring data privacy and maintaining user trust. Synthetic data, which doesn't include real personal information, can uphold this principle. It allows organizations to bypass many of the privacy concerns associated with real-world data, enabling them to test and train AI models without the risk of exposing sensitive information.

Transparency in AI involves making clear to stakeholders how an AI system works and makes decisions. When synthetic data is used, it's crucial to clearly document the process by which the use of synthetic data was implemented.

Moreover, synthetic data can help promote robustness and fairness in AI systems. Since synthetic data can be generated in large quantities and designed to represent a wide range of scenarios, it can be used to test AI systems under various conditions and edge cases. This helps ensure that the AI system performs well across diverse situations, contributing to a more robust and fair system.

Finally, using synthetic data can help with replicability in AI research. Since synthetic data can be freely shared without privacy concerns, it allows others (e.g. a validation group within an organization) to reproduce experiments and verify results, which is a key aspect of trust and transparency in a system.

The future of synthetic data is likely to be characterized by technological advancements, regulatory developments, broad industry adoption, and a tighter integration with AI development.

Improved Generation Techniques: As research progresses, we can expect to see more advanced techniques for generating synthetic data. This might include more sophisticated generative models that can create increasingly realistic and diverse synthetic data, or new techniques for ensuring privacy in synthetic data.

Regulation and Standards: As the use of synthetic data becomes more widespread, we might see the introduction of regulations and standards related to synthetic data. This could include standards for how synthetic data should be generated and used, or regulations to ensure that the use of synthetic data respects privacy and ethical considerations.

Broad Adoption Across Industries: As organizations become more aware of the benefits of synthetic data, we're likely to see increased adoption across a variety of industries and different sizes of organizations.

Integration with AI Development: Synthetic data is particularly useful for training and testing AI models, and we can expect to see tighter integration between synthetic data and AI development.

At MOSTLY AI we are constantly pushing the boundaries of what’s possible to make sure that synthetic data becomes the standard of how organizations work with and share data. Synthetic data - better than real data!