Compute distance using categorical methodology

Oct 13, 2025 by talynt.org 18 views

Problem Description

The DCRBaselineProtection and DCROverfittingProtection metrics are crucial in measuring the privacy of synthetic data by determining how far away it is from the real data. These metrics rely on a computation that measures the "distance" between two rows of data, as defined in issue #720. However, this computation can lead to issues when dealing with numerical data that has a range of 0.

The Problem: Numerical Data with a Range of 0

Numerical data can sometimes have a range of 0, indicating that all values in the column were exactly the same in the real data. This can cause problems for the DCR metrics, as they currently crash when encountering such a column. The issue arises from the fact that col_range=0 leads to a ZeroDivisionError.

Expected Behavior

To resolve this issue, we propose that if the real column range is 0 for a numerical data column, the distance should be computed using the same methodology as for categorical columns. This means that if s_j == d_j, then distance(s_j, d_j) = 0. Otherwise, the distance is 1. This approach ensures that the DCR metrics can handle numerical data with a range of 0 without crashing.

Why Constant Numerical Columns Matter

Constant numerical columns are not uncommon in real-world data. They can occur due to various reasons, such as:

Data normalization: When data is normalized to a specific range, all values in a column may become the same.
Data aggregation: When data is aggregated, all values in a column may become the same.
Data quality issues: In some cases, data quality issues can lead to all values in a column being the same.

By allowing the possibility of constant numerical columns, the DCR metrics can provide more accurate and reliable results. This is particularly important in applications where data privacy is a critical concern.

Proposed Solution

To address this issue, we propose the following changes to the DCR metrics:

Add a check for col_range == 0: Before computing the distance, check if the column range is 0. If it is, proceed to the next step.
Compute distance using categorical methodology: If col_range == 0, compute the distance using the same methodology as for categorical columns. This means that if s_j == d_j, then distance(s_j, d_j) = 0. Otherwise, the distance is 1.

Benefits of the Proposed Solution

The proposed solution offers several benefits:

Improved accuracy: By handling constant numerical columns correctly, the DCR metrics can provide more accurate results.
Increased reliability: The proposed solution ensures that the DCR metrics do not crash when encountering numerical data with a range of 0.
Better data handling: The proposed solution demonstrates a better understanding of data handling and the importance of considering edge cases.

Conclusion

In conclusion, the DCR metrics should allow the possibility of constant numerical columns. By doing so, we can ensure that the metrics provide accurate and reliable results, even when dealing with numerical data that has a range of 0. The proposed solution is a simple yet effective way to address this issue and improve the overall quality of the DCR metrics.

Future Work

Future work can focus on:

Extending the proposed solution to other metrics: The proposed solution can be extended to other metrics that rely on the same computation.
Improving data handling: Further improvements can be made to data handling to ensure that the DCR metrics can handle a wide range of data scenarios.

References

Issue #720: link to issue
DCRBaselineProtection and DCROverfittingProtection metrics: link to documentation
DCR Metrics: Frequently Asked Questions =============================================

Q: What is the DCRBaselineProtection metric?

A: The DCRBaselineProtection metric is a measure of the privacy of synthetic data by determining how far away it is from the real data. It relies on a computation that measures the "distance" between two rows of data.

Q: What is the DCROverfittingProtection metric?

A: The DCROverfittingProtection metric is another measure of the privacy of synthetic data by determining how far away it is from the real data. It also relies on the same computation as the DCRBaselineProtection metric.

Q: Why do the DCR metrics crash when encountering numerical data with a range of 0?

A: The DCR metrics crash when encountering numerical data with a range of 0 because col_range=0 leads to a ZeroDivisionError. This is due to the fact that the computation relies on dividing by the column range.

Q: What is the proposed solution to address this issue?

A: The proposed solution is to add a check for col_range == 0 before computing the distance. If the column range is 0, the distance should be computed using the same methodology as for categorical columns.

Q: Why is it important to handle constant numerical columns correctly?

A: It is essential to handle constant numerical columns correctly because they can occur due to various reasons, such as data normalization, data aggregation, or data quality issues. By handling them correctly, the DCR metrics can provide more accurate and reliable results.

Q: What are the benefits of the proposed solution?

A: The proposed solution offers several benefits, including improved accuracy, increased reliability, and better data handling.

Q: Can the proposed solution be extended to other metrics?

A: Yes, the proposed solution can be extended to other metrics that rely on the same computation.

Q: What is the next step in addressing this issue?

A: The next step is to implement the proposed solution and test it thoroughly to ensure that it works correctly and provides accurate results.

Q: Who should be involved in addressing this issue?

A: The following individuals should be involved in addressing this issue:

Data scientists and engineers who work with the DCR metrics
Developers who maintain the DCR metrics code
Quality assurance testers who ensure that the metrics work correctly

Q: What are the potential risks of not addressing this issue?

A: The potential risks of not addressing this issue include:

Inaccurate results due to incorrect handling of constant numerical columns
Increased risk of errors and crashes
Decreased reliability and trust in the DCR metrics

Q: What are the potential benefits of addressing this issue?

A: The potential benefits of addressing this issue include:

Improved accuracy and reliability of the DCR metrics
Increased trust and confidence in the metrics
Better data handling and reduced risk of errors and crashes

Q: What is the timeline for addressing this issue?

A: The timeline for addressing this issue will depend on the availability of resources and the priority of the issue. However, it is essential to address this issue as soon as possible to ensure that the DCR metrics continue to provide accurate and reliable results.

Q: How can I get involved in addressing this issue?

A: If you are interested in getting involved in addressing this issue, please contact the relevant individuals or teams who are working on the DCR metrics. They will be able to provide more information on how to get involved and contribute to the solution.