Temporal Input Configuration

Oct 13, 2025 by talynt.org 128 views

Introduction

Temporal workflows are designed to handle complex data processing tasks efficiently. However, in some cases, these workflows may exhibit unexpected behavior, leading to increased load on the database. In this article, we will explore the issue of XMIN flow ignoring waitBetweenBatchesSeconds and its impact on database load.

Understanding XMIN Flow

XMIN flow is a Temporal workflow designed to synchronize data between two databases. The workflow uses the XMIN (xmin) column to track the minimum transaction ID (xmin) in the source database. This information is used to determine the watermark for the destination database. The XMIN flow is typically used for incremental data replication, where only the changes made to the source database are replicated to the destination database.

Temporal Input Configuration

The Temporal input configuration for the XMIN flow is as follows:

{
  "flowJobName": "svc_payment_payment_xmin",
  "destinationTableIdentifier": "payment_xmin",
  "query": "SELECT * FROM \"public\".\"payment\"",
  "watermarkTable": "public.payment",
  "watermarkColumn": "xmin",
  "maxParallelWorkers": 4,
  "waitBetweenBatchesSeconds": 900,
  "writeMode": {
    "writeType": "QREP_WRITE_MODE_UPSERT",
    "upsertKeyColumns": [
      "id_payment"
    ]
  },
  "numRowsPerPartition": 100000,
  "setupWatermarkTableOnDestination": true,
  "syncedAtColName": "_PEERDB_SYNCED_AT",
  "softDeleteColName": "_PEERDB_IS_DELETED",
  "sourceName": "rds_svc_payment",
  "destinationName": "dl_svc_payment",
  "parentMirrorName": "svc_payment_payment_xmin"
}

Issue with XMIN Flow

The XMIN flow is designed to run every 900 seconds, as specified in the waitBetweenBatchesSeconds configuration. However, in some cases, the workflow may fire continuously, leading to a high load on the database. This issue has been observed on both v0.25.6 and v0.26.0 versions of the Temporal workflow.

Analysis of XMIN Flow Code

The XMIN flow code is located in the xmin_flow.go file. Upon analyzing the code, it appears that the waitBetweenBatchesSeconds configuration is not being taken into account. This is because the xmin_flow function does not check the waitBetweenBatchesSeconds value before executing the workflow.

// xmin_flow is a Temporal workflow that synchronizes data between two databases.
func xmin_flow(ctx context.Context, flow *flow.Flow, input *flow.Input) (*flow.Output, error) {
    // ...
    // The workflow is executed without checking the waitBetweenBatchesSeconds value.
    // ...
}

Impact on Database Load

The continuous firing of the XMIN flow leads to a high load on the database. This is because the workflow is executing multiple times, resulting in increased database queries and transactions.

Conclusion

In conclusion, the XMIN flow ignoring waitBetweenBatchesSeconds and its impact on database load is a critical issue that needs to be addressed. The Temporal workflow code needs to be modified to take into account the waitBetweenBatchesSeconds configuration. Additionally, the database load needs to be optimized to prevent high load issues.

Recommendations

Modify XMIN Flow Code: Modify the XMIN flow code to check the waitBetweenBatchesSeconds value before executing the workflow.
Optimize Database Load: Optimize the database load by reducing the number of database queries and transactions.
Monitor Workflow Performance: Monitor the workflow performance to detect any issues and take corrective action.

Future Work

In the future, we plan to:

Implement a Retry Mechanism: Implement a retry mechanism to handle temporary failures and prevent continuous workflow execution.
Improve Workflow Performance: Improve the workflow performance by optimizing database queries and transactions.
Enhance Monitoring and Logging: Enhance monitoring and logging to detect issues and take corrective action.
XMIN Flow Ignoring WaitBetweenBatchesSeconds, High Load on DB: Q&A ====================================================================

Q: What is the XMIN flow and how does it work?

A: The XMIN flow is a Temporal workflow designed to synchronize data between two databases. It uses the XMIN (xmin) column to track the minimum transaction ID (xmin) in the source database. This information is used to determine the watermark for the destination database.

Q: What is the purpose of the `waitBetweenBatchesSeconds` configuration in the XMIN flow?

A: The waitBetweenBatchesSeconds configuration specifies the time interval between batches of data being processed by the XMIN flow. This configuration is used to control the frequency at which the workflow executes.

Q: Why is the XMIN flow ignoring the `waitBetweenBatchesSeconds` configuration?

A: The XMIN flow code does not check the waitBetweenBatchesSeconds value before executing the workflow. This means that the workflow is executing continuously, leading to a high load on the database.

Q: What are the consequences of the XMIN flow ignoring the `waitBetweenBatchesSeconds` configuration?

A: The continuous firing of the XMIN flow leads to a high load on the database. This is because the workflow is executing multiple times, resulting in increased database queries and transactions.

Q: How can I modify the XMIN flow code to take into account the `waitBetweenBatchesSeconds` configuration?

A: You can modify the XMIN flow code to check the waitBetweenBatchesSeconds value before executing the workflow. This can be done by adding a conditional statement to check if the specified time interval has passed before executing the workflow.

Q: What are some best practices for optimizing the database load caused by the XMIN flow?

A: Some best practices for optimizing the database load caused by the XMIN flow include:

Reducing the number of database queries and transactions
Optimizing database indexing and caching
Implementing a retry mechanism to handle temporary failures
Monitoring workflow performance to detect issues and take corrective action

Q: How can I monitor the workflow performance to detect issues and take corrective action?

A: You can monitor the workflow performance by:

Using monitoring tools such as Prometheus and Grafana to track workflow metrics
Implementing logging and error handling mechanisms to detect issues
Using alerting mechanisms to notify administrators of issues

Q: What are some future plans for improving the XMIN flow and reducing the database load?

A: Some future plans for improving the XMIN flow and reducing the database load include:

Implementing a retry mechanism to handle temporary failures
Improving workflow performance by optimizing database queries and transactions
Enhancing monitoring and logging to detect issues and take corrective action

Q: How can I get help if I am experiencing issues with the XMIN flow or database load?

A: You can get help by:

Reaching out to the Temporal community for support
Contacting the Temporal development team for assistance
Using online resources such as documentation and tutorials to troubleshoot issues

Conclusion

In conclusion, the XMIN flow ignoring waitBetweenBatchesSeconds and its impact on database load is a critical issue that needs to be addressed. By modifying the XMIN flow code, optimizing database load, and monitoring workflow performance, you can reduce the high load on the database and improve the overall performance of the XMIN flow.