Machine and Process Resources

Oct 13, 2025 by talynt.org 88 views

Introduction

In this article, we will explore the issue of epoll_ctl returning EBADF and crashing with SIGABRT in a Python program running in a Docker container. We will discuss the symptoms, the environment in which the issue occurred, and the possible causes of the problem.

Symptoms

The program was running in a Docker container using the nikolaik/python-nodejs:python3.13-nodejs23 image. The container was removed along with its volume as soon as the process exited, making it impossible to provide a core dump. The exact code that caused the issue is not available due to its large size and private nature.

However, the program heavily uses asyncio and has approximately 20,000 tasks running, with around 20 tasks running simultaneously. The code has many dependencies, but only two of them might be relevant: uvloop and curl_cffi, which is used in async mode.

Pre-Crash Behavior

About 1-2 hours before the crash, the program started behaving strangely. Logs were printed in spurts, and network connections were disconnected due to timeouts. Upon inspecting the process using htop, it was discovered that the process had 0.0% CPU usage when no logs were being printed. This behavior was followed by a wave of text being printed, and then the process would return to its normal state.

This behavior is similar to the program sleeping or waiting for some IO synchronously, but nothing in the program does it. This is the second or third time this has happened, and most of the time, the program works normally.

Machine and Process Resources

The machine and process had enough free CPU and RAM available to use, so it is not a resource problem. The SSD and network were also not busy, ruling out resource-related issues.

Strace Output

The issue was caught using strace in htop, but unfortunately, it was not possible to dump the output to a file. However, the last lines of the strace output and the last lines filtered with the fd that caused the issue are available.

Last Lines of Strace Output

Last Lines of Strace Output Filtered with FD 119

CPython Versions Tested

The issue was tested on the following CPython versions:

3.13

Operating Systems Tested

The issue was tested on the following operating systems:

Linux

Output from Running 'python -VV' on the Command Line

Python 3.13.2 (main, Feb 25 2025, 05:25:21) [GCC 12.2.0]

Possible Causes

Based on the symptoms and the environment in which the issue occurred, there are several possible causes of the problem:

uvloop: The uvloop library is a high-performance event loop for Python. It is possible that the issue is related to a bug in uvloop or a misconfiguration of the library.
curl_cffi: The curl_cffi library is a CFFI wrapper for the libcurl library. It is possible that the issue is related to a bug in curl_cffi or a misconfiguration of the library.
asyncio: The asyncio library is a built-in Python library for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, and implementing network clients and servers. It is possible that the issue is related to a bug in asyncio or a misconfiguration of the library.
epoll_ctl: The epoll_ctl system call is used to control the epoll file descriptor. It is possible that the issue is related to a bug in the epoll_ctl system call or a misconfiguration of the epoll file descriptor.

Conclusion

In conclusion, the issue of epoll_ctl returning EBADF and crashing with SIGABRT in a Python program running in a Docker container is a complex problem that requires further investigation. The possible causes of the problem include bugs in the uvloop, curl_cffi, and asyncio libraries, as well as misconfigurations of these libraries. Further testing and debugging are required to determine the root cause of the issue and to resolve it.

Recommendations

Based on the symptoms and the environment in which the issue occurred, the following recommendations are made:

Upgrade to a newer version of Python: The issue may be related to a bug in the Python 3.13.2 version. Upgrading to a newer version of Python may resolve the issue.
Upgrade to a newer version of uvloop: The issue may be related to a bug in the uvloop library. Upgrading to a newer version of uvloop may resolve the issue.
Upgrade to a newer version of curl_cffi: The issue may be related to a bug in the curl_cffi library. Upgrading to a newer version of curl_cffi may resolve the issue.
Reconfigure the epoll file descriptor: The issue may be related to a misconfiguration of the epoll file descriptor. Reconfiguring the epoll file descriptor may resolve the issue.
Use a different event loop: The issue may be related to a bug in the asyncio library. Using a different event loop, such as the select event loop, may resolve the issue.

Future Work

Further investigation and testing are required to determine the root cause of the issue and to resolve it. The following tasks are planned:

Upgrade to a newer version of Python: Upgrade to a newer version of Python to see if the issue is resolved.
Upgrade to a newer version of uvloop: Upgrade to a newer version of uvloop to see if the issue is resolved.
Upgrade to a newer version of curl_cffi: Upgrade to a newer version of curl_cffi to see if the issue is resolved.
Reconfigure the epoll file descriptor: Reconfigure the epoll file descriptor to see if the issue is resolved.
Use a different event loop: Use a different event loop, such as the select event loop, to see if the issue is resolved.
Q&A: epoll_ctl returns EBADF and crashes with SIGABRT =====================================================

Q: What is epoll_ctl and why is it relevant to this issue?

A: epoll_ctl is a system call used to control the epoll file descriptor. In the context of this issue, epoll_ctl is relevant because it is used by the asyncio library to manage the epoll file descriptor. The issue is related to a bug in the epoll_ctl system call or a misconfiguration of the epoll file descriptor.

Q: What is the difference between EBADF and SIGABRT?

A: EBADF is an error code returned by the epoll_ctl system call when the file descriptor is invalid. SIGABRT is a signal sent to the process when it attempts to access an invalid file descriptor.

Q: What are the possible causes of this issue?

A: The possible causes of this issue include:

uvloop: The uvloop library is a high-performance event loop for Python. It is possible that the issue is related to a bug in uvloop or a misconfiguration of the library.
curl_cffi: The curl_cffi library is a CFFI wrapper for the libcurl library. It is possible that the issue is related to a bug in curl_cffi or a misconfiguration of the library.
asyncio: The asyncio library is a built-in Python library for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, and implementing network clients and servers. It is possible that the issue is related to a bug in asyncio or a misconfiguration of the library.
epoll_ctl: The epoll_ctl system call is used to control the epoll file descriptor. It is possible that the issue is related to a bug in the epoll_ctl system call or a misconfiguration of the epoll file descriptor.

Q: How can I troubleshoot this issue?

A: To troubleshoot this issue, you can try the following:

Upgrade to a newer version of Python: The issue may be related to a bug in the Python 3.13.2 version. Upgrading to a newer version of Python may resolve the issue.
Upgrade to a newer version of uvloop: The issue may be related to a bug in the uvloop library. Upgrading to a newer version of uvloop may resolve the issue.
Upgrade to a newer version of curl_cffi: The issue may be related to a bug in the curl_cffi library. Upgrading to a newer version of curl_cffi may resolve the issue.
Reconfigure the epoll file descriptor: The issue may be related to a misconfiguration of the epoll file descriptor. Reconfiguring the epoll file descriptor may resolve the issue.
Use a different event loop: The issue may be related to a bug in the asyncio library. Using a different event loop, such as the select event loop, may resolve the issue.

Q: How can I prevent this issue from occurring in the future?

A: To prevent this issue from occurring in the future, you can try the following:

Regularly update your dependencies: Regularly updating your dependencies, such as uvloop and curl_cffi, may help to prevent issues related to outdated libraries.
Use a different event loop: Using a different event loop, such as the select event loop, may help to prevent issues related to the asyncio library.
Reconfigure the epoll file descriptor: Reconfiguring the epoll file descriptor may help to prevent issues related to the epoll file descriptor.
Monitor your process for errors: Monitoring your process for errors may help to detect issues related to the epoll file descriptor or other system calls.

Q: What are the implications of this issue?

A: The implications of this issue are:

Process crashes: The issue may cause the process to crash, resulting in data loss and other issues.
System instability: The issue may cause system instability, resulting in other issues and errors.
Security vulnerabilities: The issue may create security vulnerabilities, allowing attackers to exploit the system.

Q: How can I get help with this issue?

A: To get help with this issue, you can try the following:

Post on a programming forum: Posting on a programming forum, such as Stack Overflow, may help you to get help from other developers.
Contact the library maintainers: Contacting the library maintainers, such as the uvloop and curl_cffi maintainers, may help you to get help with the issue.
Contact a professional developer: Contacting a professional developer may help you to get help with the issue and to resolve it.