Different request and response listeners

Oct 13, 2025 by talynt.org 144 views

[🐛 Bug]: Some dev tool request captured in 'Network.requestWillBeSent()' in request sent listener but not in 'Network.responseReceived()' response received listener

As web developers, we often rely on various tools and techniques to monitor and analyze the performance of our web applications. One such technique is using the DevTools API in Selenium to capture network requests and responses. However, in this article, we will discuss a peculiar issue where some dev tool requests are captured in the 'Network.requestWillBeSent()' listener but not in the 'Network.responseReceived()' listener.

What happened?

Our team was trying to check all API response times using Selenium. We have different domains like media, impulse, and application domains. When we added the 'Network.requestWillBeSent()' listener, we got all domains including media, impulse, and application domains. However, when we added the 'Network.responseReceived()' listener, we only got the application domain URL. We didn't get the media and impulse URL in the latter listener.

How can we reproduce the issue?

To reproduce this issue, you can try running the following code on any application that includes different domains URL:

devtool.addListener(Network.requestWillBeSent(), req -> {
    System.out.println(req.getRequest().getUrl());
});

devtool.addListener(Network.responseReceived(), response -> {
    Response res = response.getResponse();
    String responseTime = String.valueOf(res.getHeaders().get("x-response-time"));
    System.out.println(req.getRequest().getUrl() + " " + responseTime);
});

Relevant log output

Unfortunately, we don't have any relevant log output to share as the issue is not related to any specific error message.

Operating System

We have tried running this code on both Windows and Linux operating systems.

Selenium version

We are using Java 21 as our Selenium version.

Browser(s) and version(s)

We are experiencing this issue on Chrome browser.

Browser driver(s) and version(s)

We are using Chrome Driver 132.0.

Are you using Selenium Grid?

Yes, we are using Selenium Grid version 4.28.1.

Possible causes

After investigating this issue, we have found a few possible causes:

Different request and response listeners: The 'Network.requestWillBeSent()' listener is triggered before the request is sent, while the 'Network.responseReceived()' listener is triggered after the response is received. This might be the reason why we are not getting the media and impulse URL in the latter listener.
Request and response headers: The 'x-response-time' header might be set only for the application domain URL, which is why we are not getting the media and impulse URL in the 'Network.responseReceived()' listener.
Selenium Grid configuration: The Selenium Grid configuration might be causing this issue. We are using Selenium Grid version 4.28.1, which might have some bugs or limitations that are causing this issue.

Solution

To solve this issue, we can try the following:

Use a single listener: Instead of using two separate listeners, we can use a single listener that captures both the request and response events.
Check the request and response headers: We can check the request and response headers to see if the 'x-response-time' header is set for all domains.
Update Selenium Grid configuration: We can update our Selenium Grid configuration to see if it resolves the issue.

In this article, we discussed a peculiar issue where some dev tool requests are captured in the 'Network.requestWillBeSent()' listener but not in the 'Network.responseReceived()' listener. We have found a few possible causes for this issue, including different request and response listeners, request and response headers, and Selenium Grid configuration. To solve this issue, we can try using a single listener, checking the request and response headers, and updating our Selenium Grid configuration.

In the future, we plan to investigate this issue further and provide a more detailed solution. We also plan to update our Selenium Grid configuration to see if it resolves the issue.

Here is the code that we used to reproduce this issue:

devtool.addListener(Network.requestWillBeSent(), req -> {
    System.out.println(req.getRequest().getUrl());
});

devtool.addListener(Network.responseReceived(), response -> {
    Response res = response.getResponse();
    String responseTime = String.valueOf(res.getHeaders().get("x-response-time"));
    System.out.println(req.getRequest().getUrl() + " " + responseTime);
});

Note: The code above is just a sample code and might not work as expected in your environment. You should modify it according to your needs and requirements.
[🐛 Bug]: Some dev tool request captured in 'Network.requestWillBeSent()' in request sent listener but not in 'Network.responseReceived()' response received listener

Q: What is the issue with the dev tool request captured in 'Network.requestWillBeSent()' in request sent listener but not in 'Network.responseReceived()' response received listener?

A: The issue is that some dev tool requests are captured in the 'Network.requestWillBeSent()' listener but not in the 'Network.responseReceived()' listener. This means that we are not getting the media and impulse URL in the latter listener.

Q: What are the possible causes of this issue?

A: The possible causes of this issue are:

Different request and response listeners: The 'Network.requestWillBeSent()' listener is triggered before the request is sent, while the 'Network.responseReceived()' listener is triggered after the response is received. This might be the reason why we are not getting the media and impulse URL in the latter listener.
Request and response headers: The 'x-response-time' header might be set only for the application domain URL, which is why we are not getting the media and impulse URL in the 'Network.responseReceived()' listener.
Selenium Grid configuration: The Selenium Grid configuration might be causing this issue. We are using Selenium Grid version 4.28.1, which might have some bugs or limitations that are causing this issue.

Q: How can we solve this issue?

A: To solve this issue, we can try the following:

Use a single listener: Instead of using two separate listeners, we can use a single listener that captures both the request and response events.
Check the request and response headers: We can check the request and response headers to see if the 'x-response-time' header is set for all domains.
Update Selenium Grid configuration: We can update our Selenium Grid configuration to see if it resolves the issue.

Q: What are the benefits of using a single listener?

A: The benefits of using a single listener are:

Simplified code: Using a single listener simplifies the code and reduces the complexity of the code.
Improved performance: Using a single listener can improve the performance of the code by reducing the number of listeners and the overhead associated with them.
Easier maintenance: Using a single listener makes it easier to maintain the code and update it as needed.

Q: What are the benefits of checking the request and response headers?

A: The benefits of checking the request and response headers are:

Improved accuracy: Checking the request and response headers improves the accuracy of the code and ensures that we are getting the correct data.
Reduced errors: Checking the request and response headers reduces the number of errors and exceptions that occur in the code.
Improved reliability: Checking the request and response headers improves the reliability of the code and ensures that it works as expected.

Q: What are the benefits of updating Selenium Grid configuration?

A: The benefits of updating Selenium Grid configuration are:

Improved performance: Updating Selenium Grid configuration can improve the performance of the code by reducing the overhead associated with it.
Improved reliability: Updating Selenium Grid configuration can improve the reliability of the code and ensure that it works as expected.
Improved accuracy: Updating Selenium Grid configuration can improve the accuracy of the code and ensure that we are getting the correct data.

Q: What are the best practices for debugging this issue?

A: The best practices for debugging this issue are:

Use a debugger: Use a debugger to step through the code and identify the source of the issue.
Use print statements: Use print statements to print out the values of variables and identify the source of the issue.
Use a logging framework: Use a logging framework to log out the values of variables and identify the source of the issue.

Q: What are the best practices for testing this issue?

A: The best practices for testing this issue are:

Use a testing framework: Use a testing framework to write unit tests and integration tests for the code.
Use a mocking framework: Use a mocking framework to mock out dependencies and isolate the code.
Use a continuous integration tool: Use a continuous integration tool to automate the testing process and ensure that the code works as expected.