Telemetry Streaming Configuration

Oct 13, 2025 by talynt.org 40 views

Unable to Send Logs to a HTTP Endpoint (Elasticsearch) via Telemetry Streaming

Introduction

Telemetry streaming is a feature in BIG-IP that allows you to stream logs to a remote endpoint, such as Elasticsearch. However, some users have reported issues with sending logs to a HTTP endpoint via telemetry streaming. In this article, we will explore the steps to reproduce the behavior, the environment, and the expected behavior.

Environment

Telemetry Streaming Version: 1.37.0-1
BIG-IP Version: 15.1.10.4

Steps to Reproduce

Configure telemetry streaming on BIG-IP to send logs to a HTTP endpoint.
Create a logging source system log with the GUI.
Configure 4 vhosts with request logging profile.
Check the log in /var/log/restnoded/restnoded.log to see if the logs are being sent.
Use Postman to send a request to the event listener to test if the logs are being sent.

Telemetry Streaming Configuration

The telemetry streaming configuration is as follows:

{
    "class": "Telemetry",
    "controls": {
         "class":"Controls",
         "logLevel": "error",
         "debug" : true
     },
       "My_System": {
        "class": "Telemetry_System",
        "systemPoller": {
            "interval": 60
        }
    },
   "My_Listener": {
        "class": "Telemetry_Listener",
        "port": 6514,
        "trace": [
            {
                "type": "input"
            },
            {
                "type": "output"
            }
        ]
    },
    "My_Consumer": {
        "class": "Telemetry_Consumer",
        "type": "Generic_HTTP",
        "host": "SOC_IP",
        "allowSelfSignedCert": true,
        "protocol": "http",
        "trace": true,
        "path": "/",
        "method": "POST",
        "port": "9570",
        "headers": [
            {
                "name": "content-type",
                "value": "application/json"
            }
        ]
    }
}

AS3 Logging Sources Configuration

The AS3 logging sources configuration is as follows:

{
    "class": "ADC",
    "schemaVersion": "3.10.0",
    "remark": "Example depicting creation of BIG-IP module log profiles",
    "Common": {
    	"class": "Tenant",
        "Shared": {
            "class": "Application",
            "template": "shared",
            "telemetry_local_rule": {
                "remark": "Only required when TS is a local listener",
                "class": "iRule",
                "iRule": "when CLIENT_ACCEPTED {\n  node 127.0.0.1 6514\n}"
            },
            
            "telemetry_local": {
                "remark": "Only required when TS is a local listener",
                "class": "Service_TCP",
                "virtualAddresses": [
                    "255.255.255.254"
                ],
                "virtualPort": 6514,
                "iRules": [
                    "telemetry_local_rule"
                ]
            },
            
            "telemetry": {
                "class": "Pool",
                "members": [
                    {
                        "enable": true,
                        "serverAddresses": [
                            "255.255.255.254"
                        ],
                        "servicePort": 6514
                    }
                ],
                "monitors": [
                    {
                        "bigip": "/Common/tcp"
                    }
                ]
            },
            "telemetry_hsl": {
                "class": "Log_Destination",
                "type": "remote-high-speed-log",
                "protocol": "tcp",
                "pool": {
                    "use": "telemetry"
                }
            },
            "telemetry_formatted": {
                "class": "Log_Destination",
                "type": "splunk",
                "forwardTo": {
                    "use": "telemetry_hsl"
                }
            },
            "telemetry_publisher": {
                "class": "Log_Publisher",
                "destinations": [
                    {
                        "use": "telemetry_formatted"
                    }
                ]
            },
            "telemetry_traffic_log_profile": {
                "class": "Traffic_Log_Profile",
                "requestSettings": {
                    "requestEnabled": true,
                    "requestProtocol": "mds-tcp",
                    "requestPool": {
                        "use": "telemetry"
                    },
                    "requestTemplate": "event_source=\"request_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\""
                },
                "responseSettings": {
                    "responseEnabled": true,
                    "responseProtocol": "mds-tcp",
                    "responsePool": {
                        "use": "telemetry"
                    },
                    "responseTemplate": "event_source=\"response_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\",http_statcode=\"$HTTP_STATCODE\",http_status=\"$HTTP_STATUS\",response_ms=\"$RESPONSE_MSECS\""
                }
            }
        }
    }
}

Expected Behavior

The expected behavior is that the logs are sent to the HTTP endpoint via telemetry streaming and can be seen in Kibana.

Actual Behavior

The actual behavior is that the logs are not sent to the HTTP endpoint via telemetry streaming and cannot be seen in Kibana.

Troubleshooting

To troubleshoot this issue, we can check the following:

Check the telemetry streaming configuration to ensure that it is correct.
Check the AS3 logging sources configuration to ensure that it is correct.
Check the log in /var/log/restnoded/restnoded.log to see if the logs are being sent.
Use Postman to send a request to the event listener to test if the logs are being sent.
Check the firewall rules on the Security Onion server to ensure that the F5 IP address is allowed to send logs to the HTTP endpoint.

Conclusion

In conclusion, the issue of not being able to send logs to a HTTP endpoint via telemetry streaming is a complex one that requires a thorough investigation. By following the steps outlined in this article, we can troubleshoot the issue and identify the root cause.
Q&A: Unable to Send Logs to a HTTP Endpoint (Elasticsearch) via Telemetry Streaming

Q: What is telemetry streaming and how does it work?

A: Telemetry streaming is a feature in BIG-IP that allows you to stream logs to a remote endpoint, such as Elasticsearch. It works by sending logs from the BIG-IP system to a designated endpoint, where they can be processed and analyzed.

Q: What are the common issues that can cause telemetry streaming to fail?

A: Some common issues that can cause telemetry streaming to fail include:

Incorrect configuration of the telemetry streaming settings
Issues with the AS3 logging sources configuration
Problems with the firewall rules on the Security Onion server
Issues with the Elasticsearch endpoint itself

Q: How can I troubleshoot issues with telemetry streaming?

A: To troubleshoot issues with telemetry streaming, you can follow these steps:

Check the telemetry streaming configuration to ensure that it is correct
Check the AS3 logging sources configuration to ensure that it is correct
Check the log in /var/log/restnoded/restnoded.log to see if the logs are being sent
Use Postman to send a request to the event listener to test if the logs are being sent
Check the firewall rules on the Security Onion server to ensure that the F5 IP address is allowed to send logs to the HTTP endpoint

Q: What are some common error messages that can indicate issues with telemetry streaming?

A: Some common error messages that can indicate issues with telemetry streaming include:

"Error: unable to send logs to endpoint"
"Error: invalid configuration"
"Error: unable to connect to endpoint"
"Error: invalid AS3 logging sources configuration"

Q: How can I ensure that my telemetry streaming configuration is correct?

A: To ensure that your telemetry streaming configuration is correct, you can follow these steps:

Double-check the configuration settings to ensure that they are correct
Verify that the AS3 logging sources configuration is correct
Test the telemetry streaming configuration by sending a request to the event listener
Check the log in /var/log/restnoded/restnoded.log to see if the logs are being sent

Q: What are some best practices for implementing telemetry streaming?

A: Some best practices for implementing telemetry streaming include:

Ensuring that the telemetry streaming configuration is correct
Verifying that the AS3 logging sources configuration is correct
Testing the telemetry streaming configuration before deploying it in production
Monitoring the telemetry streaming logs to ensure that they are being sent correctly

Q: Can I use telemetry streaming with other logging solutions?

A: Yes, you can use telemetry streaming with other logging solutions, such as Splunk or Sumo Logic. However, you will need to configure the telemetry streaming settings to work with the specific logging solution you are using.

Q: How can I get help if I am experiencing issues with telemetry streaming?

A: If you are experiencing issues with telemetry streaming, you can try the following:

Check the BIG-IP documentation for troubleshooting information
Contact F5 support for assistance
Join the F5 community forum to ask for help from other users
Contact your F5 partner or reseller for assistance