Troubleshooting

edit

Use the information in this section to troubleshoot common problems and find answers for frequently asked questions. As a first step, ensure your stack is compatible with the Agent’s supported technologies.

Don’t worry if you can’t figure out what the problem is; we’re here to help. If you are an existing Elastic customer with a support contract, please create a ticket in the Elastic Support portal. If not, post in the APM discuss forum.

Please attach your debug logs so that we can analyze the problem. Upload the complete logs to a service like https://gist.github.com. The logs should include everything from the application startup up until the first request has been executed.

No data is sent to the APM Server

edit

If neither errors nor performance metrics are being sent to the APM Server, it’s a good idea to first check your logs and look for output just as the app starts.

If you don’t see anything suspicious in the agent logs (no warning or error), it’s recommended to turn the log level to Trace for further investigation.

Collecting agent logs

edit

The way to collect logs depends on the setup of your application.

ASP.NET Core

edit

When the Agent is activated with UseAllElasticApm or UseElasticApm, it will integrate with the ASP.NET Core logging infrastructure. This means the Agent will pick up the configured logging provider and log as any other component logs.

ASP.NET Classic

edit

Unlike ASP.NET Core, ASP.NET (classic) does not have a predefined logging system. However, if you have a logging system in place, like NLog, Serilog, or similar, you can direct the agent logs into your logging system by creating a bridge between the agent’s internal logger and your logging system.

First implement the IApmLogger interface from the Elastic.Apm.Logging namespace:

internal class ApmLoggerBridge : IApmLogger
{
	private readonly Lazy<Logger> _logger;
	public bool IsEnabled(ApmLogLevel level)
	{
		// Check for log level here.
		// Typically you just compare the configured log level of your logger
		// to the input parameter of this method and return if it's the same/higher or not
	}

	public void Log<TState>(ApmLogLevel apmLogLevel, TState state, Exception e, Func<TState, Exception, string> formatter)
	{
		// You can log the given log into your logging system here.
	}
}

An example implementation for NLog can be seen in our GitHub repository.

Then tell the agent to use the ApmLoggerBridge.

For this in ASP.NET (classic) you need to place the following code into the Application_Start method in the HttpApplication implementation of your app which is typically in the Global.asx.cs file:

AgentDependencies.Logger = new ApmLoggerBridge();

The AgentDependencies class lives in the Elastic.Apm.AspNetFullFramework namespace. During initialization, the agent checks if an additional logger was configured—​the agent only does this once, so it’s important to set it as early in the process as possible (typically in the Application_Start method).

General .NET applications

edit

If none of the above cases apply to your application, you can still use a bridge and redirect agent logs into a .NET logging system (like NLog, Serilog, or similar).

For this you’ll need an IApmLogger implementation (see above) which you need to pass to the Setup method during agent setup:

Agent.Setup(new AgentComponents(logger: new ApmLoggerBridge()));

An InstanceAlreadyCreatedException exception is thrown

edit

In the early stage of a monitored process, the Agent might throw an InstanceAlreadyCreatedException exception with the following message: "The singleton APM agent has already been instantiated and can no longer be configured." This happens when you attempt to initialize the Agent multiple times, which is prohibited. Allowing multiple Agent instances per process would open up problems, like capturing events and metrics multiple times for each instance, or having multiple background threads for event serialization and transfer to the APM Server.

Take a look at the initialization section of the Public Agent API for more information on how agent initialization works.

As an example, this issue can happen if you call the Elastic.Apm.Agent.Setup method multiple times, or if you call another method on Elastic.Apm.Agent that implicitly initializes the agent, and then you call the Elastic.Apm.Agent.Setup method on the already initialized agent.

Another example might be when you use the Public Agent API in combination with the IIS module or the ASP.NET Core NuGet package, where you enable the agent with the UseElasticApm or UseAllElasticApm methods. Both the first call to the IIS module and the UseElasticApm/UseAllElasticApm methods internally call the Elastic.Apm.Agent.Setup method to initialize the agent.

You may use the Public Agent API with the Elastic.Apm.Agent class in code that can potentially execute before the IIS module initializes or the UseElasticApm/UseAllElasticApm calls execute. If that happens, those will fail, as the Agent has been implicitly initialized already.

To prevent the InstanceAlreadyCreatedException in these scenarios, first use the Elastic.Apm.Agent.IsConfigured method to check if the agent is already initialized. After the check, you can safely use other methods in the Public Agent API. This will prevent accidental implicit agent initialization.

Startup hooks failure

edit

If the startup hook integration throws an exception, additional detail can be obtained by setting the ELASTIC_APM_STARTUP_HOOKS_LOGGING environment variable before starting the application

set ELASTIC_APM_STARTUP_HOOKS_LOGGING=1

and then running the application in a context where the environment variable will be visible. In setting this value, an ElasticApmAgentStartupHook.log file is written to in the directory containing the startup hook assembly, in addition to writing to standard output.