Golang Application Performance Monitoring with Elastic APM
Monitoring your application performance is an important subject that must be put in mind when we have our application running in Production. This kind of monitoring helps us to be aware if there is a slow down or performance issue within our system. Without any performance monitoring, it is hard for us to pin point which process takes longer to finish.
There are a lot of APM services out there, but let us try one open source elastic APM with golang this time. This writing will be focus on:
- Setting up easy APM server on localhost with default configuration
- Writing simple Golang REST API that covers the APM basic substance for transaction tracing
Elastic APM Installation
For a quick installation guide, I followed the guide based on this reference:
Basically you need docker to quickly install elasticsearch, kibana, and apm-server in one pack via docker-compose.yml file. We will use all default configuration since it’s not the main focus of what we’re trying to discuss here. After you put the docker-compose.yml file, run it by docker-compose up command. Try accessing http://localhost:5601 to access the web UI.
Writing the Golang Application
We’re going to set an example of simple REST API that is using gin golang framework.
Create an API with GET method for “/example” path. Then inside the router handler, we will simulate some processes in which consists of 3 functions.
- processingRequest function that will sleep 15 miliseconds
- doSomething function that will sleep 20 miliseconds
- getTodoFromAPI function that will fetch data a JSON placeholder API (https://jsonplaceholder.typicode.com/todos/1)
First let’s dive from the router first
Since we’re using gin framework, we can utilize the apmgin.Middleware function that will automatically wrap any router handler created by us and send it to the APM server so you don’t have to define for each API its transaction one by one manually. We can continuing the span created by the middleware by golang context via c.Request.Context() function. Then we create new span (extends) from it. Next let us write some code for the processingRequest, getTodoFromAPI function.
Once again for simplicity we’re putting up all code in the same main.go file, but in the real case it’s better to do more clean code by dividing it into its own package and domain. As you can see, every function that you want to trace by the APM need to have golang context parameter in it’s function parameter so it can extends the parent context. Speaking of apm.StartSpan method itself receives three parameters:
- Context, commonly using the parent context in this case, or you can create a new one if it’s the first span
- Name of the span (use anything that helps you to identify the span)
- Type of the span (request, custom, or anything)
Combining all of those and we have our main go function ready to run :)
Try to build the file and run it. Then access the http://localhost:8080/example using your browser.
After a few hits to our API, let us look at the APM server, open up http://localhost:5601/app/apm. You will see your application name (in my case it’s test-apm) in the services list. Since we’re using default configuration, the service name will be our executable script name. If you want specific name for the service you can set the ELASTIC_APM_SERVICE_NAME environment variable for your golang app.
Go to the bottom page, and you can see that our GET /example API is there.
If we’re having let say 10 APIs, all will be listed here if there are hits on those API, it’s automatically added when we’re adding apm gin middleware. The transaction name format will be “[METHOD] [API_PATH]”
As we click the example API, we can see more detailed tracing of the processes inside the router handler.
Clicking on the root span, we’ll get the span metadata.
Additional, we can also add some custom metadata to the transaction, for example let’s add these lines on the router handler.
span, ctx := apm.StartSpan(c.Request.Context(), "PingHandler", "request")defer span.End()// add the below lines// let say for example you get some metadata of the request payload heretoken := c.Query("token")username := c.Query("username")trx := apm.TransactionFromContext(ctx)trx.Context.SetCustom("request_param", map[string]string{"token": token,"username": username,})
Run the code again and try accessing it with additional parameter in the URL, ie: http://localhost:8080/example?username=jason&token=abc123ghj456.
As you can see that we can add custom data if you need it for more tracing / debugging on your end. Let say you want to know the username of people in which the transaction response time is more than 1 second. On our case since the url is also being sent to the APM it’s quite useless to add the custom data like the above lines of code (from URL query param), but it will be useful for POST API method since the requested data is in the body. Please be put in mind that all of these data are being stored in elastic, therefore having more custom data also means to increase in the elastic data size.
Additional Notes:
- There are still a lot of features related to elastic APM, anomaly detection (this one is not free but you can have a free trial for sometime), APM errors, distributed tracing, APM metrics (cpu, memory, etc), Logging. So it’s not only transactions and span
- This example is only using default configuration for the elastic stack, on real production case, more configuration will need to be tweaked (setting up elasticsearch server, nodes, indices, rollup policy, etc).
Have a nice day :)
