Logging is a huge and interesting topic with a lot of different aspects and elements that deserve their own discussion. So come along on a journey with us as this is the start of a series of blogposts about Logging 😉
Logging is basically serializing data about events that occur during the run of an application. These event logs are usually written into a log file or to the console. Logging is a fundamental part of being able to maintain a project, let’s see why!
Logs provide an insight into what happens inside the application. If done right, they can help a whole lot when trying to understand what executing the code actually does. They are even more helpful if someone has to work with a project unknown to them. They obviously provide much help with debugging. This is the first and most widely known usage of logs, and this category is called diagnostics.
The second category might not be obvious at first. Logs can document user interactions and transactions as well which tell you who did what and when. This way you can see who accessed or changed certain data, or who gave privileges to which user. You can check if someone misuses their privileges or detect if there was a security breach. This type helps with security aspects of maintaining the application and is called auditing.
There can be a heck load of logs from an application and for different purposes you want to see different types and amounts of logs - when you debug, you want a lot of information on how the app is processing things but on a production system you don’t need to get every itty-bitty detail anymore. Logging levels are basically a categorization of logs based on their urgency. You can use log levels in your code to specify how a line of log should be viewed. Below you can see the most typically used log levels and their description:
This level is rarely used. It should give you basically every possible detail about the behavior of the application. It can give you a step by step rundown of an algorithm or of a chain of function calls etc.
The goal is to help debugging the application and for diagnosing problems. Should be usable for developers as well as other IT people or sysadmins who may need to understand it in the future.
Should tell you the progress of the application (start, stop, events, etc.). This is data that you normally don’t look at often but should have at hand in case you need to look at something in particular.
This level is for potentially harmful situations that probably don’t effect the customer yet and are not absolutely urgent but need to be looked into.
This level should be used when the application is not functioning properly. It is still working but has a major problem. Customers are also possibly affected. Resolving the issue needs human interaction.
The application stopped working. This is an absolute emergency. Catastrophe ensues if nothing is done.
If the framework you use allows it, you can define your own level of logging if you have a specific need that none of the above levels accomplish.
First of all when logging, you need to provide context. Without context the log doesn’t really mean anything, it’s not searchable or filterable really.
Don’t write log lines that provide no actual information as to where or how something happened!
logger.warn("No csv found in the given directory") // No context, we don't get a clue about the problem logger.warn(s"No csv found in the given directory $dir") //We know which directory didn't have any csv in it, that's a starting point
logger.warn("Document not found") // Again, absolutely no context logger.warn(s"PSO document not found in the RewriteDocument process! docId: $docId") //We know what id the missing document has, and we know the surrounding circumstances
This exercise is useful especially in microservices or apps run on multiple threads. This helps with following the steps of a single process without having to cherry pick from all the other logs.
If the logging framework you work with provides this option, create a logger separately for your classes (or if seen necessary for your functions). This ensures that no two logger will be created with the same name (in case of loggers by class), it’s an easy convention to follow and it provides extra contextual information as to where the event happened.
Do NOT log confidential data from your app. Passwords and other touchy subjects should be left out of logging for obvious security reasons.
Should be a common practice throughout an application or its services. The exact format you use will depend on the framework of your choice and the way you want to manage your logs afterwards. A good goal in all cases though is for it to be easily readable both by machines and by humans.
Multiline logs like stack traces might not be ideal for machines to work with. If you use a format like json or xml this will not be an issue though.
On the oher hand logs too long will make it hard for the human eyes to process so you might want to take this into account. Also if you need to go through logs by hand, it will be useful to order the fields of the data so that it is easy to comprehend - for example date, severity, class, message.
There are some things that should be common knowledge considering a project. Ideally these should be agreed on before development starts and followed by the developers. These include things like when to use which logging level, how to phrase the logs, naming conventions, etc.
// Both can be fine but it should be unified logger.info(s"Document not found in the RewriteDocument process. docId: $docId") logger.warn(s"Document not found in the RewriteDocument process. docId: $docId")
// Both can be fine but it should be unified logger.warn(s"Document not found in the RewriteDocument process. docId: $docId") logger.warn(s"ReadDocument could not find document with id $docId")
To be continued…
Future blogposts in the series will talk about unique identification of processes, frameworks, centralized logging, tools for that, amount of logs and log retention, and some other topics surrounding logging. Stay tuned! 🙂