Useful diagnostics explain what happened, where it happened and the circumstances around the operation. A report that an operation failed is a starting point. The supporting information should help someone trace the cause, understand the current condition and decide what happens next.
The design depends on the system. A small application may need only well-structured text logs. A complex product may need decoding tools, searchable records or a database. The aim is sufficient, usable evidence for investigation.
Record the location and operational context
For a developer investigating an error, the source file and line can be particularly useful when available. Record the software version too, so that the location can be interpreted against the correct build. The reported location identifies where the problem was detected; the surrounding evidence helps establish why it occurred.
Use consistent fields for the information needed to understand an event:
- What happened, including the operation and its outcome.
- Where it happened, including the component and relevant source location.
- When it happened and its place in the sequence of events.
- The relevant state, configuration and supporting operational information.
- The severity and whether the system recovered, stopped or needs intervention.
A stack trace is often useful when the call path matters. It need not accompany every event if the other evidence already explains the failure. Choose the detail according to what will help the investigation and what the platform can provide.
Choose the format and tools for the system
Structured logging does not require a particular file format. Text can work well when entries follow a consistent layout and provide enough detail. XML, JSON or CSV can also be appropriate, especially where records are decoded or processed by separate logging tools.
Keep the record format and its interpretation clear. If a decoder is needed, it should understand records from the software versions being supported. Information that has been captured but cannot be interpreted reliably is of limited practical value.
As complexity increases, filtering and searching become increasingly useful. Being able to isolate a component, severity, operation or time period helps reduce a large collection of events to the sequence that matters. A database can support that investigation where justified, but a simpler arrangement may be sufficient for a smaller system.
Give severity a consistent meaning
Severity is an important part of error handling as well as logging. Define what each level means for the product and apply it consistently. The same condition should not appear as a routine warning in one component and a fatal error in another without a deliberate reason.
The record should distinguish the seriousness of the event from the outcome of recovery. A failed operation that succeeded on retry still happened; its successful recovery is useful information too. Repeated recovery may indicate a developing problem even while the product continues to function.
A severity label should accompany an explicit response policy. Some conditions permit continued operation, others require human intervention, and a fatal error may require the application to terminate. Define those decisions against the actual consequences for the system.
Document when retries are permissible
Retrying can be appropriate when the operation is safe to repeat and the failure may be temporary. That permission should be established in the design. An operation may have partly completed before the error was reported, so repeating it requires consideration of its effects.
A retry policy should document the eligible conditions, the maximum attempts, any delay, how success is recognised and what happens when attempts are exhausted. In some systems a configurable limit is useful; permitted values should remain within the bounds established for that system.
The correct policy may be one failure followed immediately by human intervention. It may instead allow a limited number of attempts. The choice depends on severity, the operation and the system's requirements. A mechanical movement, for example, requires equipment-specific assessment before another attempt is permitted.
Record the original failure, retry attempts and final outcome so that recovery remains visible. For networked operations, AWS guidance on retry behaviour explains why repeatability, limits and delays need deliberate design.
Define how the system stops
Where continued operation is unacceptable, specify what must stop and what intervention is needed. Depending on the product, that may mean ending an operation, disabling a component or terminating the application. The response should leave the system in its required condition; simply logging a fatal message is not the response itself.
Include the reason for stopping and enough context to support investigation. Preserve that evidence where possible without allowing logging to delay a time-critical response.
Keep diagnostics dependable and proportionate
Logging consumes storage and processing time. Define limits and rollover behaviour, and test what happens when the destination is full or unavailable. Avoid allowing a flood of diagnostic messages to disrupt the system being investigated.
Supporting information should be selected carefully. Exclude credentials and unnecessary personal data, control access to logs and make retention and export behaviour clear. OWASP's logging guidance provides further detail on event context and protecting diagnostic records.
Test the diagnostic and recovery paths together: the evidence produced by a failure, whether records can be decoded and searched, the retry limit, and the response when intervention is required. A useful logging design is one that still helps when the system is under the conditions it was designed to explain.