This article discusses the importance of implementing a uniform and scalable log management platform for network and storage systems across your organization to address security, compliance and operational issues.

The What: Logs

Recently, security has grown to be on the list of top priorities for many organizations. Despite such prominence, many organizations have not managed to take control over what happens on their network and systems. In fact, CIOs and IT administrators seldom realize that there is an effective way to do this: log management.

All users and systems create traces of their activity in the form of log files. Logs are generated at an astounding rate by IT components such as firewalls, routers, server and client operating systems, databases, and even business applications. As a result, mountains of log data accumulate and, often, nobody looks at them despite their usefulness for detecting and troubleshooting security and system operations issues.

Actively monitoring log data will help protect businesses not only from external security threats but also from potential threats lingering inside the organization. Whether it’s a purposeful data leak by a disgruntled employee or an accidental loss of information in a misplaced laptop, there is always a risk from within. Unfortunately, collecting log data is still commonly considered only a housekeeping task for IT managers rather than the first of many bricks for the information risk fortress. This perception persists despite the fact that a slew of regulations actually mandate log management.

To deal with the above challenges, businesses should adopt a log management solution that can capture 100 percent of log data from a variety of sources across the entire company. However, collecting log is only half of the battle. Monitoring the logs, analyzing them and reporting on them where most of the value lies.

The Why: Business Performance, Security, and Regulatory Compliance

Now we must explain the deeper driving forces behind implementing a log management solution. Such driving forces fall under one of the three pillars of log management: compliance, security and IT operations.

Beyond security, log data monitoring can be applied across an organization to improve business performance. For example, log management plays a key role in Identity Management (IdM) and Business Process Management (BPM). For IdM, log data indicates when new identities are created, how they are managed, and when they are deleted, as well as when and where users are accessing password-protected information. Having a keener understanding of logs can also improve BPM through defining, measuring and optimizing business processes.

Compliance and industry regulations are also major drivers toward locking down networks and databases with log management and intelligence. The Payment Card Industry Data Security Standard (PCI DSS) was developed by the major credit card companies to help prevent credit card fraud, hacking and various additional security vulnerabilities. PCI consists of twelve requirements, one of which (Requirement 10) is entirely dedicated to logging and log management (others also have references to logging). Under this requirement, logs for all system components must be reviewed daily, and these log reviews must include all in-scope systems. PCI-DSS also states that “an organization must ensure the integrity of their logs and maintain strong access control measures to guarantee that logs cannot be altered without generating alerts.“

Other industry regulations and standards that require organizations to deploy log management systems include the Health Insurance Portability and Accountability Act (HIPAA) for health-related industries, the Federal Information Security Management Act (FISMA), for federal agencies as well as many others. In these instances, log management is required to protect customer data and is considered a key incident response and forensics tool in the event of a data breach.

The How: Applying Log Management to Your Network, Servers, and Databases

To summarize, organizations need to paint a picture of total IT infrastructure activity and meet the various logging requirements of recent regulations. Achieving these goals requires more than mere log data generation and collection. IT professionals, in addition to configuring their system to generate and save logs, must also find a way to perform real-time log analysis and in-depth reporting and searching of logs.

The only practical way to do this is to put in place a log management platform. It is clear that manually tending to logs, especially for larger organizations, is, at best, costly, time-consuming, and labor intensive; it truly becomes impossible as log volume grows further.

Among all of the benefits of implementing a log management solution, centralized log collection and retention across diverse log sources reigns paramount. Why?

The traditional approach to log management goes like this: an incident occurs, followed by a review of logs on their individual points of origin. This approach is inefficient, complex, and potentially very expensive, given the massive amount of log data from all different devices that is generated by even smaller companies. You can imagine that by the time all of the data has been found and manually assessed in the context of other data, the breach has already inflicted maximum damage to the company, both in terms of stolen data and a dented reputation.

However, logs from disparate sources reviewed in the context of other logs in a centralized log data warehouse offer situational awareness that is essential to managing both security incidents and a company’s day-to-day IT operations. When responding to an incident, for example, all available evidence must be examined to determine what happened, which means all the logs from all the affected and suspect systems (as well as others connected to them!). Imagine the difference in incident response time (especially if the incident involved more than one part of a distributed IT infrastructure) between having to retrieve logs from each individual source and manually review them together and being able to submit one query across all logs and having the actual facts of the breach nicely lined up for review.

Considering day-to-day operations, centralized log retention makes short work of troubleshooting issues across all systems and of running high-level trend reports across all systems in any given business unit. Additionally, a single point of control means that with the click of a button, all relevant information can be at sys admin fingertips and all logging configurations can be updated with the evolution of security policies, threats and compliance mandates.

To conclude, log management is vital to protecting and maintaining the resiliency of IT operations, upholding security posture as well as satisfying compliance mandates. Without that key building block, a company’s business is likely to come tumbling down from either malicious or accidentals risks.

About the Author
Dr. Anton Chuvakin is a recognized security expert and book author. His current role is Chief Logging Evangelist with LogLogic, a log management company. He is an author and contributor to several security books including "Security Warrior", "Know Your Enemy II", "Information Security Management Handbook", "Hacker's Challenge 3" and "PCI Compliance." Chuvakin has also published numerous papers on security issues; he is also an active blogger (see www.SecurityWarrior.org). He participates in various security industry initiatives and standards organizations.