E2.0, Enterprise Information Management, and Archiving
Web and Enterprise 2.0 talk is all the buzz these days. The social web, its features, capabilities, and “sexy” factor is arguably the defining decision driver in technology considerations. Unfortunately, that leaves important but inherently un-sexy technologies behind. While strategists and organizational decision makers are paid not to be blinded by flashy user interfaces and promises of communal utopia, the reality is that the expectations set by the fun user interaction available on the public web are infiltrating RFPs and RFIs for even mundane, back-end capabilities.
Let’s be honest for a moment. Most back end technology is simply not fun. Important? Yes. Vital? Absolutely. Interesting? Possibly. Necessary – you bet! But fun? No. It’s hard to describe headless applications as fun. Archiving, disaster recovery, legal preparedness, audit and compliance systems fall squarely into this un-fun realm. Archiving technologies and markets have been around for years. However, recent years have seen significant evolution in both technical capabilities, as well as user demands. The need for updated, coherent, and comprehensive enterprise archiving strategies has never been greater.
End user expectations of ubiquitous availability of information and (nearly) limitless storage space are at the core of this demand. The perception engendered by the proliferation of Web 2.0 technologies on the public web is that information is immediately available regardless of how old it may be (e.g. the Internet Archive’s Wayback Machine) and that there is enough space to store anything you want; from images (e.g. on Flickr and Picasa) to email (e.g. Yahoo mail’s unlimited storage) to PowerPoint slides (e.g. slideshare.net). The expectation of end users is that their employers can do at least what a bunch of college students working out of their dorms can do. That this expectation might be based on some erroneous assumptions is not typically acknowledged by the end users clamoring for the capabilities they get for free. Neither is IT endeared to their user-customers by pointing out the flawed basis for the expectations.
It is then left to us to straddle the often oppositional realms of end-user expectations and IT reality, doing the best we can with the technology that is available to us on our side of the firewall. To be successful we need to:
- Understand the drivers and demands of compliance and IT risk management alongside the needs of our users.
- Develop a strategy that leverages the best technology available and makes use of the infrastructure investments we have already made.
- Implement the strategy by merging business-critical systems with end-user expectations on a foundation of stable and scalable enterprise information management.
This three-pronged strategy, focusing on enterprise information management (EIM) through the lens of archiving, risk management, and compliance systems is explored below. Throughout, we will be aware of Web 2.0-based employee expectations as they become Enterprise 2.0 requirements.
EIM Archiving Drivers and Demands in an Enterprise 2.0 World
There are several specific factors that are driving the need for better enterprise information management archiving strategies. First and foremost is the exponential growth of information. While cost per volume of electronic storage continues to drop, most companies are continuing to see their overall storage costs rise as the increasing volumes outstrip any gains from lower price.
Employees are increasingly interconnected in their personal lives and have come to expect similar experiences from corporate applications. Information from crowd-based sources like blogs, wikis, discussion threads, distribution lists, and web comments are adding to the informational volume with which businesses must deal. Furthermore, these sources of content are naturally disparate and easily orphaned, making the job of managing them considerably more difficult. Even in the absence of Web 2.0 technologies, email volumes alone are experiencing explosive growth, with some estimates predicting 25-30 percent annual growth. Leaving all company email on expensive, Tier-1 server discs is neither practical for performance reasons nor cost effective. In many cases leaving all company email anywhere is expensive and risky.
In addition, the complexities of managing an organization in our networked and globalized economy are leading to an exponential production of collaborative content as companies partner and work together. Advances in computing mean that automated reports and transactional data are being generated by systems and processes and then stored. All of this information falls into the archive candidate pile.
In conjunction with the information explosion that we see every day, there has been a similar increase in compliance requirements--whether legal or industry, risk mitigation practices imposed by corporations, and ever-changing legal structures like the US Federal Rules of Civil Procedure (FRCP). The corporate scandals of this decade have made a huge impact on legislative leadership, which has in turn led to increasing regulations in the form of Sarbanes-Oxley, new requirements for HIPAA, new SEC regulations, etc. While many companies choose to use a “keep everything” strategy for compliance, this is not practical and really must be viewed as detrimental. To paraphrase the Miranda warning, anything you keep may be used against you. Your enterprise ecosystem must address the dual and competing demands of ensuring accessibility to all relevant information while applying defensible policies for the disposal of obsolete, outdated, and irrelevant information.
Another factor that has affected adoption rates for archiving is eDiscovery, and particularly the costs associated with eDiscovery. Recent court actions based on new FRCP regulations have found that neither high cost nor difficulty in retrieval are valid defenses during an eDiscovery process. During a legal proceeding, organizations are fully expected to be able to quickly locate and comprehensively provide all data related to the case at hand. Needless to say, without a robust archiving strategy and system in place, it can be very costly to locate all relevant emails, wikis, blogs and electronic documents. In addition, unexpected eDiscovery activities can become great burdens on IT staff (to the detriment of other IT initiatives) if a fast and accurate search tool is not available, not to mention the high cost of legal staff doing manual document review.
The Enterprise Information Management Archiving Strategy
These EIM and compliance drivers have come together to create renewed focus on information archiving. It is important to understand that regular backups of enterprise information systems is not an archiving strategy. According to industry analysts, simply copying everything to a storage medium, even if it is immutable, on a regular basis is a thousand miles away from having an intentional and consistent policy, consistently implemented, that drives a corporate archive. To have a complete EIM system in place, businesses should focus on keeping the right information around for only as long as necessary while ensuring that the information is accessible and easily available when needed. When choosing an enterprise information strategy as well as the supporting technology, there are a number of key items to consider.
First, determine what your current and anticipated archiving needs are. Some questions to consider:
- Is there poorly managed or orphaned unstructured content in your company?
- What are your legal, compliance, and eDiscovery requirements and how might they be served through an enterprise archive?
- Is there information or content generated from enterprise applications that could benefit from archiving?
- Are Enterprise 2.0 applications, such as wikis, blogs and social networks generating content that should be managed and archived?
- Are your users frustrated with an email quota policy?
- What are your concerns about email storage costs?
- Is your current email archiving strategy akin to backing up personal email collections (Danger! Danger!)?
- Have you recently or will you soon go through a merger?
- How are you planning on rationalizing your file storage?
- How will you triage your information?
The answers to these questions will help you understand what features and capabilities will be most important in the tools you evaluate.
One of the most vital characteristics of an enterprise archive system is the capability of the system to dynamically archive the appropriate information. Scheduled backups are no longer a complete solution. They miss highly dynamic information and do not scale with the organization. Remember, Web and Enterprise 2.0 information interaction patterns mean that important data is coming from new, novel, and unanticipated sources. Furthermore, this information is exploding. Consequently, the ability to handle increasing volumes of information is vital. Capabilities like compression, de-duplication, hierarchical storage management, single-instance storage, and federation of storage in a single logical structure rather than blindly writing everything off to hard media like tapes or adding more vaults is important. With email leading the shock wave of the information explosion and also being a vital archiving target, the need for robust real-time online archiving is not a luxury; it is a necessity.
Strategically speaking, archiving activity should be closely related to, and perhaps a full subset of, enterprise records and retention management strategy. It is important to ensure close coordination with the records management staff. The governing information retention policies must be consistently enforced to be legally defensible in the event of a legal action. This means that full integration between records and retention management policy engines and archiving policy management consoles is needed. Ideally, a central policy server would be in place where records classifications could be applied to both records systems and archives, including email. Ad hoc integration of point solutions is one way to go, but it is costly and difficult to maintain. Platform vendors and pre-integrated technologies should be strongly considered as the strategic considerations come together. In the long run, these platform-based systems will provide the EIM foundation upon which next-generation systems rely.
EIM Archiving Implementation as E2.0 Foundation
It’s hard to dress up an elephant in haute couture. It looks like a silly elephant. But put a super model in one of a kind custom tailoring and something striking and emotive is revealed. Archival systems relegated to the old ways of off line separation and backups are the elephant. It doesn’t matter how spectacular the stuff hanging off it really is--it’s still cumbersome, bumbling and silly. But put an archival skeleton that is sized correctly, able to dynamically scale, and manage offload information intelligently, underneath and suddenly the custom look and feel you have hanging off that core is flattering, engaging, and desirable.
When organizations move from an EIM skeleton of simplistic backups to a skeleton of dynamic online archiving, they gain the strength of scalable enterprise systems (tape backup shelves in an IT closet are not scalable!) and the agility of Enterprise 2.0 systems. Consider one parting example.
Social networks are all the rage right now. Web 2.0 has been called the social web. Enterprise 2.0--the bringing in of Web 2.0 technologies, practices and paradigms behind the corporate firewall--has a deep focus on collaboration and social enablement. Many organizations are spending lots of money to build or buy social networking applications for their employees. Yet those same organizations often ignore the latent social networks that already exist inside the organization. Email is intrinsic to the ways we work. Dynamic email archives contain huge volumes of email in parse-able formats. This means that by leveraging the email archive you can parse out the from and to header field values. Filtering those values against a criteria like my name and you immediately have a dynamically generated and organic (growing) social graph. Mapping those values into a visualization tool and you have a display that shows clusters and nodes of activities, experts, collaborators, and trends to be tapped. Only platform-based enterprise information management dynamic archiving technologies can perform the aggregation, filtering, and compositing with other applications that are built upon that same EIM archiving foundation.
This is simply one example, a brainstorm activity really, of how a strategic approach to dynamic enterprise information archiving produces a sound and scalable foundation upon which Enterprise 2.0 tools can be built to meet the ever-growing requirements and expectations of our Web 2.0 world.
Understanding the drivers, determining your enterprise archiving strategy and implementing an appropriate, platform-based enterprise archive system is a significant process affecting all areas of your business. Done properly, it can save lots of money, time, and frustration; done improperly, legal and compliance issues can end up costing considerably more than the system implementation. By following the guidelines above, you can clarify your objectives, fully evaluate the myriad options available, and get off on the right foot with your archiving strategy.
Billy Cripe is the author of Reshaping Your Business with Web 2.0 (McGraw-Hill October 2008), an Enterprise 2.0 Evangelist, and Director of Product Management for Oracle.
Wayne Boerger is a Product Manager and Advocate for Oracle Enterprise Content Management and Enterprise 2.0.