Send your Storage question to David Hill today!

See other Storage tips from David

What seems to be the problem? All bits are not equal. They may occupy the same physical space, but some bits have more value than others. Only ordered (i.e. non-random) bits can have value. Those ordered bits are data, which means bits organized into recognizable structures, such as files and database records. File systems, database management systems, and software applications need those bits, as data, in order to function effectively. And how that data is effectively managed and used depends upon the data type.

Knowing your blood type is important for people; knowing your data types is important for IT. Decisions on how you protect your data, how you go about archiving it, and how you can best organize the data not only for its current uses, but also how to get more out of it (such as search) depend upon your understanding the data type. Yet IT organizations tend to lump data into only two types -- structured and unstructured. That is wrong. Those ordered bits we call data can come in three general types -- structured, semi-structured, and unstructured. Understanding the difference among these three types is necessary because the storage administration for each of the three types differs.

What do you need to know? Managing data used to be easy. IT is used to dealing with database data that forms the heart of online transaction processing systems. Then along came e-mail, word processing documents, electronic presentations, spreadsheets, and HTML. In addition, CAD/CAM data, videos, audio recordings, and medical images are also now digitized. Even though they are all sets of ordered bits, they have to be managed differently. Data can be classified into three distinct categories:

Table: The Different Data Types

Source: Mesabi Group August 2006

Everyone agrees with the structured category. However, if the semi-structured category is used at all, it is used only in conjunction with e-mail. While e-mail is semi-structured data, so are word processing documents and presentations. Perhaps vendors are thinking of Microsoft Exchange and Microsoft Outlook, which is a hybrid, so the term semi-structured might be used as a compromise between structured and unstructured. However, we are dealing with data, not with the applications that use that data. A search engine can search both e-mail and a word processing document, which requires that more than file metadata is used with the files.

The same ability to search content cannot be said about true unstructured data. Unstructured data are typically BLOBs (Binary Large Objects). Native (i.e., without metadata beyond file metadata) unstructured files cannot be searched with a Boolean search engine.

One reason for the confusion is both semi-structured data (say word processing documents) and unstructured data (say a digital photograph) are files. So saying that all files are unstructured is easy, but wrong. You can open a word processing file and make a keyword index of its contents. You cannot do that with a digital photograph. The ability to index is important, for example, if you need to do eDiscovery to find specific e-mails for legal discovery purposes.

What can you do about it? Vendors are likely to use only the two categories (structured and unstructured). You need not follow their classification; you simply have to figure out how to map what the vendor can do against the types of data that you want their software to work with. You may want a tool that is universal; that is a tool that can work across all data types. On the other hand, you may want a tool that works only with one data type, say semi-structured files, or with one application, say Microsoft Exchange.

Before a vendor comes, you can mentally qualify the data created or used by key applications into one of the three categories defined in the Table. (Yes, there are some applications that use a composite of data types, but try to keep the analysis process simple at first.) Then simply present your data to the vendor and see if the vendor can handle it. The key is that for semi-structured files the vendor tool has to go beyond using just file metadata. The whole process should not be too hard if you keep the table in mind.

Failing to recognize that there are three general types of data -- structured, semi-structured, and structured -- can lead to confusion when trying to analyze vendor products. Understanding the three types clearly gives you a perceptual lens to assist you in the process of evaluating whether or not the product meets your needs.

storage.itworld.com

• Email
• Print
• Share
  • Slashdot
  • Digg
  • Stumble
  • Reddit
  • Newsvine