Welcome!

Open Source Cloud Authors: Pat Romanski, Elizabeth White, Ram Sonagara, Liz McMillan, Jeev Trika

Related Topics: Linux Containers, Open Source Cloud

Linux Containers: Article

Linux Cover Story — Multidimensional Tagging

Finding information naturally

Multidimensional tagging, a key component in social sharing sites, can potentially help enterprises manage large stores of information. In this article, I'll examine the ways that multidimensional tagging will be implemented using Open Source tools.

As storage costs have continued to decrease, organizations create and retain more information. It is easy - and useful - to keep information online, just in case it's needed. The challenge has become managing information not simply storing it. Structured systems, such as relational database management systems (RDBMS), have well-developed tools for indexing, locating, and retrieving information. Unfortunately not all information fits well into structured systems. Graphics, presentations, spreadsheets, and word processor documents have all proliferated, no longer bound by storage costs. This is especially true of desktop systems where 200GB hard drives are common.

Finding exactly the information needed at the right moment has become difficult. A typical desktop drive has tens of thousands of files. A file server or NAS device could easily have hundreds of thousands or even millions of files. Even if system files and applications are discounted, that's still a lot of files to wade through.

File Systems and Search Engines Provide Some Relief
File systems impose a certain degree of order on unstructured information. By allowing information to be placed in hierarchical directories, file systems group files that share some relationship into meaningful categories. The downside to categorization using a file system is complexity for the user. The user is forced to descend through more and more layers of directories to find what he wants. Just remembering where the correct directory is becomes a chore. Information organization via the file system becomes less useful when information is spread across an enterprise. Incompatible systems, individual ways of building directory structures, and shear scale make this a difficult way to manage information assets across an organization, even a small one.

Search engines provide some respite from our information organizational woes. By indexing keywords and storing references to their source in a database, it's possible to look quickly through all the available files. The vendors of searching engine technology, drawing from vast experience in indexing hundreds of millions of Web sites, provide tools that let users find information spread across a huge number of desktop disks and enterprise storage systems.

Tagging Provides Necessary Clues
The trouble with the typical search engine is that the user must first know what he's looking for. He has to have some idea what keywords are indexed for a particular piece of information. If the indexed words don't match the words that a user thinks apply to the information, then the search engine won't find what the user is looking for. A marketing brochure, for example, may not have any words in it that say "marketing brochure" but that's how the user thinks of it. The way the information is categorized in the users head doesn't always match the strict keyword index of the engine.

This has already become something of a problem for Web sites that allow users to share large amounts of information. Cues are needed to help visitors who come to a site find what interests them without knowing the exact nature of the content. In typical Internet fashion an organic solution has arisen called multidimensional tagging, labeling, social tagging, folksonomy, or just simply tagging. It's multidimensional because a single piece of information can have many different tags, reflecting the different dimensions that users apply to it. Tagging lets users assign a set of categories to a piece of information when they create it. The tag system, also known as the tag cloud, grows as people use the information and see relationships in it that the original author might have missed. Users categorize information according to how they view the information, which makes it useful for groups of people who don't always think alike, such as engineers and marketing people.

Tagging is a key feature in social sharing sites such as Yahoo's del.icio.us and Flickr as well as You Tube, and Userscripts.org. Whether it's sharing interesting Web pages, photos, video, or Greasemonkey scripts, all of these sites rely on user categorization. Without tagging, no one would find anything of interest and the site would fail. Unlike simple storage sites (such as Yahoo Photos and Yahoo Briefcase), they require a way of presenting information to users that lets them find it quickly. Users can find information even when they're not really sure what they want. Social sites don't abandon search engines. Instead they integrate searching with tagging to provide a breadth of information retrieval options. Most let users search the tag cloud as well as scanned keywords, providing a rich search environment.

Tagging: New to the Enterprise
Tagging technology for the enterprise environment is new and not widely deployed in products. That's unfortunate. Not only is it extremely useful for finding information, it's also a natural way to do it. This is especially true for people used to social sharing sites. The tagging methodology facilitates the efficient sharing of information across many users and a large file space. It's exactly what enterprises need to make best use of the great stores of unstructured information on their corporate networks.

There are three ways that tagging is being implemented in corporate environments: integrated into applications; as a part of a standalone information management system; and, eventually, as a file system feature. The first kind of implementation is readily available. Image management systems, even ones directed at the desktop environment like Google's Picasa, include tagging as a core feature. The next version of the Thunderbird e-mail client (version 2.0) is expected to include e-mail tagging, augmenting its current search capabilities. Of course, once users get used to tagging for managing certain types of information, they will wonder why they can't use it for all the information that they need to access. They'll expect tag clouds that span all kinds of information in the enterprise.

Tagging is also being implemented in targeted information management tools. Tools for searching large stores of information in a corporate network are still at an early stage but tagging should be expected to make an appearance in information management and search engine tools in the near future. Consider this, Yahoo uses tagging in Flickr and del.icio.us as well as the upcoming My Web 2.0. Is it a stretch to expect it to implement tagging in the corporate search arena? The same is true for Google, which uses tagging on its eBlogger site, GMail Web-based mail service, and Picasa image management tool.

Finally, tagging can be expected to become a feature of the file system and operating system. Some aspects of tagging already exist in operating systems. The ability to attach keywords to files in Microsoft Windows is an example of file system-level tagging. These keywords are currently read by Microsoft's desktop search engine, creating a crude multidimensional tagging feature. Of course, entering and displaying tags is clunky and tags can't be displayed in and of themselves, rendering it more of a hack than a real feature. It does, however, point the way to future features of the operating system. Fully integrated into an operating system as normal metadata and using standard visual cues such those used on social sites, tagging will become a typical part of most corporate environments.

Tools Exist, File System Hooks Don't
The tools for corporate tagging capabilities already exist in the Open Source community. Most of it is encapsulated in the tools used by social bookmarking sites, which are often based on the LAMP stack. They're typically written in common scripting languages, such as Perl or Python, or Java. One such Open Source tool is unalog. Ostensibly a social bookmarking system, it's written in Python and the source is readily available on SourceForge. While the core tools exist, the hooks into the file system are still mostly missing.

A somewhat different but innovative approach is evident with Flickrfs or the Flickr File System. Based on FUSE, it creates a virtual file system with tagging for the Flickr digital photo management service. A fusion of file system and service, Flickrfs lets Linux users access the Flickr service as if it were any other mounted Linux file system. Photos can be accessed through the same tags available on Flickr using standard Linux commands such as cp. Flickrfs represents another way that tagging may come to information management - as a specific application or service but integrated into the normal file system.

Conclusion
Multidimensional tagging provides an opportunity to let users manage information more in line with their natural way of thinking. By sharing tags across the enterprise, users will spend less time looking for information and more time making use of it. Unlike other collaborative systems, users do all the work without legions of editors making decisions that users find mystifying. The social sites on the Internet have shown this to be a viable information management model. It's a matter of how and when, not if, these features become available to the corporate enterprise.

References

More Stories By Tom Petrocelli

Tom Petrocelli, president of Technology Alignment Partners, is a veteran of over 21 years in the technology arena. His background encompasses software engineering, marketing, IT, sales, marketing, and general management. He has worked in various industries including defense, digital signal processing, call center/CRM, networking, and data storage and storage networking. Tom is also the author of a new book entitled Data Protection and Information Lifecycle Management, published by Prentice Hall.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Linux News Desk 05/21/06 09:33:34 AM EDT

Multidimensional tagging, a key component in social sharing sites, can potentially help enterprises manage large stores of information. In this article, I'll examine the ways that multidimensional tagging will be implemented using Open Source tools.

@ThingsExpo Stories
IoT is rapidly changing the way enterprises are using data to improve business decision-making. In order to derive business value, organizations must unlock insights from the data gathered and then act on these. In their session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, and Peter Shashkin, Head of Development Department at EastBanc Technologies, discussed how one organization leveraged IoT, cloud technology and data analysis to improve customer experiences and effi...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
Big Data engines are powering a lot of service businesses right now. Data is collected from users from wearable technologies, web behaviors, purchase behavior as well as several arbitrary data points we’d never think of. The demand for faster and bigger engines to crunch and serve up the data to services is growing exponentially. You see a LOT of correlation between “Cloud” and “Big Data” but on Big Data and “Hybrid,” where hybrid hosting is the sanest approach to the Big Data Infrastructure pro...
A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
With 15% of enterprises adopting a hybrid IT strategy, you need to set a plan to integrate hybrid cloud throughout your infrastructure. In his session at 18th Cloud Expo, Steven Dreher, Director of Solutions Architecture at Green House Data, discussed how to plan for shifting resource requirements, overcome challenges, and implement hybrid IT alongside your existing data center assets. Highlights included anticipating workload, cost and resource calculations, integrating services on both sides...
"We are a well-established player in the application life cycle management market and we also have a very strong version control product," stated Flint Brenton, CEO of CollabNet,, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Unless your company can spend a lot of money on new technology, re-engineering your environment and hiring a comprehensive cybersecurity team, you will most likely move to the cloud or seek external service partnerships. In his session at 18th Cloud Expo, Darren Guccione, CEO of Keeper Security, revealed what you need to know when it comes to encryption in the cloud.
We're entering the post-smartphone era, where wearable gadgets from watches and fitness bands to glasses and health aids will power the next technological revolution. With mass adoption of wearable devices comes a new data ecosystem that must be protected. Wearables open new pathways that facilitate the tracking, sharing and storing of consumers’ personal health, location and daily activity data. Consumers have some idea of the data these devices capture, but most don’t realize how revealing and...
What are the successful IoT innovations from emerging markets? What are the unique challenges and opportunities from these markets? How did the constraints in connectivity among others lead to groundbreaking insights? In her session at @ThingsExpo, Carmen Feliciano, a Principal at AMDG, will answer all these questions and share how you can apply IoT best practices and frameworks from the emerging markets to your own business.
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
Traditional IT, great for stable systems of record, is struggling to cope with newer, agile systems of engagement requirements coming straight from the business. In his session at 18th Cloud Expo, William Morrish, General Manager of Product Sales at Interoute, outlined ways of exploiting new architectures to enable both systems and building them to support your existing platforms, with an eye for the future. Technologies such as Docker and the hyper-convergence of computing, networking and sto...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, discussed the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filterin...
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently. In his session at @ThingsExpo, ...
Early adopters of IoT viewed it mainly as a different term for machine-to-machine connectivity or M2M. This is understandable since a prerequisite for any IoT solution is the ability to collect and aggregate device data, which is most often presented in a dashboard. The problem is that viewing data in a dashboard requires a human to interpret the results and take manual action, which doesn’t scale to the needs of IoT.
Internet of @ThingsExpo has announced today that Chris Matthieu has been named tech chair of Internet of @ThingsExpo 2016 Silicon Valley. The 6thInternet of @ThingsExpo will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.