Comments on: Metadata Schmetadata, Relevance and Reality http://webdogs.org/2009/03/26/metadata-schmetadata-relevance-and-reality/ site sentient since 2006 Tue, 26 Apr 2011 20:57:58 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Daniel Tunkelang http://webdogs.org/2009/03/26/metadata-schmetadata-relevance-and-reality/comment-page-1/#comment-19686 Daniel Tunkelang Sat, 28 Mar 2009 04:45:19 +0000 http://www.webdogs.org/findability/?p=755#comment-19686 Brian, thanks for responding. I hope you don't mind that I've been mirroring this discussion at my blog, The Noisy Channel: http://thenoisychannel.com/2009/03/27/does-metadata-matter/ The discussion there is unmoderated, and I encourage you (or readers here) to chime in. Brian, thanks for responding. I hope you don’t mind that I’ve been mirroring this discussion at my blog, The Noisy Channel:

http://thenoisychannel.com/2009/03/27/does-metadata-matter/

The discussion there is unmoderated, and I encourage you (or readers here) to chime in.

]]> By: Brian Lawlor http://webdogs.org/2009/03/26/metadata-schmetadata-relevance-and-reality/comment-page-/#comment-19685 Brian Lawlor Sat, 28 Mar 2009 01:15:55 +0000 http://www.webdogs.org/findability/?p=755#comment-19685 Daniel, your observations are spot-on about the evaluation of user need and experience. Our evaluations of whether our users can find what they are looking for are, admittedly, limited to basic user-experience surveys, as well as a considerable amount of first-hand observations of users conducting actual searches. But what we have done is not as thorough or precise as the basic frameworks you suggest. We readily agree that one cannot generalize in any definitive fashion about metadata from our experience. We are not inviting others to do so. We are, however, reporting on our experience and what we do think legal services and other non-profits can and should question, including whether the initial cost and ongoing investment in creating and managing metadata is a practical way to go. What we are discovering is that our users are finding what they are looking for without the apparent need to add metadata to our targets, most of which do not have added metadata or we could not add it if we wanted. (For example, our targeted Google Sites content.) What is left unstated in the post are other things we are exploiting in the Google Search Appliance, including an array of filters and collections for narrowing search results in a way very easily understood by our users, selective use of Keymatch, and a set of OneBox modules that are very effective in helping our users find some of the most common things they are searching for. In the context of our project, those options strike us as more efficient and less costly ways for us to "help" the search. Daniel, your observations are spot-on about the evaluation of user need and experience. Our evaluations of whether our users can find what they are looking for are, admittedly, limited to basic user-experience surveys, as well as a considerable amount of first-hand observations of users conducting actual searches. But what we have done is not as thorough or precise as the basic frameworks you suggest. We readily agree that one cannot generalize in any definitive fashion about metadata from our experience. We are not inviting others to do so. We are, however, reporting on our experience and what we do think legal services and other non-profits can and should question, including whether the initial cost and ongoing investment in creating and managing metadata is a practical way to go.

What we are discovering is that our users are finding what they are looking for without the apparent need to add metadata to our targets, most of which do not have added metadata or we could not add it if we wanted. (For example, our targeted Google Sites content.) What is left unstated in the post are other things we are exploiting in the Google Search Appliance, including an array of filters and collections for narrowing search results in a way very easily understood by our users, selective use of Keymatch, and a set of OneBox modules that are very effective in helping our users find some of the most common things they are searching for. In the context of our project, those options strike us as more efficient and less costly ways for us to “help” the search.

]]> By: Daniel Tunkelang http://webdogs.org/2009/03/26/metadata-schmetadata-relevance-and-reality/comment-page-1/#comment-19684 Daniel Tunkelang Fri, 27 Mar 2009 12:36:13 +0000 http://www.webdogs.org/findability/?p=755#comment-19684 An interesting article. But perhaps I missed an explanation of how you performed your evaluation. Did you assign tasks to your users and compare their effectiveness on the two systems? Did you ask them to express their subjective satisfaction with the system? Did you have some productivity measure external to the system, such as efficiency at completing projects? It may be that a simple out-of-the box ranked search approach, with no annotation, manual or automatic, of your documents, is exactly what your organization need. But it's very hard to generalize from your experience without understanding better what exactly you where evaluating. An interesting article. But perhaps I missed an explanation of how you performed your evaluation. Did you assign tasks to your users and compare their effectiveness on the two systems? Did you ask them to express their subjective satisfaction with the system? Did you have some productivity measure external to the system, such as efficiency at completing projects?

It may be that a simple out-of-the box ranked search approach, with no annotation, manual or automatic, of your documents, is exactly what your organization need. But it’s very hard to generalize from your experience without understanding better what exactly you where evaluating.

]]>