The infusion of new, future content into the simplified structural taxonomy we created is a separate challenge we will be posting about later. Dealing with our existing files is more immediate, more concrete. Groking those files has been one of the more interesting, at times hilarious parts of this project.

For those legal services programs interested in how the existing files from our eight local offices break out, here are the percentages for the seven most common “document” types:

• 67% – WordPerfect (WPD)
• 18% – Word (DOC)
• 9% – Portable Document Format (PDF)
• 3% – Excel (XLS)
• 1% – Text (TXT)
• 0.9% – Rich Text Format (RTF)
• 0.6% – PowerPoint (PPT)

(Discretion prohibits us from detailing the other file flotsam discovered on local office servers. That said, allow us to observe that some within our organization have extraordinarily good taste in photos taken by National Geographic, and not such good taste in music.)

We are totally on track for targeting most of our planned GSA targets: The existing office archive files listed above have long been targeted (although we still have a lot of work left to fit them into our structural taxonomy); over the last several months we have worked very hard to refresh and update (and remove, as warranted) the targeted content at LSNC’s various public websites; and we are very pleased with the quality of the GSA results we are getting out of Google Sites.

This is all good news. In addition, we are putting in place a few more content channels: Targeting the content in our organization’s seven private Google discussion groups, and a program-wide canvass for select hard-copy training resource materials for digital conversion and addition to the shared repository. It’s all good.

We have had one major disappointment: We discovered that there are significant, unanticipated technical challenges unique to the Pika CMS that thus far have prevented effective use of the GSA to target Pika content. The problem is not the GSA itself or configuring the GSA to target Pika. The GSA by design performs wholly benign, non-destructive crawls as it indexes targeted records. We did a huge amount of target testing and SERP evaluation, and we were very pleased — actually, thrilled is a better word — with the results we were getting from Pika. The unanticipated problem is that the current version of Pika is not well optimized for use as an enterprise search target. There are code anomalies in Pika that, among other things, cause it to auto-generate new case intakes and case records when it is crawled by the GSA.

After an assessment by Pika Software, it is now apparent it will take something in the neighborhood of 200 hours of work to make Pika more receptive, shall we say, to an external crawl by the GSA. (Ouch!) So for now, we have to put that part of the project to the side. File under: Lessons Learned.

Other posts of possible interest...

• Selecting GSA targets – Part Two: The Practical Realities
• Revised: What the LSNC Shared Portal now looks like
• TIG final evaluation report for The Findability Project
• Metadata Schmetadata, Relevance and Reality

The RTR will be our way to recognize and promote among all our staff the changes in how documents and other files are made easily and intuitively findable, and given a new level of access and usability throughout our non-profit organization. After all, that is the core purpose of enterprise search. And a key element of all this is changing deeply rooted individual notions or assumptions about what can or should be “shareable.”

In working on this project within a non-profit environment, we have learned that most employees have an inclination to undershare, not overshare. Not because they are selfish or secretive; rather, because the type of transparent sharing that enterprise search makes possible is foreign to most of them. It is familiar to them to be asked to provide a document to others on request in person, by phone or by email. It is foreign to them to decide in advance that a document they created or have received from someone else should be transparent to the rest of the entire organization. The concepts of creation and possession are severed from the concept of findability.

To be sure, the increasing use of collaborative web-based document tools within our organization — principally our adoption two years ago of Google Apps — has helped us on this journey. Most staff at this point are familiar with the concept, if not the practice in their individual work, of creating or editing or uploading documents that can be “shared” from a common web location. They get that, even if they don’t do it themselves, because increasingly others demand they do so… when they get a “share” message email from Google Docs about a document someone created or edited there; when they get an email with a link to something someone else posted in our domain’s Google Sites; or when they get a message to fill out a Google Docs form for, well, whatever.

As we prepare for the RTR, the team working on this project have brainstormed about what we can say or demonstrate to the staff in each office, to prompt them to rethink (OK, in some cases just think) what types of documents should be shared with others by adding them to the new document repositories.

We now refer to this as “presumptive shareability.” In particular situations, it may not be appropriate to make the document or file transparent through enterprise search, but in most cases it will be because all are situations where the document or file has served a shareable purpose, i.e, use by more than one person or re-use by one or more persons.

Among the situations we think should trigger staff to think to add the document or file in question to the shared repository are the following:

• An attachment to an email message you send or forward to someone else.
• You request or receive a file as an email attachment from someone within the organization.
• You receive a non-confidential file attachment from someone outside the organization.
• Every time you re-use a document or form as part of your work.
• You learn that the PowerPoint (or other presentation format) for a training or conference event you attended is now available for viewing or downloading.
• You lug home substantive hard-copy handouts distributed from a training or conference.
• Can you say, “presentation” and/or “portable”? Whatever it is, if it is a PDF or PPT file it is presumptively shareable.
• If it is the “final” version of a case-related pleading, memorandum, exhibit or correspondence and you think others may find it usable, share it.
• Usable documents you discover and think to save to your desktop as part of research on the Web, regardless of file type (PDF, DOC, XLS, etc.)
• Similarly, when doing work-related research on the Web, anytime you think to bookmark a web page or save the page to your desktop as an HTML or TXT file.
• … you get the drift.

Shareability promotes findability. That’s our story and we’re stickin’ to it.

Other posts of possible interest...

• The Findability Project Taxonomy – Part Two: The Practice
• TFP Taxonomy – Part Four: Revisions to the project's structural taxonomy
• The search box as a findability (design) concept
• Selecting GSA targets – Part Two: The Practical Realities

As far as it goes, this abstract list of GSA target sets fairly summarizes what we, as an organization, want to make transparent via enterprise search, which is to say make “findable” in ways not practicable without the GSA. This abstract list of GSA targets, however, fails to convey what we have done at non-abstract, practical level to make those targets useful to our larger search goals.

So, let me hit a few notes about several practical decisions we’ve made at launch as we target the GSA at real files offering real search results.

As described in Part One, when we first unpacked our GSA and aimed it, uh, somewhat aimlessly at any and every file on one of our local servers, the GSA did its job in killer fashion… and blew out our file limit. While one can proceed that way, we were always mindful that we had to sort out how to organize and structure the shared content that we seriously wanted to make searchable and findable. So, one of the first tasks we confronted on this project was to work out our thinking about “taxonomy,” resulting in the basic directory structures we have adopted.

That taxonomic “organization” step was essential to this project, but completing that particular project objective doesn’t translate directly into searchable content organized in a particular way. You see, there is this pesky little detail: Real people need to actually identify the existing and/or newly created files to be included and then somehow get the files in the directories on the shared document repository that are the target of the GSA.

Easier said than done.

In our case — particularly given the limited IT and support staff resources available to us as a typical legal services field program — we had to come up with some practical approaches to move existing files from any number of different locations to the designated shared locations or document repositories. (I will discuss how we handle adding newly created files, in a later post.) Here’s what we did with our existing files to fold them into the content targeted by our GSA:

1. Initially, include all existing “staff-specific” content, with an opt-out

We did find ourselves on the receiving end of a lot of staff enthusiasm about this project. Truly. But it is impractical and unrealistic to expect your individual legal services advocates and other staff to comb through all their thousands of files and then move them over to a different file server location. (Maybe it should be realistic to expect them to do this, but in our experience it just ain’t gonna happen. No way.) But there are tons of content gold in them thar files, so we had to figure out a way to initially get all that good stuff in place, even if not parsed out in a taxonomic sense, so we could target it.

To accomplish this, we first vetted with, and got buy-in from, all our local offices to do the following:

On the local project file server for each local office, we created a special project “archive” directory. Then each local office manager copied each individual staff member’s files wholesale over to a user-specific directory in this so-called archive directory. Having an unequivocal “opt-out” option was important to the success of this approach. Again and again, in formal meetings and informal discussions, we reminded office staff that they could ask that any or all files to be removed from these initial archive-file targets. No questions asked. There were a few such requests, but not a lot: One advocate asked that her files not be targeted at all, so we removed all her files; two others had less than a dozen files they wanted removed as targets, so we did so. No biggie.

The net effect is that this makes the targeted advocate files initially non-taxonomic, but in short order you have a huge repository that has a (allow me to exaggerate here, for literary effect) 99% chance of including pretty much everything the individual staff members would add if they “woulda, coulda, shoulda,” so to speak. In our case, this initially amounts to about 300,000 document files, the vast majority of which are advocate-generated files.

At launch, this does mean that these office-specific, bulk compilations of existing files added as targets include a significant number of drafts and duplicates that one would normally not include as a shared file if it were being added as a newly created file. For example, within our office culture it is not only common but actually expected that advocates not work in isolation on major cases. (We discourage the “lone eagle” model.) So, our early search-results testing shows that often the same file shows up in more than one target location because more than one advocate has a copy of the file in their archive.

It bears mentioning two other factors we kept in mind as part of this initial targeting of shared files: We double- and triple-checked with all management staff to assure nothing management-sensitive or -confidential was moved to a location where it could be targeted. Also, before we moved anything over wholesale, as described, we asked all staff to remove certain types of files that no one would reasonably expect to be part of the searchable content. Examples: Family photos, MP3 music downloads, YouTube videos, yada yada yada. Enough said.

We do have an approach in mind for “peeling off” these office-specific archives over time, to separate out the drafts and duplicates and place them within our taxonomic directory structures. More about that later.

2. Using Google Sites as the platform for our existing intranet content

I recall having a passing conversation with Gabrielle Hammond at last year’s TIG conference about how we were holding off on further intranet development while waiting to see how Google implements its JotSpot-based wiki application, now known as Google Sites.

Well, people, we now know what Google Sites is all about and we love it! For the last several years we had been using MediaWiki as the publication platform for our intranet, but we are in the process of replacing it for our internal wiki needs. We are about half way through that process, which should be completed shortly after the first of the year.

One big bonus of moving our intranet content to Google Sites is that it is quasi-tailor made to work with both the GSA and Google Analytics. I say “quasi” because the interactions between them are good but hardly optimal at the moment. For example, only days ago Google Analytics for Google Apps was rolled out, but the quality of the data we are getting so far is not so easy to get a handle on. More importantly, Google promises GSA integration with Google Sites, but it is still a buggy implementation. We have easily targeted test site pages within our domain’s Google Sites, but have hit a wall with getting the GSA to properly return search results on the indexed content within files uploaded to Google Sites. Turns out we are one of several organizations that have identified this problem and Google Enterprise support assures it will have a fix with its next software upgrade, in about a month or two. We (and our GSA consultant) are confident this will work in due time, but it’s one of those details we have to wait on for now.

3. Updating our public web content

Over the last 10 years, LSNC has placed an enormous amount of its advocate content out on the public Web. But one recent example is the California Food Stamp Guide, a prime example of public content that our advocates can search at that site, but would want to be able to search directly via our GSA shared portal. It is also one example of a content cluster that can be part of or its own GSA collection. (“Collections are logical views of information in the index, as defined by URL patterns. This allows you, for example, to index the entire contents of your intranet, but then divide it up into logical groups of content.”)

Implementation of The Findability Project has prompted some public housekeeping. Our target testing of our public content, predictably, reveals that we have stuff out there that is, well… past its shelf-life, shall we say. So we are working on a systematic way to thoroughly review and clean up that public content. It is obvious but important: Current and correct public content means better search results via the GSA. (Apologies to the larger legal services community for not doing it sooner.)

4. Targeting our case management system

We consider our Pika case management system a key, long-term GSA target. But we are not there yet. We have prioritized getting all the other targeted content organized and in position, with clear protocols in place. We also are busy reworking on our shared portal, which will integrate the GSA search functions and provide users with (hopefully) intuitive ways to filter their search results, search select content collections, and provide the users with some nice Google GSA touches like OneBox searches, among other features.

That all said, being able to target our case management system is a total no-brainer and perhaps the most practical of necessities. In a given day, there is likely nothing more common or more vital to our work for clients than the search for information within our case management system. The native search functions built into the current version 3.07 of Pika are good. But we are optimistic that we can exploit the GSA to make those searches even better. And certainly more integrated with everything else in our new enterprise search universe.

Other posts of possible interest...

• Revised: What the LSNC Shared Portal now looks like
• Metadata Schmetadata, Relevance and Reality
• Selecting GSA targets – Part Three: Quantification, Revision and Finalization
• Selecting GSA targets – Part One: Four abstract targets

• Download: Best practices and protocols for GSA Documents
• Download: Organization of advocate-user directories

As I so often like to say, allow me to explain:

In making practical decisions about handling files targeted by our Google Search Appliance (GSA), we look both backward and forward in time. This dichotomy between the past and the future is one that Google Enterprise itself promotes with its cursory recommendations that its customers decide for themselves where to locate existing content and new content.

Based on our experience working on this project, there are considerable differences in how to handle the “past.” A separate post on those issues will be coming forth, soon. But going forward into the “future,” we have thrashed out the practices and protocols detailed in the two memos linked, above. While there are institutional contexts for some things described in the memos that may be lost on those not part of our organization, the memos are (hopefully) self-explanatory. There are other practical details about the document protocols that will be expanded on in later posts, including how the Shared Repository web interface works, the integration of our metadata models, and so on. (All good things come to those who wait, at least with this project.)

Among the most practical observations in these memos, I think, is breaking through the common but incorrect perception that one needs to save a document to “the” correct directory, as opposed to “a” correct directory, however it is done. And while staff are instintively bewildered, somewhat, by concepts of “taxonomy” and “metadata” and wonder how they will be able to find things if they are not located in “the” correct directory, it is also extraordinarily reassuring to them to know that we’re talking about Google here. Even if they do not understand how it all works, typically they have great faith that Google search will find it for them, as described in an earlier ancecdote.

The memos also attempt to address some of the practical realities and limits of a non-profit, legal services work environment. LSNC has neither the resources nor motivation to micro-manage how users organize their own file directories. Life is too short. But as detailed in the “advocate-user directories” memo, we do now require all LSNC staff to have a user-specific, user-named directory, and that the name used be the user’s full name. The primary motivation for this requirement is a practical need to standardize directory-name conventions throughout the organization, so that the location and targeting of files is predictable, manageable and findable. And, if for no other reason, doing so eliminates the need to guess whether something is located in the Shareen, Shari, Shelly or Sherri directory — a real-world example from our Auburn office, illustrated to the right.

Other posts of possible interest...

• Presumptive Shareability
• Selecting GSA targets – Part Two: The Practical Realities
• Don't overthink file naming conventions
• The Findability Project Taxonomy – Part Two: The Practice

• Download: File-naming “best practices”

If you take a look at the files in your advocate staff directories, you are likely to see individualized albeit typical patterns in the file names. There is a discernible Darwinism to the conventions individual users adopt: They use both directory structures and name files in a way that makes them “findable” for them later, if not for others. (OK, basically “unfindable” by others, in a lot of instances.) Common naming patterns include, in almost all instances, at least a generic name descriptive of the type of document (e.g., petition or complaint or writ), plus other descriptive elements that help the user to later locate it, such as a client or project name, the date of the document, and/or whether the file is a draft or a final or a version copy.

There are any number of ways one can go with file naming conventions, as well illustrated in the article at CompuJurist, Are there any recognized “best” practices for file naming conventions? Akin to what is discussed in that article, we’ve adopted the following template for use by advocates to name their files:

[draft/final]  [document type]  [party/case]  [subject]  [date] [file extension]

Yes, there are other longstanding concerns about how files are named, beyond what may concern your attorneys and other advocates. These concerns include naming conventions driven by the demands of particular operating systems, all too often non-intuitive but technical project requirements, or the recommendations of 800-pound gorillas like Google (explaining why Google favors dashes over underscores).

All that said, if you look at the memo linked above, you may notice that the examples promote the use of underscores, rather than spaces, between words in the filenames.

Our thinking is this:

• As a practical matter, using spaces between words in file names creates file transfer problems when moving the file from one server to the other. Not a good thing. Especially when you are dealing with relocating files that count in the hundreds of thousands.
• Using spaces creates readability problems when viewing the path of the file in a GSA search result, because the GSA normalizes the URL by inserting special characters wherever the file has a space in its name. Even if you didn’t know what to call it, you’ve seen this phenomenon. Here’s a real world example in a GSA search result from one of our test bed sites:
  
  /Ukiah%20Office/Former%20Staff/Kan's%20Transfer%20File/letter%20to%20jake.wpd
  
  Look familiar?
• File names with underscores are easier to read than files with dashes. They just are, OK? To be fair, not everyone is going to agree with that proposition. But we did some admittedly unscientific user testing (hey, Glenn, you get what you pay for), where we asked staff to read the same file name listed three ways: With spaces, with underscores, and with dashes. Without exception, our crack team of testers said they found it easiest to read the file names if they had spaces (duh!); less easy to read if there were underscores; and least easy where dashes were used.
• A usability corollary: If you use underscores, a linked file name in a search result is easier to read because as a link the file name appears underlined, so words appear as if they have spaces, which is the easiest of the three formats to read (see test results, above).

This is fairly prosaic stuff and bears some thought, but is not worth overthinking. Or much of an enforcement regime. The project goal is not to have nice, neat, compliant file names. That’s an objective. It is not the goal. We are not investing a lot of time worrying about those who paint outside the lines. The point of the project is to get the files targeted properly so that users can find the content they contain.

We are confident that, as users see file names displayed in the GSA search results, it will sink in why it matters how one names the files, and they will adapt.

Other posts of possible interest...

• Presumptive Shareability
• Selecting GSA targets – Part Two: The Practical Realities
• Going Forward: Document “best practices” and protocols
• The Findability Project Taxonomy – Part Two: The Practice

I’ll just give you the end-product upfront and then explain how LSNC sorted out the basic taxonomic structures for its shared document repository. The two PDF files linked below are copies of what was distributed at a program-wide meeting a few weeks ago to address and resolve what the basic organization structures would look like.

• Download: 29 Top-level Project Directories
• Download: Simplified Project Taxonomy

It was actually quite easy to come up with an initial (if bloated) proposed list of likely substantive advocacy content targets, their location, and how the content would be organized, but even that required process.

LSNC has what it calls a “regional counsel” model, which means there are three designated advocacy leaders with senior substantive, litigation and advocacy experience who are expected to provide just that, “leadership.” (One of the three, by the way, is Mona Tawatao who is the recipient of the 2007 NLADA Reginald Heber Smith award.) The regional counsel, with feedback from other management leadership (including the executive director, a few local office managing attorneys interested in this particular project detail, and the senior office manager representing support staff interests) worked up the list, later vetted more broadly with the entire management team, who in turn vetted it with each of their local offices or other program unit.

In the initial proposal, the substantive advocacy content was organized based on the ten LSC Problem categories in current use by legal services programs, plus roughly an additional 30 or so other general categories. The latter included additional substantive categories (economic development, disaster relief, etc.), practice matters (e.g., federal and state court practice issues, discovery, etc.), and other work-related content (self-help clinic content, specialized training materials, etc.) that reflect what LSNC and other legal services field programs actually do for a living. In response to any number of discussions and comments by the smaller group thrashing the details out, the list at times expanded and contracted, went deeper and then sometimes more shallow. This initial organization structure also included targeted content related to local office and central administrative office work. A similar vetting process was undertaken by the senior office manager with all the other office managers in all the core local offices, as well as administrative and business office managers. As mentioned earlier, each of those, in turn, were asked to vet the structures with their respective staff.

This process did not operate in a project vacuum. As not only one of the three regional counsel but also the person responsible for managing this project, I also did what I think managers should always do: I talk to the people affected. I took the time, a lot of it, to speak directly and individually with all of the forgoing to explain the overall project and its technical demands, and in a non-technical fashion (well, at least I tried) the significance of developing an organizational structure, and other, related issues, such as the use metadata models to attribute value to the targeted content, and so on. The point being, to take the time to assure leadership understood from more than a memo what the project is about, why it matters, and answer their questions or concerns. In response to the vetting and these dialogs, real changes were made in the proposed organization and additional content targets were identified. Time investments paid dividends, at least in this case.

By the time our GSA consultant showed up for a scheduled three-day thrashing of our test-bed installation in Sacramento, we had a taxonomy with over 40 top-level directories and a lot of two- and three-level deep subdirectories. He looked at this, in a non-committal fashion said “that’s fine,” and then began to suggest reasons why it should be simplified. This push by the GSA consultant was prompted by notions of usability and manageability of the content areas. As mentioned in the prior post on project taxonomy, there are not significant advantages or improvements to search results in a repository structure beyond a second-level directory. The consultant also emphasized that most users are not likely to locate or use a directory substructure below the second level. (This has to do with users navigating directory structures to add, remove or modify files, for whatever reason.)

Since a significant portion of the metadata models we are adopting rely on the organization structures in order to build logical, searchable “collections,” we simplified the structures in response to the consultant’s recommendation in this regard. Hence, the 29 top-level directories and the “simplified” taxonomy you can see in the memos linked at the beginning of this post, and the reliance on only one-level deeper for those directories.

As we get life experience with this organization structure, my guess is that we may expand to add a few additional top-level directories but not many, if any. I think we have things pretty much covered at the top-level, at this point. But apart from the rigid yet practical exploitation of the dated — but undeniably familiar — LSC Problem Codes for a large chunk of the substantive organization, my guess also is that the one-level down subdirectory structures will likely change as users give us feedback, and we discover that some subdirectories are not particularly used or useful. Proof’s in the pudding, people.

This all came full circle with our program-wide meeting a few weeks back. By the time of that meeting, every manager within LSNC had seen the organization proposals, every manager had a one-on-one conversation with project staff about the project and the organization structure, every manager had vetted the proposal to his or her people, and the memos you see linked here had been distributed to all offices.

That’s how we roll.

Other posts of possible interest...

• TFP Taxonomy – Part Four: Revisions to the project's structural taxonomy
• Presumptive Shareability
• The Findability Project Taxonomy – Part One: The Theory
• Metadata Schmetadata, Relevance and Reality