Google and the death of Serendipity

Serendipity : the faculty or phenomenon of finding valuable or agreeable things not sought for; http://www.merriam-webster.com/dictionary/serendipity

I've been thinking about serendipity. Mostly, about whether serendipity can survive the age of search engines. Don't get me wrong - there is power in being able to find something when you seek it. There is nothing more frustrating than knowing something is out there and not being able to find it. But how do you find something you don't know exists?

At least since the practice of open stacks became available, individuals have been able to randomly walk into the stacks and pull any book off the shelf. For past generations this was allowed them to learn beyond a specialize domain. In some sense, it created the continual learning associated with being educated. You could find almost anything and potentially draw influence from something unrelated.

Paper - because it is not easily queried to the word, lends itself to serendipity. A book is pulled from a shelf cracked and assessed - a magazine's contents are scanned. Something unexpected, but interesting enters into an unintended audience with unintended wonderful consequences: something new is created. Some new perspective is gained. Innovation takes place.

How do you preserve serendipity when content is recorded in bits? Web surfing - the now dead practice of following links between sites, is almost extinct. With the rise of search engines it is far easier to look for something and trust the search engine tuners.

It is inevitable; we are moving from the bulk of paper to the access of electrons. The mass of information being produced would overwhelm any attempt to store it in paper. We have more available, but it is better? Are people able to learn from the random? Are there still meaningful joyous surprises happening? How do you preserve serendipity?

Managing Documents

Three things have come together in the last six weeks that have reminded me of the importance of document management:

• I ran a mini-financial audit for a non-profit to prepare for a more formal audit.
• I prepared my personal income taxes.
• I reviewed database usage to assessing which subscriptions to renew.

Each is in a different knowledge domain. Each has multiple dimensions that need to be considered. Each requires deciding when to keep records. 

The first two cases are relatively easy. Financial rules are fairly well defined by accounting principles and various laws and regulations. It is easy to sit down with an accountant, bookkeeper, or lawyer and review the record keeping requirements. In the case of the audit, I was looking for a list of defined records that were need to back specific transactions. The organization has guidelines for the types of receipts and other documents that need to be kept and how to handle exceptions. The organization has guidelines for how long records need to be kept. Similarly, with taxes there are regulations for how long you need to keep records based on how long the IRS or state taxing authority can audit for supporting documents - I am certain the organizational guidelines are governed by these regulations. The database subscriptions are a bit more tricky. There are no clear regulations. Instead there is an organizational need to support organizational goals. In this case, the goals are teaching and research. 

I'm trying to put together questions, I can use to decide when to keep and when to discard documents in a collection. This is my tentative list, and I would think in any organization could create a similar list with the help of financial, IT, and legal departments.

1. Will the documents be used? Are the documents being used?
2. How much does it cost to create and retain documents?
3. Is a document or collection available from somewhere else? What will it cost me to get it?
4. What does it cost to discard or discontinue a document?
5. How long should a document be kept? How long will it have value?

Keep in mind cost in my list is not necessarily just a financial cost, you might also need to consider good-will, convenience, or risk as costs. Is there a better way to think about and manage information?

Knowledge Management vs. Information Technology

Managing data is easily accomplished with basic information technology like databases. This sometimes creates the illusion that we can manage very large amounts of data with little cost. Managing to turn data into information, by giving it a meaningful context, is becoming an area of intense research. Some organizations try to manage so called "big data".₁ Big data requires specialized tools and expertise. Often an organization needs two or three highly specialized and skilled individuals to create information out of data. Comparatively, giving data a context to turn it into information is child's play when compared to the task of being able to actually use the information and create knowledge. Managing knowledge is virtually impossible. Because knowledge is created by the individual, directly managing knowledge requires structuring every interaction between every individual in an organization and the organization's information. This is best called by its common name, micro-management. Micro-management is not knowledge management. I define knowledge management as

Creating an information environment where

• Individuals have access to the information they need to create knowledge 
• Individuals record information that is likely to be needed by the organization
• Information is viewed as a valuable asset by the organization

From my definition knowledge management encompasses aspects that can not be addressed by information technology alone. Knowledge management encompasses Information Management and Document Management. It has an important role to play in an information intense environment. Knowledge management professionals need to consult and work closely with other key departments in an organization, such as Information Technology, Marketing, Legal, and Financial departments.

There are several strategies for creating such an information environment. I would like to examine some strategies for creating this kind of information environment in future posts.

¹ "Big Data" is any unstructured dataset that requires special tools and insight to make meaningful. The term is deliberately vague and depends on the organizations ability to use the data.

A Manifesto for Library School Graduates

Some have suggested Librarian is a dying profession. Some Libraries are cutting back on professional staff. Some organizations are eliminating Librarian positions and hope to replace these positions with sophisticated software tools. I believe these organizations are being foolish and short sighted. I believe Library School Graduates have key skills and tools that should be valuable assets for any organization that uses information.

Library School Graduates face serious professional challenges. There is a trend to see software as tools that can replace knowledge workers¹ in much the same way machines replaced industrial workers. For example, Google is seen by some as a replacement for reference services. Developers are working furiously to create software that can query large bodies of unstructured data to replace indexing. And social media is seen by some as a way to replace collection development by providing relevance. Corporate librarians are often seen as expensive professionals that do not contribute to the bottom line. They are not creating services used by direct customers and so are seen as a cost. Knowledge management appears to be evolving into an technology management role.

If we were to only look at these challenges the out look is grim. However, there are reasons Library School Graduates should be hopeful. The amount of information being created is increasing geometrically. Some people are starting to realize the amount of information that can be retained is greater than the organization ability to use it. Information technology is often taxed by the expectations to manage large amounts of data. I believe some organizations are coming to the realization that information technology (IT) can not fill the gap alone and are looking to various professionals to help them manage their information.

I believe Library School Graduates have an important opportunity in fill this gap:

1. Librarians have, as a profession, managed information for hundreds of years. Librarians create processes to successfully access information. This experience and training is still relevant and compliments the tools being created by IT. Library tools like indexing, controlled vocabularies, and facets have a role to play in solving information challenges face by many organizations. Often the skills need to use these tools effectively need to be taught to the users of IT tools. Library School Graduates could  have a part in filling this role.
2. Information literacy is becoming an indispensable skill. Librarians have taken on the challenge of teaching this skill. As organizations rely more heavily on  knowledge workers it is imperative each knowledge worker has this skill within their domain of expertise. Library School Graduates should be leading the effort to train and mentor these skills.
3. Librarians have experience and framing that brings a different context for understanding processes. Developing effective information processes is a difficult complex task. There are deep social, political and legal dimensions that must be tackled by any organization. Solutions require information technology, financial, and process components. Good solutions require the expertise of financial experts, software developers, legal experts, and information experts. Library School graduates should be actively participating in solving these challenges.

I believe the Library profession should be growing. I believe there are specific steps we should be taking:

1. Expand the focus of the job titles we include in the profession. Naomi House maintains a list of keywords for finding library jobs. The list includes a wide variety of key words including Social Media, UX Designer, and Data Management Analyst.² I would also encourage people to look at Technical Writing and Business Analyst positions for requirements that match library skills.
2. Learn the technical languages of other professionals. Almost every organization has its own unique technical language. Being able to communicate effectively to a wide variety of people in terms is imperative.  With this ability you can make the case that your contribution is valuable.
   
   I first became aware of this when listening to a seminar by Dr. Juran. Dr. Juran was speaking to Quality Engineers about the need to talk to corporate leaders in financial terms so their work would be appreciated. Library graduates need to do the same thing. Learn a little about finances so you can speak to financial professionals. Learn a little about programming so you can speak to software professionals. Take time to understand copyright and intellectual property issues so you can speak with legal professionals.
3. Learn basic statistics. Statistics are hard. I'm not advocating becoming a statistician. I am advocating getting enough of an understanding of statistics that you are "statistically literate". You should understand and rate any graph or number.  You should know the right questions to ask to rate the value of the information being presented. Think of this skill as statistical information literacy. This skill is invaluable in most organizations and help keep everyone honest.

By taking these kinds of steps, I believe librarians could become invaluable partners in education, government and business. I believe library school graduates have much to offer and will continue to make meaningful contributions for years to come. If the profession of "librarian" becomes irrelevant we will only have ourselves to blame.

¹Peter Ducker coined the term "knowledge worker" to denote professionals who functioned independently and created value by using their expertise with an organizations information.
  ²Naomi House INALJ (I Need a Library Job) http://inalj.com

The Best Job Search Engine

Searching for appropriate job postings is like searching for a needle in a pile of steel shavings. All the obvious ways to find the needle won't work. Specialized job search engines purport to help find appropriate postings for job seekers. Generally, search engines measure success by the number of results returned. In my experiences the results are largely not good matches for the individual.

Job search engines have some serious challenges:

• Job descriptions and requirements do not have standard formats and vocabulary.
• Job titles are different across companies and industries for comparable skills and functions.
• Industry is key for some job functions and totally irrelevant for others. (For example, do you really care if your payroll expert has construction experience, or are you satisfied that they have payroll experience working with salaried, hourly, and seasonal employees?)

Success for the job seeker is finding jobs that match their particular skills and interests regardless of the challenges the search engines present. Based on my ability to do that with the search engines the best search engines are:

1. Dice.com - Dice has three strengths that made it outstanding during my job search:
   
   
   • A checkbox to restrict the search to job title only
   • The ability to quickly restrict to a region
   • The ability to filter out old jobs
   
   These two strengths let me zero in quickly on a half dozen jobs that I was qualified for and interested in. I may have missed jobs that would have been acceptable, but the jobs I was looking at were always dead on. This is much better than wading through 60 or 70 "matches" that weren't even close, to find one might be close.
   
   I also had a much higher percentage of interviews stemming from postings in Dice than in other job boards/search engines.
   
2. USAjobs.gov, Governmentjobs.com, Virginia Jobs - the government websites have two things to recommend them:
   
   
   • Relevant jobs appeared at the top of the list
   • It is easy in all of them to restrict the region and government agency
   
   The government websites still returned too many irrelevant results and sometimes it is difficult to separate the jobs that are restricted to internal people only.

I'm still looking for a job engine that will let you eliminate jobs that require a security clearance by the level and type of clearance. I'm also still looking for standardized job titles, or some way to quickly distinguish between to different jobs that have the same title for example,"business analyst" - a business analyst seems to be either someone with accounting knowledge who determines the health of the business, or someone who writes requirements for software based on a businesses needs and requests. Few people could do both. I would also like to see job search engines drop the notion of "industry". Most jobs are industry independent.

So, next time I'm looking for a job, I'll pass on most search engines. I'll use the few that work for me. I will sill check specialized job lists. 

Change, Repent, or Progress?

I have a problem with the word "evolution". The way it is being used in the popular science press is a bit disconcerting. It can be used to mean "change", "progress", "adaption", or in the worst cases "God" or "Providence". It can be fun to take an author's words and start substituting potential synonyms. However, this doesn't tell us what the author intended.

Here is the proposal - the popular press stops using the word "evolution" and all of its derivatives like "evolve" and "evolutionary". Instead, I challenge authors to use the word they mean. Substitute progress, adapt, change, or repent. Choose the word that makes the most sense.

Day 100!

William did it! 100 days straight playing the violin. And he had a great recital today as well! Not bad for someone who just turned 8... Woohoo!