Skip to main content Accessibility help
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Knowledge Management, Databases and Data Mining
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

1911 results in Knowledge Management, Databases and Data Mining

Page 91 of 96

  • Preface

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp xv-xxii

  • Summary

    As recently as the 1990s, studies showed that most people preferred getting information from other people rather than from information retrieval (IR) systems. Of course, in that time period, most people also used human travel agents to book their travel. However, during the last decade, relentless optimization of information retrieval effectiveness has driven web search engines to new quality levels at which most people are satisfied most of the time, and web search has become a standard and often preferred source of information finding. For example, the 2004 Pew Internet Survey (Fallows 2004) found that “92% of Internet users say the Internet is a good place to go for getting everyday information.” To the surprise of many, the field of information retrieval has moved from being a primarily academic discipline to being the basis underlying most people's preferred means of information access. This book presents the scientific underpinnings of this field, at a level accessible to graduate students as well as advanced undergraduates.

    Information retrieval did not begin with the Web. In response to various challenges of providing information access, the field of IR evolved to give principled approaches to searching various forms of content. The field began with scientific publications and library records but soon spread to other forms of content, particularly those of information professionals, such as journalists, lawyers, and doctors.

  • 17 - Hierarchical clustering

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 346-368

  • Summary

    Flat clustering is efficient and conceptually simple, but as we saw in Chapter 16 it has a number of drawbacks. The algorithms introduced in Chapter 16 return a flat unstructured set of clusters, require a prespecified number of clusters as input and are nondeterministic. Hierarchical clustering (or hierarchic clustering) outputs a hierarchy, a structure that is more informative than the unstructured set of clusters returned by flat clustering. Hierarchical clustering does not require us to prespecify the number of clusters and most hierarchical algorithms that have been used in information retrieval (IR) are deterministic. These advantages of hierarchical clustering come at the cost of lower efficiency. The most common hierarchical clustering algorithms have a complexity that is at least quadratic in the number of documents compared to the linear complexity of K-means and EM (cf. Section 16.4, page 335).

    This chapter first introduces agglomerative hierarchical clustering (Section 17.1) and presents four different agglomerative algorithms, in Sections 17.2 through 17.4, which differ in the similarity measures they employ: single-link, complete-link, group-average, and centroid similarity. We then discuss the optimality conditions of hierarchical clustering in Section 17.5. Section 17.6 introduces top-down (or divisive) hierarchical clustering. Section 17.7 looks at labeling clusters automatically, a problem that must be solved whenever humans interact with the output of clustering. We discuss implementation issues in Section 17.8. Section 17.9 provides pointers to further reading, including references to soft hierarchical clustering, which we do not cover in this book.

  • 5 - Index compression

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 78-99

  • Summary

    Chapter 1 introduced the dictionary and the inverted index as the central data structures in information retrieval (IR). In this chapter, we employ a number of compression techniques for dictionary and inverted index that are essential for efficient IR systems.

    One benefit of compression is immediately clear. We need less disk space. As we will see, compression ratios of 1:4 are easy to achieve, potentially cutting the cost of storing the index by 75%.

    There are two more subtle benefits of compression. The first is increased use of caching. Search systems use some parts of the dictionary and the index much more than others. For example, if we cache the postings list of a frequently used query term t, then the computations necessary for responding to the one-term query t can be entirely done in memory. With compression, we can fit a lot more information into main memory. Instead of having to expend a disk seek when processing a query with t, we instead access its postings list in memory and decompress it. As we will see below, there are simple and efficient decompression methods, so that the penalty of having to decompress the postings list is small. As a result, we are able to decrease the response time of the IR system substantially. Because memory is a more expensive resource than disk space, increased speed owing to caching – rather than decreased space requirements – is often the prime motivator for compression.

  • 4 - Index construction

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 61-77

  • Summary

    INDEXING INDEXER

    In this chapter, we look at how to construct an inverted index. We call this process index construction or indexing; the process or machine that performs it the indexer. The design of indexing algorithms is governed by hardware constraints. We therefore begin this chapter with a review of the basics of computer hardware that are relevant for indexing. We then introduce blocked sort-based indexing (Section 4.2), an efficient single-machine algorithm designed for static collections that can be viewed as a more scalable version of the basic sort-based indexing algorithm we introduced in Chapter 1. Section 4.3 describes single-pass in-memory indexing, an algorithm that has even better scaling properties because it does not hold the vocabulary in memory. For very large collections like the web, indexing has to be distributed over computer clusters with hundreds or thousands of machines. We discuss this in Section 4.4. Collections with frequent changes require dynamic indexing introduced in Section 4.5 so that changes in the collection are immediately reflected in the index. Finally, we cover some complicating issues that can arise in indexing – such as security and indexes for ranked retrieval – in Section 4.6.

    Index construction interacts with several topics covered in other chapters. The indexer needs raw text, but documents are encoded in many ways (see Chapter 2). Indexers compress and decompress intermediate files and the final index (see Chapter 5).

  • 9 - Relevance feedback and query expansion

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 162-177

  • Summary

    SYNONYMY

    In most collections, the same concept may be referred to using different words. This issue, known as synonymy, has an impact on the recall of most information retrieval (IR) systems. For example, you would want a search for aircraft to match plane (but only for references to an airplane, not a woodworking plane), and for a search on thermodynamics to match references to heat in appropriate discussions. Users often attempt to address this problem themselves by manually refining a query, as was discussed in Section 1.4; in this chapter, we discuss ways in which a system can help with query refinement, either fully automatically or with the user in the loop.

    The methods for tackling this problem split into two major classes: global methods and local methods. Global methods are techniques for expanding or reformulating query terms independent of the query and results returned from it, so that changes in the query wording will cause the new query to match other semantically similar terms. Global methods include:

  • Query expansion/reformulation with a thesaurus or WordNet (Section 9.2.2)

  • Query expansion via automatic thesaurus generation (Section 9.2.3)

  • Techniques like spelling correction (discussed in Chapter 3)

    • Local methods adjust a query relative to the documents that initially appear to match the query. The basic methods here are:

  • Relevance feedback (Section 9.1)

  • Pseudorelevance feedback, also known as blind relevance feedback (Section 9.1.6)

  • (Global) Indirect relevance feedback (Section 9.1.7)

  • 21 - Link analysis

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 421-440

  • Summary

    The analysis of hyperlinks and the graph structure of the Web has been instrumental in the development of web search. In this chapter, we focus on the use of hyperlinks for ranking web search results. Such link analysis is one of many factors considered by web search engines in computing a composite score for a web page on any given query. We begin by reviewing some basics of the Web as a graph in Section 21.1, then proceed to the technical development of the elements of link analysis for ranking.

    Link analysis for web search has intellectual antecedents in the field of citation analysis, aspects of which overlap with an area known as bibliometrics. These disciplines seek to quantify the influence of scholarly articles by analyzing the pattern of citations among them. Much as citations represent the conferral of authority from a scholarly article to others, link analysis on the Web treats hyperlinks from a web page to another as a conferral of authority. Clearly, not every citation or hyperlink implies such authority conferral; for this reason, simply measuring the quality of a web page by the number of in-links (citations from other pages) is not robust enough. For instance, one may contrive to set up multiple web pages pointing to a target web page, with the intent of artificially boosting the latter's tally of in-links. This phenomenon is referred to as link spam.

  • 8 - Evaluation in information retrieval

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 139-161

  • Summary

    We have seen in the preceding chapters many alternatives in designing an information retrieval (IR) system. How do we know which of these techniques are effective in which applications? Should we use stop lists? Should we stem? Should we use inverse document frequency weighting? IR has developed as a highly empirical discipline, requiring careful and thorough evaluation to demonstrate the superior performance of novel techniques on representative document collections.

    In this chapter, we begin with a discussion of measuring the effectiveness of IR systems (Section 8.1) and the test collections that are most often used for this purpose (Section 8.2). We then present the straightforward notion of relevant and nonrelevant documents and the formal evaluation methodology that has been developed for evaluating unranked retrieval results (Section 8.3). This includes explaining the kinds of evaluation measures that are standardly used for document retrieval and related tasks like text classification and why they are appropriate. We then extend these notions and develop further measures for evaluating ranked retrieval results (Section 8.4) and discuss developing reliable and informative test collections (Section 8.5).

    We then step back to introduce the notion of user utility, and how it is approximated by the use of document relevance (Section 8.6). The key utility measure is user happiness. Speed of response and the size of the index are factors in user happiness.

  • 12 - Language models for information retrieval

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 218-233

  • Summary

    A common suggestion to users for coming up with good queries is to think of words that would likely appear in a relevant document, and to use those words as the query. The language modeling approach to information retrieval (IR) directly models that idea: A document is a good match to a query if the document model is likely to generate the query, which will in turn happen if the document contains the query words often. This approach thus provides a different realization of some of the basic ideas for document ranking which we saw in Section 6.2 (page 107). Instead of overtly modeling the probability P(R = 1|q, d) of relevance of a document d to a query q, as in the traditional probabilistic approach to IR (Chapter 11), the basic language modeling approach instead builds a probabilistic language model Md from each document d, and ranks documents based on the probability of the model generating the query: P(q|Md).

    In this chapter, we first introduce the concept of language models (Section 12.1) and then describe the basic and most commonly used language modeling approach to IR, the query likelihood model (Section 12.2). After some comparisons between the language modeling approach and other approaches to IR (Section 12.3), we finish by briefly describing various extensions to the language modeling approach (Section 12.4).

    Language models

    Finite automata and language models

    What do we mean by a document model generating a query?

  • 11 - Probabilistic information retrieval

  • Christopher D. Manning, Stanford University, California, Prabhakar Raghavan, Hinrich Schütze, Universität Stuttgart
  • Book:
    Introduction to Information Retrieval
    Published online:
    05 June 2012
    Print publication:
    07 July 2008, pp 201-217

  • Summary

    During the discussion of relevance feedback in Section 9.1.2, we observed that if we have some known relevant and nonrelevant documents, then we can straightforwardly start to estimate the probability of a term t appearing in a relevant document P(t|R = 1), and that this could be the basis of a classifier that decides whether documents are relevant or not. In this chapter, we more systematically introduce this probabilistic approach to information retrieval (IR), which provides a different formal basis for a retrieval model and results in different techniques for setting term weights.

    Users start with information needs, which they translate into query representations. Similarly, there are documents, which are converted into document representations (the latter differing at least by how text is tokenized, but perhaps containing fundamentally less information, as when a nonpositional index is used). Based on these two representations, a system tries to determine how well documents satisfy information needs. In the Boolean or vector space models of IR, matching is done in a formally defined but semantically imprecise calculus of index terms. Given only a query, an IR system has an uncertain understanding of the information need. Given the query and document representations, a system has an uncertain guess of whether a document has content relevant to the information need. Probability theory provides a principled foundation for such reasoning under uncertainty.

  • 2 - Basic Description Logics

  • Edited by Franz Baader, Technische Universität, Dresden, Diego Calvanese, Freie Universität Bozen, Bolzano, Deborah L. McGuinness, Rensselaer Polytechnic Institute, New York, Daniele Nardi, Università degli Studi di Roma 'La Sapienza', Italy, Peter F. Patel-Schneider , AT&T Bell Laboratories, New Jersey
  • Book:
    The Description Logic Handbook
    Published online:
    06 July 2010
    Print publication:
    09 August 2007, pp 47-104

  • Summary

    Abstract

    This chapter provides an introduction to Description Logics as a formal language for representing knowledge and reasoning about it. It first gives a short overview of the ideas underlying Description Logics. Then it introduces syntax and semantics, covering the basic constructors that are used in systems or have been introduced in the literature, and the way these constructors can be used to build knowledge bases. Finally, it defines the typical inference problems, shows how they are interrelated, and describes different approaches for effectively solving these problems. Some of the topics that are only briefly mentioned in this chapter will be treated in more detail in subsequent chapters.

    Introduction

    As sketched in the previous chapter, Description Logics is the most recent name for a family of knowledge representation (KR) formalisms that represent the knowledge of an application domain (the “world”) by first defining the relevant concepts of the domain (its terminology), and then using these concepts to specify properties of objects and individuals occurring in the domain (the world description). As the name Description Logics indicates, one of the characteristics of these languages is that, unlike some of their predecessors, they are equipped with a formal, logic-based semantics. Another distinguished feature is the emphasis on reasoning as a central service: reasoning allows one to infer implicitly represented knowledge from the knowledge that is explicitly contained in the knowledge base.

  • 13 - Medical Informatics

  • Edited by Franz Baader, Technische Universität, Dresden, Diego Calvanese, Freie Universität Bozen, Bolzano, Deborah L. McGuinness, Rensselaer Polytechnic Institute, New York, Daniele Nardi, Università degli Studi di Roma 'La Sapienza', Italy, Peter F. Patel-Schneider , AT&T Bell Laboratories, New Jersey
  • Book:
    The Description Logic Handbook
    Published online:
    06 July 2010
    Print publication:
    09 August 2007, pp 436-457

  • Summary

    Abstract

    Description Logics and related formalisms are being applied in at least five applications in medical informatics – terminology, intelligent user interfaces, decision support and semantic indexing, language technology, and systems integration. Important issues include size, complexity, connectivity, and the wide range of granularity required – medical terminologies require on the order of 250,000 concepts, some involving a dozen or more conjuncts with deep nesting; the nature of anatomy and physiology is that everything connects to everything else; and notions to be represented range from psychology to molecular biology. Technical issues for expressivity have focused on problems of part–whole relations and the need to provide “frame-like” functionality – i.e., the ability to determine efficiently what can sensibly be said about any particular concept and means of handling at least limited cases of defaults with exceptions. There are also significant problems with “semantic normalization” and “clinical pragmatics” because understanding medical notions often depends on implicit knowledge and some notions defy easy logical formulation. The two best-known efforts – Open Galen and Snomed-rt – both use idiosyncratic Description Logics with generally limited expressivity but specialized extensions to cope with issues around part–whole and other transitive relations. There is also a conflict between the needs for re-use and the requirement for easy understandability by domain expert authors. OpenGalen has coped with this conflict by introducing a layered architecture with a high level “Intermediate Representation” which insulates authors from the details of the Description Logic, which is treated as an “assembly language” rather than the primary medium for expressing the ontology.

  • 9 - Implementation and Optimization Techniques

  • Edited by Franz Baader, Technische Universität, Dresden, Diego Calvanese, Freie Universität Bozen, Bolzano, Deborah L. McGuinness, Rensselaer Polytechnic Institute, New York, Daniele Nardi, Università degli Studi di Roma 'La Sapienza', Italy, Peter F. Patel-Schneider , AT&T Bell Laboratories, New Jersey
  • Book:
    The Description Logic Handbook
    Published online:
    06 July 2010
    Print publication:
    09 August 2007, pp 329-374

  • Summary

    Abstract

    This chapter will discuss the implementation of the reasoning services which form the core of DL-based knowledge representation systems. To be useful in realistic applications, such systems need both expressive logics and fast reasoners. As expressive logics inevitably have high worst-case complexities, this can only be achieved by employing highly optimized implementations of suitable reasoning algorithms. Systems based on such implementations have demonstrated that they can perform well with problems that occur in realistic applications, including problems where unoptimized reasoning is hopelessly intractable.

    Introduction

    The usefulness of Description Logics in applications has been hindered by the basic conflict between expressiveness and tractability. Realistic applications typically require both expressive logics, with inevitably high worst-case complexities for their decision procedures, and acceptable performance from the reasoning services. Although the definition of acceptable may vary widely from application to application, early experiments with Description Logics indicated that, in practice, performance was a serious problem, even for logics with relatively limited expressive powers [Heinsohn et al., 1992].

    On the other hand, theoretical work has continued to extend our understanding of the boundaries of decidability in Description Logics, and has led to the development of sound and complete reasoning algorithms for much more expressive logics. The expressive power of these logics goes a long way towards addressing the criticisms leveled at Description Logics in traditional applications such as ontological engineering [Doyle and Patil, 1991] and is sufficient to suggest that they could be useful in several exciting new application domains, for example reasoning about database schemas and queries [Calvanese et al., 1998f; 1998a] and providing reasoning support for the so-called Semantic Web [Decker et al., 2000; Bechhofer et al., 2001b].

  • 14 - OWL: a Description-Logic-Based Ontology Language for the Semantic Web

  • Edited by Franz Baader, Technische Universität, Dresden, Diego Calvanese, Freie Universität Bozen, Bolzano, Deborah L. McGuinness, Rensselaer Polytechnic Institute, New York, Daniele Nardi, Università degli Studi di Roma 'La Sapienza', Italy, Peter F. Patel-Schneider , AT&T Bell Laboratories, New Jersey
  • Book:
    The Description Logic Handbook
    Published online:
    06 July 2010
    Print publication:
    09 August 2007, pp 458-486

  • Summary

    Abstract

    It has long been realized that the web could benefit from having its content understandable and available in a machine processable form. The Semantic Web aims to achieve this via annotations that use terms defined in ontologies to give well defined meaning to web accessible information and services. OWL, the ontology language recommended by the W3C for this purpose, was heavily influenced by Description Logic research. In this chapter we review briefly some early efforts that combine Description Logics and the web, including predecessors of OWL such as OIL and DAML+OIL. We then go on to describe OWL in some detail, including the various influences on its design, its relationship with RDFS, its syntax and semantics, and a range of tools and applications.

    Background and history

    The World Wide Web, while wildly successful in growth, may be viewed as being limited by its reliance on languages such as HTML that are focused on presentation (i.e., text formatting) rather than content. Languages such as XML do add some support for capturing the meaning of web content (instead of simply how to render it in a browser), but more is needed in order to support intelligent applications that can better exploit the ever increasing range of information and services accessible via the web. Such applications are urgently needed in order to avoid overwhelming users with the sheer volume of information becoming available.

Page 91 of 96