Package | Description |
---|---|
org.apache.nutch.analysis.lang |
Text document language identifier.
|
org.apache.nutch.any23 |
This packages uses the Apache Any23 library
for parsing and extracting structured data in RDF format from a
variety of Web documents.
|
org.apache.nutch.crawl |
Crawl control code and tools to run the crawler.
|
org.apache.nutch.fetcher |
The Nutch robot.
|
org.apache.nutch.hostdb | |
org.apache.nutch.indexer |
Index content, configure and run indexing and cleaning jobs to
add, update, and delete documents from an index.
|
org.apache.nutch.indexer.anchor |
An indexing plugin for inbound anchor text.
|
org.apache.nutch.indexer.basic |
A basic indexing plugin, adds basic fields: url, host, title, content, etc.
|
org.apache.nutch.indexer.feed |
Indexing filter to index meta data from RSS feeds.
|
org.apache.nutch.indexer.filter | |
org.apache.nutch.indexer.geoip |
This plugin implements an indexing filter which takes
advantage of the
GeoIP2-java API.
|
org.apache.nutch.indexer.jexl |
This plugin implements a dynamic indexing filter which uses JEXL
expressions to allow filtering based on the page's metadata
|
org.apache.nutch.indexer.links | |
org.apache.nutch.indexer.metadata |
Indexing filter to add document metadata to the index.
|
org.apache.nutch.indexer.more |
A more indexing plugin, adds "more" index fields:
last modified date, MIME type, content length.
|
org.apache.nutch.indexer.replace |
Indexing filter to allow pattern replacements on metadata.
|
org.apache.nutch.indexer.staticfield |
A simple plugin called at indexing that adds fields with static data.
|
org.apache.nutch.indexer.subcollection |
Indexing filter to assign documents to subcollections.
|
org.apache.nutch.indexer.tld |
Top Level Domain Indexing plugin.
|
org.apache.nutch.indexer.urlmeta |
URL Meta Tag Indexing Plugin
|
org.apache.nutch.metadata |
A Multi-valued Metadata container, and set
of constant fields for Nutch Metadata.
|
org.apache.nutch.microformats.reltag |
A microformats Rel-Tag
Parser/Indexer/Querier plugin.
|
org.apache.nutch.net.urlnormalizer.protocol | |
org.apache.nutch.net.urlnormalizer.slash | |
org.apache.nutch.protocol |
Classes related to the
Protocol interface,
see also org.apache.nutch.net.protocols . |
org.apache.nutch.protocol.file |
Protocol plugin which supports retrieving local file resources.
|
org.apache.nutch.protocol.ftp |
Protocol plugin which supports retrieving documents via the ftp protocol.
|
org.apache.nutch.protocol.htmlunit |
Protocol plugin which supports retrieving documents via the http protocol.
|
org.apache.nutch.protocol.http.api |
Common API used by HTTP plugins (
http ,
httpclient ) |
org.apache.nutch.protocol.okhttp |
Protocol plugin based on okhttp, supports http, https, http/2.
|
org.apache.nutch.scoring |
The
ScoringFilter interface. |
org.apache.nutch.scoring.depth |
Scoring filter to stop crawling at a configurable depth
(number of "hops" from seed URLs).
|
org.apache.nutch.scoring.link |
Scoring filter used in conjunction with
WebGraph . |
org.apache.nutch.scoring.opic |
Scoring filter implementing a variant of the Online Page Importance Computation
(OPIC) algorithm.
|
org.apache.nutch.scoring.orphan |
Scoring filter to modify score or status of orphaned pages (no inlinks found
for a configurable amount of time).
|
org.apache.nutch.scoring.similarity | |
org.apache.nutch.scoring.similarity.cosine |
Implements the cosine similarity metric for scoring relevant documents
|
org.apache.nutch.scoring.tld |
Top Level Domain Scoring plugin.
|
org.apache.nutch.scoring.urlmeta |
URL Meta Tag Scoring Plugin
|
org.apache.nutch.scoring.webgraph | |
org.apache.nutch.segment |
A segment stores all data from on generate/fetch/update cycle:
fetch list, protocol status, raw content, parsed content, and extracted outgoing links.
|
org.apache.nutch.tools |
Miscellaneous tools.
|
org.apache.nutch.tools.warc |
Tools to import / export between Nutch segments and
WARC archives.
|
org.apache.nutch.util |
Miscellaneous utility classes.
|
org.creativecommons.nutch |
Sample plugins that parse and index Creative Commons medadata.
|
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
AbstractFetchSchedule
This class provides common methods for implementations of
FetchSchedule . |
AdaptiveFetchSchedule
This class implements an adaptive re-fetch algorithm.
|
CrawlDatum |
FetchSchedule
This interface defines the contract for implementations that manipulate fetch
times and re-fetch intervals.
|
Generator.SelectorEntry |
Inlink |
Inlinks
A list of
Inlink s. |
NutchWritable |
Signature |
Class and Description |
---|
CrawlDatum |
NutchWritable |
Class and Description |
---|
CrawlDatum |
NutchWritable |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
NutchWritable |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
NutchWritable |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Class and Description |
---|
NutchWritable |
Class and Description |
---|
CrawlDatum |
NutchWritable |
Class and Description |
---|
Generator.SelectorEntry |
Class and Description |
---|
NutchWritable |
Class and Description |
---|
CrawlDatum |
Class and Description |
---|
CrawlDatum |
Inlinks
A list of
Inlink s. |
Copyright © 2021 The Apache Software Foundation