public class IndexingFiltersChecker extends AbstractChecker
echo "http://www.lemonde.fr" | $NUTCH_HOME/bin/nutch indexchecker -stdin
Modifier and Type | Field and Description |
---|---|
protected boolean |
checkRobotsTxt |
protected boolean |
doIndex |
protected boolean |
dumpText |
protected boolean |
followRedirects |
protected HashMap<String,String> |
metadata |
protected URLNormalizers |
normalizers |
keepClientCnxOpen, stdin, tcpPort, usage
Constructor and Description |
---|
IndexingFiltersChecker() |
Modifier and Type | Method and Description |
---|---|
static void |
main(String[] args) |
protected int |
process(String url,
StringBuilder output) |
int |
run(String[] args) |
getProtocolOutput, parseArgs, processSingle, processStdin, processTCP, run
getConf, setConf
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getConf, setConf
protected URLNormalizers normalizers
protected boolean dumpText
protected boolean followRedirects
protected boolean checkRobotsTxt
protected boolean doIndex
protected int process(String url, StringBuilder output) throws Exception
process
in class AbstractChecker
Exception
Copyright © 2021 The Apache Software Foundation