public class Http extends HttpBase
Modifier and Type | Field and Description |
---|---|
protected static org.slf4j.Logger |
LOG |
accept, acceptCharset, acceptLanguage, BUFFER_SIZE, COOKIE, enableCookieHeader, enableIfModifiedsinceHeader, maxContent, maxCrawlDelay, maxDuration, partialAsTruncated, proxyException, proxyHost, proxyPort, proxyType, RESPONSE_TIME, responseTime, storeHttpHeaders, storeHttpRequest, storeIPAddress, timeout, tlsCheckCertificate, tlsPreferredCipherSuites, tlsPreferredProtocols, useHttp11, useHttp2, useProxy, userAgent
X_POINT_ID
Constructor and Description |
---|
Http()
Public default constructor.
|
Modifier and Type | Method and Description |
---|---|
protected Response |
getResponse(URL url,
CrawlDatum datum,
boolean redirect) |
static void |
main(String[] args) |
void |
setConf(Configuration conf)
Set the
Configuration object. |
getAccept, getAcceptCharset, getAcceptLanguage, getConf, getCookie, getMaxContent, getMaxDuration, getProtocolOutput, getProxyHost, getProxyPort, getRobotRules, getTimeout, getTlsPreferredCipherSuites, getTlsPreferredProtocols, getUseHttp11, getUserAgent, isCookieEnabled, isIfModifiedSinceEnabled, isStoreHttpHeaders, isStoreHttpRequest, isStoreIPAddress, isStorePartialAsTruncated, isTlsCheckCertificates, logConf, main, processDeflateEncoded, processGzipEncoded, useProxy, useProxy, useProxy
public void setConf(Configuration conf)
Configuration
object.setConf
in interface Configurable
setConf
in class HttpBase
conf
- protected Response getResponse(URL url, CrawlDatum datum, boolean redirect) throws ProtocolException, IOException
getResponse
in class HttpBase
ProtocolException
IOException
Copyright © 2021 The Apache Software Foundation