public class HttpWebClient extends Object
Constructor and Description |
---|
HttpWebClient() |
Modifier and Type | Method and Description |
---|---|
static void |
cleanUpDriver(WebDriver driver) |
static RemoteWebDriver |
createChromeRemoteWebDriver(URL seleniumHubUrl,
boolean enableHeadlessMode) |
static WebDriver |
createChromeWebDriver(String chromeDriverPath,
boolean enableHeadlessMode) |
static RemoteWebDriver |
createDefaultRemoteWebDriver(URL seleniumHubUrl,
boolean enableHeadlessMode) |
static RemoteWebDriver |
createFirefoxRemoteWebDriver(URL seleniumHubUrl,
boolean enableHeadlessMode) |
static WebDriver |
createFirefoxWebDriver(String firefoxDriverPath,
boolean enableHeadlessMode) |
static WebDriver |
createOperaWebDriver(String operaDriverPath,
boolean enableHeadlessMode) |
static RemoteWebDriver |
createRandomRemoteWebDriver(URL seleniumHubUrl,
boolean enableHeadlessMode) |
static WebDriver |
getDriverForPage(String url,
Configuration conf) |
static String |
getHtmlPage(String url) |
static String |
getHtmlPage(String url,
Configuration conf)
Function for obtaining the HTML BODY using the selected selenium
webdriver There are a number of configuration properties within
nutch-site.xml which determine whether to take screenshots of
the rendered pages and persist them as timestamped .png's into HDFS. |
public static WebDriver getDriverForPage(String url, Configuration conf)
public static WebDriver createFirefoxWebDriver(String firefoxDriverPath, boolean enableHeadlessMode)
public static WebDriver createChromeWebDriver(String chromeDriverPath, boolean enableHeadlessMode)
public static WebDriver createOperaWebDriver(String operaDriverPath, boolean enableHeadlessMode)
public static RemoteWebDriver createFirefoxRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
public static RemoteWebDriver createChromeRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
public static RemoteWebDriver createRandomRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
public static RemoteWebDriver createDefaultRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
public static void cleanUpDriver(WebDriver driver)
public static String getHtmlPage(String url, Configuration conf)
nutch-site.xml
which determine whether to take screenshots of
the rendered pages and persist them as timestamped .png's into HDFS.url
- the URL to fetch and renderconf
- the Configuration
Copyright © 2021 The Apache Software Foundation