IronWebScraper - The C# Web Scraping Library
WebScraper Properties |
The WebScraper type exposes the following members.
Properties| Name | Description | |
|---|---|---|
 | FailedUrls |
Gets the number of failed http requests which have exceeded their total maximum number of retries.
|
| HttpRetryAttempts |
The number of times WebScraper will retry a failed URL (normally with a new identity) before considering it non-scrapable.
|
| HttpTimeOut |
Gets or the time after-which a HTTP request will be considered failed or lost.
(non-contactable or Dns unavailable)
|
| MaxHttpConnectionLimit |
Gets or sets the total number of allowed open HTTP requests (threads)
|
| OpenConnectionLimitPerHost |
Gets or sets the allowed number of concurrent HTTP requests (threads) per hostname or IP address. This helps protect hosts against too many requests.
|
| RateLimitPerHost |
Gets or sets minimum polite delay (pause) between request to a given domain or IP address.
|
| SuccessfulFileDownloadCount |
Gets the number of successful http downloads using the DownloadFile and DownloadImage methods..
|
| SuccessfulfulRequestCount |
Gets the number of successful http requests.
|
| ThrottleMode |
Makes the WebSraper intelligently throttle requests not only by hostname, but also by host servers' IP addresses. This is polite in-case multiple scraped domains are hosted on the same machine.
|
See Also