WebScraper.SetSiteSpecificCrawlRateLimit Method

IronWebScraper

previous page next page

WebScraperSetSiteSpecificCrawlRateLimit Method

Set a throttle limit for a specific domain

Namespace:  IronWebScraper
Assembly:  IronWebScraper (in IronWebScraper.dll) Version: 4.0.4.25470 (4.0.4.3)
Syntax
Copy
public void SetSiteSpecificCrawlRateLimit(
	string hostName,
	TimeSpan crawlRate
)
Public Sub SetSiteSpecificCrawlRateLimit ( 
	hostName As String,
	crawlRate As TimeSpan
)

Parameters

hostName
Type: SystemString
The http host (domain name).
crawlRate
Type: SystemTimeSpan
The maximum frequency of http requests for the given hostName.
See Also
previous page start next page