Added fetchers concept: seperate scripts to fetch the feeds
Fetchers claim to be a certain client. They try to send the same headers as the original client. That's better than a simple curl request with a fake user agent, because curl doesn't send the other headers like the original client and therefore its traffic stands out.
This commit is contained in:
1
fetchers/chrome_agents
Normal file
1
fetchers/chrome_agents
Normal file
@@ -0,0 +1 @@
|
||||
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.78 Safari/537.36
|
Viittaa uudesa ongelmassa
Block a user