Added fetchers concept: seperate scripts to fetch the feeds

Fetchers claim to be a certain client. They try to send the same
headers as the original client. That's better than a simple curl request
with a fake user agent, because curl doesn't send the other headers like
the original client and therefore its traffic stands out.
This commit is contained in:
2017-08-11 12:57:30 +02:00
parent 8a80aa0d6d
commit 3a723b9440
6 changed files with 40 additions and 9 deletions

2
fetcherslist Normal file
View File

@ -0,0 +1,2 @@
fetchers/firefox
fetchers/chrome