This topic was automatically generated from Slack. You can find the original thread here.
What would be an appropiate way to make an HTTP trigger ignore this kind of requests? (robots.txt, favicon, …) there is a “remove favicon requests” but not for robots? 
Hey , I believe your request make sense. I’ve created a ticket here to improve HTTP sources
In the mean time, you can try to use Pipedream Filter actions to filter out events that have the path /robots.txt
Thanks for the tip!
That most likely means the URL what shared somewhere where a web crawler was able to discover it.
: instead of filtering out an infinitely large list of exceptions, what could be even better & simpler would be the ability for HTTP triggers to accept POST requests only (or basically to select which HTTP methods to allow or decline).
Robots and the average web user/browser should never send a POST request arbitrarily (unless submitting a form, or something like that).
It was published on a forum, as it is an usage example https://community.getgrist.com/t/pdf-rendering-with-apitemplate-io-pipedream/4984
One sensible thing to do would be to add a parameter of “only accept requests to paths matching…” And make that a Regex. A second parameter could be “O ly accept HTTP methods…”
This avoids making assumptions, having an infinite arbitrary list of exclusions, or limiting methods.
(basically, make a “limited” filter built onto the trigger)
Path matching is an amazing idea! 
Even better than just matching on the HTTP method.
Thanks folks. I’ve updated the issue accordingly! The new prop should filter by path
Yes I wanted this as well as few of my HTTP trigger by SEO / robots. It failed due to unexpected trigger POST data. Hope this will be up soon!
