Every now and then I find myself needing to automate some web requests, either to download using something a little more robust than a web browser, scrape some web content, or to maintain a session. That automation can be a bit of a pain if there’s a form submission involved, because it means opening up the page source, finding the form and any connected javascript code, and figuring out what fields are submitted.
A little utility I just put together does that for you: [Post Grabber](https://github.com/michaeltyson/PostGrabber) detects POST data and generates an equivalent “curl” command that can be used in shell or Automator scripts.
Post Grabber works with its own internal browser, so it can intercept POST submissions directly. That means it works with HTTPS, unlike the traditional web sniffer approach.
[Download the app](https://github.com/downloads/michaeltyson/PostGrabber/Post%20Grabber%201.2.zip), or see the [source on GitHub](https://github.com/michaeltyson/PostGrabber).
Wow, this would have saved me about 8 hours of time a few weeks back.
This product is almost perfect.
Is there an option or a hack to change the User Agent :-)
One could definitely add one pretty easily – it’s just a matter of adding a call to [webView setCustomUserAgent:] in the awakeFromNib method of PGMainWindowController, if you felt like it =)