What webhooks are and why you should care
Webhooks are user-defined HTTP callbacks. Here’s a common example: You go to github. There’s a textbox for their code post webhook. You drop in a URL. Now when you post your code to github, github will HTTP POST to your chosen URL with details about the code post. There is no simpler way to allow open ended integration with arbitrary web services.
This tiny interface is used in obvious ways: bug tracking integration, sms messaging, IRC and twitter.
The same tiny interface is also used in non-obvious ways, like Run Code Run which offers to build and run your project’s tests for you. All by just plugging a runcoderun.com URL into GitHub.
Webhooks today offer a lot of value as an instant notification mechanism. Have events your users care about? Give them a webhook for those events and you’ve given them the power and flexibility to integrate that event stream into their life.
For all of that power, webhooks are impressively simple to implement. It’s a one liner in almost every language.
urllib.urlopen(user.webhook.url)
While there’s a lot of value in webhooks today, it’s the future that really interests me. Webhooks are composable. You’ll point a webhook at a site that will call other webhooks. It might process the data, record it, fork it off to multiple webhooks or something stranger still. Yahoo Pipes tried to do this, but ultimately you were limited to what Yahoo Pipes was designed to do. Webhooks can be integrated and implemented everywhere. They piggyback the fundamental decentralized nature of the web.
I imagine a future where twitter feed updates instantly call a webhook. I’ve pointed that webhook at a service that does bayesian filtering. The filtering has been set up to determine if the tweet looks time-sensitive “Anyone interested in getting dinner tonight?” vs time-insensitive “Webhooks are cool.” Time sensitive posts call another webhook, this time set to sms my phone. Note that nowhere in this future am I writing any code. I don’t have to.
It’s important that we get to this level of customization for the masses. It’s also important for adoption that we use the web’s native verbs. We understand HTTP on a fundamental level. It’s simple, scales and makes sense.
You should care because webhooks will be ubiquitous. You should care because they’re going to reshape the internet. You should care because webhooks are the next step in the evolution of communication on the internet and nothing will be left untouched.
I’m looking for the “notify me of follow-up comments via web hook”, this is as dissapointing as last night, with your mother.
http://webhooks.pbwiki.com/
So it’s like user-specified XML-RPC…
Sounds like an implementation of database triggers for the web world. Neat.
Cool. Never used github, but that’s a great idea. Thanks.
YHVH: If or when I eventually move off of wordpress.com (note that I started this blog three days ago) then I’ll probably add webhooks to it.
JohnW: Yeah, that’s a good analogy. I didn’t talk about the bi-directional nature, on Devjavu (SVN/Trac as a service) there’s a pre-commit webhook that lets you cancel an svn hook with a message (like “Sorry, we don’t allow hard tabs in commits”) and in that way they’re even more like database triggers.
Unfuddle supports this too: http://www.unfuddle.com/blog/2008/5/18/git-hosting-and-repository-improvements
Not directly affiliated with them, just a satisfied customer. -pete
Uche Ogbuji in fact thought of a very similar idea and in fact called it Web Triggers.
http://notes.4suite.org/Bright_Content%3ADesign%3AWeb_Triggers
old news for any web developer who has ever coded before ever. the new part is the catchy name
long live webhooks!
Ivan: Respectfully, I disagree. There are old examples of webhooks, like Paypal’s Instant Payment Notification; however, there are hardly any non-developer webhooks. This is about bringing the power of instant notification, integration and composability to the end-users, not just programmers.
Webhooks are cool.
Anyone interested in getting dinner tonight?
I just started using Chris Pearson’s premium theme, Thesis, a few days ago on my self-hosted blog. It uses hooks extensively and so far, I like doing things that way. Unhook, hook. Easy.
i think it’s a great idea but what you describe still strikes me as too low level for the average non-technical user. composability is huge. great term too
. but what are your thoughts on creating an interface (and/or the literacy i suppose) that an average, busy user (think: a teacher; a hill staffer; a non tech-based small business…) will intuitively know HOW to compose? the example of inserting a url into a text box still requires the user understand in a low level way the notion of composability, and be familiar with what is out there to compose WITH. perhaps it’s building various options into the right hand click menu of your browser:
right click –> send to service…
which pulls up a selection of categories and choices in those categories. one mght be ‘notify’ (a more sophisticated middleware solution); another might be ‘publish’ with a list of blogs you have registered… i’m sure there are tons more.
your AI service example strikes me as falling in the middleware category. most users dont think about bayesian filters, they think in terms of ’service = notify’. ‘notify’ would bring up a dialogue box where the user can select the threshold of timeliness they’re interested in.
the middleware is where the fun is for the developers
, and where really cool innovative evolutions of what the web can DO come in. but i think the user experience needs to be abstracted away from that.
dunno, thoughts?
Jessy, I agree. I’ve been playing with a lot of prototypes on making the user experience better, one similar to what you’re describing. However, I don’t believe it’s that important of a problem to solve now. But I appreciate more people thinking about it. As the value of web hooks becomes obvious from advanced users/early adopters using them, they will slowly become more usable for regular users. It’s just sort of inevitable for useful technology to evolve, at the very least, usable enough.
Disagree that webhooks are easy to implement – your one-liner will leave you open to abuse, where malicious users input the URL to a deliberately slow loading page. This will quickly tie up your web serving processes.
Scaling webhooks properly requires offline processing of some sort, probably using a message queue. I’d be interested to know how GitHub handles this problem.
Good point. But that is why there are timeouts right ?
The interesting challenge then is the fact that the callee may not be up all the time. And distinguishing between a genuine callee availability issue and a malicious callee becomes impossible to the caller.
But there is a simple “good enough” solution for that too. Just put the callee behind a highly available Amazon SQS system.
The caller then just implements a timeout based callback without a retry.
From your comment on Reddit I can see you’ve already thought about this problem. I’d like to see your memcache protocol compliant webhook forwarder, sounds like a neat solution.
BTW, Trackback is an example of a user-land webhook of sorts..
Simon Willison: If you look at the github-services project on github, I think you can find out. I seem to recall this code was open sourced on there.
At GitHub our webhooks are run as part of the post-receive queued job on non-web slices. The code is quite simple, in essence wrapping up some JSON and posting to the webhook URL. Here are the details if you’re interested in a reference:
http://github.com/guides/post-receive-hooks
We also have a bunch of pre-built services that users can activate to send their commit info to places like Twitter, Campfire, IRC, etc. All they have to do is enter a few connection details into a form. This code is open source:
http://github.com/pjhyett/github-services
Many of these services have been contributed by our users, so that they can have easy integration with their favorite app!
Just trying to wrap my head around this — so, are webhooks kind of like the pipe | on a Unix command line? A way to connect the output of one process/service to another, and chain things together?
“I imagine a future where twitter feed updates instantly call a webhook”
I’m already using a free service that does just this, Gnip. See http://gnipcentral.com/ for a whole bunch of services that you can to have a webhook called whenever they update.
The only downside is that Twitter are being stingy with their data, and only allow a link to the status update to be sent, which you then have to fetch from their servers to get the full message.
Maybe Twitter don’t send the full message because it can be edited (at least for a while) by the user, and you might miss out on this.
Nice post. Web Hooks are an ideal way to leverage the extremely vetted HTTP backbone for the traditional “event callback” pattern. Long overdue. Gnip’s filter “postURL” callback setting leverages the pattern heavily. Would love feedback on it.
Dustin Beltramo: Yeah, that’s a decent analogy. My twitter example would kinda be like: “twitter timothyfitz | bayes “time critical” | sms”. There are obvious differences because we’ve got a stream of requests instead of a file descriptor, but the analogy for what you can do to leverage them is vaild (and in fact Webhook evangelist Jeff Lindsay has used that analogy many times before).
Pete Warden/Jud Valeski: I haven’t actually used Gnip yet, but Jeff Lindsay and I have talked about it quite a bit. Just digging around on the website and reading the API docs, the very first thing that jumps out is that this still isn’t *quite* webhooks. It’s still targeting developers and it’s not just as easy as dropping a url into a textbox. It would be cool to see a single server site for whatever your largest use case is. Hypothetically, if it’s twitter then you’d have a webpage where you gave your twitter username and password and a webhook url (with a list of sample / common urls) and a submit button. Taking it to that level of simplicity has the potential to open the concept up to an order-of-magnitude larger market.
[...] assembla, joe gregoria, jon udell, timothy fitz, triggers Timothy Fitz recently wrote on What webhooks are and why you should care. It’s a very clear and straightforward description of just that. It helps to have people [...]
[...] a service to HTTP, and it becomes a team player in our ecosystem. Let’s revolutionize the last of our dinosaur protocols and move on. [...]
I’ve coded something similar, sort of, at http://qerio.com, using the web rack of “virtual devices” from http://virtual-devices.net. It adds the GUI element to the mix. The webhooks, in this case, are the patch cords in the back of the rack.
[...] What webhooks are and why you should care While there’s a lot of value in webhooks today, it’s the future that really interests me. Webhooks are composable. You’ll point a webhook at a site that will call other webhooks. It might process the data, record it, fork it off to multiple webhooks or something stranger still. Yahoo Pipes tried to do this, but ultimately you were limited to what Yahoo Pipes was designed to do. Webhooks can be integrated and implemented everywhere. They piggyback the fundamental decentralized nature of the web. [...]
Timothy, I haz teh lazy. Please take my content and sell it as ur own in a future post, I know you like it: consuming webhook posts is so simple, even a bent snake could do it.
I think a two-liner consumer would nicely compliment you one one-liner for posting.
@easy_consume(80)
def handle(req): return “meow”+req.args
[...] http://timothyfitz.wordpress.com/2009/02/09/what-webhooks-are-and-why-you-should-care/ : cet article explique le principe de webhooks et cite quelques exemples d’utilisation [...]
See the Twitter Streaming API documentation at: http://apiwiki.twitter.com/Streaming-API-Documentation
A different mechanism, same result.
[...] [upmod] [downmod] What webhooks are and why you should care « Timothy Fitz (timothyfitz.wordpress.com) 1 points posted 3 months ago by jeethu tags webdev webhooks hooks [...]
It will be awesome if you can chain these together. Like some sort of yahoo pipes.
Timothy— I notice you mention in a comment that you’d like to add webhooks to your blog if/when you move off of wordpress.com. If you enjoy WordPress and would like to incorporate webhooks, I recently released a wordpress.org plugin called HookPress which adds a simple interface to register webhooks against WordPress actions and filters. If/when you move to a self-served WordPress install, I invite you to take a look.
http://mitcho.com/code/hookpress/