Skip to content

Tools: Pipes 1/2

04/04/2012

Let’s do some scraping: (getting the data from web pages in a format we can handle and for example put in Manyeyes.

Mashup: (http://en.wikipedia.org/wiki/Mashup_%28web_application_hybrid%29)

In Web development, a mashup is a Web page or application that uses and combines data, presentation or functionality from two or more sources to create new services.

links from:

http://onlinejournalismblog.com/2010/07/07/an-introduction-to-data-scraping-with-scraperwiki/

http://www.poynter.org/how-tos/digital-strategies/e-media-tidbits/102589/how-to-scrape-websites-for-data-without-programming-skills/

outhub: for example getting tables from wikipedia: (about wind energy)

and you can export this in a format you prefer.

Yahoo Pipes:

http://www.daybarr.com/blog/yahoo-pipes-tutorial-an-example-using-the-fetch-page-module-to-make-a-web-scraper

So pipes offers more than Google Image Search alone, you can combine Google Image search with rss feed etc.

http://pipes.yahoo.com/pipes/

for this you need a yahoo account!

The simplest pipe is an rss feed:

http://pipes.yahoo.com/pipes/pipe.info?_id=9437f833c53c7204b81129ad5f7f357e

And here is a pipe that discovers the rss feed itself on a page, see image.
For more complex pipes, just search on the pipes page if it is already there!!!

For instance Google Image Search…
Getting data from Pachube is more complex, you need to get your key in, but the example pipe is very helpful! Getting your key in and the number of the Pachube feed and you are ready.

From the twitter pipeline a lot can be learnt, http://pipes.yahoo.com/pipes/pipe.edit?_id=ac45e9eb9b0174a4e53f23c4c9903c3f

And yes, mashup, that is combining things, here is a nice example using “union”:

http://pipes.yahoo.com/pipes/pipe.info?_id=1qj9XFW32xGgtAairuymrA

Advertisements

From → tools

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: