Lab 3 – Map Reduce Streaming Data With Python

This is a tutorial to demonstrate mapper and reducer python scripts to convert data from one input format to a required output format.

Follow the steps from the tutorial below:

https://dbaumgartel.wordpress.com/2014/04/10/an-elastic-mapreduce-streaming-example-with-python-and-ngrams-on-aws/

Create the mapper.py file and the reducer.py file and have all files in the same windows directory.

Then run ->

type googlebooks-eng-all-1gram-20120701-x | mapper.py | reducer.py

set PATH=”C:\Program Files\R\R-3.2.2\bin\x64″;%PATH%

type googlebooks-eng-all-1gram-20120701-x | mapper.py | reducer.py | rscript grapher.r

Create a graph based on the data via R:

Create a graph with r – piping data from standard input: