Memo for Big Data

Python
Crawler
Scrapy
http://scrapy.org/

Data Analysis

pandas

http://pandas.pydata.org/

https://pda.readthedocs.org/en/latest/chp5.html

Machine Learning

scikit-learn

http://scikit-learn.org/stable/

Graphs Analysis

NetworkX

http://networkx.github.io/

Data Base

Mongo

http://www.mongodb.org/

Distributed Computing Framework

disco

http://discoproject.org/

DPark

https://github.com/douban/dpark/

Spark

http://spark-project.org/docs/latest/python-programming-guide.html

happy

https://code.google.com/p/happy/

A Guide to Python Frameworks for Hadoop

http://blog.cloudera.com/blog/2013/01/a-guide-to-python-frameworks-for-hadoop/

Visualization

matplotlib

http://matplotlib.org/

R

RHIPE

http://www.datadr.org/

other articles
http://datacommunitydc.org/blog/2013/03/getting-started-with-python-for-data-scientists/

http://www.r-bloggers.com/stepping-up-to-big-data-with-r-and-python-a-mind-map-of-all-the-packages-you-will-ever-need/