Big Data
Coursera
An eleven-module Coursera course from O.P. Jindal Global University covering Hadoop, HDFS, MapReduce, Hive, Pig, HBase and Apache Spark. Graded assignments use PySpark, Spark SQL and MLlib on real datasets, so you finish able to build distributed processing pipelines.
More resources on Big Data Tools
hadoop.apache.org
Official Apache Hadoop project page for the open-source framework enabling distributed storage (HDFS) and processing (MapReduce/YARN) of big data. It offers documentation, downloads, getting-started guides, and ecosystem resources to help you deploy and use Hadoop.
kafka.apache.org
Apache Kafka's official site for the distributed event streaming platform. It provides project overview, downloads, and extensive documentation plus quickstarts and tutorials for using Kafka, its Streams and Connect ecosystem to build real-time data pipelines.
Designing Data-Intensive
Martin Kleppmann's O'Reilly survey of the systems behind modern databases: storage engines, replication, partitioning, transactions, consensus and stream processing. Readers finish able to reason about the tradeoffs when choosing or combining data stores at scale.
Apache Spark with Scala
Learn Apache Spark with Scala! Master big data processing with hands-on examples from expert Frank Kane.
Apache Spark Documentation
Apache Spark's official documentation: programming guides for RDDs, DataFrames and Spark SQL, Structured Streaming and MLlib, plus cluster deployment, configuration and tuning references. Readers can write, submit and tune distributed data-processing jobs in Python, Scala, Java or R.
Introduction to Big Data
Interested in increasing your knowledge of the Big Data landscape? This course is for those new to data science and interested in understanding why the Big Data Era has come to be. It is for those who want to become conversant with the terminology and the core concepts behind big data problems, applications, and systems. It is for those who want to start thinking about how Big Data might be useful in their business or career. It provides an introduction to one of the most common frameworks, Hadoop, that has made big data analysis easier and more accessible -- increasing the potential for data to transform our world! At the end of this course, you will be able to: * Describe the Big Data landscape including examples of real world big data problems including the three key sources of Big Data: people, organizations, and sensors. * Explain the V’s of Big Data (volume, velocity, variety, veracity, valence, and value) and why each impacts data collection, monitoring, storage, analysis and reporting. * Get value out of Big Data by using a 5-step process to structure your analysis. * Identify what are and what are not big data problems and be able to recast big data problems as data science questions. * Provide an explanation of the architectural components and programming models used for scalable big data analysis. * Summarize the features and value of core Hadoop stack components including the YARN resource and job management system, the HDFS file system and the MapReduce programming model. * Install and run a program using Hadoop! This course is for those new to data science. No prior programming experience is needed, although the ability to install applications and utilize a virtual machine is necessary to complete the hands-on assignments. Hardware Requirements: (A) Quad Core Processor (VT-x or AMD-V support recommended), 64-bit; (B) 8 GB RAM; (C) 20 GB disk free. How to find your hardware information: (Windows): Open System by clicking the Start button, right-clicking Computer, and then clicking Properties; (Mac): Open Overview by clicking on the Apple menu and clicking “About This Mac.” Most computers with 8 GB RAM purchased in the last 3 years will meet the minimum requirements.You will need a high speed internet connection because you will be downloading files up to 4 Gb in size. Software Requirements: This course relies on several open-source software tools, including Apache Hadoop. All required software can be downloaded and installed free of charge. Software requirements include: Windows 7+, Mac OS X 10.10+, Ubuntu 14.04+ or CentOS 6+ VirtualBox 5+.