Apache Spark Documentation
Apache Software Foundation
The official MLlib guide from the Spark project, documenting DataFrame-based APIs for feature extraction, classification, regression, clustering, and pipelines. Includes code in Scala, Java, Python, and R, so readers can build and tune distributed models directly.
More resources on Big Data Analytics
hadoop.apache.org
Official Apache Hadoop project page for the open-source framework enabling distributed storage (HDFS) and processing (MapReduce/YARN) of big data. It offers documentation, downloads, getting-started guides, and ecosystem resources to help you deploy and use Hadoop.
Apache Spark Documentation
Apache Spark's official documentation: programming guides for RDDs, DataFrames and Spark SQL, Structured Streaming and MLlib, plus cluster deployment, configuration and tuning references. Readers can write, submit and tune distributed data-processing jobs in Python, Scala, Java or R.
Introduction to Big Data
Interested in increasing your knowledge of the Big Data landscape? This course is for those new to data science and interested in understanding why the Big Data Era has come to be. It is for those who want to become conversant with the terminology and the core concepts behind big data problems, applications, and systems. It is for those who want to start thinking about how Big Data might be useful in their business or career. It provides an introduction to one of the most common frameworks, Hadoop, that has made big data analysis easier and more accessible -- increasing the potential for data to transform our world! At the end of this course, you will be able to: * Describe the Big Data landscape including examples of real world big data problems including the three key sources of Big Data: people, organizations, and sensors. * Explain the V’s of Big Data (volume, velocity, variety, veracity, valence, and value) and why each impacts data collection, monitoring, storage, analysis and reporting. * Get value out of Big Data by using a 5-step process to structure your analysis. * Identify what are and what are not big data problems and be able to recast big data problems as data science questions. * Provide an explanation of the architectural components and programming models used for scalable big data analysis. * Summarize the features and value of core Hadoop stack components including the YARN resource and job management system, the HDFS file system and the MapReduce programming model. * Install and run a program using Hadoop! This course is for those new to data science. No prior programming experience is needed, although the ability to install applications and utilize a virtual machine is necessary to complete the hands-on assignments. Hardware Requirements: (A) Quad Core Processor (VT-x or AMD-V support recommended), 64-bit; (B) 8 GB RAM; (C) 20 GB disk free. How to find your hardware information: (Windows): Open System by clicking the Start button, right-clicking Computer, and then clicking Properties; (Mac): Open Overview by clicking on the Apple menu and clicking “About This Mac.” Most computers with 8 GB RAM purchased in the last 3 years will meet the minimum requirements.You will need a high speed internet connection because you will be downloading files up to 4 Gb in size. Software Requirements: This course relies on several open-source software tools, including Apache Hadoop. All required software can be downloaded and installed free of charge. Software requirements include: Windows 7+, Mac OS X 10.10+, Ubuntu 14.04+ or CentOS 6+ VirtualBox 5+.
Big Data
An eleven-module Coursera course from O.P. Jindal Global University covering Hadoop, HDFS, MapReduce, Hive, Pig, HBase and Apache Spark. Graded assignments use PySpark, Spark SQL and MLlib on real datasets, so you finish able to build distributed processing pipelines.
Big Data: Principles and Best Practices
Nathan Marz's presentation of the Lambda Architecture: an immutable master dataset with batch and speed layers serving precomputed views. Readers learn a design approach for scalable, fault-tolerant analytics systems and how tools like Hadoop and Storm fit it.
Hadoop: The Definitive Guide
O'Reilly's reference on Hadoop, covering HDFS, MapReduce job design, cluster setup and administration, and the surrounding ecosystem including Pig, Hive, and HBase. Readers can write MapReduce jobs and reason about how data moves across a cluster.