# Hadoop - Course Contents 1 Basics What is Big data? How Data is Organised and Analyzed Different characteristics of Big data Reasons behind learning Hadoop What is Hadoop? Hadoop History and Evolution Principles of Hadoop Different Features of Hadoop 2 HDFS Concepts What is HDFS? Understanding Blocks in HDFS Understanding DataNode Understanding NameNode Metadata in NameNode Understanding Secondary NameNode How Read Operation works in HDFS How Write Operation works in HDFS 3 Linux Commands Introduction to a Linux Shell Environment - What is Shell? Learning User commands like pwd, who, etc. Learning commands related to Calendar Commands related to creating a new file Commands related to displaying the contents of a file Commands related to listing directories and files Commands related to creating a new directory Commands related to copy, move and comparing different files Commands related to counting lines, words and characters in a file Commands related to running Processes and their Statistics including grep Practice sessions on Linux Commands 4 HDFS Commands Commands to start and stop dfs and yarn Commands to list directories and files in Hadoop Commands to find size and empty space in directories Commands to create directories, put files into HDFS and later remove files Commands to move files, get files and list the files in HDFS Commands to view content of a file in HDFS Commands to copyFromLocal, copyToLocal, moveFromLocal and moveToLocal Commands to checksum and count the number of directories, files and bytes Commands related to Access Control Lists (ACLs) Commands to merge different files Commands to change group and their permissions Commands to list file statistics Commands to how to append a file to another file Commands to create a zero byte file Commands to empty the trash folder in HDFS Commands to reduce the size of a file (using truncate) Commands to check file system consistency Commands to check the classpath of HDFS Commands to verify missing and corrupted blocks of a file Commands to get configurations of NameNode and secondary NameNode Commands to create, list and delete snapshots Practice Sessions on HDFS Commands 5 Hadoop I/O Introduction to working of Serialization and Deserialization Will learn about Writable Interface Will learn about Writable Comaparables and Comparators Will learn about Writable classes Will learn about IntWritable class and methods Will learn about FloatWritable class and methods Will learn about ByteWritable and BooleanWritable classes and methods Will learn about Text class and methods Will learn about BytesWritable, NullWritable and ObjectWritable classes Different practice programs on Writable classes and methods Will learn about Configuration, Job and JobConf classes 6 MapReduce What is MapReduce? Different terminology used in MapReduce Working Procedure of MapReduce Will learn about JobClient in MapReduce Working process of JobTracker in MapReduce TaskTracker procedure in MapReduce Will learn about MapTask in MapReduce Will learn about ReduceTask in MapReduce How the Combiner works in MapReduce Advantages of Partitioners in MapReduce 7 Solving WordCount Problem in MapReduce Understanding a WordCount problem Working with WordCount program Will learn to write the code in Driver class Will learn to write the code in Mapper class Will learn to write the code in Reducer class 8 Solving Marks Computation Problem in MapReduce Understanding a Marks Average problem Will learn to write a program on how to find average marks of each student Will learn to write the code in Driver, Mapper and Reducer classes for similar class of problems 9 Solving Electricity Units Consumption Problem in MapReduce Understanding a problem related to consumption of Average Electricity Units Will learn to write a program on how to find Average Consumption of Electricity Units greater than 30 Will learn to write the code in Driver, Mapper and Reducer classes for similar class of problems 10 Solving Maximum Temperature Problem in MapReduce Understanding a problem related to computation of Maximum Temperature Will learn to write a program on how to find Maximum Temperature of any year using MapReduce Will learn to write the code in Driver, Mapper and Reducer classes for similar class of problems 11 Solving WordCount Problem using Multiple Reducers in MapReduce Understanding a problem related to WordCount using Multiple Reducers Will learn to write a program using Partitioner Will learn to write the code in Driver, Mapper, Reducer and Partitioner classes for similar class of problems 12 Solving Votes Count Problem using MapReduce Understanding a problem related to Counting Votes Will learn to write a program on counting of votes in a General Election using MapReduce Will learn to write the code in Driver, Mapper and Reducer classes for similar class of problems 13 Solving Matrix Multiplication Problem using MapReduce Understanding a problem related to Matrix Multiplication Will learn to write a program on Matrix Multiplication Problem using MapReduce Will learn to write the code in Driver, Mapper and Reducer classes for similar class of problems 14 Apache Pig Introduction Understanding Pig and PigLatin Different Datatypes in PigLatin Will learn about Schema in PigLatin Will learn about Schema definition for Tuple, Bag and Map datatype 15 Grunt Shell in Pig Will learn to execute PigLatin in Local mode and MapReduce moode Will learn executing PigLatin Script Will learn ShShell Commands in Grunt Will learn FsShell Commands in Grunt 16 Operators in Pig Understanding how to use Load Operator Will learn about Store and Dump Operator Will learn how to use Describe Operator Will learn about Foreach Operator with Filter Operator Will learn usage of Group and Cogroup Operators in Pig Understanding Order By Operator Will learn how to use Distinct Operator Will learn about Join Operator Working with Limit Operator Will learn usage of Parallel Operator in Pig Understanding Flatten Operator Will learn how to use Specialized Join Operator Will learn to work with Union and Cross Operator 17 Hive Understanding Hive Architecture and different components of Hive Numeric Datatypes in Hive String Datatypes in Hive Date and Miscellaneous datatypes Will learn about Datatype conversions in Hive Will learn different Complex Datatypes Understanding Arithematic Operators Understanding Predicate and Logical Operators Will learn different Mathematical and Aggreagate functions Will learn Collection and Date functions Will learn different String functions 18 Hive Queries Queries on Create, Alter and Drop Databases Creating different types of tables like managed, external and partitioned Queries on Alter and Drop Tables Queries on how to load input data to table from Local filesystem and HDFS Queries using SELECT clause Queries on how to Insert data into tables Queries using WHERE clause Queries using CREATE TABLE AS SELECT(CTAS) Queries using LIMIT clause Queries using Nested SELECT Queries on creating column INDEX of a table Queries on Inner JOIN and Outer JOIN Queries using ORDER BY and UNION ALL clause The complete course includes extensive coding and practice exercises.