WHAT IS BIG DATA?

  100% Pass and No Plagiarism Guaranteed

WHAT IS BIG DATA?

Introduction:

This assignment will provide a summative assessment of your understanding of Big Data Systems and related technologies. Three mini-tasks that are to be completed have the following aims:

• Introduce Big Data in the context of a given organisation (See Task 1)
• Understand the problems of working with Big Data and describe technologies that specialize in catering for Big Data (See Task 2)
• Use a software package that is designed for Big Data Systems to perform a simple analytical task (See Task 3)

Task 1 – Introduce Big Data

In The Context of Amazon; Amazon is an online book retailer that has expended its retail offering far beyond books over the last decade (www.amazon.com).

• Define Big Data in terms of the four V’s. Describe how each V could apply to Amazon. (E.g. ‘Volume’ is one of the V’s. What data would Amazon likely to be capturing to qualify?)
• Give an example from Amazon to illuminate your points for each of the 4 V’s discussed above                                                                                                                 (12 Marks)

Task 2 – Big Data Technologies

Hadoop is a technological framework that enables processing of large datasets at the scale of Big Data. Your task is to research and understand Hadoop. Your description should include:

• What is Hadoop?
• What are the technological challenges of working with Big Data?
• How does Hadoop framework overcome abovementioned challenges? (10 Marks)

Task 3 – Big Data Analytics with Orange Software Package

The dataset that we will be using is contained in the file Titanic.tab that is made available on CloudDeakin under Resources->Assignment 3->Titanic.tab

This by no means is a Big Data set. In order to simplify the analytical task (as promised in lectures) we will settle for using a smaller and simpler dataset. Your task is to:

• Analyse the full dataset using Orange and try to get an insight.
• Take a random sample of 200 records and perform the same analysis. State your findings. Are your conclusions similar to what you have found previously? Explain why or why not.
• Under what circumstances would it be permissible to use a random sample from a full dataset for analysis? Under what circumstances would it raise red flags? (11.3 Marks)