|
Challenges in Mining Big DataBog'liq 2.1 ga oidChallenges in Mining Big Data
The mining of Big Data involves multiple processes that
facing a lot of challenges, like [3] [4]:
•
Heterogeneity and Incompleteness:
-
the data do not have a particular format. it is a mixed
data based on different patterns or rules. Data can be both struc-
tured and unstructured. 80% of the data generated by organiza-
tions are unstructured like images, pdf documents, video, audio
etc. and they cannot be stored in row/ column format as struc-
tured data. so, it needs sophisticated technology that can deal
with heterogeneous data.
-
Incomplete data refers to the missing of data field val-
ues. While most modern data mining algorithms have inbuilt
solutions to handle missing values it seeks to impute missing val-
ues in order to produce improved models.
•
Scale and complexity:
The traditional technologies are not enough for man-
aging the increasing volumes of data. Big data analysis is consid-
ered as a challenge due to scalability and complexity of data that
needs to be analyzed.
•
Speed/Velocity:
Big data has a speed/velocity in data. and it needs to be
processed within a certain period of time, otherwise, the results
will become not valuable.
•
Privacy and Security:
Security and privacy play a significant role in big data
research and technology, especially in social media, bank trans-
actions and health information. Developing algorithms that deal
with personal data is a major challenge.
Techniques for Big Data Mining
Big data analysis is the complex process of examin-
ing large and varied data sets to get useful information, extract
knowledge and hidden patterns, that can help companies or ap-
plication make informed business decisions and manage their
problems. For analysis the huge amount of data requires sophis-
ticated technologies. Emerging technologies such as the Hadoop
framework and MapReduce offer new and exciting ways to pro-
cess and transform complex, unstructured and large amounts of
data, into meaningful knowledge [3] [4] [5].
|
| |
