Big Data Mining

Big data mining has emerged in the last few years as a means to harvest, process, and analyze huge amounts of data. While many techniques have existed for the past two decades to mine data sets, the volume of data being mined is growing exponentially. As a result, these methods are no longer sufficient to ensure high-quality results. Big Data mining refers to the use of advanced machine learning algorithms to search for patterns in massive data sets. The enormous volume of data available today has created an opportunity for organizations to harness the power of their own data resources in order to gain insights that can be used for decision support, customer support, product innovation and business optimization.

Data mining is the process of examining and analysing vast amounts of data in order to uncover patterns in big data. The methods were derived from statistics and artificial intelligence (AI), with a dash of database management tossed in for good measure. Data mining usually has one of two goals: classification or prediction. Classification is the act of sorting data into groups or 'classes' that share some common attribute. For example, in marketing, companies use data mining to automatically assign households to demographic groups such as 'soccer moms', 'dinner-party couples' or 'blue-collar dads'. This allows them to personalise their sales pitches in order to appeal more directly to each group. Classification can also be used for security, fraud detection and medical diagnosis.
Prediction is the process of extrapolating from past observations to make a reasonable guess about what the future holds. Data miners are trying to predict things like customer behaviour, disease outbreaks or credit risk. They'll take a set of information -- called a dataset -- and look for patterns that suggest what might happen in the future. A data miner will look at millions of records of people's buying habits, for example, and try to find clusters of customers who have similar tastes. Then she will make a prediction about the kinds of products they are likely to buy together.

The kinds of Big Data that can be mined are:
Flat Files
Relational Databases
DataWarehouse
Transactional Databases
Multimedia Databases
Spatial Databases
Time Series Databases
World Wide Web(WWW)

The areas where Data Mining is widely used are as follows: Financial Data Analysis: Financial data in the banking and financial industries is generally credible and of high quality that requires systematic data analysis and data mining. Retail Industry: In the retail industry, data mining aids in the identification of client buying patterns and trends, resulting in improved customer service and increased customer retention and satisfaction. Telecommunication Industry: Data mining in the telecommunications industry aids in the detection of telecommunication patterns, the detection of fraudulent actions that promotes better utilisation of resources, and the improvement of service quality. Biological Data Analysis: Bioinformatics includes biological data mining as a significant component. Data mining contributes in analysis of genomic networks and protein pathways, the discovery of structural patterns and so on. Intrusion Detection: It is used for developing data mining algorithms for intrusion detection and to construct discriminating qualities, use association and correlation analysis, as well as aggregation.

There are various procedures involved in data mining. The procedures are as follows: Data Preparation: The cleansing of a huge volume of data to get the values to an amendable form is referred to as data preparation. It is accomplished by employing statistical approaches to derive simple profits from complex ones. Model Building: It has to do with selecting a specific model for calculating performance. It entails a variety of strategies in order to get the best results from the data mining process. Composition: This is the final stage that allows you to select the prior model to apply to your new data in order to achieve the desired results.

The 5 data mining techniques are:

Analysis:This method is used to find vital and relevant data and metadata. It's used to divide data into separate categories.

Association: It refers to a technique for identifying meaningful relationships (dependency modelling) among various variables in massive databases. This technique can assist you in uncovering hidden patterns in the data that can be used to discover variables within the data as well as the co-occurrence of many variables that appear frequently in the dataset.

Anomaly:This is the observation of data objects in a dataset that do not follow a predictable pattern or behave in a predictable manner. Outliers, novelties, noise, deviations, and exceptions are all terms used to describe anomalies. They frequently provide crucial and useful data.

Clustering: Clustering analysis is the process of identifying groups and clusters in data so that the degree of association between two objects is highest if they belong to the same group and lowest if they do not. Customer profiles can be created as a result of this analysis.

Regression: The technique of discovering and analysing the relationship between variables is known as regression analysis. If one of the independent variables is changed, it can help you comprehend how the characteristic value of the dependent variable changes.