Describe the Data Mining Functions Data Mining Techniques?

Data mining refers to the broadly-defined set of techniques involving finding meaningful patterns - or information - in large amounts of raw data.

At a very high level, data mining is performed in the following stages (note that terminology and steps taken in the data mining process varies by data mining practitioner):

1. Data collection: gathering the input data you intend to analyze

2. Data scrubbing: removing missing records, filling in missing values where appropriate

3. Pre-testing: determining which variables might be important for inclusion during the analysis stage

4. Analysis/Training: analyzing the input data to look for patterns

5. Model building: drawing conclusions from the analysis phase and determining a mathematical model to be applied to future sets of input data

6. Application: applying the model to new data sets to find meaningful patterns

Data mining can be used to classify or cluster data into groups or to predict likely future outcomes based upon a set of input variables/data.

Common data mining techniques and tools include, for example:

a. decision tree learning

b. Bayesian classification

c. neural networks

During the analysis phase (sometimes also called the training phase), it is customary to set aside some of the input data so that it can be used to cross-validate and test the model, respectively. This is an important step taken in order to to avoid "over-fitting" the model to the original data set used to train the model, which would make it less applicable to real-world applications.