**Machine Learning**

**Machine Learning Algorithms**

**Application**

**Life Cycle**

**AI VS ML**

**Data Science vs Machine Learning**

**Machine Learning vs Deep Learning**

**How to get Datasets**

**Data Pre-processing**

**Supervised Machine Learning**

**Unsupervised Machine Learning**

**Supervised Vs Unsupervised Machine Learning**

**Regression Analysis**

**Linear Regression**

**Simple Linear Regression**

**Multiple Linear Regression**

**Backward Elimination**

**Polynomial Regression**

**Classification Algorithm**

**Logistic Regression**

**K-NN Algorithm**

**Support Vector Machine Algorithm**

**Naive Bayes Classifier**

**Regression vs Classification**

**Linear Regression vs Logistic Regression**

**Decision Tree Classification**

**Random Forest Algorithm**

**Clustering**

**Hierarchical Clustering**

**K-Means Clustering**

**Apriori Algorithm**

**Association Rule**

**Confusion Matrix**

**Cross Validation**

**Dimensionality Deduction Technique**

**Overfitting & Underfitting**

**Principal Component Analysis**

**P Value**

**Regularization**

**Examples**

**Semi Supervised Learning**

**Essential Mathematics**

**Overfitting**

**Encoding Techniques**

# Principal Component Analysis

**
In this page, we will learn What is Principal Component Analysis (PCA)?, Some common terms used in PCA algoritm,
Principal Components in PCA, Steps for PCA algorithm, Applications of Principal Component Analysis.**

## What is Principal Component Analysis (PCA)?

Principal Component Analysis is an unsupervised learning algorithm used in machine learning to reduce dimensionality. With the help of orthogonal transformation, it is a statistical technique that turns observations of correlated features into a set of linearly uncorrelated data. The Principal Components are the newly altered features. It's one of the most widely used programs for exploratory data analysis and predictive modeling. It's a method for extracting strong patterns from a dataset by lowering variances.

PCA seeks out the lowest-dimensional surface on which to project the high-dimensional data.

The variance of each characteristic is taken into account by PCA since the high attribute indicates a good separation between the classes and so minimizes dimensionality. Image processing, movie recommendation systems, and optimizing power allocation in multiple communication channels are some of the real-world uses of PCA. Because it is a feature extraction technique, it keeps the significant variables while discarding the less important ones.

The PCA algorithm is based on the following mathematical concepts:

- Variance and Covariance
- Eigenvalues and Eigen factors

## Some common terms used in PCA algoritm:

**Dimensionality:** The number of features or variables in
a dataset is referred to as its dimensionality. It's the
amount of columns in the dataset, to put it simply.

**Correlation:** It is a term that describes how closely
two variables are related to one another. For example, if one
variable changes, the other variable changes as well. The
correlation value can be anywhere between -1 and +1. If the
variables are inversely proportional to each other, the result
is -1, and if the variables are directly proportional to each
other, the result is +1.

**Orthogonal:** It denotes that the variables are unrelated
to one another, and so the correlation between them is zero.

**Eigenvectors:** If you have a square matrix M and a
non-zero vector v, you have eigenvectors. If Av is the scalar
multiple of v, then v is an eigenvector.

**Covariance Matrix:** The Covariance Matrix is a matrix
that contains the covariance between two variables.

## Principal Components in PCA

The Principal Components are the converted new features or the result of PCA, as stated above. The number of PCs in this dataset is either equal to or less than the number of original features in the dataset. The following are some of the properties of these main components:

- The linear combination of the original features must be the main component.
- These components are orthogonal, which means there is no link between two variables.
- When going from 1 to n, the importance of each component declines, indicating that the 1 PC is the most important and the n PC is the least important.

## Steps for PCA algorithm

**Getting the dataset**

To begin, we must divide the input dataset into two halves, X
and Y, with X being the training set and Y being the
validation set.

**Representing data into a structure**

Now we'll use a structure to represent our data. As an
example, the two-dimensional matrix of independent variable X
will be represented. Each column correlates to the Features,
and each row corresponds to the data elements. The dataset's
dimensions are determined by the number of columns.

**Standardizing the data**

We'll normalize our data in this step. In a given column, for
example, features with a large variance are more essential
than features with a lower variance.

If the importance of features is unaffected by the variance of the feature, we shall divide each data item in a column by the column's standard deviation. The matrix will be referred to as Z in this case.

**Calculating the Covariance of Z**

We will take the matrix Z and transpose it to get the
covariance of Z. We'll multiply it by Z after it's been
transposed. The Covariance matrix of Z will be the output
matrix.

**Calculating the Eigen Values and Eigen Vectors**

The eigenvalues and eigenvectors for the resulting covariance
matrix Z must now be calculated. The directions of the axes
with high information are called eigenvectors or the
covariance matrix. The eigenvalues are defined as the
coefficients of these eigenvectors.

**Sorting the Eigen Vectors**

We'll take all of the eigenvalues and arrange them in
decreasing order, from largest to smallest, in this phase. In
the eigenvalues matrix P, sort the eigenvectors in the same
order. P* will be the name of the resulting matrix.

**Calculating the new features Or Principal Components**

We'll calculate the new features here. We'll do this by
multiplying the P* matrix by the Z. Each observation in the
resulting matrix Z* is a linear combination of the original
features. The Z* matrix's columns are all independent of one
another.

**Remove less or unimportant features from the new
dataset.**

We'll determine what to keep and what to discard now that the
new feature set has arrived. It indicates that only relevant
or crucial features will be kept in the new dataset, while
unimportant ones will be deleted.

## Applications of Principal Component Analysis

- PCA is mostly utilized as a dimensionality reduction technique in AI applications like computer vision and picture compression.
- If the data includes a lot of dimensions, it can also be utilized to find hidden patterns. Finance, data mining, psychology, and other fields employ PCA.