Study Card: t-SNE (t-distributed Stochastic Neighbor Embedding)

Direct Answer

• t-SNE is a dimensionality reduction technique specifically designed for visualizing high-dimensional data in lower dimensions (typically 2D or 3D). It aims to preserve the local structure of the data, meaning that points close together in the high-dimensional space remain close together in the low-dimensional embedding. t-SNE uses probability distributions to model the relationships between data points, minimizing the Kullback-Leibler (KL) divergence between the high-dimensional and low-dimensional distributions.
• Key points: Dimensionality Reduction for Visualization, Preserving Local Structure, Probability Distributions, KL Divergence.
• Practical Applications: Visualizing clusters in high-dimensional data, exploring patterns in complex datasets, understanding the relationships between data points. Used in fields like genomics, image processing, and natural language processing.

Key Terms

• Dimensionality Reduction: The process of reducing the number of variables in a dataset while preserving important information.
• Local Structure: The relationships between nearby points in a high-dimensional space.
• Probability Distribution: A function that describes the likelihood of different outcomes. t-SNE uses Gaussian and t-distributions.
• Kullback-Leibler (KL) Divergence: A measure of how one probability distribution differs from a second, expected probability distribution. t-SNE minimizes the KL divergence between the high-dimensional and low-dimensional distributions.
• Perplexity: A hyperparameter in t-SNE that controls the effective number of neighbors considered for each point.

Example

Imagine you have a dataset of thousands of images, each represented by a high-dimensional vector of pixel values. t-SNE can be used to map these high-dimensional vectors to a 2D plane, where similar images are clustered together visually. This allows you to see patterns and clusters in the image data that would be difficult or impossible to discern in the original high-dimensional space. For instance, images of cats might cluster in one region, images of dogs in another, and images of cars in a third.

Code Implementation

import numpy as np
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
from sklearn.datasets import load_digits

# Load a sample dataset (digits dataset)
digits = load_digits()
X = digits.data
y = digits.target

# Apply t-SNE
tsne = TSNE(n_components=2, perplexity=30.0, learning_rate=200.0, n_iter=1000, random_state=42) # Example parameters. Tuning might be needed.
X_embedded = tsne.fit_transform(X)

# Plot the embedded data
plt.figure(figsize=(10, 8))
plt.scatter(X_embedded[:, 0], X_embedded[:, 1], c=y, cmap='Spectral')
plt.colorbar(label="Digit Label")
plt.title("t-SNE Visualization of Digits Dataset")
plt.xlabel("t-SNE Component 1")
plt.ylabel("t-SNE Component 2")
plt.show()

Related Concepts

• PCA (Principal Component Analysis): Another dimensionality reduction technique, but PCA focuses on preserving global structure (variance) rather than local structure. Interviewers might ask you to compare and contrast t-SNE and PCA. Follow up: When would you prefer t-SNE over PCA and vice-versa?
• UMAP (Uniform Manifold Approximation and Projection): A newer dimensionality reduction technique that is often compared to t-SNE. UMAP is generally faster and better at preserving global structure than t-SNE. Follow-up questions could be around the relative advantages and disadvantages of UMAP and t-SNE.
• Manifold Learning: t-SNE is a type of manifold learning, which assumes that high-dimensional data lies on a lower-dimensional manifold. Follow-up question: what is a manifold? What other manifold Learning techniques are there?
• Visualization Techniques: Interviewers might ask about other visualization techniques for high-dimensional data, such as parallel coordinates plots, scatterplot matrices, or glyphs. They may ask for specific examples and which are suitable for particular datasets and data exploration questions.