Artificial Intelligence (AI) spans a wide range of techniques, tools, and theories rooted in computer science, mathematics, and cognitive science. This glossary presents essential terms frequently used in the AI and machine learning community — not only buzzwords, but also the mathematical and algorithmic foundations that drive intelligent systems.
Artificial Intelligence (AI)
A branch of computer science aimed at building machines capable of performing tasks that typically require human intelligence, such as perception, reasoning, decision-making, and language understanding.
Machine Learning (ML)
A subset of AI that focuses on developing algorithms that enable systems to learn patterns from data and make predictions or decisions without being explicitly programmed for each task.
Deep Learning
A class of machine learning algorithms that use multi-layered artificial neural networks to model high-level abstractions in data, often achieving state-of-the-art results in image, speech, and text tasks.
Neural Network
A computational model inspired by the structure of biological brains, composed of layers of nodes (neurons) connected by weighted links. Each neuron processes inputs using an activation function and passes the output forward.
Backpropagation
A supervised learning algorithm used to train neural networks. It computes the gradient of the loss function with respect to each weight by applying the chain rule in reverse through the network, enabling efficient weight updates.
Gradient Descent
An optimization algorithm used to minimize the loss function by iteratively updating model parameters in the opposite direction of the gradient, proportional to a learning rate.
Loss Function
A mathematical function that quantifies the difference between the predicted outputs of a model and the actual targets. Common examples include Mean Squared Error and Cross-Entropy Loss.
Activation Function
A non-linear function applied to the output of each neuron in a neural network, enabling the model to learn complex patterns. Examples include ReLU, Sigmoid, and Tanh.
Supervised Learning
A learning paradigm where the model is trained on input-output pairs (labeled data), learning to map inputs to known targets.
Unsupervised Learning
A method where the model learns patterns from data without labels, often used for clustering, dimensionality reduction, and anomaly detection.
Reinforcement Learning
A learning framework where an agent interacts with an environment to maximize cumulative rewards through trial and error, guided by a reward signal.
Transfer Learning
A technique where a model trained on one task is adapted to a different but related task, typically by fine-tuning the later layers of a neural network.
Fine-Tuning
The process of continuing the training of a pre-trained model on new data to adapt it to a more specific task or domain.
Embedding
A dense, continuous vector representation of discrete inputs (like words or images) that captures semantic or structural similarity in a lower-dimensional space.
Vector Database
A specialized data store designed to index and retrieve embedding vectors using similarity measures such as cosine distance, enabling efficient search and recommendation systems.
Natural Language Processing (NLP)
A field of AI focused on enabling machines to understand, generate, and manipulate human language, including tasks like translation, sentiment analysis, and question answering.
Transformer
A neural network architecture that uses self-attention mechanisms to process sequences in parallel, forming the backbone of large language models like GPT.
Large Language Model (LLM)
A type of transformer-based model trained on massive text corpora to generate and understand human-like language, capable of answering questions, summarizing content, and more.
Token
A unit of text used by language models, often representing words, subwords, or characters, depending on the tokenization strategy.
Prompt Engineering
The practice of designing effective inputs (prompts) to guide the behavior and output of language models, particularly in zero-shot or few-shot learning scenarios.
Zero-Shot Learning
A model’s ability to perform tasks it was never explicitly trained for, using generalization learned from other tasks or contexts.
Model Inference
The phase where a trained model is used to make predictions or generate outputs on new, unseen data.
Training
The process of learning the optimal parameters of a model using data, usually through iterative optimization based on feedback from a loss function.
Overfitting
A modeling issue where a model learns the training data too well, including noise, and fails to generalize to new data.
Underfitting
Occurs when a model is too simple to capture the underlying structure of the data, resulting in poor performance on both training and test data.
Explainability (XAI)
A set of methods that make AI systems more transparent by clarifying how inputs influence predictions or decisions, often critical in high-stakes applications.
Bias
Unintended and systematic errors in AI models caused by imbalanced data, flawed assumptions, or social/cultural influences, leading to unfair or inaccurate outcomes.
Hallucination
When a generative model produces outputs that appear coherent or factual but are actually false, fabricated, or misleading.
Grounding
The process of anchoring AI model outputs to factual, verified, or domain-specific information, often using retrieval or structured knowledge bases.
Retrieval-Augmented Generation (RAG)
An approach that enhances generative models by combining them with document retrieval, ensuring outputs are based on up-to-date, verifiable sources.
Conclusion
Understanding these foundational terms is critical for navigating the field of AI, whether you're developing systems, analyzing their impacts, or simply trying to keep up with rapid innovation. As AI continues to expand across industries, fluency in this language is an asset for anyone working in or alongside the technology.