When Large Language Model Meets Optimization

Optimization algorithms and large language models (LLMs) enhance decision-making in dynamic environments by integrating artificial intelligence with traditional techniques. LLMs, with extensive domain knowledge, facilitate intelligent modeling and strategic decision-making in optimization, while optimization algorithms refine LLM architectures and output quality. This synergy offers novel approaches for advancing general AI, addressing both the computational challenges of complex problems and the application of LLMs in practical scenarios. This review outlines the progress and potential of combining LLMs with optimization algorithms, providing insights for future research directions.

• Publication year

  2024

• Venue

  Swarm and Evolutionary Computation

• Publication date

  2024-05-16

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.48550/arXiv.2405.10098arXiv 2405.10098
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Developing a hybrid system for stock selection and portfolio optimization with many-objective optimization based on deep learning and improved NSGA-III

  2024cited by this paper
• Training-Free Transformer Architecture Search With Zero-Cost Proxy Guided Evolution

  2024cited by this paper
• Large Language Model-Based Evolutionary Optimizer: Reasoning with elitism

  2024cited by this paper
• PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Preference Alignment

  2024cited by this paper
• Large Language Model Agent for Hyper-Parameter Optimization

  2024cited by this paper
• RETRACTED: Enhancing large language models with evolutionary fine-tuning for news summary generation

  2024cited by this paper
• When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges

  2024influential reference
• Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

  2024influential reference
• Evolving code with a large language model

  2024cited by this paper
• On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)

  2024cited by this paper
• Enhancing Large Language Models-Based Code Generation by Leveraging Genetic Improvement

  2024cited by this paper
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization

  2024cited by this paper
• A Survey on Multimodal Large Language Models for Autonomous Driving

  2023cited by this paper
• Large Language Models as Evolutionary Optimizers

  2023cited by this paper
• Enhancing Genetic Improvement Mutations Using Large Language Models

  2023cited by this paper
• InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators

  2023cited by this paper
• LLM Performance Predictors are good initializers for Architecture Search

  2023cited by this paper
• Quality-Diversity through AI Feedback

  2023cited by this paper
• Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models

  2023cited by this paper
• Large Language Model for Multi-objective Evolutionary Optimization

  2023cited by this paper
• Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning

  2023cited by this paper
• OptiMUS: Optimization Modeling Using MIP Solvers and large language models

  2023cited by this paper
• An evolutionary model of personality traits related to cooperative behavior using a large language model

  2023cited by this paper
• Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach

  2023cited by this paper
• Can pruning make Large Language Models more efficient?

  2023cited by this paper
• Use Your INSTINCT: INSTruction optimization usIng Neural bandits Coupled with Transformers

  2023cited by this paper
• Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution

  2023cited by this paper
• EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers

  2023cited by this paper
• Language Models as Black-Box Optimizers for Vision-Language Models

  2023cited by this paper
• Structural Pruning of Large Language Models via Neural Architecture Search

  2023cited by this paper
• Large Language Models as Optimizers

  2023cited by this paper
• Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models

  2023cited by this paper
• Black-box Prompt Tuning for Vision-Language Model as a Service

  2023cited by this paper
• Genetic Prompt Search via Exploiting Language Model Probabilities

  2023cited by this paper
• Leveraging Large Language Models for the Generation of Novel Metaheuristic Optimization Algorithms

  2023cited by this paper
• CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models

  2023cited by this paper
• A Survey on Evaluation of Large Language Models

  2023cited by this paper
• PatternGPT :A Pattern-Driven Framework for Large Language Model Text Generation

  2023cited by this paper
• LLMatic: Neural Architecture Search Via Large Language Models And Quality Diversity Optimization

  2023cited by this paper
• LLM-Pruner: On the Structural Pruning of Large Language Models

  2023cited by this paper
• When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario

  2023cited by this paper
• Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives

  2023cited by this paper
• Black-Box Prompt Tuning With Subspace Learning

  2023cited by this paper
• Automatic Prompt Optimization with "Gradient Descent" and Beam Search

  2023cited by this paper
• AutoML-GPT: Automatic Machine Learning with GPT

  2023cited by this paper
• Reliable Gradient-free and Likelihood-free Prompt Tuning

  2023cited by this paper
• Can GPT-4 Perform Neural Architecture Search?

  2023cited by this paper
• MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation

  2023cited by this paper
• Gradient-Free Textual Inversion

  2023cited by this paper
• Sparks of Artificial General Intelligence: Early experiments with GPT-4

  2023cited by this paper
• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

  2023cited by this paper
• EvoPrompting: Language Models for Code-Level Neural Architecture Search

  2023cited by this paper
• Local Model-Based Pareto Front Estimation for Multiobjective Optimization

  2023cited by this paper
• Language Model Crossover: Variation through Few-Shot Prompting

  2023cited by this paper
• SEED: Simple, Efficient, and Effective Data Management via Large Language Models

  2023cited by this paper
• The OpenELM Library: Leveraging Progress in Language Models for Novel Evolutionary Algorithms

  2023cited by this paper
• Program Synthesis with Generative Pre-trained Transformers and Grammar-Guided Genetic Programming Grammar

  2023cited by this paper
• Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation

  2023cited by this paper
• Emergence of Scale-Free Networks in Social Interactions among Large Language Models

  2023cited by this paper
• DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions

  2023cited by this paper
• StableYolo: Optimizing Image Generation for Large Language Models

  2023cited by this paper
• Explaining Data Patterns in Natural Language with Language Models

  2023cited by this paper
• Using Large Language Models for Hyperparameter Optimization

  2023cited by this paper
• Algorithm Evolution Using Large Language Model

  2023cited by this paper
• Multimodal Large Language Models: A Survey

  2023cited by this paper
• Evolution through Large Models

  2022cited by this paper
• A Review on Evolutionary Multitask Optimization: Trends and Challenges

  2022cited by this paper
• Black-Box Tuning for Language-Model-as-a-Service

  2022cited by this paper
• Black-box Prompt Learning for Pre-trained Language Models

  2022cited by this paper
• GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models

  2022cited by this paper
• PanGu Drug Model: learn a molecule like a human

  2022cited by this paper
• PaLM: Scaling Language Modeling with Pathways

  2022cited by this paper
• BBTv2: Towards a Gradient-Free Future with Large Language Models

  2022cited by this paper
• LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models

  2022cited by this paper
• Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards

  2022cited by this paper
• GPS: Genetic Prompt Search for Efficient Few-Shot Learning

  2022cited by this paper
• An improved differential evolution by hybridizing with estimation-of-distribution algorithm

  2022cited by this paper
• Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models

  2022cited by this paper
• MIPLIB 2017: data-driven compilation of the 6th mixed-integer programming library

  2021cited by this paper
• SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions

  2021cited by this paper
• A Survey of Advances in Evolutionary Neural Architecture Search

  2021cited by this paper
• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

  2021cited by this paper
• AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models

  2021cited by this paper
• AutoBERT-Zero: Evolving BERT Backbone from Scratch

  2021cited by this paper
• Evaluating Large Language Models Trained on Code

  2021cited by this paper
• An Adaptive Covariance Scaling Estimation of Distribution Algorithm

  2021cited by this paper
• Solve routing problems with a residual edge-graph attention neural network

  2021cited by this paper
• Genetic Algorithm Based Deep Learning Neural Network Structure and Hyperparameter Optimization

  2021cited by this paper
• Adaptive Dynamic Programming for Control: A Survey and Recent Advances

  2021cited by this paper
• Evolutionary Transfer Optimization - A New Frontier in Evolutionary Computation Research

  2021cited by this paper
• Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey

  2021cited by this paper
• Hybridizing particle swarm optimization with simulated annealing and differential evolution

  2020cited by this paper
• Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum

  2020cited by this paper
• Hybridizing salp swarm algorithm with particle swarm optimization algorithm for recent optimization functions

  2020cited by this paper
• A reinforcement learning approach for optimizing multiple traveling salesman problems over graphs

  2020cited by this paper
• Deep Reinforcement Learning for Traveling Salesman Problem with Time Windows and Rejections

  2020cited by this paper
• A Survey on Evolutionary Neural Architecture Search

  2020cited by this paper
• A hybrid Harris Hawks optimization algorithm with simulated annealing for feature selection

  2020cited by this paper
• A Comprehensive Survey of Neural Architecture Search

  2020cited by this paper
• Language Models are Few-Shot Learners

  2020cited by this paper

• A survey on large language models driven meta-optimizers for automated intelligent optimization

  2026cites this paper
• A comprehensive review on deep learning for multi-objective optimization

  2026cites this paper
• AgenticPruner: MAC-Constrained Neural Network Compression via LLM-Driven Strategy Search

  2026cites this paper
• Score-Guided Identification of Optimal Explanations for Conditional Text Similarity

  2026cites this paper
• Enhancing Mathematical Problem Solving in LLMs through Execution-Driven Reasoning Augmentation

  2026cites this paper
• Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows

  2025cites this paper
• RideAgent: An LLM-Enhanced Optimization Framework for Automated Taxi Fleet Operations

  2025cites this paper
• A bi-level optimization framework for household distributed energy systems: Integrating multiple flexible loads

  2025cites this paper
• EvoContext: Evolving Contextual Examples by Genetic Algorithm for Enhanced Hyperparameter Optimization Capability in Large Language Models

  2025cites this paper
• Synergistic integration of metaheuristics and machine learning: latest advances and emerging trends

  2025cites this paper
• REMoH: A Reflective Evolution of Multi-objective Heuristics approach via Large Language Models

  2025cites this paper
• Digital intelligence enhancing scheduling optimization in process industries: overview and perspectives

  2025cites this paper
• AI Large Models for Power System: A Survey and Outlook

  2025cites this paper
• Instruction Learning Paradigms: A Dual Perspective on White-box and Black-box LLMs

  2025cites this paper
• iPLAN: Redefining Indoor Wireless Network Planning Through Large Language Models

  2025cites this paper
• Adaptive Hybrid PSO-Embedded GA for neuroevolutionary training of multilayer perceptron controllers in VSC-based islanded microgrids

  2025cites this paper
• Multi-objective optimization for smart cities: a systematic review of algorithms, challenges, and future directions

  2025cites this paper
• Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimization

  2025cites this paper
• Natural Language Dispatch of Active Distribution Networks through Sequential LLM Agents

  2025cites this paper
• EvoCut: Strengthening Integer Programs via Evolution-Guided Language Models

  2025cites this paper
• Cooperative Design Optimization through Natural Language Interaction

  2025cites this paper
• Leveraging large language models for the automatic implementation of problems in optimization frameworks

  2025cites this paper
• Integrating Large Language Models into Fluid Antenna Systems: A Survey

  2025cites this paper
• Integrating Digital and Hybrid Learning Models in Environmental Literature Education: Enhancing Speaking Skills Through Virtual Platforms in the Digital Era

  2025cites this paper
• Comprehensive survey on communication structure adaptive control and collaborative optimization for multiagent systems based on deepseek large language model

  2025cites this paper
• A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

  2025cites this paper
• The Rise of Large Language Models: Evolution, Applications, and Future Directions

  2025cites this paper
• Large Language Models in Operations Research: Methods, Applications, and Challenges

  2025cites this paper
• SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling

  2025cites this paper
• AccelerQ: Accelerating Quantum Eigensolvers with Machine Learning on Quantum Simulators

  2025cites this paper
• Refining Hybrid Genetic Search for CVRP via Reinforcement Learning-Finetuned LLM

  2025influential citation
• Large Language Models and Operations Research: A Structured Survey

  2025cites this paper
• Research on a General Solution Framework for Large-scale Optimization Problems Based on Large Language Models

  2025cites this paper
• Enabling Interactive Education With Low‐Latency Large Language Models

  2025cites this paper
• Environment-adaptive robust control of variable speed AGVs using regional pole placement

  2025cites this paper
• A Comprehensive Review on the Application of Large Language Models (LLMs) in Power Systems

  2025cites this paper
• Dynamic Freight Order Allocation Optimization Under Carbon Tax Constraints in SAGIN: Multi‐Objective Optimization and Sustainability Assessment

  2025cites this paper
• Optimizing μPMUs Placement in Power Distribution Networks with Large Language Models

  2025cites this paper
• Large Language Models for Knowledge-Free Network Management: Feasibility Study and Opportunities

  2025cites this paper
• Decision Information Meets Large Language Models: The Future of Explainable Operations Research

  2025cites this paper
• EquivaMap: Leveraging LLMs for Automatic Equivalence Checking of Optimization Formulations

  2025cites this paper
• Combinatorial Optimization for All: Using LLMs to Aid Non-Experts in Improving Optimization Algorithms

  2025cites this paper
• Evaluating the Robustness of Object Detectors Via Delta Debugging, Quickxplain, and A Combinatorial Hybrid

  2025cites this paper
• MambaGas: A Mamba-Based Model for Daily Natural Gas Consumption Forecasting

  2025cites this paper
• An LLM-Based Modeling and Decision Optimization for User-Centric Electric Vehicle Charging

  2024cites this paper
• Nature-Inspired Intelligent Computing: A Comprehensive Survey

  2024cites this paper
• A Systematic Survey on Large Language Models for Algorithm Design

  2024cites this paper
• A comprehensive systematic review on machine learning application in the 5G-RAN architecture: Issues, challenges, and future directions

  2024cites this paper
• LLM-based Optimization of Compound AI Systems: A Survey

  2024cites this paper
• Accelerating Quantum Eigensolver Algorithms With Machine Learning

  2024cites this paper