WES: Agent-based User Interaction Simulation on Real Infrastructure

We introduce the Web-Enabled Simulation (WES) research agenda, and describe FACEBOOK's WW system. We describe the application of WW to reliability, integrity and privacy at FACEBOOK1, where it is used to simulate social media interactions on an infrastructure consisting of hundreds of millions of lines of code. The WES agenda draws on research from many areas of study, including Search Based Software Engineering, Machine Learning, Programming Languages, Multi Agent Systems, Graph Theory, Game AI, and AI Assisted Game Play. We conclude with a set of open problems and research challenges to motivate wider investigation.

• Publication year

  2020

• Venue

  International Conference on Software Engineering

• Publication date

  2020-04-11

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/3387940.3392089arXiv 2004.05363
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• The Assessor's Dilemma: Improving Bug Repair via Empirical Game Theory

  2021cited by this paper
• How Relevant Is the Turing Test in the Age of Sophisbots?

  2019cited by this paper
• Bisecting commits and modeling commit risk during testing

  2019cited by this paper
• Reinforcement learning

  2019cited by this paper
• RecSim: A Configurable Simulation Platform for Recommender Systems

  2019cited by this paper
• Grandmaster level in StarCraft II using multi-agent reinforcement learning

  2019influential reference
• Causality

  2019cited by this paper
• SapFix: Automated End-to-End Repair at Scale

  2019cited by this paper
• Deploying Search Based Software Engineering with Sapienz at Facebook

  2018cited by this paper
• Model Checking

  2018cited by this paper
• From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis

  2018cited by this paper
• Genetic Improvement of Software: A Comprehensive Survey

  2018cited by this paper
• Trade-offs in continuous integration: assurance, security, and flexibility

  2017cited by this paper
• Robotic Testing of Mobile Apps for Truly Black-Box Automation

  2017cited by this paper
• A Survey of App Store Analysis for Software Engineering

  2017cited by this paper
• The N-Tuple bandit evolutionary algorithm for automatic game improvement

  2017cited by this paper
• Program Synthesis

  2017cited by this paper
• Causal impact analysis for app releases in google play

  2016cited by this paper
• The Oracle Problem in Software Testing: A Survey

  2015cited by this paper
• Efficient dependency detection for safe Java test acceleration

  2015cited by this paper
• Search based software engineering for software product line engineering: a survey and directions for future work

  2014cited by this paper
• Co-evolutionary automatic programming for software development

  2014cited by this paper
• An empirical analysis of flaky tests

  2014cited by this paper
• Community standards

  2013cited by this paper
• How to Approximate a Set without Knowing Its Size in Advance

  2013cited by this paper
• Software ecosystems - A systematic literature review

  2013cited by this paper
• Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost

  2013cited by this paper
• A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

  2013cited by this paper
• Game AI revisited

  2012cited by this paper
• Search-based software engineering: Trends, techniques and applications

  2012influential reference
• Automatically generating realistic test input from web services

  2011cited by this paper
• Constraint‐based human resource allocation in software projects

  2011cited by this paper
• Cooperative Co-evolutionary Optimization of Software Project Staff Assignments and Job Scheduling

  2011cited by this paper
• Game theory.

  2011cited by this paper
• Multi-Agent Systems and Simulation: A Survey from the Agent Commu- nity’s Perspective

  2009cited by this paper
• A systematic review of software fault prediction studies

  2009cited by this paper
• Model checking

  2009cited by this paper
• An experiment in automatic game design

  2008cited by this paper
• Software Agents

  2008cited by this paper
• The road ahead for Mining Software Repositories

  2008cited by this paper
• Software Testing Research: Achievements, Challenges, Dreams

  2007cited by this paper
• Methods for Empirical Game-Theoretic Analysis

  2006cited by this paper
• Designing Economic Mechanisms

  2006cited by this paper
• Realistic load testing of Web applications

  2006cited by this paper
• Supply chain simulation tools and techniques: a survey

  2005cited by this paper
• Mutation Testing Using Genetic Algorithms: A Co-evolution Approach

  2004cited by this paper
• A post-placement side-effect removal algorithm

  2002cited by this paper
• Software product lines - practices and patterns

  2002cited by this paper
• A fast and elitist multiobjective genetic algorithm: NSGA-II

  2002cited by this paper
• Software Product Lines: Practices and Patterns

  2001cited by this paper
• Agent Theories, Architectures, and Languages: A Survey

  1995cited by this paper
• Introduction to Graph Theory

  1995cited by this paper
• Software agents

  1994cited by this paper
• Reinforcement Learning

  1992influential reference

• Just-in-Time Catching Test Generation at Meta

  2026cites this paper
• Breaking Single-Tester Limits: Multi-Agent LLMs for Multi-User Feature Testing

  2025cites this paper
• Harden and Catch for Just-in-Time Assured LLM-Based Software Testing: Open Research Challenges

  2025cites this paper
• Investigating Tax Evasion Emergence Using Dual Large Language Model and Deep Reinforcement Learning Powered Agent-based Simulation

  2025cites this paper
• Assured LLM-Based Software Engineering

  2024cites this paper
• Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling

  2024cites this paper
• Social and Ethical Risks Posed by General-Purpose LLMs for Settling Newcomers in Canada

  2024cites this paper
• The 13th International Workshop on Genetic Improvement(GI @ ICSE 2024)

  2024cites this paper
• Enhancing Testing at Meta with Rich-State Simulated Populations

  2024influential citation
• Simulation-Driven Automated End-to-End Test and Oracle Inference

  2023cites this paper
• Socialz: Multi-Feature Social Fuzz Testing

  2023influential citation
• Polarization in Social Media: A Virtual Worlds-Based Approach

  2023cites this paper
• Challenges of Testing an Evolving Cancer Registration Support System in Practice

  2023cites this paper
• SciNet: Codesign of Resource Management in Cloud Computing Environments

  2023cites this paper
• Search-Based Software Engineering in the Era of Modern Software Systems

  2023cites this paper
• Evaluating recommender systems in relation to the dissemination of illegal and harmful content in the UK

  2023cites this paper
• Multiagent Simulators for Social Networks

  2023cites this paper
• Synthetic End-User Testing: Modeling Realistic Agents Based on Behavioral Examples

  2022cites this paper
• Online testing of RESTful APIs: promises and challenges

  2022cites this paper
• SAT-based techniques for Approximate Circuit Design

  2022influential citation
• Prismal view of ethics

  2022cites this paper
• Testing non-testable programs using association rules

  2022cites this paper
• Scaling Genetic Improvement and Automated Program Repair

  2022cites this paper
• Undulate: A framework for data-driven software engineering enabling soft computing

  2022cites this paper
• Modelling Cooperation in Network Games with Spatio-Temporal Complexity

  2021cites this paper
• Test’n’Mo: a collaborative platform for human testers and intelligent monitoring agents

  2021cites this paper
• Facebook’s Cyber–Cyber and Cyber–Physical Digital Twins

  2021influential citation
• Metamorphic Testing for Hybrid Simulation Validation

  2021cites this paper
• Platform ecosystems and its modelling: Russian evidence

  2021cites this paper
• Testing Web Enabled Simulation at Scale Using Metamorphic Testing

  2021influential citation
• RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems

  2021cites this paper
• Measurement Challenges for Cyber Cyber Digital Twins: Experiences from the Deployment of Facebook's WW Simulation System

  2021influential citation
• Artificial Intelligence inspired methods for the allocation of common goods and services

  2021cites this paper
• Genetic Improvement @ ICSE 2020

  2020cites this paper
• Evolving sqrt into 1/x via software data maintenance

  2020cites this paper
• Open Loop In Natura Economic Planning

  2020cites this paper
• FITNESS PROPAGATION AND SELECTION OF SEED NODE FOR MESSAGE DIFFUSION IN SOCIAL NETWORKS

  year unknowncites this paper
• A Review of Social Media Simulation using Multi-agent Models

  year unknowncites this paper
• FAUSTA: Scaling Dynamic Analysis with Traffic Generation at WhatsApp

  year unknowncites this paper