POMDPStressTesting.jl

Adaptive stress testing of black-box systems, implemented within the POMDPs.jl ecosystem.

See the documentation for more details.

Citation

If you use this package for research purposes, please cite the following:

@article{moss2021pomdpstresstesting,
  title = {{POMDPStressTesting.jl}: Adaptive Stress Testing for Black-Box Systems},
  author = {Robert J. Moss},
  journal = {Journal of Open Source Software},
  year = {2021},
  volume = {6},
  number = {60},
  pages = {2749},
  doi = {10.21105/joss.02749}
}

Interface

To stress test a new system, the user has to define the GrayBox and BlackBox interface outlined in src/GrayBox.jl and src/BlackBox.jl.

GrayBox Interface

The GrayBox simulator and environment interface includes:

• GrayBox.Simulation type to hold simulation variables
• GrayBox.environment(sim::Simulation) to return the collection of environment distributions
• GrayBox.transition!(sim::Simulation) to transition the simulator, returning the log-likelihood

BlackBox Interface

The BlackBox system interface includes:

• BlackBox.initialize!(sim::Simulation) to initialize/reset the system under test
• BlackBox.evaluate!(sim::Simulation) to evaluate/execute the system under test
• BlackBox.distance(sim::Simulation) to return how close we are to an event
• BlackBox.isevent(sim::Simulation) to indicate if a failure event occurred
• BlackBox.isterminal(sim::Simulation) to indicate the simulation is in a terminal state

Functions ending with ! may modify the Simulation object in place.

Solvers

Several solvers are implemented.

Reinforcement learning solvers

• MCTSPWSolver

Deep reinforcement learning solvers¹

• TRPOSolver
• PPOSolver

Stochastic optimization solvers

• CEMSolver

Baseline solvers

• RandomSearchSolver

Example

An example implementation of the AST interface is provided for the Walk1D problem:

• Julia source: test/Walk1D.jl
• Jupyter notebook: Walk1D.ipynb
• Descriptive tutorial-style write-up: walk1d.pdf (created using TeX.jl)

Installation

Install the POMDPStressTesting.jl package via:

] add POMDPStressTesting

Testing

To run the test suite, you can use the Julia package manager.

] test POMDPStressTesting

Contributing

We welcome contributions! Please fork the repository and submit a new Pull Request.

---

Package maintained by Robert Moss: [email protected]

¹ TRPO and PPO thanks to Shreyas Kowshik's initial implementation.