Problems connecting to AutomotvieSimulator.jl

[ll7]

Hello, I am currently trying to implement an example of the POMDPStressTesting.jl with the environment of AutomotiveSimulator.jl. Unfortunately, I am very new to Julia, and I am struggling to find my mistake.

I tried to guide my attempts along the Walk1D.jl example.

What I wanted to achieve is a simple scenario with two vehicles where the adversarial tries to modify the initial speed of the vehicle and the distance metric is literally the distance between the two vehicles on the straight roadway.

This is the code I want to execute:

using POMDPStressTesting # this package
using Distributions      # for the Normal distribution
using Parameters         # for @with_kw default struct parameters
using AutomotiveSimulator
using AutomotiveVisualization

# initialize the simulation based on https://sisl.github.io/AutomotiveSimulator.jl/dev/tutorials/straight_roadway/

# * GrayBox Environment Setup

# https://nbviewer.jupyter.org/github/sisl/POMDPStressTesting.jl/blob/master/notebooks/Walk1D.ipynb#GrayBox.Simulation

@with_kw mutable struct AutoSimSetup
    roadway:Roadway = gen_straight_roadway(1, 2000.0)
    scene::Scene = Scene([
        Entity(VehicleState(VecSE2(10.0,0.0,0.0), roadway, 8.0), VehicleDef(), 1),
        Entity(VehicleState(VecSE2(50.0,0.0,0.0), roadway, 12.5), VehicleDef(), 2),
        ])
    models::Dict{Int, LaneFollowingDriver} = Dict{Int, LaneFollowingDriver}(
        1 => StaticLaneFollowingDriver(0.0), # always produce zero acceleration
        2 => IntelligentDriverModel(v_des=12.0), # default IDM with a desired speed of 12 m/s
    )
    timestep::Float64 = 0.1 # typical time step of the simulation
    nticks::Int64 = 100
    endtime::Float64 = timestep * nticks
    eventdistance::Float64 = 30.0
end

# stores simulation-related values
@with_kw mutable struct AutoSim <: GrayBox.Simulation
    params::AutoSimSetup = AutoSimSetup() # Setup Parameters
    distance_to::Float64 = 1000 # distance between the two vehicles
    init_velocity2::Float64 = 12.5
    scene::Scene =  Scene([
        Entity(VehicleState(VecSE2(10.0,0.0,0.0), params.roadway, 8.0), VehicleDef(), 1),
        Entity(VehicleState(VecSE2(50.0,0.0,0.0), params.roadway, init_velocity2), VehicleDef(), 2),
        ])
    time::Float64 = 0.0 # current time
    distribution::Distribution = Normal(12.5, 1)
end

GrayBox.environment(sim::AutoSim) = GrayBox.Environment(:init_velocity2 => sim.distribution)

function GrayBox.transition!(sim::AutoSim, sample::GrayBox.EnvironmentSample)
    sim.time += sim.params.timestep
    sim.init_velocity2 = sample[init_velocity2].value
    sim.scene = simulate(sim.scene, sim.params.roadway, sim.params.models, 1, sim.params.timestep)
    return logpdf(sample)::Real
end

# * BlackBox System Setup: https://nbviewer.jupyter.org/github/sisl/POMDPStressTesting.jl/blob/master/notebooks/Walk1D.ipynb#Black-box-System

function BlackBox.initialize!(sim::AutoSim)
    sim.params.roadway = gen_straight_roadway(1, 2000.0)
    sim.params.scene = Scene([
        Entity(VehicleState(VecSE2(10.0,0.0,0.0), sim.params.roadway, 8.0), VehicleDef(), 1),
        Entity(VehicleState(VecSE2(50.0,0.0,0.0), sim.params.roadway, 12.5), VehicleDef(), 2),
    ])
    sim.params.models = Dict{Int, LaneFollowingDriver}(
        1 => StaticLaneFollowingDriver(0.0), # always produce zero acceleration
        2 => IntelligentDriverModel(v_des=12.0), # default IDM with a desired speed of 12 m/s
    )
    sim.params.timestep = 0.1
    sim.params.endtime = 10.0
end

function BlackBox.distance(sim::AutoSim)
    return 40.0
end

BlackBox.isevent(sim::AutoSim) = false #extract_featuresdistance_to(2, sim.params.roadway, sim.params.scene, [1]) <= sim.params.eventdistance

BlackBox.isterminal(sim::AutoSim) = BlackBox.isevent(sim) || sim.time >= sim.params.endtime

function BlackBox.evaluate!(sim::AutoSim, sample::GrayBox.EnvironmentSample)
    logprob::Real = GrayBox.transition!(sim)
    d::real = BlackBox.distance(sim)
    event::Bool = BlackBox.isevent(sim)
    return (logprob::Real, d::Real, event::Bool)
end

# * AST Setup and Running: https://nbviewer.jupyter.org/github/sisl/POMDPStressTesting.jl/blob/master/notebooks/Walk1D.ipynb#AST-Setup-and-Running

function setup_ast(seed=0)
    # Create gray-box simulation object
    sim::GrayBox.Simulation = AutoSim()

    # AST MDP formulation object
    mdp::ASTMDP = ASTMDP{ASTSampleAction}(sim)
    mdp.params.debug = true # record metrics
    mdp.params.top_k = 10   # record top k best trajectories
    mdp.params.seed = seed  # set RNG seed for determinism

    # Hyperparameters for MCTS-PW as the solver
    solver = MCTSPWSolver(n_iterations=1000,        # number of algorithm iterations
                          exploration_constant=1.0, # UCT exploration
                          k_action=1.0,             # action widening
                          alpha_action=0.5,         # action widening
                          depth=sim.params.nticks) # tree depth

    # Get online planner (no work done, yet)
    planner = solve(solver, mdp)

    return planner
end

planner = setup_ast();

action_trace = search!(planner)

final_state = playback(planner, action_trace, sim->sim.x)

failure_rate = print_metrics(planner)

d3tree = visualize(planner) # re-runs the search to output the tree, then visualizes it

I am using the following packages with Julia 1.6.1 in Ubuntu 20.04

julia> import Pkg; Pkg.status()
     Project scpsLL v0.1.0
      Status `~/git/SCPS-LL/scpsLL/Project.toml`
  [0f7eb3e9] AdversarialDriving v0.1.0 `https://github.com/sisl/AdversarialDriving.jl#master`
  [6aa42d20] AutomotiveSimulator v0.1.1
  [0faf7fd0] AutomotiveVisualization v0.1.1
  [4c6ed407] BayesianOptimization v0.2.5
  [31c24e10] Distributions v0.23.12
  [bb4c363b] GridInterpolations v1.1.2
  [7073ff75] IJulia v1.23.2
  [a40420fb] LocalApproximationValueIteration v0.4.2
  [db97f5ab] LocalFunctionApproximation v1.1.0
  [8314cec4] PGFPlotsX v1.2.10
  [182e52fb] POMDPPolicies v0.4.1
  [e0d0a172] POMDPSimulators v0.3.12
  [6fc570d8] POMDPStressTesting v0.5.0 `https://github.com/sisl/POMDPStressTesting.jl#master`
  [a93abf59] POMDPs v0.9.3
  [d96e819e] Parameters v0.12.2
  [91a5bcdd] Plots v1.15.2
  [3065fe53] RLInterface v0.3.7 `https://github.com/JuliaPOMDP/RLInterface.jl#master`
  [71555da5] Reel v1.3.2

I have the following bug, where I don't quite understand how to solve the bug.

Exception has occurred: MethodError
MethodError: Cannot `convert` an object of type Nothing to an object of type Real
Closest candidates are:
  convert(:Type{T}, !Matched::Gray24) where T<:Real at /home/luttkule/.julia/packages/ColorTypes/6m8P7/src/conversions.jl:114
  convert(::Type{T}, !Matched::Gray) where T<:Real at /home/luttkule/.julia/packages/ColorTypes/6m8P7/src/conversions.jl:113
  convert(::Type{T}, !Matched::Base.TwicePrecision) where T<:Number at twiceprecision.jl:250
  ...

Stacktrace:
 [1] default_action(unused#::MCTS.ExceptionRethrow, mdp::ASTMDP{ASTSampleAction}, s::ASTState, ex::MethodError)
   @ MCTS ~/.julia/packages/MCTS/ww2qH/src/default_action.jl:3
 [2] action_info(p::MCTS.DPWPlanner{ASTMDP{ASTSampleAction}, ASTState, ASTSampleAction, typeof(POMDPStressTesting.AST.rollout), MCTS.RandomActionGenerator{Random._GLOBAL_RNG}, typeof(POMDPStressTesting.AST.go_to_state), Random._GLOBAL_RNG}, s::ASTState; tree_in_info::Bool)
   @ MCTS ~/.julia/packages/MCTS/ww2qH/src/dpw.jl:79
 [3] (::POMDPModelTools.var"#action_info#kw")(::NamedTuple{(:tree_in_info,), Tuple{Bool}}, ::typeof(POMDPModelTools.action_info), p::MCTS.DPWPlanner{ASTMDP{ASTSampleAction}, ASTState, ASTSampleAction, typeof(POMDPStressTesting.AST.rollout), MCTS.RandomActionGenerator{Random._GLOBAL_RNG}, typeof(POMDPStressTesting.AST.go_to_state), Random._GLOBAL_RNG}, s::ASTState)
   @ MCTS ~/.julia/packages/MCTS/ww2qH/src/dpw.jl:19
 [4] search!(planner::MCTS.DPWPlanner{ASTMDP{ASTSampleAction}, ASTState, ASTSampleAction, typeof(POMDPStressTesting.AST.rollout), MCTS.RandomActionGenerator{Random._GLOBAL_RNG}, typeof(POMDPStressTesting.AST.go_to_state), Random._GLOBAL_RNG}; return_tree::Bool, verbose::Bool)
   @ POMDPStressTesting ~/.julia/packages/POMDPStressTesting/zbluM/src/solvers/mcts.jl:80
 [5] search!(planner::MCTS.DPWPlanner{ASTMDP{ASTSampleAction}, ASTState, ASTSampleAction, typeof(POMDPStressTesting.AST.rollout), MCTS.RandomActionGenerator{Random._GLOBAL_RNG}, typeof(POMDPStressTesting.AST.go_to_state), Random._GLOBAL_RNG})
   @ POMDPStressTesting ~/.julia/packages/POMDPStressTesting/zbluM/src/solvers/mcts.jl:76
 [6] top-level scope
   @ ~/git/SCPS-LL/scpsLL/src/AutomotiveSimulator/pomdp-test.jl:109

In the default_action.jl of the MCTS package is the exception thrown on line 3:

default_action(:ExceptionRethrow, mdp, s, ex) = rethrow(ex)

I get the following expressions for s and mdp.

s
ASTState
  t_index: Int64 1
  parent: Nothing nothing
  action: Nothing nothing
  state: Nothing nothing
  hash: UInt64 0x037b19cd36727d90
  q_value: Float64 0.0
  terminal: Bool false

mdp
ASTMDP{ASTSampleAction}
  params: ASTParams
  sim: AutoSim
  sim_hash: UInt64 0x037b19cd36727d90
  t_index: Int64 2
  current_seed: Nothing nothing
  top_paths: DataStructures.PriorityQueue{Any, Float64, Base.Order.ForwardOrdering}
  metrics: POMDPStressTesting.AST.ASTMetrics
  dataset: Array{Any}0,
  rate: Float64 NaN
  predict: Nothing nothing

I assume that the error is that parent, action and state are all nothing in s

default_action is called in the dpw.jl file in the action_info function.

s:ASTState

goes into the function and is originally passed as initstate in the mcts.jl file of the POMDPStressTesting package.

initstate = AST.initialstate(mdp)

So something must be wrong in the initstate definition, but I don't know how to find out what.

Any help or idea is really appreciated.

Thanks and kind regards

[mossr]

Nice to see you're combining this package with AutomotiveSimulator.jl (so am I at the moment 😁)

The bug you're running into is actually quite simple:

Change this (inside the function BlackBox.evaluate! definition):

logprob::Real = GrayBox.transition!(sim)

To this:

logprob::Real = GrayBox.transition!(sim, sample)

i.e., actually pass the sample into the GrayBox.transition! function you've defined. You're getting the conversion error because the transition! function without the sample as input is not defined, so it's using the stubbed-out empty interface function that returns nothing.