iLOCuS

Incentivizes vehicles to optimize sensing distribution in crowd sensing system

Documentation

• RL
• Reaction
• Basics

RL

The reinforcement learning standard core code

• agent
• core
• environment
• model
• objectives
• policy

agent.py

The agent for RL

• calc_q_value
  • Given a state (or batch of states) calculate the Q-values.
• update_policy
  • Update your policy.
• fit
  • Fit your model to the provided environment.
• evaluate
  • Test your agent with a provided environment.

core.py

The core classes needed for RL

• Sample
  • Represents a reinforcement learning sample. Used to store observed experience from an MDP. Represents a standard (s, a, r, s', terminal) tuple.
• ReplayMemory
  • Interface for replay memories.

environment.py

The environment for training

• step
  • Given an action, compute the next state and reward, i.e. progress the state.
• _compute_reawrd
  • Compute the reward given the current distribution of taxis and desired distribution (KL divergence).

model.py

Standard CNN, with activtion fucntion relu

objectives.py

The loss functions, calculating mean huber loss

policy.py

RL Policy classes, we are using LinearDecayGreedyEpsilonPolicy

Reaction

• distance
• drivers

distance.py

Utility class to calculate distance and paths from index values

drivers.py

Simulator for driver reactions

• step
  • Lottery pick to match requests and drivers. If not assigned, go to the best possible adjacent grid, or remain unmoved.

Basics

Basic structures and utilities to visualize data

• crowdsourcer
• model
• view

crowdsourcer.py

The ultimate agent to make decisions, place holder

model.py

Environment simulator, provides simulated data from real data

view.py

Visualize data on canvas, helpful to see distributions