Hence, a very broad range of signal transition graphs can be synthesized. The only requirement is that the corresponding initial state graph is finite, connected, and has a consistent state assignment. Unlike previous methods, the initial STG need not be a live, safe, nor a free choice net. Performing transformations at the state graph level has the advantage that the requirements imposed on the initial STG are very weak. A constraint satisfaction framework is proposed that can guarantee necessary and sufficient conditions for a state graph assignment to result in a transformed state graph that is free of critical races. Moreover, we show that our learned Laplacian representations lead to more exploratory options and better reward shaping.In this article, we propose a global assignment theory for encoding state graph transformations. We validate this via comprehensive experiments on a set of gridworld and continuous control environments. It enables learning high-quality Laplacian representations that faithfully approximate the ground truth. To solve this problem, we reformulate the graph drawing objective into a generalized form and derive a new learning objective, which is proved to have eigenvectors as its unique global minimizer. As a result, their learned Laplacian representation may differ from the ground truth.
To approximate the Laplacian representation in large (or even continuous) state spaces, recent works propose to minimize a spectral graph drawing objective, which however has infinitely many global minimizers other than the eigenvectors. Such representation captures the geometry of the underlying state space and is beneficial to RL tasks such as option discovery and reward shaping. The Laplacian representation recently gains increasing attention for reinforcement learning as it provides succinct and informative representation for states, by taking the eigenvectors of the Laplacian matrix of the state-transition graph as state embeddings.
0 kommentar(er)
