Abstract

Some extremely low-dimensional yet crucial geometric eigen-lengths often determine the success of some geometric tasks. For example, the height of an object is important to measure to check if it can fit between the shelves of a cabinet, while the width of a couch is crucial when trying to move it through a doorway.
Humans have materialized such crucial geometric eigen-lengths in common sense since they are very useful in serving as succinct yet effective, highly interpretable, and universal object representations. However, it remains obscure and underexplored if learning systems can be equipped with similar capabilities of automatically discovering such key geometric quantities from doing tasks.
In this work, we therefore for the first time formulate and propose a novel learning problem on this question and set up a benchmark suite including tasks, data, and evaluation metrics for studying the problem. We focus on a family of common fitting tasks as the testbed for the proposed learning problem. We explore potential solutions and demonstrate the feasibility of learning eigen-lengths from simply observing successful and failed fitting trials. We also attempt geometric grounding for more accurate eigen-length measurement and study the reusability of the learned eigen-lengths across multiple tasks. Our work marks the first exploratory step toward learning crucial geometric eigen-lengths and we hope it can inspire future research in tackling this important yet underexplored problem.

Learning Framework

Given a fitting task instance, we aim to learn a set of eigen-lengths, modeled as neural networks, from which we can determine the feasibility of the task. We do so by creating an information bottleneck, where we first map the geometry input to a small number of eigen-lengths, then execute a task program only based on their values to compute the final output. We only supervise the binary task output during training.

Application: Embodied Visual Navigation

Consider an embodied visual navigation scenario, where we aim to find a collision-free navigation policy for a cylinder-shaped robot vacuum with a mounted depth camera. We can apply our eigen-length learning framework to predict the feasibility of moving one step forward based on the current point cloud observation.

The above video shows a robot vaccum navigating an unseen test scene. We adopt a simple policy: move forward when network predicts positive, turn clockwise otherwise. The predicted environment eigen-lengths are visualized with a rectangle, indicating the size of the navigable space. The color (green/red) represents the final output (positive/negative) obtained by comparing the eigen-lengths of the robot and the environment.

Citation

Acknowledgements

This research is supported by a grant from the Stanford Human-Centered AI Center, a grant from the TRI University 2.0 program, and a Vannevar Bush Faculty Fellowship
The website template was borrowed from Michaël Gharbi.