Paper Link: https://arxiv.org/pdf/2409.08276v1
Introduction
Paper claims that the reason that tactile is often overlooked is due to the cost of the sensors, expensive fabrication process for pressure based sensors
Inconvenient to use on different robot platforms, need to be custom built for specific robot end-effectors
Learnings
- Every modality becomes a token with the same d_size (width)
- Images go through ResNet
- 15 AnySkin Readings go through a small MLP
- Transformer keeps the tensor shape
[B, L, D]N tokens in give N vectors out - Action token is an input, learned vector appended to each timestep’s tokens
- Exists because keeps sensor tokens from having to serve as the readout
- Action chunking predicts about 10 future actions at once
- Trains with a regression loss agains tthe demonstrations
- executes the chunk from the newest timestep
- Tactile policy is only useful if a new sensor produces the same signal as the old one