SimTac

Tactile sensing in biological organisms is deeply intertwined with morphological form, such as human fingers, cat paws, and elephant trunks, which enables rich and adaptive interactions through a variety of geometrically complex structures. In contrast, vision-based tactile sensors in robotics have been limited to simple planar geometries, with biomorphic designs remaining underexplored. To address this gap, we present SimTac, a physics-based simulation framework for the design and validation of biomorphic tactile sensors. SimTac consists of particle-based deformation modeling, light-field rendering for photorealistic tactile image generation, and a neural network for predicting mechanical responses, enabling accurate and efficient simulation across a wide range of geometries and materials. We demonstrate the versatility of SimTac by designing and validating physical sensor prototypes inspired by biological tactile structures and further demonstrate its effectiveness across multiple Sim2Real tactile tasks, including object classification, slip detection, and contact safety assessment. Our framework bridges the gap between bio-inspired design and practical realisation, expanding the design space of tactile sensors and paving the way for tactile sensing systems that integrate morphology and sensing to enable robust interaction in unstructured environments.

Background

Bioinspiration

Simulation Framework

Simulation Principles

Deformation and Rendering — **The principle of sensor membrane deformation simulation and optical rendering simulation**. Here we use a finger-shaped GelTip sensor as an example. a-b, The pipeline of the tactile image simulation. We amplify the contact deformation and sparsify the particle density to facilitate easier understanding; **a particle-based iteration method** is employed for **simulating membrane deformation**strong>, while **optical rendering**strong> is achieved using **a light field-based lighting model**. c, Offline light field generation. d, Real-time online image rendering.

Force — **The principle of mechanical response simulation**. We amplify the contact deformation and sparsify the particle density to facilitate easier understanding; **A Sparse Tensor Network** is used to **predict force/deformation fields**, where the deformation data iterated from the Material Point Method (MPM) serves as input, while the force/deformation data computed from FEM serves as the ground truth. This framework enables rapid and accurate mechanical response simulations that approximate FEM-level precision.

Simulation Performance

The performance evaluation of optical response simulation. a, The comparison of tactile images collected from the real world and those generated from the proposed simulator under varying contact positions and indenter shapes. b, The quantitative analysis results of the optical simulation under contact with different indenters, evaluated using: (i) Structural Similarity Index (SSIM); (ii) Mean Squared Error (MSE); (iii) Mean Absolute Error (MAE); (iv) Peak Signal-to-Noise Ratio (PSNR). c, The optical simulation results when the sensor interacts with different textures.

Mechanical Response — **The performance evaluation of mechanical response simulation**. a, **Comparison of 3D deformation and force fields** between SimTac simulation and ground truth from FEM on the test set across different contact scenarios. b,**Comparison of marker motion** between the SimTac simulation and ground truth from the real world across different contact scenarios. c, **The quantitative analysis results** of the mechanical response simulation when in contact with different indenters. (i) MAE of deformation field; (ii) MAE of force field; (iii) Percentage error of the total force.

Simulation Flexibility

Shape — **The shape flexibility of the SimTac simulator**. The proposed simulator can be applied to tactile sensors **with diverse biomorphic shapes**, and is also capable of simulating **both RGB-based and marker-based tactile sensors**.

Material — **The material flexibility of the SimTac simulator**. a, Deformation of membranes with varying hardness under identical contact conditions. Softer membranes exhibit larger deformation and lower force. b, The trained network can be applied to **sensor membranes of different materials** through a **fine-tuning process**, requiring only a small additional FEM ground truth dataset. c, The MAE of (i) deformation field; (ii) force field and (iii) total force in the X, Y, and Z directions between the simulated and ground truth values, evaluated across GelTip sensor membranes of different materials.

Applications - Sensor Prototype Development

Applications - Sim2Real Tasks

Supplementary Videos

Elephant-Trunk-Shaped Tactile Sensor

Sim2Real Task - Contact Object Classification

Sim2Real Task - Slip Detection

Sim2Real Task - Contact Safety Assessment

Sim2Real Task - Dynamic Slip Detection

Citation

If you find our model helpful, feel free to cite it:


  @article{zhang2025simtac,
    title={SimTac: A Physics-Based Simulator for Vision-Based Tactile Sensing with Biomorphic Structures},
    author={Zhang, Xuyang and Jiang, Jiaqi and Chen, Zhuo and Zhao, Yongqiang and Yang, Tianqi and Gomes, Daniel Fernandes and Wang, Jianan and Luo, Shan},
    journal={arXiv preprint arXiv:2511.11456},
    year={2025}
  }