Bioelastic state recovery for haptic sensory substitution

a, Fabrication of the core on a motorized stage. Each coil was wound to eight layers along the 2.75-mm shaft of the bobbin to reach approximately 63% fill density. The resistance of each coil was measured to verify consistency and all transducers described in this article lie within 13–14 Ω (R_coil = 13.39 ± 0.384 Ω; n = 18). b, Assembled transducer. c, Disassembled view of the transducer with key mechanical components labelled. d, Scale illustration of bistable transducer components and the variable harness system. e, 3D illustration of the three harness variants used in this study.

a, Schematic illustration of the mesh-reinforced regions of the adhesive layer. b, Integration of the array with skin using the adhesive layer. c, Layer-by-layer fabrication of the adhesive coupling, schematic illustrations and photographs. d, Photograph of two transducer units positioned inside the controller module and schematic illustrations demonstrating how each modular interconnect layer routes signals through the hexagonal array. During assembly, the internal contacts fold into the core, in which they can be soldered to the coil. e–k, Mechanical response testing for the array. e, Model geometry for mechanical simulation of strain under a unit actuator during skin bending (50 mm radius). f, Simulated strain using a substrate without mesh. g, Simulated strain with a mesh-reinforced substrate. h, Photograph of the haptic array under 20% lateral stretching. i, Numerical simulation of strain within the Dragon Skin/mesh adhesive layer, polyimide interconnects (fracture strain, >7%; ref. ³⁸) and copper traces (fracture strain, >1%; ref. ³⁹) of the array during 20% lateral stretching. j, Photograph of the haptic array under 50-mm bending radius. k, Numerical simulation of strain within the Dragon Skin/mesh adhesive layer, polyimide interconnects and copper traces of the array during bending (50 mm radius). l, Transducer resonance measured across the array periodically by the embedded inductance measurement unit during cyclic deformations of stretching and bending (mean across 19 transducers; bars, standard deviation). m, Torsion angle induced by a kirigami structure (8.5 mm initial height) on a skin phantom (E = 31 kPa) measured periodically during cyclical translation of the top panel by 3 mm (setup shown in inset).

a, Demagnetization curve of an N48 neodymium magnet. Arnold Magnetic Technologies N48 from the ANSYS material library was used for the permanent magnet part of the armature with a modification of B_r to 1 T in its B–H curve. b, B–H curve of iron–cobalt (VACOFLUX 50 Solid) and PDMS–MNP. The PDMS–MNP diaphragm used a self-defined material with a relative permeability of 1.04 and a bulk conductivity of 4 S m⁻¹. c, Example quasi-magnetostatic simulation showing the magnetic field strength and direction as a colour map and quiver plot (I_coil = 400 mA) for the compressed and relaxed states (*||B|| = 7.4 mT, stray magnetic field at 6.6 mm radial distance from central axis). d, Model geometry and boundary conditions for solid mechanical processes. A downward displacement was applied to the armature and the corresponding reaction forces from the skin phantoms were evaluated. e, Example simulations showing strain distribution within a skin phantom (45:1, E = 43 kPa) for the relaxed and compressed states (2 mm effective indentation). f, Schematic of the mass–spring–damper vibration model for the transducer and the periodic loading force applied to it. g, Stress distribution within the kirigami structure in the relaxed state (PET yield stress, 80 MPa). h, Stress distribution within the kirigami structure in the compressed state. i, In-plane strain of the skin phantom in the compressed state. A stiff skin phantom was used here to evaluate the upper bound of stress in the structure (E = 5.1 MPa). j, In-plane deformation of the skin phantom in the compressed state.

a, Mechanical bistability evaluation in healthy individuals. Measurements were performed in five skin locations (depicted on the left). The absolute value of current required to transition the armature between relaxed and compressed states was measured for n = 6 participants (three males, three females, ages 20–36 years). The indentation depth was adjusted by performing experiments with harnesses of varying heights. b–i, Characterization of force as a function of extension was performed using a motorized stage (ESM303, Mark-10) and a force gauge (M5-2, Mark-10) with a 10 N capacity. b, Experimental and simulated forces measured as a function of the longitudinal position of the armature for several applied current values (excluding PDMS–MNP diaphragm and skin compressive force). We defined the origin as where the base of the armature and the core are in contact and we determined this using the force meter. For experimental results, the solid lines and shaded areas correspond to the means and standard deviations, respectively (n = 8). The square markers correspond to the maximum recorded force for each current, averaged across all transducers. The inset shows the experimental setup. c, Forces measured as a function of the longitudinal position of the armature rod (excluding force from core) for 40:1, 45:1, 50:1 and 55:1 skin phantoms with thicknesses of d = 2 mm (inset shows experimental setup). d, Forces measured as a function of the longitudinal position of the armature rod for 35:1, 40:1, 45:1, 50:1 and 55:1 skin phantoms with thicknesses of d = 5 mm. e, Skin reactive forces measured as a function of the longitudinal position of the armature rod (excluding force from core) for the ventral aspect of the forearm for n = 6 subjects (three males, three females, ages 20–36 years; experimental setup shown in inset). f, Skin reactive forces measured for the dorsal aspect of the palm above the adductor pollicis for n = 6 subjects (three males, three females, ages 20–36 years; ___location indicated in inset). g, Skin reactive forces measured for the dorsal aspect of the palm above metacarpal III for n = 6 subjects (three males, three females, ages 20–36 years; ___location indicated in inset). h, Tensile-peeling-force measurements of the harness attached to the silicone–mesh composite adhesive mounted on the forearm of n = 4 subjects (two males, two females, ages 26–32 years). The horizontal line shows the holding force of the indentation actuator. i, Tensile peeling force for a Ø7 mm disc pre-rotated to 30° and mounted on the forearm of n = 4 subjects (two males, two females, ages 26–32 years) with 3M 9699 double-sided adhesive. The horizontal line shows the holding force of the torsion actuator.

a, Photograph of the experimental setup. b, Representative photograph of image sequence showing speckle pattern. c, 3D-reconstructed points versus true points. d, Normal deformation of skin phantoms measured in the relaxed and compressed states with DIC. e–g, Temporal profiles (top) and frequency-___domain transformations (bottom) of skin phantom indentation (45:1 PDMS phantom, E = 43 kPa; I_coil-pk = 250 mA) measured at the centre of the contacting rod of the transducer with DIC during square-wave perturbation of the coil. e, Relaxed-state perturbation. f, Compressed-state perturbation. g, Full-transition vibration.

a, Basic split-crease unit cell design and fully folded Kresling-pattern-inspired kirigami structure. Parameters α and β are discussed in the Supplementary Methods. b, Design of a variant cell that uses a combination of straight and curved creases, which further increases the internal space. c, Curved crease design using the Bézier curve method, in which P1–P5 are control points (discussed in the Supplementary Methods). d, Kirigami structure with only straight valley creases in the folded state (h = 6.75 mm). e, Variant kirigami structure with 2 mm straight crease length (h = 6.75 mm; folded state). f, Crease length, 4 mm. g, Two-tiered kirigami structure illustrating switchable motion states. h, Bottom view of the contacting elements (ring and disc). i, 2D patterns of the lower and upper tier kirigami modules for fabrication. j, Panels used to make the kirigami structure, consisting of one layer of PET plastic sheet and two layers of PU thin-film tape. k, Photograph of the flat kirigami panel used for fabricating the 3D structure. l, Photograph of the assembled lower and upper tier kirigami modules.

a,b, Vibration current amplitudes of perception thresholds (n = 12, six males, six females, ages 20–36 years). a, Compressed-state vibration. b, Relaxed-state vibration. c, Relative perception of vibration in the transducer with respect to a commercial LRA (dotted line, equivalent perception; n = 12, six males, six females, ages 20–36 years). d–g, Perceptual intensity evaluated according to its definition in Methods for n = 14 human participants (eight males, six females, ages 23–31 years) under four ring–disc configurations. The left column shows mean values reported by each subject (n = 5 repeat measurements; bars, standard deviation) and the right column reflects all subjects (n = 14; bars, 1.5 interquartile range; shaded box, range between 25th and 75th percentiles; horizontal line, median; ×, mean; ◆, outliers from 1.5 interquartile range). According to the Friedman test, a statistically significant difference can be seen within each ring–disc configuration (P < 0.0001) and each showed a moderate positive correlation between ratings and rotation angles (Spearman correlation, ρ ≥ 0.58), which was highly statistically significant (P < 0.0001). d, Outcomes reported for rotation and indentation of the ring, in which the ring was rotated to angles of 0.33°, 0.66° and 1° with a static indentation of 0.2 mm. e, Outcomes reported for rotation of the disc, in which the disc was rotated to angles of 5°, 10° and 15° with a static indentation of 0 mm. f, Outcomes reported for rotation and indentation of the disc, in which the disc was rotated to angles of 5°, 10° and 15° with a static indentation of −0.4 mm. g, Outcomes reported for rotation and indentation of the ring and disc, in which the ring and disc were rotated to angles of (0.33°, 5°), (0.66°, 10°) and (1°, 15°) with a static indentation of (0.2 mm, −0.4 mm). h–j, Torsion and indentation array discrimination experiments. h, Illustration and photograph of the apparatus setup, which includes three torsion actuators (A, D and E) and two indentation actuators (B and C). i, Discrimination between indentation and torsion, combined results for 15 healthy participants (seven males, eight females, ages 23–31 years). Accuracy was averaged over 15 subjects with ten repeated trials (n = 150). j, Discrimination between patterns of torsion, combined results for 12 healthy participants (seven males, five females, ages 23–31 years).

a–e, Miniaturization of the indentation structure. The coil was wound to 12 Ω. a, Photograph of the original and miniaturized transducers. b, Scale illustration of the miniaturized transducer. c, Experimental and simulated forces measured as a function of the longitudinal position of the armature for several applied current values. Measurements were performed using a motorized stage (ESM303, Mark-10) and a force gauge (M5-2, Mark-10) with a 10 N capacity. d, Simulated phase diagram for I_coil → ∞ showing regions across the parameter space of skin modulus and indentation depth for successful bistability. The shaded regions correspond to the boundaries of bistability, outside which only one state is attainable (monostable, compressed or relaxed). e, The minimum applied current required to overcome the energy barrier between each state plotted against the total indentation depth for a range of Young’s moduli, E (simulated). f–k, Miniaturization of the torsion structure. f, Photograph of the single-tiered kirigami structure. g, Scale illustration of the one-tiered kirigami structure. h, Volumetric illustration of freely moving mechanical elements of the one-tiered structure. i, Photograph of a haptic array incorporating units for both indentation and torsion. j, Finite element analysis of stress driven by creasing within the kirigami structure (PET yield stress, 80 MPa; E = 31 kPa, skin). k, DIC out-of-plane (left) and in-plane (right) strains on a skin phantom during the compressed state of the single-tier transducer (E = 31 kPa; arrows, direction and magnitude of in-plane deformation). The inner disc rotated 7.5° and translated 0.25 mm vertically relative to the outer ring.

a–h, Visual sensory substitution task. Confusion matrix of participant performance on object identification task (n = 7), with the colour map indicating the frequency of occurrence. The P-value was P = 0.0178, testing the null hypothesis that selections were made at random between six possible choices (Wilcoxon signed-rank test; n = 7). The effect size was large, r = 0.896, according to rank-biserial correlation. a, Experimental setup of the visual sensory substitution task. b, Subject 1, female, age 21 years. c, Subject 2, female, age 23 years. d, Subject 3, female, age 19 years. e, Subject 4, male, age 37 years. f, Subject 5, male, age 32 years. g, Subject 6, male, age 30 years. h, Subject 7, male, age 31 years. i–s, Balance sensory substitution task. Participant performance on sharpened Romberg task, measured as the time duration before losing balance. The P-value across repeated trials and subjects was P = 0.00103, testing the null hypothesis that that no differences exist in group means. The vertical bars indicate standard deviation, shaded boxes indicate the interquartile range, the horizontal lines above each box indicate medians and filled circles correspond to the means. i, Stimulation patterns superimposed on smartphone display during rotation around the forward axis for the balance sensory substitution task. The virtual mesh highlights the LiDAR-reconstructed surface. j, Subject 1, female, age 35 years. k, Subject 2, female, age 35 years. l, Subject 3, female, age 20 years. m, Subject 4, female, age 29 years. n, Subject 5, female, age 32 years. o, Subject 6, male, age 36 years. p, Subject 7, male, age 20 years. q, Subject 8, male, age 34 years. r, Subject 9, male, age 33 years. s, Subject 10, male, age 24 years.

Evaluation of the foot-strike sensory substitution system. a, Experimental setup depicting reference axes. As part of the task, four separate surfaces were presented at orientations (ψ, φ) ∈ {(0°, 0°), (15°, 60°), (15°, −60°), (30°, 0°)}. b, Stimulation patterns superimposed on smartphone display during rotation around the forward axis. The virtual mesh highlights the LiDAR-reconstructed surface. c–e, Unit normal vectors of foot and variable surface centred on a common origin during task with feedback. The significance of the results for each individual was evaluated using the Wilcoxon signed-rank test over repeat trials (across all three phases), given the null hypothesis that the median of the data is identical to the control. Effect size was characterized according to rank-biserial correlation. c, Subject 1, male, age 37 years (n = 57, P < 0.0001, r = −0.59). d, Subject 2, female, age 35 years (n = 55, P < 0.0001, r = −0.75). e, Subject 3, male, age 21 years (n = 60, P = 0.00027, r = −0.47). f–h, Unit normal vectors of foot and variable surface centred on a common origin during task without feedback (control). f, Subject 1, male, age 37 years. g, Subject 2, female, age 35 years. h, Subject 3, male, age 21 years.