Haptic Fidelity Framework

The Haptic Fidelity Framework providing the means to describe, understand and compare haptic feedback systems. The framework locates a system in the spectrum of providing realistic or abstract haptic feedback using the Haptic Fidelity dimension. It comprises 14 objective criteria that either describe foundational or limiting factors. A second Versatility dimension captures the current trade-off between highly realistic but application-specific and more abstract but widely applicable feedback.

The paper is available here.

Haptic Fidelity

Haptic Fidelity describes an objective measure for the qualities regarding the realism of a haptic rendering system. It takes into account how the haptics are rendered and which haptic receptors are addressed but does not describe how a user will experience the haptics. It provides a measure of how realistic the system can reproduce a haptic experience through its rendering mechanisms, thus the potential for a realistic perception from the user.

The Haptic Fidelity dimension incorporates 14 independent factors to assess Haptic Fidelity for virtual reality. They are divided into three categories: Sensing, Hardware and Software. The factors are further differentiated between foundational factors (F) that describe the features of a system and the value they provide, and limiting factors (L) that comprise factors negatively impacting the perception. The set of foundational factors (such as Magnitude or Sensory Integrity) represent the added value of the system. Limiting factors on the other hand can merely diminish this value. If a limitation is only minor or does not apply to a system at all, the overall value of the system does not change, while major limitations can drastically impair the whole system. We consider the latency of a system, for example, as a limiting factor, because it can only have a negative impact though never enhance a system. Factors that do not apply to every feedback system are also included in the limiting factors as they do not impact the value of a system if they are not applicable. Combining all individual factors into one overall score, the Haptic Fidelity dimension provides a single measure for haptic rendering systems that describes how abstract or realistic the system can potentially provide haptic feedback to a user for a particular use case. The Haptic Fidelity dimension can only be evaluated regarding what is intended to be conveyed by the system. If a system is designed to simulate the feedback of punches from boxing, it should only be evaluated how realistic these punches can be represented but not how realistic the system can, for example, represent touching flowers. Therefore, Haptic Fidelity and its factors are relative scales that can only be evaluated regarding the intent of the system.

In the following, the individual factors are described in detail. The descriptions contain the phrase "The degree to which the same ..." where "the same" refers to how or where the intended haptics would be perceived in reality.

Sensing

This factor describes where on the user's body the haptic feedback is created by the system and to what extent this is in line with where one would perceive the stimulus in the natural occurrence of the intended haptics. This scale is grounded in the human ability to localize a haptic stimulus on the body.

Example: Using EMS on the upper and lower arm to simulate lifting a virtual box, like the system by Lopes et al. does, would get a medium-high score as it correctly involves the arms to simulate the weight of the box, but does not give feedback on the fingers and hands where the box is touched.

This factor describes how well the system provides haptic feedback to the same extent of area on the user's body where one would get feedback in the natural occurrence of the intended haptics. This scale is grounded in the spatial resolution of haptic receptors in the human body and the ability to relate stimuli to each other, forming a consistent sensory impression. The differences in density of haptic receptors should be considered for this factor. Hence, a system that intends to convey haptics to the fingertips should match the area more precisely than a system affecting the upper arm.

Example: Simulating the haptic feedback of a boxing punch with a small, actuated plate on the forearm of a user, like the Impacto System does, involves a significantly smaller area than a boxing glove would impact. Therefore, such a system would get a medium-low score.

This factor describes how well the system stimulates the same haptic receptors as they would be stimulated in the natural occurrence of the intended haptics. It is grounded in the existence of different haptic receptors in the human body. Each kind of receptor responds to different forms of stimuli, making it possible to perceive and differentiate a variety of haptic properties. While each kind of receptor responds to specific low-level stimuli, perceptual psychology classified haptic cues into higher-level perceptions that can be distinguished by humans.

Example: Using the vibration of a VR controller to give feedback for the contact force of virtual walls, as it is done in one of the conditions in this paper, involves completely different haptic receptors leading to a low score.

This factor describes how well the system creates the same strength (e.g., same force) or variation (e.g., same texture) of haptic stimuli compared to the natural occurrence of the intended haptics. The scale is grounded in the ability of human haptic receptors to perceive different variations of stimuli. When rating this scale, the just noticeable difference in sensations from a haptic system should be considered

Example: Simulating the haptic feedback of a boxing punch, like the Impacto System does with EMS and a small, actuated plate on the forearm, can only represent the force to a medium degree and the pressure applied to the forearm to a low degree. Therefore, such a system would get a medium-low score.

This factor describes to what extent the haptic stimuli match with the perception of the other senses that are also addressed by the system and if this match is in line with the intent. It is based on the human ability to integrate all senses into a consistent perception of the world. The senses are weighted differently for this integration; especially vision has been found to be weighted more strongly. This visual dominance effect leads to the possibility that the visual perception can influence how the haptics are perceived.

Example: A physical sandbox with a VR visualization as it is described in this paper, display water evoking the expectation that the haptic perception will be consistent with it and feel like water, but the user will only perceive the haptics of sand. This can be rated with a medium degree of integrity as it does not match to full extent but is also better than perceiving or a solid surface or nothing at all.

This factor describes if the system creates different haptic cues that are usually perceived together in nature, e.g., weight together with weight distribution when lifting an object. The fact that these dependent haptic cues are naturally perceived together create the expectation to always perceive dependent haptic cues together. Therefore, this factor is based on the integration of different haptic stimuli and the learned expectations which haptic stimuli are generally perceived together. This factor is a limiting factor because not all haptic cues have other dependent cues.

Example: An force feedback glove can be used to render the weight, contact force, volume and shape of objects that can be touched and lifted. But it does not provide dependent haptic cues like the texture or temperature of the object, which would be perceivable when touching an object in reality. Therefore, it has some limitations and a medium-low impact on the haptic perception can be assumed.

This factor describes if different physical properties that are intended to be conveyed by the system can be distinguished by either targeting different haptic receptors or through spatial or temporal separation. Different haptic receptors and the integration of different haptic stimuli makes it possible to distinguish a variety of haptic properties. While in nature each object has individual haptic properties, haptic feedback systems are sometimes only capable of providing a limited number of haptic stimuli. Remapping haptic stimuli to represent other physical properties is often used in these cases, e.g., using vibration to represent the contact force of an object. This might work well with a distinct mapping where users can clearly identify and learn the remapping, but can lead to confusion and unrealistic sensations from ambiguous mappings. This factor is a limiting factor because not all systems intend to render multiple physical properties or use any kind of remapping.

Example: A system that uses vibration to render texture and contact force of an object when touching it would make it challenging to distinguish which of the two physical properties is currently rendered or varied. Therefore, such a system would get a high rating due to its limitations.

Hardware

This factor describes if the hardware of the system provides at least the same number of degrees of freedom (DoF) as in the natural occurrence of the intended haptics. The system can have more degrees of freedom than they require making the system more versatile. Naturally, objects that can be freely moved and rotated have 6 DoF while slide doors would only have 1 DoF and walls none. This factor is based on the fact that haptics are naturally present in multiple dimensions which must be represented by technical solutions. If the system provides a certain number of DoF but the user's range of motion is limited within these through the system, it is not considered in this factor but part of the factor Constraints below.

Example: The Aero-Plane system is a custom controller with two vertical propellers meant to simulate the forces of a ball rolling on a plane, or the motion of food in a pan. The system offers two DoF through the actuation of the propellers creating forces in the left/right and up/down directions. To properly simulate the intended scenarios, a third DoF would be necessary to also create forces in the front/back direction. Therefore, the system receives a medium-high score.

This factor describes to what extent the system can reproduce the detail of haptic feedback compared to the natural occurrence of the intended haptics. It is based on the necessity of mechanical components with varying and unavoidably finite accuracy. For rendering shape this can include the resolution of building blocks to touch, or how detailed a passive haptic object resembles the object. Related aspects that can influence the precision, e.g., calibration imprecision or joint tolerance, should also be considered for this factor.

Example: The precision of a pin-based system to render the shape of objects is dependent on the number and size of the individual pins. The PoCoPo system uses a small number of 1 cm wide pins which is a low resolution compared to the sensory resolution of the hand. It would therefore get a medium-low score. A study by Muender et. al. compares three types of passive haptic props with different degrees of detail. The differences in prop detail can be expressed in this factor.

This factor describes the influence of the hardware latency on the haptic perception. It is based on the fact that technical solutions often require time to transmit signals and mechanically change their states. While in nature haptics are almost always perceived immediately to an action, the latency of haptic feedback systems can lead to a delay in the haptic perception and therefore result in unrealistic sensations. The time until a delay is noticed and the influence of delay on the perception depends on the kind of stimuli and situation. Delays over 25 ms to 100 ms are commonly reported to be noticeable and have an impact on performance. This is a limiting factor because latency can only have a negative impact on the haptic perception and will never enhance a system.

Example: The Shifty system actuates a weight up and down in a tube to simulate different weight distributions, which takes up to 2.8 s. Since this latency is not constantly noticeable but only when a different virtual object is picked up, and since it is mitigated by adjusting the weight already before grasping, the system would get a medium score.

This factor describes how the haptic perception is influenced by any side effects that the hardware of the system creates. This can include the vibration of motors, pain from EMS, or forces from moving parts. It is based on the fact that technical devices can produce unintended haptic cues which may lead to distraction, irritation, and unrealistic sensations. This is a limiting factor because side effects can only have a negative impact on the haptic perception and never enhance a system.

Example: EMS systems, such as by Lopes et al., can cause unpleasant traction, tickle, or even pain leading to a medium score. Other systems that have actuated parts, as the Shifty system, will create unintended counter forces and inertia from the device adjusting, leading to a low score.

This factor describes how the haptic perception is influenced by constraints that the hardware imposes on the user's range of motion. This could be caused by joint limits, wire length or colliding parts of the hardware. It does not consider if the system intentionally limits the user's range of motion to render haptics. This factor is based on the fact that technical solutions might restrict the user's freedom of movement because of the system design or mechanical components such as wearables or wires. This is a limiting factor because constraints can only have a negative impact on the haptic perception and will never enhance a system.

Example: An exoskeleton or robotic arm that limits the range of motion of the user’s arms due to its joint limits or world-grounding, as it is the case in this system, would get a medium score

Software

This factor describes how accurately the software can calculate the haptic feedback to be rendered. This can include how accurate colliders represent the shape of an objects or how accurate a physical simulation can calculate the compliance of an object. It is based on the fact that the feedback has to be calculated by software before it can be displayed by hardware.

Example: If an exoskeleton is used to render the contact force of an object but the collider to represent the object uses an inaccurate low-resolution mesh, the hardware can at most render the haptics with this low resolution. Depending on how low this resolution is, the system would have a low or medium-low score.

This factor describes the influence that the software latency has on the haptic perception. It is based on the fact that the necessary calculations take time and may cause a noticeable delay. The same threshold for noticeable delays as in Hardware Latency apply for this factor. This is a limiting factor because the presence of latency can only have a negative impact on the haptic perception and will never enhance a system.

Example: A system that takes 500 ms to simulate the compliance of an object creates a noticeable delay. A latency of 500 ms can be interpreted as a medium impact on the haptic perception and therefore would get a medium score.

Versatility

The Versatility dimension of the framework describes a measure on how specific a system is in providing haptic feedback for a particular application. This dimension exists orthogonal to the Haptic Fidelity dimension as it represents the trade-off between highly realistic but application-specific and more abstract but widely applicable feedback. As the specificity of a system is not an objective measure to the same degree as the factors from Haptic Fidelity are, we provide categories describing the five levels of this scale.

Systems that provide abstract feedback are naturally more widely applicable for different application scenarios as the feedback can be repurposed to represent any kind of other more specific feedback (e.g., vibration feedback is used to represent contact force or weight). Realistic feedback, on the other hand, can be achieved by designing systems that are custom-made for a particular application. Only addressing receptors at the exact body position necessary for the intended application makes these systems extremely specific to this scenario. Placing feedback systems in the space of these two dimensions should give researchers and designers a better understanding on how the realism of feedback is connected to the possible use cases and how more abstract feedback can be applied to different applications. While Haptic Fidelity was assessed by individual factors, the Versatility dimension provides a single factor rating haptic feedback systems on a scale from specific to generic. How a system is rated on this scale does not have any implication for the overall quality of a system but rather represents how versatile it can be used.

These categories provide the means to congruently rate the versatility of haptic feedback systems and generate a score for the Versatility dimension. Similar to the Haptic Fidelity this dimension has to be rated relative to the intended application of the haptic feedback. This is necessary to generate valid ratings that are consistent between the two dimensions. A standard VR controller with vibration would normally be considered very generic in its feedback. But if the scenario is to render the haptic of a vibrating phone, for example, it has to be considered as more specific for this particular use case.

  1. (specific) The systems feedback is specific for one particular variation of a use case (e.g., for a particular climbing wall, other variations cannot be haptically rendered).
  2. The systems feedback is specific for a particular use case (e.g., boxing) and supports different variations (e.g., the system could provide feedback for other variations like different boxing fights or even other kinds of martial arts).
  3. The systems feedback is quite specific for a group of use cases to which the particular application belongs to (e.g., all scenarios with tangibles on a desk).
  4. The systems feedback is unspecific for a particular use case and generic to a larger group of applications (e.g., a force feedback glove or an exoskeleton).
  5. (generic) The systems feedback is completely generic and not specific for almost any application (e.g., vibration from VR controller as feedback for climbing).
Haptic Fidelity Score 1.0
Versatility Score 1.0
Factor-Code
Set all scores from Factor-Code
Factor-Code