Floating-Point 8-bit (FP8) Precision

• Floating-point 8-bit (FP8) precision is an 8-bit format using configurations like E5M2 and E4M3, enabling efficient AI computations with reduced resource usage.
• Dynamic per-tensor and static scaling methods, including μnit Scaling, optimize FP8’s numerical range while maintaining sufficient accuracy in deep neural networks.
• Innovative hardware designs, such as specialized tensor cores and transprecision FPUs, accelerate FP8 operations and reduce energy consumption in large-scale AI deployments.

Floating-point 8-bit (FP8) precision refers to the representation and use of 8-bit floating-point numbers in numerical computing, with dominant applications in deep neural network (DNN) training and inference, and emerging support in modern hardware accelerators. FP8 offers substantial reductions in model memory footprint, bandwidth, and compute energy, enabling efficient deployment and scaling of large-scale AI systems while retaining sufficient numerical fidelity for most modern models.

1. Format Definitions and Encodings

FP8 denotes any 8-bit floating-point format with explicit allocations for sign, exponent, and mantissa bits. The dominant encodings are E4M3 and E5M2:

• E5M2: 1 sign bit, 5 exponent bits (bias 15), 2 mantissa bits. Dynamic range approximately $[\pm 2^{-14}, \pm 2^{+16}]$, machine epsilon $2^{-2}=0.25$. Supports full IEEE-754 treatment of subnormals, $\pm 0$, $\pm\infty$, and NaN (Micikevicius et al., 2022).
• E4M3: 1 sign bit, 4 exponent bits (bias 7), 3 mantissa bits. Dynamic range approximately $[\pm 2^{-6}, \pm 2^{+8}]$, machine epsilon $2^{-3}=0.125$. Overflow saturates at largest finite; a single NaN, no $\pm\infty$ (Micikevicius et al., 2022).

Other variants include binary8 (1/5/2), E3M4, E2M5 (Zhang et al., 2023), and the flexible FFP8 where sign, exponent width, mantissa width, and bias are configurable per-tensor (Huang et al., 2021). HiFloat8 (HiF8) introduces a tapered format with variable exponent/mantissa allocation according to value magnitude, yielding nearly FP16-level dynamic range (38 binades) in only 8 bits (Luo et al., 2024).

For a normalized value, the general reconstruction is:

$$x = (-1)^s \times 2^{e-\text{bias}} \times (1 + m/2^M)$$

where $s$ is the sign, $e$ is the stored exponent, and $m$ is the stored mantissa.

2. Quantization Methodologies and Scaling

Essential to practical FP8 use is the selection of per-tensor scaling factors that adapt tensor magnitudes to the limited representable range; in most workflows, distinct formats or scales are allocated for weights, activations, and gradients. The two major approaches are:

• Dynamic per-tensor (amax) scaling: Compute $S=2^{b_\text{scale}}$ by $$b_\text{scale} = \lfloor \log_2 (\text{max}_\text{num}/\text{amax}) \rfloor$$ where $\text{amax} = \max_i |x_i|$, and $\text{max}_\text{num}$ is the largest normal in the target FP8 format. Normalize before quantization, multiply by $S$; dequantize by dividing (Perez et al., 2023, Micikevicius et al., 2022, Kim et al., 3 Feb 2025).
• Static scaling and $\mu$nit Scaling: Use closed-form, static multipliers ensuring activations/weights have approximately unit variance at every layer, with variance-preserving attention (e.g., square-root softmax), residual post-norm, and layerwise learnable mixing, enabling direct FP8 mapping without dynamic scale tracking (Narayan et al., 9 Feb 2025).
• Flexible Bias: In FFP8 and related schemes, bias is learned or set to align most values’ log-magnitude with the normal region; layer-wise tuning ensures >99% of tensor values are represented without overflow/underflow (Huang et al., 2021).

Specialized methodologies (S2FP8) learn "shift and squeeze" parameters per-tensor to optimize the log-domain mapping of values into the FP8 representable grid, removing manual loss scaling (Cambier et al., 2020).

3. Hardware Implementations and ISA Extensions

FP8 has catalyzed custom low-precision hardware designs across diverse RISC-V, FPGA, and ASIC contexts. Major design patterns include:

• Expanding Sum-of-Dot-Product Units (ExSdotp): Enables FP8 input (w=8), FP16 (2w=16) accumulation in a single fused pipeline stage, packing/unpacking subwords in 64-bit lanes; reduces datapath area and critical path by ~30% compared to cascaded expanding FMAs (Bertaccini et al., 2022).
• Transprecision FPUs: Architectures (e.g., FPnew) provide parallel FP8 SIMD datapaths; area and energy scale sublinearly with word width (FP8 energy up to 16x lower than FP64), enabling ultra-low-power operation (Mach et al., 2020, Tagliavini et al., 2017).
• MAC and Accumulation Design: FP8 multipliers feeding into FP12 accumulators with hardware stochastic rounding achieve near-baseline accuracy at ~23% delay and ~9% area reduction versus FP16; 18-bit random sources for rounding are critical to suppress swamping error (Ali et al., 2024).
• Dual-Precision Weight Encoding: Approaches like NestedFP decompose each FP16 weight into FP8-compatible primary/auxiliary bytes, enabling runtime-switchable FP8/FP16 matrix multiplications without extra storage (Lee et al., 29 May 2025).

Vendor hardware (Nvidia H100, Intel Gaudi 2) offers native or emulated FP8 tensor cores with 2x peak throughput vs. FP16 and acceleration of memory-bound inference phases (Kim et al., 3 Feb 2025).

4. Training, Inference, and Downstream Applications

FP8 has been validated for both end-to-end training and inference, including models up to 175B parameters. Key findings include:

• Mixed-Precision Training: Weights, activations, gradients stored and multiplied in FP8 (often E5M2 for gradients, E4M3 for activations/weights), with FP16 or FP32 for partial sum accumulation and master weight updates. Enhanced loss-scaling and stochastic rounding are crucial to suppress gradient underflow and quantization noise (Mellempudi et al., 2019, Wang et al., 2018, Micikevicius et al., 2022).
• Inference & Post-Training Quantization (PTQ): FP8 PTQ matches or outperforms INT8 PTQ on transformers and CNNs, especially for models with heavy-tailed activations where uniform scale INT8 fails catastrophically; FP8 PTQ maintains accuracy within 0.3% of full-precision (Li et al., 2023, Huang et al., 2021, Kuzmin et al., 2022).
• Federated and Edge Learning: On-device FP8 client training in federated scenarios achieves ~3-6x communication compression vs. FP32, with minor accuracy loss, leveraging fixed-point or stochastic quantization (Wang et al., 2024).
• LLMs: FP8 training (with per-tensor E4M3/E5M2, or via μnit Scaling) matches BF16/FP16 quality at scales up to 70B, typically with 1-2x speedup (Perez et al., 2023, Narayan et al., 9 Feb 2025, Kim et al., 3 Feb 2025).
• Alternative 8-bit Designs: HiFloat8 (HiF8) employs tapered precision to combine wide dynamic range (38 binades, nearly FP16) with sufficient mantissa for training and inference across vision, NLP, and LLM workloads (Luo et al., 2024).

5. Numerical Error Analysis and Practical Guidelines

FP8 introduces significantly higher rounding and quantization error than FP16, but this can be carefully managed:

• Accumulation (FP8→FP16 or FP12): Fused expanding units (ExSdotp) and chunk-based accumulations suppress error accumulation seen with naive cascading FMAs, achieving relative errors in dot products of $<10^{-3}$ for sequence lengths up to 2000 (Bertaccini et al., 2022, Wang et al., 2018).
• Quantization-Aware Training (QAT) and stochastic rounding close the gap to FP32, with format sensitivity largely eliminated after training (Kuzmin et al., 2022).
• Subnormals: Enabling subnormal number support is critical for accuracy—dropping subnormals reduces ResNet-50 accuracy from ~76% to ~58% in INT8/FP8; subnormals ensure graceful underflow (Zhang et al., 2023).
• Loss scaling: For FP8, loss scales are larger and more dynamic (“back-off” or “adaptive” schedules) versus FP16 due to reduced subnormal range (Mellempudi et al., 2019).
• Bias (scaling): Adaptive per-layer or layerwise-tuned bias (dynamic or flexible as in FFP8) ensures that the vast majority of values land within normalized range, minimizing representational loss (Huang et al., 2021, Micikevicius et al., 2022).
• Model recommendations: E4M3 is preferred for activations/weights in many settings (especially LLMs), E5M2 for gradients, with adjustment based on outlier statistics observed in each network (Kuzmin et al., 2022, Kim et al., 3 Feb 2025).

6. Performance, Energy, and Datacenter Impact

FP8 adoption yields measurable efficiency improvements:

• Throughput/Performance: Modern accelerators achieve up to 2x GEMM throughput using FP8 compared to FP16 (e.g., 1.5–2× speedup in LLM serving). Measured silicon efficiency of 575 GFLOP/W (FP8→FP16) and up to 2.95 TFLOPs/W on RISC-V and FD-SOI FPGAs (Bertaccini et al., 2022, Mach et al., 2020, Kim et al., 3 Feb 2025, Lee et al., 29 May 2025).
• Energy and Area: FP8 SIMD datapaths are 20–30% smaller and shorter in critical path than FP16, and over 50% smaller than FP32. Power draw is proportionally reduced (Bertaccini et al., 2022, Ali et al., 2024, Tagliavini et al., 2017).
• Datacenter TCO: FP8 quantization in LLM inference, especially using dynamic row-wise scaling, reduces datacenter TCO by minimizing energy and enabling more dense and efficient accelerator utilization; major CSPs recommend splitting decode-phase to lower-power accelerators optimized for small-matrix utilization (Kim et al., 3 Feb 2025).
• On-device Training and Edge Deployment: Communication and memory costs drop 3–6x using FP8 for federated aggregation and model updates (Wang et al., 2024).

7. Limitations, Controversies, and Future Directions

Despite broad success, FP8 adoption is not universal:

• Stability Concerns: In LLM pretraining, pure FP8 training presently yields higher loss-sharpness and more frequent loss divergence than BF16, unless using advanced, format-aware techniques (e.g., μnit Scaling, staged precision, sharpness monitoring). Training robustness across seeds and hyperparameters is lower than in BF16 (Lee et al., 2024).
• Precision–range trade-off: Choice of E/M is critical; for models dominated by outliers, higher exponent count (e.g., E5M2 or E4M3) is needed, while smoother distributions favor more mantissa bits (Kuzmin et al., 2022).
• Hardware Support: Not all platforms yet offer native FP8 tensor-cores or arithmetic units; software emulation or on-the-fly casting can bottleneck throughput if not carefully integrated (Lee et al., 29 May 2025).
• Algorithmic Sensitivities: Learning rate schedules, scaling strategies, and residual-branch parameterization must be carefully managed under FP8 constraints (Narayan et al., 9 Feb 2025, Cambier et al., 2020).

Open research includes automatic scale/bias selection, design of accumulators with still lower precision, and general-purpose hardware for fully programmable 8-bit floating-point formats supporting all major AI workloads (Huang et al., 2021, Kuzmin et al., 2022, Luo et al., 2024).