MayaFlux 1.0.0
Digital-First Multimedia Processing Framework
Loading...
Searching...
No Matches
TensorConvert.hpp
Go to the documentation of this file.
1#pragma once
2
3#include "Tensor.hpp"
4
5#include <expected>
6
7namespace MayaFlux::Kakshya {
8
9/**
10 * @brief Empty when a tensor can be converted to a type, otherwise the reason it cannot.
11 *
12 * Whole-byte integers (8 to 64 bits, signed and unsigned), bool, IEEE half,
13 * bfloat16, float and double, and complex of float and double are supported.
14 * Complex and real types do not convert to each other. Other types (float8,
15 * float6, float4, sub-byte integers and multi-lane elements) are refused.
16 *
17 * Quantization is only held by integer storage of at most 32 bits. A source
18 * with quantization is dequantized; @p quantization asks for the result to be
19 * quantized and needs an integer target.
20 *
21 * @param source Layout of the tensor to convert.
22 * @param target Element type of the result.
23 * @param quantization Quantization the result should carry, if any.
24 */
25[[nodiscard]] MAYAFLUX_API std::string diagnose_conversion(
26 const TensorSpec& source, TensorDType target,
27 const std::optional<TensorQuantization>& quantization = std::nullopt);
28
29/**
30 * @brief Convert the elements of a tensor to another element type.
31 *
32 * The result is a dense C-order tensor with the source's name and axes. A
33 * strided source is read through its strides.
34 *
35 * Integer to integer conversion is exact and saturates. A float becomes an
36 * integer by rounding to nearest, ties to even, with NaN as zero and
37 * saturation at the range. A float narrows by rounding to nearest, ties to
38 * even, and overflows to infinity. Anything non-zero becomes true in a bool.
39 *
40 * Values are real numbers when a quantization is involved: a source with
41 * quantization is read as scale * (stored - zero_point), and a requested
42 * quantization stores clamp(round(value / scale) + zero_point), with NaN
43 * stored as the zero point. A source quantization is not carried to the
44 * result unless it is requested again.
45 *
46 * Usage:
47 * @code
48 * auto weights = convert_tensor(*bf16_weights.access(), TensorDType::of<float>());
49 * auto packed = convert_tensor(*activations.access(), TensorDType::of<int8_t>(),
50 * TensorQuantization { .scale = { 0.05 }, .zero_point = { 3 }, .axis = std::nullopt });
51 * @endcode
52 *
53 * @param source Tensor to read.
54 * @param target Element type of the result.
55 * @param quantization Quantization the result should carry, if any.
56 * @return The converted tensor, or the reason it could not be converted.
57 */
58[[nodiscard]] MAYAFLUX_API std::expected<TensorData, std::string> convert_tensor(
59 const TensorAccess& source, TensorDType target,
60 const std::optional<TensorQuantization>& quantization = std::nullopt);
61
62}
Validated, non-owning view of tensor memory and its layout.
std::string diagnose_conversion(const TensorSpec &source, TensorDType target, const std::optional< TensorQuantization > &quantization)
Empty when a tensor can be converted to a type, otherwise the reason it cannot.
std::expected< TensorData, std::string > convert_tensor(const TensorAccess &source, TensorDType target, const std::optional< TensorQuantization > &quantization)
Convert the elements of a tensor to another element type.
Element type as kind, bit width and lane count.
Complete description of a tensor: type, shape, layout, meaning.