Fixed-point encoding, quantisation and error budgets

By UK A Level Computer Science · independent study · myAudience

Original generated teaching illustration, built-in imagegen,2026-10-04; mathematical model,not documentary photograph. · Original generated artwork; operator provenance/rights and independent scientific acceptance pending.

Fixed-point encoding, quantisation and error budgets

A fixed-point format places the binary point at a declared position. The bits do not store a visible dot: the decoder supplies its meaning. In our eight-bit signed model, four positions are fractional, giving weights −8,4,2,1,1/2,1/4,1/8,1/16. The value is the signed integer encoded by the byte divided by 16. This contract permits exact fractions on a uniform lattice while excluding values between lattice points. It differs from an ordinary eight-bit integer even when the stored pattern is identical.

The minimum pattern 10000000 gives −8. The maximum 01111111 gives 7+15/16=7.9375. Adjacent representable values differ by 1/16=0.0625. Increasing fractional positions at unchanged total width decreases this spacing but reduces the available integer range. Wider storage can improve both at a cost. State signedness, width and point location together: claiming that fixed point has a particular range without these choices is incomplete. A negative weight belongs to the leading bit, not to a separate minus sign outside the field.

To encode −2.375, multiply by 16 to obtain −38, an exact integer. Its eight-bit two’s-complement pattern is 256−38=218, or 11011010. With the declared point it is 1101.1010, decoded as −8+4+1+1/2+1/8=−2.375. The encoded integer and scaled value are different units. Applying an unsigned conversion would decode 13.625 and fail the sign contract. Division by 16 should occur after the correct signed interpretation, not after discarding the leading bit.

A value is exact only if scaling yields a representable integer. Decimal 0.1×16=1.6, so the format cannot represent it exactly. Under nearest rounding with a separately defined tie rule,1.6 rounds to 2 and the stored value is 2/16=0.125. Truncation towards zero would instead choose 1/16=0.0625. Neither result makes the original value irrational. Different rounding policies give different errors and can introduce different bias; the policy must be specified before judging an implementation.

Absolute error is the magnitude of stored minus intended value. Here|0.125−0.1|=0.025. Relative error divides that magnitude by|0.1|, giving 0.25 or 25%. The same absolute error can be insignificant for a large value and serious for a small one. At intended zero the usual relative-error denominator is zero, so report a suitable absolute measure instead of inventing a percentage. A tolerance written only as a percentage may therefore be inadequate for an application whose values cross zero.

For nearest rounding within the interior of a uniform representable lattice, the absolute rounding error is at most half a step, here 1/32=0.03125. This bound excludes overflow and depends on a well-defined tie rule. Values beyond the range cannot be legitimised merely by rounding to the nearest bit pattern: clamping, wrapping or rejecting are separate outcomes. A stored value may also undergo later arithmetic that adds further errors. A bound for one conversion does not prove a bound for a complete measurement or processing pipeline.

Consider a fictional sensor requiring values from −4 to 4 with absolute conversion error at most 0.02. The current four-fractional-bit step permits an error up to 0.03125, so it cannot guarantee that target. Five fractional bits give a half-step of 0.015625 but eight signed bits then range only −4 to 3.96875, excluding positive 4. A nine-bit signed format with five fractional bits covers −8 to 7.96875 and satisfies the stipulated conversion bound. The extra bit solves a specific combined range-and-error requirement, not every source of sensor inaccuracy.

Reading the diagram: The plate aligns the 1101.1010 cells with signed weights and shows 0.1 between its two neighbouring lattice points. The arrow to 0.125 represents the specified nearest policy, not a physical measurement shift. Its error cards refer to that conversion only. A strong transfer answer uses exact fractions where possible, checks both range and tolerance and distinguishes quantisation from instrument noise or algorithmic error. It should also specify how the boundary reports an unrepresentable input without pretending that a wrapped value is a successful reading.

Worked example: Encode −2.375 and nearest-round 0.1 in the stated 8-bit signed format with 4 fractional bits. Calculate absolute and relative rounding error.

−2.375×16=−38; its byte is 218=11011010, decoded with point after four bits.

0.1×16=1.6 rounds to 2, so stored value=0.125.

Absolute error=0.025; relative error=0.025/0.1=0.25=25%.

Answer: −2.375 is exact as 1101.1010. One tenth is approximated by 0.125 under nearest rounding, with the specified errors.

Check your understanding: A fictional sensor must represent −4 through 4 with conversion error≤0.02. Compare an 8-bit signed format with 4 or 5 fractional bits and a 9-bit format with 5. Explain why a correct conversion bound does not establish total sensor accuracy and what relative error means at zero.

Answer: Four fractional bits give half-step 1/32=0.03125, exceeding the required bound. Five give 1/64=0.015625, but 8 signed bits then range −4..3.96875 and cannot represent 4. With 9 bits and 5 fractional bits the range is −8..7.96875 and the nearest-rounding bound 1/64 meets both stipulated requirements. Physical calibration, noise and later processing errors remain separate and must be budgeted if a total guarantee is required. Relative error using the intended value as denominator is undefined at zero; use an explicit absolute tolerance there. Boundary rejection should distinguish out-of-range failure from a legitimate zero.

Why: Require both range and error calculations under the same format, an explicit rounding assumption and a separation of model conversion error from real measurement accuracy. Another suitable wider representation is acceptable if justified.