Skip to content

perf: Implement DecimalEncoder that used 256-bit math to encode decimal values to the avro binary representation#366

Open
PauloHMattos wants to merge 1 commit intoch-robinson:mainfrom
PauloHMattos:perf-binary-decimal
Open

perf: Implement DecimalEncoder that used 256-bit math to encode decimal values to the avro binary representation#366
PauloHMattos wants to merge 1 commit intoch-robinson:mainfrom
PauloHMattos:perf-binary-decimal

Conversation

@PauloHMattos
Copy link
Copy Markdown
Contributor

This is something I had on my fork because my application primarily works with decimal for its numbers.
My objective was to reduce the allocations associated with the use of BigInteger to encode the decimal values, but the change also had a positive impact in throuput.

| Method   | Job                | Runtime            | Mean     | Error     | StdDev    | Median   | Ratio | RatioSD | Allocated | Alloc Ratio |
|--------- |------------------- |------------------- |---------:|----------:|----------:|---------:|------:|--------:|----------:|------------:|
| Baseline | .NET 8.0           | .NET 8.0           | 1.448 ms | 0.1946 ms | 0.5707 ms | 1.689 ms |  0.35 |    0.14 |      48 B |       0.000 |
| Baseline | .NET Framework 4.8 | .NET Framework 4.8 | 4.142 ms | 0.0903 ms | 0.2647 ms | 4.063 ms |  1.00 |    0.09 | 2727936 B |       1.000 |
  1. I think this change could also be applied to the non NET6_0_OR_GREATER path, but i decided to keep netstandard2.0 as is.
  2. The 48 bytes allocated are due to the allocation of the enumerator for the list. This is solved by perf: Avoid enumerator boxing and virtual calls during array serialization #365

I don't know, if the extra complexity of this change is worth to take, but decided to open the PR anyway, in case you think it is.

Benchmark code:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using Chr.Avro.Abstract;
using Chr.Avro.Serialization;

[SimpleJob(RuntimeMoniker.Net48, baseline: true)]
[SimpleJob(RuntimeMoniker.Net80)]
[MemoryDiagnoser]
public class AvroSerializeListOfObjects {
    private ObjectWithCollection _toSerialize;
    private MemoryStream _memoryStream;
    private Chr.Avro.Serialization.BinaryWriter _writer;
    private BinarySerializer<ObjectWithCollection> _serializer;

    [GlobalSetup]
    public void GlobalSetup() {
        // Large enough to fit the serialized value
        _memoryStream = new MemoryStream(500 * 1024 * 1024);
        _writer = new Chr.Avro.Serialization.BinaryWriter(_memoryStream);
        var rnd = new Random();
        _toSerialize = new ObjectWithCollection() {
            Numbers = Enumerable.Range(0, 10000).Select(i => (decimal)rnd.NextDouble()).ToList(),
        };

        var serializerBuilder = new BinarySerializerBuilder();
        var schemaBuilder = new SchemaBuilder();
        var schema = schemaBuilder.BuildSchema<ObjectWithCollection>();
        _serializer = serializerBuilder.BuildDelegate<ObjectWithCollection>(schema);
    }

    [IterationSetup]
    public void IterationSetup() => _memoryStream.Position = 0;


    [Benchmark]
    public void Baseline() => _serializer(_toSerialize, _writer);
}

public class ObjectWithCollection {
    public List<decimal> Numbers { get; set; }
}

@PauloHMattos PauloHMattos changed the title perf: Implement DecimalEncder that used 256-bit math to encode decimal values to the avro binary representation perf: Implement DecimalEncoder that used 256-bit math to encode decimal values to the avro binary representation Dec 1, 2025
@PauloHMattos
Copy link
Copy Markdown
Contributor Author

This code code be simplified a lot (and maybe even have a better performance) by using something like https://github.com/NethermindEth/int256

I decided against it to avoid adding a new dependency

@PauloHMattos
Copy link
Copy Markdown
Contributor Author

PauloHMattos commented Dec 4, 2025

If the target framework of the library was updated to net8.0 this could be simplified by using UInt128 and maybe vectorized using the new cross-platform Vector128/Vector256 (both introduced in net7.0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant