Potential regression with bigdecimal_as_decimal

[calderhayes]

Hello!

Upon updating our app from 3.16.12=>3.16.13 we noticed that some of our tests were failing. We were expecting a decimal value and instead received a string version of that decimal value.

We suspect that the update may have a regression which ignores some default options set in the initializer. Our json initializer looks like this:

require "oj"
require "active_support/json/encoding"

Oj.default_options = { mode: :compat, bigdecimal_as_decimal: true, safe: true }
Oj.optimize_rails

We serve this JSON using jbuilder if that makes a difference.

Do you think this is a bug? Is there a work around?

Thank you,

Calder

[ohler55]

The changes between the two version were test updates, SIMD changes, and rails array and hash subclass fixes. Nothing changes regarding floating point parsing. Can you provide and example that works in v3.16.12 and fails in v3.16.13?

[omeH]

Hi, maybe I can provide examples.

First example with correct encoding:

ActiveSupport: 8.0.3
Oj: 3.16.12

initializers/oj.rb
Oj.optimize_rails
Oj.default_options = { bigdecimal_as_decimal: true, bigdecimal_load: :bigdecimal, decimal_class: BigDecimal }

Oj::Rails::Encoder.new.encode(test: BigDecimal("10.0")) => "{"test":10.0}"

Second example with incorrect encoding:

ActiveSupport: 8.0.3
Oj: 3.16.13

initializers/oj.rb
Oj.optimize_rails
Oj.default_options = { bigdecimal_as_decimal: true, bigdecimal_load: :bigdecimal, decimal_class: BigDecimal }

Oj::Rails::Encoder.new.encode(test: BigDecimal("10.0")) => "{"test":"10.0"}"

You can see that results are different with different gem version.

[omeH]

Also, recently I tried to update ActiveSupport version up to 8.1.1. In this version there was changed behavior of ActiveSupport::JSON.encode method. And I had similar issue.

initializers/oj.rb
Oj.default_options = { bigdecimal_as_decimal: true, bigdecimal_load: :bigdecimal, decimal_class: BigDecimal }
Oj.optimize_rails

ActiveSupport::JSON::Encoding.encode_without_options({ test: BigDecimal("10.0") }) => "{"test":"10.0"}"
Oj::Rails::Encoder.new.encode(test: BigDecimal("10.0")) => "{"test":10.0}"

[ohler55]

It seems that ActiveSupport 8.1.1 now emits a BigDecimal as a string if without options. To "fix" oj I'll need to know what should be emitted with each ActiveSupport version and how the encode_without_options effects the result as well.

[omeH]

ActiveSupport::JSON::Encoding.encode_without_options is an instance of Oj::Rails::Encoder 🤔 . https://github.com/rails/rails/blob/main/activesupport/lib/active_support/json/encoding.rb#L229
If I understand correctly Oj::Rails.set_encoder sets up this encoder.

[ohler55]

Correct me if I am wrong but there is also an encoder with options. It sounds like the desire is for supporting both.

[omeH]

No, you are right. There are two defined instance of Oj::Rails::Encoder and Oj::Rails::Encoder itself. How it handles now you can check here https://github.com/rails/rails/blob/main/activesupport/lib/active_support/json/encoding.rb#L47

[ohler55]

With a little more time on the holidays I finally dug into this issue. The underlying issue is that activesupport has a as_json method on Hash which since defined, gets called. In that call BigDecimal is converted to a string before being handed back to Oj. The work around is to monkey patch BigDecimal like this:

class BigDecimal
  def as_json
    self
  end
end

I'm not sure if that break anything else. I'm experimenting with having the method look at the Oj defaults to determine how to encode the results.

[gstokkink]

@ohler55 we're running into this as well after upgrading to Rails 8.1.1, your workaround fixes it for us. Note that we had to define it as def as_json(*) since arguments can get passed in.

[ohler55]

Bottom line is Oj can't really really ignore the as_json methods unless a special case is made for Arrays and Hash. Is that something folks are interested in?

[gstokkink]

@ohler55 sure

[lizdeika]

Hello, any plans to fix this? 😅
Currently stuck with 3.16.12 because in newer versions decimals are strings?

I try every new version of oj but rspec catches an error in the test suite.

     Failure/Error: expect(items.first["amount"]).to eq(item.amount)

       expected: 0.1e2
            got: "10.0"

       (compared using ==)

[gstokkink]

We decided to drop the oj gem because of this issue and others. See also https://byroot.github.io/ruby/json/2024/12/29/optimizing-ruby-json-part-4.html (and related posts by byroot).

[ohler55]

@lizdeika , byt fix do you mean monkey patching BigDecimal or making a special case for Array and Hash?