We have upgraded from v0.25.0 to a higher version lately and see issues with newly created tables.
We use options.InferSchema(msg) to create and update schemas for our tables.
We use options.Marshal(msg) to marshal the messages.
With version v.26.0+, some rows cannot be ingested and BQ rejects them with an error:
some.field._some_one_of: no such field: _some_one_of."; Reason: "invalid"
We use these Options:
protobq.SchemaOptions{
UseEnumNumbers: false,
UseOneofFields: true,
UseDateTimeWithoutOffset: true,
}
When calling options.Marshal(msg), the nested map[string]bigquery.Value will contain the _some_one_of field.
When calling options.InferSchema(msg) the nested bigquery.Schema will not contain the _some_one_of field anymore.
When I disable this change:
// if oneof.IsSynthetic() {
// continue
// }
everything works as expected.
How do you handle this case?
We have upgraded from v0.25.0 to a higher version lately and see issues with newly created tables.
We use
options.InferSchema(msg)to create and update schemas for our tables.We use
options.Marshal(msg)to marshal the messages.With version v.26.0+, some rows cannot be ingested and BQ rejects them with an error:
We use these Options:
When calling
options.Marshal(msg), the nestedmap[string]bigquery.Valuewill contain the_some_one_offield.When calling
options.InferSchema(msg)the nestedbigquery.Schemawill not contain the_some_one_offield anymore.When I disable this change:
everything works as expected.
How do you handle this case?