Validate conversion of ViewDefinotion collections into Avro

[bashir2]

There seems to be a mismatch between how we create the Avro schema for a collection in a ViewDefinition here vs how we set collection values in an Avro record here. We should add unit-tests and confirm that this is working as intended (or fix it).

[pratt4]

HI @bashir2

I think there may be a schema/value mismatch risk in ViewSchema.java and wanted to confirm the intended contract before changing behavior.

In getAvroSchema(), codebase only emit an Avro array when columnType == null && isCollection(). If columnType != null (e.g : "type": "string"), codebase emit a scalar optionalString, even if collection=true.

In setValueInRecord() , when e.getColumnInfo().isCollection() is true we write a String[] into the record.

So for a column where collection=true and type="string" (if that’s allowed), the current logic could result in:

schema: nullable string

record value: String[] (array)

Questions to confirm intent:

Is collection=true && type!=null a valid ViewDefinition input (e.g. repeated primitives like string)?

If yes, should Avro schema always be an array whenever collection=true, regardless of type?

If no, should we enforce/validate that invariant (reject such ViewDefinitions) and add tests for it?

I can add a unit test that asserts the schema type and validates schema-record compatibility for a collection=true,type="string" column, and then (depending on the answer) either adjust schema generation or enforce the invariant.

Also, current tests seem to check that fields exist / values are set, but don’t assert that schema type matches the record value type for collections (e.g: via Avro validation or an actual write).