Iceberg v3 support - enable and initial version

[Shekharrajak]

Ref #3376

[mbutrovich]

Thanks for starting the work on this @Shekharrajak! We need extremely conservative fallbacks for table v3, because getting it wrong would yield incorrect results. For example, I don't see anything for encrypted tables.

[Shekharrajak]

Thanks for starting the work on this @Shekharrajak! We need extremely conservative fallbacks for table v3, because getting it wrong would yield incorrect results. For example, I don't see anything for encrypted tables.

Right now we are falling back to Spark for features which is not supported in comet. Isn't we should enable it for supported features ?

I can add unsupport check for encrypted tables for now similar to DV.

[Shekharrajak]

these check may slowdown a bit but eventually the plan is to support these v3 features and fully functional iceberg v3 spec support.

[Shekharrajak]

Please trigger the workflow.