Support maxLength in JSON Schema #105
Description
Is your feature request related to a problem? Please describe.
My Redshift warehouse uses this tap, and is experiencing disk size bloat. It appears to be much bigger than I'd expect (2 orders of magnitude). I suspect this is due to target-redshift creating all columns as varchar(10000), despite my source data (Postgresql) having mostly varchar(256).
Redshift docs advise to use the smallest possible column size for data (see here and here).
Describe the solution you'd like
If the input stream supplies a "maxLength" property for a string field, use that as the size of the field created. This behaviour could be turned on via a config setting.
Describe alternatives you've considered
Not sure. Possibly running a bunch of "alter table" queries after ingestion.
Additional context
I would like to use this in conjunction with a tap-postgresql change described at MeltanoLabs/tap-postgres#477