Search before asking
Paimon version
master @ 711481b
Compute Engine
Hive
Minimal reproduce step
HiveTypeUtils.HiveToPaimonTypeVisitor.atomic() (paimon-hive/paimon-hive-common/.../HiveTypeUtils.java line ~312) maps Hive timestamp to DataTypes.TIMESTAMP_MILLIS() (precision 3). This runs whenever a Hive schema is inferred into Paimon (migrate / clone / CREATE via Hive). Hive timestamp holds up to nanosecond precision, so any sub-millisecond fraction is silently truncated.
What doesn't meet your expectations?
The mapping should preserve precision like its siblings. In the same file, the local-zoned reverse path returns DataTypes.TIMESTAMP_WITH_LOCAL_TIME_ZONE() (default precision 6), and SparkTypeUtils.java:436 maps timestamp to new TimestampType() (precision 6). Only the Hive timestamp path drops to precision 3, producing an inconsistent, lossy schema.
Anything else?
Proposed fix: use DataTypes.TIMESTAMP() (precision 6) for parity with the same-file LTZ path and SparkTypeUtils. Open design point: precision 9 would be fully lossless for Hive nanosecond timestamps — worth a committer's decision. Existing persisted schemas are unaffected; only newly inferred schemas change.
Are you willing to submit a PR?
Search before asking
Paimon version
master @ 711481b
Compute Engine
Hive
Minimal reproduce step
HiveTypeUtils.HiveToPaimonTypeVisitor.atomic()(paimon-hive/paimon-hive-common/.../HiveTypeUtils.java line ~312) maps HivetimestamptoDataTypes.TIMESTAMP_MILLIS()(precision 3). This runs whenever a Hive schema is inferred into Paimon (migrate / clone / CREATE via Hive). Hivetimestampholds up to nanosecond precision, so any sub-millisecond fraction is silently truncated.What doesn't meet your expectations?
The mapping should preserve precision like its siblings. In the same file, the local-zoned reverse path returns
DataTypes.TIMESTAMP_WITH_LOCAL_TIME_ZONE()(default precision 6), andSparkTypeUtils.java:436maps timestamp tonew TimestampType()(precision 6). Only the Hivetimestamppath drops to precision 3, producing an inconsistent, lossy schema.Anything else?
Proposed fix: use
DataTypes.TIMESTAMP()(precision 6) for parity with the same-file LTZ path and SparkTypeUtils. Open design point: precision 9 would be fully lossless for Hive nanosecond timestamps — worth a committer's decision. Existing persisted schemas are unaffected; only newly inferred schemas change.Are you willing to submit a PR?