Skip to content

Databricks: timestamp precision 0-5 is now emitted (since 30.25.0), which Databricks rejects #8903

Closed

Description

@baruchoxman

Before you file an issue

  • read dialects: trino and snowflake; write dialect: databricks
  • Reproduced on 30.29.0 and on main (11bb075): DatabricksGenerator.SUPPORTS_TEMPORAL_PRECISION = True is still set. 30.24.0 is not affected.

Summary

Since 092e29a ("fix(databricks): preserve timestamp precision during SQL generation", released in 30.25.0), the Databricks generator emits any timestamp precision it is given. Databricks only accepts a parameterless TIMESTAMP, TIMESTAMP(6), or 7 to 9 when nanosecond timestamps are enabled. So a precision of 0 to 5, which other dialects produce routinely, now becomes SQL Databricks rejects. The most common source is a Trino timestamp literal without a fractional part, which parses as precision 0.

Fully reproducible code snippet

import sqlglot
from sqlglot import transpile

print(sqlglot.__version__)

# 1. A Trino timestamp literal without a fractional part
print(transpile("SELECT TIMESTAMP '2024-01-01 00:00:00'", read="trino", write="databricks")[0])

# 2. An explicit low precision from another dialect
print(transpile("SELECT CAST(x AS TIMESTAMP(3))", read="snowflake", write="databricks")[0])

# 3. Databricks to Databricks keeps a bare TIMESTAMP
print(transpile("SELECT CAST(x AS TIMESTAMP)", read="databricks", write="databricks")[0])

Output on 30.29.0:

30.29.0
SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP(0))
SELECT CAST(x AS TIMESTAMP(3))
SELECT CAST(x AS TIMESTAMP)

Output on 30.24.0, which Databricks accepts:

30.24.0
SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP)
SELECT CAST(x AS TIMESTAMP)
SELECT CAST(x AS TIMESTAMP)

Expected: a precision Databricks cannot represent is dropped, giving TIMESTAMP as in 30.24.0, while 6 and 7 to 9 are kept.

What Databricks does with the generated SQL (run on a Databricks SQL warehouse, 2026-11-04)

SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP(0)) AS t
-- [INVALID_TIMESTAMP_PRECISION] The seconds precision 0 of TIMESTAMP_LTZ is invalid. Expected an
-- integer in [7, 9] for nanosecond precision; use precision 6 or parameterless TIMESTAMP_LTZ for the
-- standard microsecond type. SQLSTATE: 22023

SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP(3)) AS t
-- same error, precision 3

SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP(6)) AS t   -- runs
SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP) AS t      -- runs

SELECT CAST('2024-01-01 00:00:00' AS TIMESTAMP(9)) AS t
-- [FEATURE_NOT_ENABLED] ... spark.sql.timestampNanosTypes.enabled

The flag that unlocks 7 to 9 does not help with 0 to 5, and on a SQL warehouse it cannot be set: SET spark.sql.timestampNanosTypes.enabled = true fails with CONFIG_NOT_AVAILABLE.

Official Documentation

Context

One shape for a fix: keep SUPPORTS_TEMPORAL_PRECISION for the precisions Databricks accepts and drop the parameter otherwise, for example in the Databricks datatype_sql. Happy to open a PR if that shape works for you.

Drafted with Claude; the output above was produced by running the snippet and the queries.

Activity

  1. ksaiharshavinay commented on Nov 4, 2026

    @ksaiharshavinay
    Contributor

    I'd like to work on this fix. My plan: reproduce with the snippet above, then narrow the Databricks generator so a precision Databricks cannot represent (0-5) is dropped to a bare TIMESTAMP, while 6 and 7-9 are preserved. I'll add regression tests in tests/dialects/test_databricks.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions