Skip to content

Linting & Formatting

Python1

We use the Ruff2 formatter to both lint and format our Python code. This is the same system that is used in Databricks, though the globally applied settings are currently unknown to us. Custom settings can be applied using the pyproject.toml file under the [tool.ruff] section. For example:

[tool.ruff]
line-length = 88
builtins = ["dbutils", "spark"]
exclude = [
    "path/to/file.py"
]

Currently, custom settings are not working on Databricks, but they should still apply when working locally in your IDE.

To install and use ruff locally, you can use the following steps:

  1. Make sure you have poetry installed on your system (documentation forthcoming)
  2. poetry add ruff
  3. Edit your pyproject.toml file to include a [tool.ruff] section like that above^
  4. poetry run ruff check to see whether your file formats are correct.
    • This is what's used as part of our current CI build process
  5. poetry run ruff check --fix to let ruff apply any easy fixes that it can find
    • Examples might include unused imports or extra whitespace at the end of lines

Note

To run ruff on Databricks, use the Edit > Format document option or press Ctrl + Shift + F. This may take a while (up to a minute) to run, so be patient. For notebooks, you can use Edit > Format notebook

SQL3

All Databricks SQL formatting settings must be stored in a file under your Workspace's home directory called .dbsql-formatter-config.json. The settings will apply globally across all SQL scripts you create. Here is a list of options that can be set:

{
  "printWidth": 80,
  "indentationStyle": "spaces",
  "indentationWidth": 4,
  "keywordCasing": "uppercase",
  "functionNameCasing": "lowercase",
  "commaPosition": "end",
  "numNewLinesBetweenStatements": 1,
  "numNewLinesBetweenClauses": 0,
  "shouldExpandExpressions": true,
  "shouldExpandCaseStatements": true,
  "shouldExpandInStatements": false,
  "shouldExpandBetweenConditions": false,
  "shouldBreakOnJoinSections": true
}
  • Fields:
  • printWidth: Max line length, defaults to 100
  • indentationStyle: "tabs" or "spaces" for indentation, default "spaces"
  • indentationWidth: Number of spaces when indentationStyle is set to "spaces", default 2
  • keywordCasing: SQL keywords in "uppercase" or "lowercase"
  • functionNameCasing: SQL functions casing as "uppercase" or "lowercase"
  • commaPosition: Comma placement in lists at the "beginning" or "end", default "end"
  • numNewLinesBetweenStatements: Number of lines between separate statements, default 1
  • numNewLinesBetweenClauses: Number of lines between clauses within a statement, default 0
  • shouldExpandExpressions: true if boolean expressions should be on separate lines, defaults to true
  • shouldExpandCaseStatements: true if CASE statements should be on separate lines, defaults to true
  • shouldExpandInStatements: true if items in an IN selection should be on separate lines, defaults to false
  • shouldExpandBetweenConditions: true if items in a BETWEEN selection should be on separate lines, defaults to false
  • shouldBreakOnJoinSections: true if JOIN's ON conditions should be on separate lines, defaults to true

  1. https://learn.microsoft.com/en-us/azure/databricks/notebooks/notebook-editor#configure-python-syntax-highlighting ↩

  2. https://docs.astral.sh/ruff/ ↩

  3. https://learn.microsoft.com/en-us/azure/databricks/sql/user/sql-editor/custom-format ↩