feature request: add auto-indent (`gg=G` / `indentexpr`) as special formatter

[chrisgrieser]

Did you check existing requests?

• I have searched the existing issues

Describe the feature

For some filetypes without formatters available, I found using a simple mapping to run gg=G useful to get at least some auto-formatting. It would be nice, if gg=G was available as a dedicated formatter for conform.nvim, similar to other special formatters like squeeze_blank or trim_newlines.

Provide background

It's another useful addition for the fallback filetype ["_"]. There are some filetypes where you either through treesitter or through "old vim" information on how to auto-indent (I believe it's opt.indentexpr?), but no formatter tooling. .scm files (treesitter query files) are one example.

What is the significance of this feature?

nice to have

Additional details

No response

[wookayin]

These two can be considered:

• Indent formatter w.r.t. indentexpr (what OP requested)
• The built-in formatter (gw), w.r.t. formatoptions

in the abstraction. Related to #269

[tigion]

I have written my custom formatter for this:

  opts = {
    formatters_by_ft = {
      gitconfig = { 'trim_whitespace', 'trim_newlines', 'my_auto_indent' },
    },
    formatters = {
      -- Custom formatter to auto indent buffer.
      my_auto_indent = {
        format = function(_, _, _, callback)
          vim.cmd.normal('m`gg=G``')
          callback()
        end,
      },
    },
  },

[chrisgrieser]

@tigion

Well that was easier than expected. Good idea! Minor modification, the snippet should use

vim.cmd.normal { "m`gg=G``", bang = true }

I'd suggest adding this one as built-in formatter via PR?

[tigion]

I think it still needs to be checked whether an area is selected and then only take = for this.

I only used it for the complete buffer so far.

[tigion]

Now with selection support in visual mode:

      my_auto_indent = {
        format = function(_, ctx, _, callback)
          -- save cursor position in ` mark -> indent -> restore cursor position
          if ctx.range == nil then
            -- no range, use whole buffer
            vim.cmd.normal({ 'm`gg=G``', bang = true })
          else
            -- use range
            vim.cmd.normal({ 'm`=``', bang = true })
          end
          callback()
        end,
      },

Why is bang = true needed?

[chrisgrieser]

Why is bang = true needed?

Without, any remapping for = or gg or m the user has well change the behavior of the normal command. See :h normal.

[tigion]

A shorter version:

      -- Custom formatter to auto indent buffer.
      -- - Indents with neovim's builtin indentation `=`.
      -- - Saves and restores cursor position in ` mark.
      my_auto_indent = {
        format = function(_, ctx, _, callback)
          -- no range, use whole buffer otherwise use selection
          local cmd = ctx.range == nil and 'gg=G' or '='
          vim.cmd.normal({ 'm`' .. cmd .. '``', bang = true })
          callback()
        end,
      },

[austinliuigi]

To note my experience, this caused problems when I used it with another formatter in the same filetype. Looking at the trim_newlines and trim_whitespace formatters, it seems that the format() function shouldn't mutate the buffer, but instead pass the formatted lines to the callback. Given that, here's the implementation that works for me:

  formatters_by_ft = {
    norg = { "vim_indent", "injected" },
  },
  formatters = {
    vim_indent = {
      format = function(_, ctx, _, callback)
        local view = vim.fn.winsaveview()

        local cmd = (ctx.range ~= nil) and "=" or "gg=G"
        vim.cmd("normal! " .. cmd)
        local out_lines = vim.api.nvim_buf_get_lines(0, 0, -1, true)
        vim.cmd("normal! u")

        callback(nil, out_lines)

        vim.fn.winrestview(view)
      end,
    },
  },

The change that made this work was saving the newly indented lines then undoing the actual indentation on the buffer, so as to not confuse conform.

[shikorido]

I know that this one unlikely will be present in conform.

But anyway, in last few days I wanted to use builtin vim formatter, but the fact that after undo cursor goes either to the beginning of the file (if simple logic like gg=G was applied, view save-restore only works after formatting, but not with undo) or to the beginning of the first change (this is conform's case with vim.lsp.util.apply_text_edits).

However, no matter how hard I've searched and tried, the only working solution to save cursor pos on undo was to replace the whole buffer with the formatted lines via vim.api.nvim_buf_set_lines. I would be glad to hear if someone knows in which cases cursor position does not change on undo like with a vim.lsp.buf.format.

Modifying the current buffer as was proposed is not a desirable solution if we take into account that formatters use stdout or temporary files, also it leaves 1 redundant change in undo history. So, in my opinion, the following solution might be more appropriate with its own flaws of course:

Collapsed due to plethora of comments

require("conform").setup({
  formatters_by_ft = {
    lua = { "stylua", "vim_indent" }
  },
    formatters = {
    -- It works, but after undo cursor moves
    -- to the beginning of the first change.
    -- It can't be avoided here because apply logic goes to conform.
    -- I don't know why, but with stylua, LSP and other formatters
    -- cursor position after undo stays where it was before formatting.
    -- It makes it more strange because conform.nvim always uses
    -- vim.lsp.util.apply_text_edits for applying formatting results.
    -- UPD. I was mistaken. If cursor position was outside changes
    -- applied by vim.lsp.util.apply_text_edits vim on undo moves it
    -- to the place of the first change.
    -- Perhaps, vim.lsp.buf.format either replaces the whole buffer or
    -- uses workarounds?
    --
    -- I can avoid this by setting all the formatted lines
    -- at once using vim.api.nvim_buf_set_lines,
    -- but still have no clue why cursor stays
    -- persistent after vim.lsp.buf.format.
    vim_indent = {
      name = "Builtin vim formatter",
      -- Probably it only applies to external formatters,
      -- not the ones that have format function.
      -- I didn't notice its impact, it is considered only
      -- by apply_format function since we define our own format function.
      -- https://github.com/stevearc/conform.nvim/blob/238f542a118984a88124fc915d5b981680418707/lua/conform/runner.lua#L182
      --range_args = true,
      -- Format fn signature.
      -- ---@cast config conform.LuaFormatterConfig
      -- local ok, err = pcall(config.format, config, ctx, input_lines, err_string_cb)
      -- https://github.com/stevearc/conform.nvim/blob/238f542a118984a88124fc915d5b981680418707/lua/conform/runner.lua#L344
      --
      -- Callback fn before wrapping.
      -- function(err, output)
      --   final_err = errors.coalesce(final_err, err)
      --   done = true
      --   result = output
      -- end
      -- https://github.com/stevearc/conform.nvim/blob/238f542a118984a88124fc915d5b981680418707/lua/conform/runner.lua#L693
      --- @diagnostic disable-next-line:unused-local (config)
      format = function(config, ctx, input_lines, callback)
        local bufnr = ctx.buf

        -- "=" for hacky, "gv=" for vimmy.
        --local cmd = (ctx.range ~= nil) and "=" or "gg=G"
        local cmd = (ctx.range ~= nil) and "gv=" or "gg=G"

        -- Create a temporary scratch buffer.
        local temp_buf = vim.api.nvim_create_buf(false, true)

        -- Copy buffer options to the temp buffer.
        local opts = {
          "ts", "sts", "sw", "et", "ft", "ff"
          -- The following options can be omitted.
          --"smarttab" -- window opt, does not work for buf.
          --"smartindent", "autoindent",
          --"cindent", "indentexpr"
        }
        for _, opt in ipairs(opts) do
          local sval = vim.api.nvim_get_option_value(opt, { buf = bufnr })
          vim.api.nvim_set_option_value(opt, sval, { buf = temp_buf })
        end

        -- Copy the lines from the original buffer to the temporary buffer.
        vim.api.nvim_buf_set_lines(temp_buf, 0, -1, false, input_lines)

        -- After range manipulations and format in a temporary buffer
        -- visual line mode gets changed to regular visual mode.
        -- Save the mode now to be able to restore it later.
        local mode = vim.api.nvim_get_mode().mode

        if not ctx.range then
          -- Run cmd in the temporary buffer.
          vim.api.nvim_buf_call(temp_buf, function() vim.cmd("silent normal! "..cmd) end)
        else
          -- Set selection area in the temporary buffer and execute cmd.
          -- Start and end chars can be omitted for %dG=%dG.
          --- @diagnostic disable-next-line:unused-local (start_char)
          local start_line, start_char = unpack(ctx.range.start)
          --- @diagnostic disable-next-line:unused-local (start_char)
          local end_line, end_char = unpack(ctx.range["end"])
          vim.api.nvim_buf_call(temp_buf, function()
            -- Simpler.
            --vim.cmd(
            --  string.format(
            --    "silent normal! %dG=%dG",
            --    start_line,
            --    end_line
            --  )
            --)
            --
            -- More hacky.
            --vim.fn.setpos(".", { 0, start_line, start_char + 1 })
            --vim.cmd("silent normal! "..mode)
            --vim.fn.setpos(".", { 0, end_line, end_char + 1 })
            --vim.cmd("silent normal! "..cmd)
            -- "=" fed key completely exits from visual mode,
            -- also involves other unintended issues
            -- including, but not limited to, restoration.
            --vim.fn.feedkeys("=", "L")
            --
            -- Vimmy.
            vim.api.nvim_buf_set_mark(0, "<", start_line, start_char, {})
            vim.api.nvim_buf_set_mark(0, ">", end_line, end_char, {})
            vim.cmd("silent normal! "..cmd)
          end)
        end

        -- Get modified lines from the temporary buffer.
        local formatted_lines
        if not ctx.range then
          formatted_lines = vim.api.nvim_buf_get_lines(temp_buf, 0, -1, false)
        else
          formatted_lines = {}
          for i = 1, #input_lines do
            formatted_lines[i] = input_lines[i]
          end
          -- Start and end chars can be omitted,
          -- but left to understand the next commented block.
          --- @diagnostic disable-next-line:unused-local (start_char)
          local start_line, start_char = unpack(ctx.range.start)
          --- @diagnostic disable-next-line:unused-local (end_char)
          local end_line, end_char = unpack(ctx.range["end"])
          for i = start_line, end_line do
            formatted_lines[i] = vim.api.nvim_buf_get_lines(temp_buf, i - 1, i, false)[1] or ""
          end
          -- The actions are redundant,
          -- callback expects a fully formatted output.
          -- Range is intended to be used by our format function.
          --formatted_lines[start_line] = formatted_lines[start_line]:sub(start_char + 1)
          --if start_line == end_line then
          --  end_char = end_char - start_char
          --end
          --formatted_lines[end_line] = formatted_lines[end_line]:sub(1, end_char + 1)

          -- This one only constructs formatted range.
          --formatted_lines = vim.api.nvim_buf_get_lines(temp_buf, start_line - 1, end_line, false)
          --formatted_lines[1] = formatted_lines[1]:sub(start_char + 1)
          --if start_line == end_line then
          --  end_char = end_char - start_char
          --end
          --formatted_lines[#formatted_lines] = formatted_lines[#formatted_lines]:sub(1, end_char + 1)
        end

        -- Clean up the temporary buffer.
        vim.api.nvim_buf_delete(temp_buf, { force = true })

        callback(nil, formatted_lines)

        -- Return to previous mode,
        -- without the actions visual line mode
        -- will be resetted to regular visual mode.
        -- We can't know whether the formatter
        -- was called asynchronously, though,
        -- ctx has no such info.
        --if not ctx.async then
        if mode == "v" or mode == "V" then
          local mode_after = vim.api.nvim_get_mode().mode
          -- The mode was completely resetted to normal by some reason.
          if mode_after ~= "v" and mode_after ~= "V" then
            vim.api.nvim_buf_call(bufnr, function()
              vim.cmd("silent normal! gv")
            end)
          else
            vim.api.nvim_buf_call(bufnr, function()
              vim.cmd("silent normal! "..mode)
            end)
          end
        end
        --end
      end
    }
  }
})

Note: if you want cursor to stay on undo consider defining your own formatting function
and use vim.api.nvim_buf_set_lines(bufnr, 0, -1, false, formatted_lines), by far this is the only solution I found that behaves nice with undos, just like vim.lsp.buf.format.

UPD. I finally figured out how the cursor behaves on undo. I'll leave the formatting function here for someone who also just like me wants cursor to stay on position where formatting was initiated when undoing an action. The undo-redo jiggling is a bad habit, probably... In the function you can choose either to replace the whole buffer or use first diff merge with a line cursor stays on (becomes expensive if first diff line is above the cursor line, no idea if it's even more efficient than just copying everything). It does not handle indentation in range, but why can't we simply use = or =gv?

Plethora of comments as always

local function FormatWithTemporaryBuffer(bufnr)
  bufnr = bufnr or 0

  -- Get all lines from the original buffer.
  local lines = vim.api.nvim_buf_get_lines(bufnr, 0, -1, false)

  -- Create a temporary buffer to work with.
  local temp_buf = vim.api.nvim_create_buf(false, true) -- create an empty buf

  -- Copy buffer options to the temp buffer.
  local opts = {
    "ts", "sts", "sw", "et", "ft", "ff"
    -- The following options can be omitted.
    --"smarttab" -- window opt, does not work for buf.
    --"smartindent", "autoindent",
    --"cindent", "indentexpr"
  }
  for _, opt in ipairs(opts) do
    local sval = vim.api.nvim_get_option_value(opt, { buf = bufnr })
    vim.api.nvim_set_option_value(opt, sval, { buf = temp_buf })
  end

  -- Copy the lines from the original buffer to the temporary buffer.
  vim.api.nvim_buf_set_lines(temp_buf, 0, -1, false, lines)

  -- Run gg=G in the temporary buffer to indent everything.
  vim.api.nvim_buf_call(temp_buf, function() vim.cmd("silent normal! gg=G") end)
  -- Instead of explicit switching.
  --vim.api.nvim_win_set_buf(0, temp_buf) -- switch to the temp buf
  --vim.cmd("silent normal! gg=G") -- indent the entire buf
  --vim.api.nvim_win_set_buf(0, bufnr) -- switch to the original buf

  -- Get modified lines from the temporary buffer.
  local formatted_lines = vim.api.nvim_buf_get_lines(temp_buf, 0, -1, false)

  -- Note: to completely fix the cursor jump issue on undo we must either
  -- replace the whole buffer with formatted lines (simplest),
  -- or merge the diffs depending on 2 cases:
  -- 1. If the cursor is on line before first diff we can simply
  --    copy this one line the cursor stays on.
  -- 2. If the cursor is on line after first diff we must merge
  --    all the consequent lines with the current cursor line.
  --    This way vim will respect our cursor pos on undo.

  local cursor_line = vim.api.nvim_buf_call(bufnr, function()
    return vim.fn.getpos(".")[2]
  end)
  -- Important assertion to significantly simplify the logic.
  assert(
    #lines == #formatted_lines,
    "FormatWithTemporaryBuffer: #lines ~= #formatted_lines"
  )
  local first_diff_start_line, first_diff_end_line
  local first_diff = {}
  -- TODO: Use vim.diff to make it faster (and simpler?).
  -- And, probably, refactor to work with vim.lsp.util.apply_text_edits
  -- by constructing text_edits table.
  for i = 1, #formatted_lines do
    if formatted_lines[i] ~= lines[i] then
      if first_diff then
        if not first_diff_start_line then
          first_diff_start_line = i
        else
          first_diff_end_line = i
        end
      else
        vim.api.nvim_buf_set_lines(bufnr, i - 1, i, false, { formatted_lines[i] })
      end
    end
    if first_diff then
      -- "==" will be reached anyway, ">=" is redundant.
      if i == cursor_line then
        if not first_diff_start_line then
          first_diff_start_line = cursor_line
        end
        first_diff_end_line = cursor_line
        for j = first_diff_start_line, first_diff_end_line do
          table.insert(first_diff, formatted_lines[j])
        end
        vim.api.nvim_buf_set_lines(bufnr, first_diff_start_line - 1, first_diff_end_line, false, first_diff)
        first_diff = nil
      end
    end
  end

  -- Or simply copy the formatted lines.
  --vim.api.nvim_buf_set_lines(bufnr, 0, -1, false, formatted_lines)

  -- Clean up the temporary buffer.
  vim.api.nvim_buf_delete(temp_buf, { force = true })
end

[leo60228]

I might be misunderstanding the exact issue you describe, but the vim_indent I'm using (tweaked from one of the other comments on this issue) seems to have the same undo behavior as for external formatters:

conform.setup({
  formatters_by_ft = {
    vim = { 'vim_indent', 'injected' },
  },
  formatters = {
    vim_indent = {
      format = function(_, ctx, _, callback)
        local view = vim.fn.winsaveview()
        local old_start_l, old_start_c = unpack(vim.api.nvim_buf_get_mark(0, '<'))
        local old_end_l, old_end_c = unpack(vim.api.nvim_buf_get_mark(0, '>'))

        if ctx.range ~= nil then
          vim.api.nvim_buf_set_mark(0, '<', ctx.range['start'][1], ctx.range['start'][2], {})
          vim.api.nvim_buf_set_mark(0, '>', ctx.range['end'][1], ctx.range['end'][2], {})
        end

        local cmd = (ctx.range ~= nil) and 'gv=' or 'gg=G'
        vim.cmd('silent normal! ' .. cmd)
        local out_lines = vim.api.nvim_buf_get_lines(0, 0, -1, true)
        vim.cmd('silent normal! u')

        callback(nil, out_lines)

        vim.api.nvim_buf_set_mark(0, '<', old_start_l, old_start_c, {})
        vim.api.nvim_buf_set_mark(0, '>', old_end_l, old_end_c, {})
        vim.fn.winrestview(view)
      end,
    },
  },
})

[shikorido]

The answer was really quick, thank you for curiosity. To observe the issue I'm talking about, move the cursor to place which won't be changed after formatting, then run full buffer formatting and make undo. Cursor will move to the first change. Then try to do the same using vim.lsp.buf.format(). The cursor won't go to the first change. Somehow vim decides to make cursor sticky if it was in place that was changed.

And yea, I was mistaken on part which compares vim.lsp.util.apply_text_edits and vim.lsp.buf.format. The first one that conform uses have the same behavior with stylua, my bad. But the statement that vim.lsp.buf.format saves cursor pos on undo - that's true. I can only assume that it replaces the whole buffer.

[shikorido]

By the way, the proposed approach that formats the current buffer, saves formatted lines, makes undo and sends the formatted lines to conform leaves 1 redundant change in undo history. If you have undotree, you will see literally two identical changes next to each other.