feat!: remove code for SQLite (#172)

* feat!: remove code for SQLite

ref [Introduce revised telescope-frecency.nvim : neovim](https://www.reddit.com/r/neovim/comments/174m8zu/introduce_revised_telescopefrecencynvim/)

I have deprecated SQLite features 4 months ago. It is the time to remove
code for them.

* test: fix test to load telescope validly

* test: remove sqlite.lua from CI settings

* test: test database as native

* fix: add lacked type from old database/sqlite.lua

* docs: remove description for SQLite3 logic

* chore: fix types

* chore: add types for Database:raw_table

.github/workflows/ci.yml

@@ -30,11 +30,6 @@ jobs:
         with:
           repository: nvim-telescope/telescope.nvim
           path: telescope.nvim
-      - name: Checkout sqlite.lua
-        uses: actions/checkout@v3
-        with:
-          repository: kkharji/sqlite.lua
-          path: sqlite.lua
       - name: Install Neovim
         uses: rhysd/action-setup-vim@v1
         id: nvim
@@ -45,7 +40,6 @@ jobs:
         env:
           PLENARY_PATH: plenary.nvim
           TELESCOPE_PATH: telescope.nvim
-          SQLITE_PATH: sqlite.lua
           DEBUG_PLENARY: 1
           EXE: ${{ steps.nvim.outputs.executable }}
         run: |-
@@ -59,14 +53,12 @@ jobs:
         env:
           PLENARY_PATH: plenary.nvim
           TELESCOPE_PATH: telescope.nvim
-          SQLITE_PATH: sqlite.lua
           DEBUG_PLENARY: 1
           EXE: ${{ steps.nvim.outputs.executable }}
         run: |-
           # HACK: This is needed because it fails to add runtimepath's.
           cp -af $PLENARY_PATH/lua/plenary/ lua/
           cp -af $TELESCOPE_PATH/lua/telescope/ lua/
-          cp -af $SQLITE_PATH/lua/sqlite/ lua/
           TEST_DIR=lua/frecency/tests/
           MINIMAL_LUA=${TEST_DIR}minimal.lua
           NVIM=$(perl -e '$_ = $ENV{EXE}; s,\\,/,g; print')

README.md

@@ -81,20 +81,11 @@ directories provided by the language server.
 - [nvim-web-devicons](https://github.com/kyazdani42/nvim-web-devicons) (optional)
 - [ripgrep](https://github.com/BurntSushi/ripgrep) or [fd](https://github.com/sharkdp/fd) (optional)
 
-**NOTE:** The former version of this plugin has used [SQLite3][] database to
-store timestamps and file records. But the current build uses Lua native code
-to store them, so you can now remove [sqlite.lua][] from dependencies. See
-[*Remove dependency for sqlite.lua*][remove-sqlite] for the detail.
-
 **NOTE:** `ripgrep` or `fd` will be used to list up workspace files. They are
 extremely faster than the native Lua logic. If you don't have them, it
 fallbacks to Lua code automatically. See the detail for `workspace_scan_cmd`
 option.
 
-[SQLite3]: https://www.sqlite.org/index.html
-[sqlite.lua]: https://github.com/kkharji/sqlite.lua
-[remove-sqlite]: #user-content-remove-dependency-for-sqlitelua
-
 ## Installation
 
 ### [Packer.nvim](https://github.com/wbthomason/packer.nvim)
@@ -224,11 +215,6 @@ See [default configuration](https://github.com/nvim-telescope/telescope.nvim#tel
 
   Determines if non-indexed files are included in workspace filter results.
 
-- `use_sqlite` (default: `false`)
-
-  Use [sqlite.lua][] with `true` or native code with `false`. See [*Remove
-  dependency for sqlite.lua*][remove-sqlite] for the detail.
-
 - `workspace_scan_cmd` (default: `nil`)
 
   This option can be set values as `"LUA"|string[]|nil`. With the default
@@ -280,7 +266,7 @@ telescope.setup {
 
 The default location for the database is `$XDG_DATA_HOME/nvim` (eg
 `~/.local/share/nvim/` on linux). This can be configured with the `db_root`
-config option.
+config option. The filename for the database is `file_frecency.bin`.
 
 ### Maintainance
 
@@ -318,21 +304,22 @@ not remove the file itself, only from DB.
 :FrecencyDelete /full/path/to/the/file
 ```
 
-### Remove dependency for [sqlite.lua][]
+### Note about the compatibility for the former version.
 
-The former version of this plugin has used SQLite3 library to store data. When
+The former version of this plugin has used SQLite3 library to store data. [#172][]
-you upgrade from such version, Neovim will silently migrate DB and inform that
+has removed the whole code for that. If you prefer the old SQLite database,
-you can remove `sqlite.lua` from dependencies.
+you should lock the version to [a3e818d][] with your favorite plugin manager.
 
-| made by default | made by `sqlite.lua` |
+[#172]: https://github.com/nvim-telescope/telescope-frecency.nvim/pull/172
-|--|--|
+[a3e818d]: https://github.com/nvim-telescope/telescope-frecency.nvim/commit/a3e818d001baad9ee2f6800d3bbc71c4275364ae
-| `~/.local/share/nvim/file_frecency.bin` | `~/.local/share/nvim/file_frecency.sqlite3` |
 
-The DB file will be migrated into a filename above, and old file (SQLite3
+```lua
-version) will still remain. If you still want to use SQLite3 version, set
+-- example for lazy.nvim
-`use_sqlite = true`.
+{
-
+  "nvim-telescope/telescope-frecency.nvim",
-Also you can explicitly migrate DB by calling `:FrecencyMigrateDB` command.
+  commit = "a3e818d001baad9ee2f6800d3bbc71c4275364ae",
+}
+```
 
 ## Highlight Groups
 

lua/frecency/database.lua

@@ -1,4 +1,10 @@
----@diagnostic disable: missing-return, unused-local
+local FileLock = require "frecency.file_lock"
+local wait = require "frecency.wait"
+local watcher = require "frecency.watcher"
+local log = require "plenary.log"
+local async = require "plenary.async" --[[@as PlenaryAsync]]
+local Path = require "plenary.path" --[[@as PlenaryPath]]
+
 ---@class FrecencyDatabaseConfig
 ---@field root string
 
@@ -6,43 +12,220 @@
 ---@field path string?
 ---@field workspace string?
 
----@class FrecencyDatabase
----@field config FrecencyDatabaseConfig
----@field filename string
----@field has_entry fun(): boolean
----@field new fun(fs: FrecencyFS, config: FrecencyDatabaseConfig): FrecencyDatabase
----@field protected fs FrecencyFS
-local Database = {}
-
----@param paths string[]
----@return nil
-function Database:insert_files(paths) end
-
----@return integer[]|string[]
-function Database:unlinked_entries() end
-
----@param files integer[]|string[]
----@return nil
-function Database:remove_files(files) end
-
----@param path string
----@return boolean
-function Database:remove_entry(path) end
-
----@param path string
----@param max_count integer
----@param datetime string?
----@return nil
-function Database:update(path, max_count, datetime) end
-
----@async
 ---@class FrecencyDatabaseEntry
 ---@field ages number[]
 ---@field count integer
 ---@field path string
 ---@field score number
 
+---@class FrecencyDatabase
+---@field config FrecencyDatabaseConfig
+---@field file_lock FrecencyFileLock
+---@field filename string
+---@field fs FrecencyFS
+---@field new fun(fs: FrecencyFS, config: FrecencyDatabaseConfig): FrecencyDatabase
+---@field table FrecencyDatabaseTable
+---@field version "v1"
+local Database = {}
+
+---@class FrecencyDatabaseTable
+---@field version string
+---@field records table<string,FrecencyDatabaseRecord>
+
+---@class FrecencyDatabaseRecord
+---@field count integer
+---@field timestamps integer[]
+
+---@param fs FrecencyFS
+---@param config FrecencyDatabaseConfig
+---@return FrecencyDatabase
+Database.new = function(fs, config)
+  local version = "v1"
+  local self = setmetatable({
+    config = config,
+    fs = fs,
+    table = { version = version, records = {} },
+    version = version,
+  }, { __index = Database })
+  self.filename = Path.new(self.config.root, "file_frecency.bin").filename
+  self.file_lock = FileLock.new(self.filename)
+  local tx, rx = async.control.channel.counter()
+  watcher.watch(self.filename, tx)
+  wait(function()
+    self:load()
+  end)
+  async.void(function()
+    while true do
+      rx.last()
+      log.debug "file changed. loading..."
+      self:load()
+    end
+  end)()
+  return self
+end
+
+---@return boolean
+function Database:has_entry()
+  return not vim.tbl_isempty(self.table.records)
+end
+
+---@param paths string[]
+---@return nil
+function Database:insert_files(paths)
+  if #paths == 0 then
+    return
+  end
+  for _, path in ipairs(paths) do
+    self.table.records[path] = { count = 1, timestamps = { 0 } }
+  end
+  wait(function()
+    self:save()
+  end)
+end
+
+---@return string[]
+function Database:unlinked_entries()
+  local paths = {}
+  for file in pairs(self.table.records) do
+    if not self.fs:is_valid_path(file) then
+      table.insert(paths, file)
+    end
+  end
+  return paths
+end
+
+---@param paths string[]
+function Database:remove_files(paths)
+  for _, file in ipairs(paths) do
+    self.table.records[file] = nil
+  end
+  wait(function()
+    self:save()
+  end)
+end
+
+---@param path string
+---@param max_count integer
+---@param datetime string?
+function Database:update(path, max_count, datetime)
+  local record = self.table.records[path] or { count = 0, timestamps = {} }
+  record.count = record.count + 1
+  local now = self:now(datetime)
+  table.insert(record.timestamps, now)
+  if #record.timestamps > max_count then
+    local new_table = {}
+    for i = #record.timestamps - max_count + 1, #record.timestamps do
+      table.insert(new_table, record.timestamps[i])
+    end
+    record.timestamps = new_table
+  end
+  self.table.records[path] = record
+  wait(function()
+    self:save()
+  end)
+end
+
 ---@param workspace string?
 ---@param datetime string?
 ---@return FrecencyDatabaseEntry[]
-function Database:get_entries(workspace, datetime) end
+function Database:get_entries(workspace, datetime)
+  local now = self:now(datetime)
+  local items = {}
+  for path, record in pairs(self.table.records) do
+    if self.fs:starts_with(path, workspace) then
+      table.insert(items, {
+        path = path,
+        count = record.count,
+        ages = vim.tbl_map(function(v)
+          return (now - v) / 60
+        end, record.timestamps),
+      })
+    end
+  end
+  return items
+end
+
+-- TODO: remove this func
+-- This is a func for testing
+---@private
+---@param datetime string?
+---@return integer
+function Database:now(datetime)
+  if not datetime then
+    return os.time()
+  end
+  local epoch
+  wait(function()
+    local tz_fix = datetime:gsub("+(%d%d):(%d%d)$", "+%1%2")
+    epoch = require("frecency.tests.util").time_piece(tz_fix)
+  end)
+  return epoch
+end
+
+---@async
+---@return nil
+function Database:load()
+  local start = os.clock()
+  local err, data = self.file_lock:with(function()
+    local err, stat = async.uv.fs_stat(self.filename)
+    if err then
+      return nil
+    end
+    local fd
+    err, fd = async.uv.fs_open(self.filename, "r", tonumber("644", 8))
+    assert(not err, err)
+    local data
+    err, data = async.uv.fs_read(fd, stat.size)
+    assert(not err, err)
+    assert(not async.uv.fs_close(fd))
+    watcher.update(stat)
+    return data
+  end)
+  assert(not err, err)
+  local tbl = loadstring(data or "")() --[[@as FrecencyDatabaseTable?]]
+  if tbl and tbl.version == self.version then
+    self.table = tbl
+  end
+  log.debug(("load() takes %f seconds"):format(os.clock() - start))
+end
+
+---@async
+---@return nil
+function Database:save()
+  local start = os.clock()
+  local err = self.file_lock:with(function()
+    self:raw_save(self.table)
+    local err, stat = async.uv.fs_stat(self.filename)
+    assert(not err, err)
+    watcher.update(stat)
+    return nil
+  end)
+  assert(not err, err)
+  log.debug(("save() takes %f seconds"):format(os.clock() - start))
+end
+
+---@async
+---@param tbl FrecencyDatabaseTable
+function Database:raw_save(tbl)
+  local f = assert(load("return " .. vim.inspect(tbl)))
+  local data = string.dump(f)
+  local err, fd = async.uv.fs_open(self.filename, "w", tonumber("644", 8))
+  assert(not err, err)
+  assert(not async.uv.fs_write(fd, data))
+  assert(not async.uv.fs_close(fd))
+end
+
+---@param path string
+---@return boolean
+function Database:remove_entry(path)
+  if not self.table.records[path] then
+    return false
+  end
+  self.table.records[path] = nil
+  wait(function()
+    self:save()
+  end)
+  return true
+end
+
+return Database

lua/frecency/database/native.lua

@@ -1,212 +0,0 @@
-local FileLock = require "frecency.file_lock"
-local wait = require "frecency.wait"
-local watcher = require "frecency.database.native.watcher"
-local log = require "plenary.log"
-local async = require "plenary.async" --[[@as PlenaryAsync]]
-local Path = require "plenary.path" --[[@as PlenaryPath]]
-
----@class FrecencyDatabaseNative: FrecencyDatabase
----@field version "v1"
----@field file_lock FrecencyFileLock
----@field table FrecencyDatabaseNativeTable
-local Native = {}
-
----@class FrecencyDatabaseNativeTable
----@field version string
----@field records table<string,FrecencyDatabaseNativeRecord>
-
----@class FrecencyDatabaseNativeRecord
----@field count integer
----@field timestamps integer[]
-
----@param fs FrecencyFS
----@param config FrecencyDatabaseConfig
----@return FrecencyDatabaseNative
-Native.new = function(fs, config)
-  local version = "v1"
-  local self = setmetatable({
-    config = config,
-    fs = fs,
-    table = { version = version, records = {} },
-    version = version,
-  }, { __index = Native })
-  self.filename = Path.new(self.config.root, "file_frecency.bin").filename
-  self.file_lock = FileLock.new(self.filename)
-  local tx, rx = async.control.channel.counter()
-  watcher.watch(self.filename, tx)
-  wait(function()
-    self:load()
-  end)
-  async.void(function()
-    while true do
-      rx.last()
-      log.debug "file changed. loading..."
-      self:load()
-    end
-  end)()
-  return self
-end
-
----@return boolean
-function Native:has_entry()
-  return not vim.tbl_isempty(self.table.records)
-end
-
----@param paths string[]
----@return nil
-function Native:insert_files(paths)
-  if #paths == 0 then
-    return
-  end
-  for _, path in ipairs(paths) do
-    self.table.records[path] = { count = 1, timestamps = { 0 } }
-  end
-  wait(function()
-    self:save()
-  end)
-end
-
----@return string[]
-function Native:unlinked_entries()
-  local paths = {}
-  for file in pairs(self.table.records) do
-    if not self.fs:is_valid_path(file) then
-      table.insert(paths, file)
-    end
-  end
-  return paths
-end
-
----@param paths string[]
-function Native:remove_files(paths)
-  for _, file in ipairs(paths) do
-    self.table.records[file] = nil
-  end
-  wait(function()
-    self:save()
-  end)
-end
-
----@param path string
----@param max_count integer
----@param datetime string?
-function Native:update(path, max_count, datetime)
-  local record = self.table.records[path] or { count = 0, timestamps = {} }
-  record.count = record.count + 1
-  local now = self:now(datetime)
-  table.insert(record.timestamps, now)
-  if #record.timestamps > max_count then
-    local new_table = {}
-    for i = #record.timestamps - max_count + 1, #record.timestamps do
-      table.insert(new_table, record.timestamps[i])
-    end
-    record.timestamps = new_table
-  end
-  self.table.records[path] = record
-  wait(function()
-    self:save()
-  end)
-end
-
----@param workspace string?
----@param datetime string?
----@return FrecencyDatabaseEntry[]
-function Native:get_entries(workspace, datetime)
-  local now = self:now(datetime)
-  local items = {}
-  for path, record in pairs(self.table.records) do
-    if self.fs:starts_with(path, workspace) then
-      table.insert(items, {
-        path = path,
-        count = record.count,
-        ages = vim.tbl_map(function(v)
-          return (now - v) / 60
-        end, record.timestamps),
-      })
-    end
-  end
-  return items
-end
-
--- TODO: remove this func
--- This is a func for testing
----@private
----@param datetime string?
----@return integer
-function Native:now(datetime)
-  if not datetime then
-    return os.time()
-  end
-  local epoch
-  wait(function()
-    local tz_fix = datetime:gsub("+(%d%d):(%d%d)$", "+%1%2")
-    epoch = require("frecency.tests.util").time_piece(tz_fix)
-  end)
-  return epoch
-end
-
----@async
----@return nil
-function Native:load()
-  local start = os.clock()
-  local err, data = self.file_lock:with(function()
-    local err, stat = async.uv.fs_stat(self.filename)
-    if err then
-      return nil
-    end
-    local fd
-    err, fd = async.uv.fs_open(self.filename, "r", tonumber("644", 8))
-    assert(not err, err)
-    local data
-    err, data = async.uv.fs_read(fd, stat.size)
-    assert(not err, err)
-    assert(not async.uv.fs_close(fd))
-    watcher.update(stat)
-    return data
-  end)
-  assert(not err, err)
-  local tbl = loadstring(data or "")() --[[@as FrecencyDatabaseNativeTable?]]
-  if tbl and tbl.version == self.version then
-    self.table = tbl
-  end
-  log.debug(("load() takes %f seconds"):format(os.clock() - start))
-end
-
----@async
----@return nil
-function Native:save()
-  local start = os.clock()
-  local err = self.file_lock:with(function()
-    self:raw_save(self.table)
-    local err, stat = async.uv.fs_stat(self.filename)
-    assert(not err, err)
-    watcher.update(stat)
-    return nil
-  end)
-  assert(not err, err)
-  log.debug(("save() takes %f seconds"):format(os.clock() - start))
-end
-
-function Native:raw_save(tbl)
-  local f = assert(load("return " .. vim.inspect(tbl)))
-  local data = string.dump(f)
-  local err, fd = async.uv.fs_open(self.filename, "w", tonumber("644", 8))
-  assert(not err, err)
-  assert(not async.uv.fs_write(fd, data))
-  assert(not async.uv.fs_close(fd))
-end
-
----@param path string
----@return boolean
-function Native:remove_entry(path)
-  if not self.table.records[path] then
-    return false
-  end
-  self.table.records[path] = nil
-  wait(function()
-    self:save()
-  end)
-  return true
-end
-
-return Native

lua/frecency/database/sqlite.lua

@@ -1,155 +0,0 @@
-local sqlite = require "frecency.sqlite"
-local log = require "plenary.log"
-local Path = require "plenary.path" --[[@as PlenaryPath]]
-
----@class FrecencySqliteDB: sqlite_db
----@field files sqlite_tbl
----@field timestamps sqlite_tbl
-
----@class FrecencyFile
----@field count integer
----@field id integer
----@field path string
----@field score integer calculated from count and age
-
----@class FrecencyTimestamp
----@field age integer calculated from timestamp
----@field file_id integer
----@field id integer
----@field timestamp number
-
----@class FrecencyDatabaseSqlite: FrecencyDatabase
----@field sqlite FrecencySqliteDB
-local Sqlite = {}
-
----@param fs FrecencyFS
----@param config FrecencyDatabaseConfig
----@return FrecencyDatabaseSqlite
-Sqlite.new = function(fs, config)
-  local self = setmetatable(
-    { config = config, buf_registered_flag_name = "telescope_frecency_registered", fs = fs },
-    { __index = Sqlite }
-  )
-  self.filename = Path.new(self.config.root, "file_frecency.sqlite3").filename
-  self.sqlite = setmetatable({}, {
-    __index = function(this, key)
-      if not rawget(this, "instance") then
-        local lib = sqlite.lib
-        rawset(
-          this,
-          "instance",
-          sqlite {
-            uri = self.filename,
-            files = { id = true, count = { "integer", default = 1, required = true }, path = "string" },
-            timestamps = {
-              id = true,
-              file_id = { "integer", reference = "files.id", on_delete = "cascade" },
-              timestamp = { "real", default = lib.julianday "now" },
-            },
-          }
-        )
-      end
-      return rawget(this, "instance")[key]
-    end,
-  })
-  return self
-end
-
----@return boolean
-function Sqlite:has_entry()
-  return self.sqlite.files:count() > 0
-end
-
----@param paths string[]
----@return integer
-function Sqlite:insert_files(paths)
-  if #paths == 0 then
-    return 0
-  end
-  ---@param path string
-  return self.sqlite.files:insert(vim.tbl_map(function(path)
-    return { path = path, count = 0 } -- TODO: remove when sql.nvim#97 is closed
-  end, paths))
-end
-
----@param workspace string?
----@param datetime string?
----@return FrecencyDatabaseEntry[]
-function Sqlite:get_entries(workspace, datetime)
-  local query = workspace and { contains = { path = { workspace .. "/*" } } } or {}
-  log.debug { query = query }
-  local files = self.sqlite.files:get(query) --[[@as FrecencyFile[] ]]
-  local lib = sqlite.lib
-  local age = lib.cast((lib.julianday(datetime) - lib.julianday "timestamp") * 24 * 60, "integer")
-  local timestamps = self.sqlite.timestamps:get { keys = { age = age, "id", "file_id" } } --[[@as FrecencyTimestamp[] ]]
-  ---@type table<integer,number[]>
-  local age_map = {}
-  for _, timestamp in ipairs(timestamps) do
-    if not age_map[timestamp.file_id] then
-      age_map[timestamp.file_id] = {}
-    end
-    table.insert(age_map[timestamp.file_id], timestamp.age)
-  end
-  local items = {}
-  for _, file in ipairs(files) do
-    table.insert(items, { path = file.path, count = file.count, ages = age_map[file.id] })
-  end
-  return items
-end
-
----@param datetime string? ISO8601 format string
----@return FrecencyTimestamp[]
-function Sqlite:get_timestamps(datetime)
-  local lib = sqlite.lib
-  local age = lib.cast((lib.julianday(datetime) - lib.julianday "timestamp") * 24 * 60, "integer")
-  return self.sqlite.timestamps:get { keys = { age = age, "id", "file_id" } }
-end
-
----@param path string
----@param count integer
----@param datetime string?
----@return nil
-function Sqlite:update(path, count, datetime)
-  local file = self.sqlite.files:get({ where = { path = path } })[1] --[[@as FrecencyFile?]]
-  local file_id
-  if file then
-    self.sqlite.files:update { where = { id = file.id }, set = { count = file.count + 1 } }
-    file_id = file.id
-  else
-    file_id = self.sqlite.files:insert { path = path }
-  end
-  self.sqlite.timestamps:insert {
-    file_id = file_id,
-    timestamp = datetime and sqlite.lib.julianday(datetime) or nil,
-  }
-  local timestamps = self.sqlite.timestamps:get { where = { file_id = file_id } } --[[@as FrecencyTimestamp[] ]]
-  local trim_at = timestamps[#timestamps - count + 1]
-  if trim_at then
-    self.sqlite.timestamps:remove { file_id = tostring(file_id), id = "<" .. tostring(trim_at.id) }
-  end
-end
-
----@return integer[]
-function Sqlite:unlinked_entries()
-  ---@param file FrecencyFile
-  return self.sqlite.files:map(function(file)
-    if not self.fs:is_valid_path(file.path) then
-      return file.id
-    end
-  end)
-end
-
----@param ids integer[]
----@return nil
-function Sqlite:remove_files(ids)
-  self.sqlite.files:remove { id = ids }
-end
-
----@param path string
----@return boolean
-function Sqlite:remove_entry(path)
-  local exists = not not self.sqlite.files:get({ where = { path = path } })[1]
-  return exists and self.sqlite.files:remove { path = path } or false
-end
-
-return Sqlite

lua/frecency/entry_maker.lua

@@ -40,6 +40,12 @@ end
 ---@field score number
 ---@field display fun(entry: FrecencyEntry): string, table
 
+---@class FrecencyFile
+---@field count integer
+---@field id integer
+---@field path string
+---@field score integer calculated from count and age
+
 ---@alias FrecencyEntryMakerInstance fun(file: FrecencyFile): FrecencyEntry
 
 ---@param filepath_formatter FrecencyFilepathFormatter

lua/frecency/frecency.lua

@@ -1,12 +1,9 @@
-local Sqlite = require "frecency.database.sqlite"
+local Database = require "frecency.database"
-local Native = require "frecency.database.native"
 local EntryMaker = require "frecency.entry_maker"
 local FS = require "frecency.fs"
-local Migrator = require "frecency.migrator"
 local Picker = require "frecency.picker"
 local Recency = require "frecency.recency"
 local WebDevicons = require "frecency.web_devicons"
-local sqlite_module = require "frecency.sqlite"
 local os_util = require "frecency.os_util"
 local log = require "plenary.log"
 
@@ -16,7 +13,6 @@ local log = require "plenary.log"
 ---@field private database FrecencyDatabase
 ---@field private entry_maker FrecencyEntryMaker
 ---@field private fs FrecencyFS
----@field private migrator FrecencyMigrator
 ---@field private picker FrecencyPicker
 ---@field private recency FrecencyRecency
 local Frecency = {}
@@ -34,7 +30,6 @@ local Frecency = {}
 ---@field show_filter_column boolean|string[]|nil default: true
 ---@field show_scores boolean? default: false
 ---@field show_unindexed boolean? default: true
----@field use_sqlite boolean? default: false
 ---@field workspace_scan_cmd "LUA"|string[]|nil default: nil
 ---@field workspaces table<string, string>? default: {}
 
@@ -56,23 +51,12 @@ Frecency.new = function(opts)
     show_filter_column = true,
     show_scores = false,
     show_unindexed = true,
-    use_sqlite = false,
     workspace_scan_cmd = nil,
     workspaces = {},
   }, opts or {})
   local self = setmetatable({ buf_registered = {}, config = config }, { __index = Frecency })--[[@as Frecency]]
   self.fs = FS.new { ignore_patterns = config.ignore_patterns }
 
-  local Database
-  if not self.config.use_sqlite then
-    Database = Native
-  elseif not sqlite_module.can_use then
-    self:warn "use_sqlite = true, but sqlite module can not be found. It fallbacks to native code."
-    Database = Native
-  else
-    self:warn "SQLite mode is deprecated."
-    Database = Sqlite
-  end
   self.database = Database.new(self.fs, { root = config.db_root })
   local web_devicons = WebDevicons.new(not config.disable_devicons)
   self.entry_maker = EntryMaker.new(self.fs, web_devicons, {
@@ -81,7 +65,6 @@ Frecency.new = function(opts)
   })
   local max_count = config.max_timestamps > 0 and config.max_timestamps or 10
   self.recency = Recency.new { max_count = max_count }
-  self.migrator = Migrator.new(self.fs, self.recency, self.config.db_root)
   return self
 end
 
@@ -104,10 +87,6 @@ function Frecency:setup()
     self:validate_database()
   end
 
-  vim.api.nvim_create_user_command("FrecencyMigrateDB", function()
-    self:migrate_database()
-  end, { desc = "Migrate DB telescope-frecency to native code" })
-
   vim.api.nvim_create_user_command("FrecencyDelete", function(info)
     local path_string = info.args == "" and "%:p" or info.args
     local path = vim.fn.expand(path_string) --[[@as string]]
@@ -157,17 +136,10 @@ end
 ---@private
 ---@return nil
 function Frecency:assert_db_entries()
-  if self.database:has_entry() then
+  if not self.database:has_entry() then
-    return
+    self.database:insert_files(vim.v.oldfiles)
-  elseif not self.config.use_sqlite and sqlite_module.can_use then
+    self:notify("Imported %d entries from oldfiles.", #vim.v.oldfiles)
-    local sqlite = Sqlite.new(self.fs, { root = self.config.db_root })
-    if sqlite:has_entry() then
-      self:migrate_database(false, true)
-      return
-    end
   end
-  self.database:insert_files(vim.v.oldfiles)
-  self:notify("Imported %d entries from oldfiles.", #vim.v.oldfiles)
 end
 
 ---@private
@@ -213,45 +185,6 @@ function Frecency:register(bufnr, datetime)
   self.buf_registered[bufnr] = true
 end
 
----@param to_sqlite boolean?
----@param silently boolean?
----@return nil
-function Frecency:migrate_database(to_sqlite, silently)
-  local function migrate()
-    if not sqlite_module.can_use then
-      self:error "sqlite.lua is unavailable"
-    elseif to_sqlite then
-      self.migrator:to_sqlite()
-      self:notify "Migration is finished successfully."
-    else
-      self.migrator:to_v1()
-      self:notify "Migration is finished successfully. You can remove sqlite.lua from dependencies."
-    end
-  end
-
-  if silently then
-    migrate()
-    return
-  end
-
-  local prompt = to_sqlite and "Migrate the DB into SQLite from native code?"
-    or "Migrate the DB into native code from SQLite?"
-  vim.ui.select({ "y", "n" }, {
-    prompt = prompt,
-    ---@param item "y"|"n"
-    ---@return string
-    format_item = function(item)
-      return item == "y" and "Yes, Migrate it." or "No. Do nothing."
-    end,
-  }, function(item)
-    if item == "y" then
-      migrate()
-    else
-      self:notify "Migration aborted"
-    end
-  end)
-end
-
 ---@param path string
 ---@return nil
 function Frecency:delete(path)

lua/frecency/migrator.lua

@@ -1,83 +0,0 @@
-local Sqlite = require "frecency.database.sqlite"
-local Native = require "frecency.database.native"
-local wait = require "frecency.wait"
-
----@class FrecencyMigrator
----@field fs FrecencyFS
----@field recency FrecencyRecency
----@field root string
-local Migrator = {}
-
----@param fs FrecencyFS
----@param recency FrecencyRecency
----@param root string
----@return FrecencyMigrator
-Migrator.new = function(fs, recency, root)
-  return setmetatable({ fs = fs, recency = recency, root = root }, { __index = Migrator })
-end
-
----@return nil
-function Migrator:to_v1()
-  local native = Native.new(self.fs, { root = self.root })
-  native.table = self:v1_table_from_sqlite()
-  wait(function()
-    native:save()
-  end)
-end
-
----@return nil
-function Migrator:to_sqlite()
-  local sqlite = Sqlite.new(self.fs, { root = self.root })
-  local native = Native.new(self.fs, { root = self.root })
-  for path, record in pairs(native.table.records) do
-    local file_id = sqlite.sqlite.files:insert { path = path, count = record.count }
-    sqlite.sqlite.timestamps:insert(vim.tbl_map(function(timestamp)
-      return { file_id = file_id, timestamp = ('julianday(datetime(%d, "unixepoch"))'):format(timestamp) }
-    end, record.timestamps))
-  end
-end
-
----@private
----@return FrecencyDatabaseNativeTable
-function Migrator:v1_table_from_sqlite()
-  local sqlite = Sqlite.new(self.fs, { root = self.root })
-  ---@type FrecencyDatabaseNativeTable
-  local tbl = { version = "v1", records = {} }
-  local files = sqlite.sqlite.files:get {} --[[@as FrecencyFile[] ]]
-  ---@type table<integer,string>
-  local path_map = {}
-  for _, file in ipairs(files) do
-    tbl.records[file.path] = { count = file.count, timestamps = { 0 } }
-    path_map[file.id] = file.path
-  end
-  -- local timestamps = sqlite.sqlite.timestamps:get { keys = { "id", "file_id", epoch = "unixepoch(timestamp)" } } --[[@as FrecencyTimestamp[] ]]
-  local timestamps = sqlite.sqlite.timestamps:get {
-    keys = { "id", "file_id", epoch = "cast(strftime('%s', timestamp) as integer)" },
-  } --[[@as FrecencyTimestamp[] ]]
-  table.sort(timestamps, function(a, b)
-    return a.id < b.id
-  end)
-  for _, timestamp in ipairs(timestamps) do
-    local path = path_map[timestamp.file_id]
-    if path then
-      local record = tbl.records[path]
-      if record then
-        if #record.timestamps == 1 and record.timestamps[1] == 0 then
-          record.timestamps = {}
-        end
-        ---@diagnostic disable-next-line: undefined-field
-        table.insert(record.timestamps, timestamp.epoch)
-        if #record.timestamps > self.recency.config.max_count then
-          local new_table = {}
-          for i = #record.timestamps - self.recency.config.max_count + 1, #record.timestamps do
-            table.insert(new_table, record.timestamps[i])
-          end
-          record.timestamps = new_table
-        end
-      end
-    end
-  end
-  return tbl
-end
-
-return Migrator

lua/frecency/sqlite.lua

@@ -1,17 +0,0 @@
----@class FrecencySqlite
----@field can_use boolean
----@field lib sqlite_lib
----@overload fun(opts: table): FrecencySqliteDB
-
-return setmetatable({}, {
-  __index = function(_, k)
-    if k == "lib" then
-      return require("sqlite").lib
-    elseif k == "can_use" then
-      return not not vim.F.npcall(require, "sqlite")
-    end
-  end,
-  __call = function(_, opts)
-    return require "sqlite"(opts)
-  end,
-}) --[[@as FrecencySqlite]]

lua/frecency/tests/database_spec.lua

@@ -1,41 +1,41 @@
 local FS = require "frecency.fs"
-local Native = require "frecency.database.native"
+local Database = require "frecency.database"
 local async = require "plenary.async" --[[@as PlenaryAsync]]
 local util = require "frecency.tests.util"
 async.tests.add_to_env()
 
-local function with_native(f)
+local function with_database(f)
   local fs = FS.new { ignore_patterns = {} }
   local dir, close = util.tmpdir()
   dir:joinpath("file_frecency.bin"):touch()
   return function()
-    local native = Native.new(fs, { root = dir.filename })
+    local database = Database.new(fs, { root = dir.filename })
-    f(native)
+    f(database)
     close()
   end
 end
 
-local function save_and_load(native, tbl, datetime)
+local function save_and_load(database, tbl, datetime)
-  native:raw_save(util.v1_table(tbl))
+  database:raw_save(util.v1_table(tbl))
   async.util.sleep(100)
-  local entries = native:get_entries(nil, datetime)
+  local entries = database:get_entries(nil, datetime)
   table.sort(entries, function(a, b)
     return a.path < b.path
   end)
   return entries
 end
 
-a.describe("frecency.database.native", function()
+a.describe("frecency.database", function()
   a.describe("updated by another process", function()
     a.it(
       "returns valid entries",
-      with_native(function(native)
+      with_database(function(database)
         assert.are.same(
           {
             { path = "hoge1.txt", count = 1, ages = { 60 } },
             { path = "hoge2.txt", count = 1, ages = { 60 } },
           },
-          save_and_load(native, {
+          save_and_load(database, {
             ["hoge1.txt"] = { count = 1, timestamps = { "2023-08-21T00:00:00+0000" } },
             ["hoge2.txt"] = { count = 1, timestamps = { "2023-08-21T00:00:00+0000" } },
           }, "2023-08-21T01:00:00+0000")

lua/frecency/tests/frecency_spec.lua

@@ -1,19 +1,21 @@
----@diagnostic disable: invisible
+-- HACK: This is needed because plenary.test_harness resets &rtp.
+-- https://github.com/nvim-lua/plenary.nvim/blob/663246936325062427597964d81d30eaa42ab1e4/lua/plenary/test_harness.lua#L86-L86
+vim.opt.runtimepath:append(vim.env.TELESCOPE_PATH)
+
+---@diagnostic disable: invisible, undefined-field
 local Frecency = require "frecency.frecency"
 local Picker = require "frecency.picker"
 local util = require "frecency.tests.util"
 local log = require "plenary.log"
 local Path = require "plenary.path"
 
-local use_sqlite
-
 ---@param files string[]
 ---@param callback fun(frecency: Frecency, finder: FrecencyFinder, dir: PlenaryPath): nil
 ---@return nil
 local function with_files(files, callback)
   local dir, close = util.make_tree(files)
   log.debug { db_root = dir.filename }
-  local frecency = Frecency.new { db_root = dir.filename, use_sqlite = use_sqlite }
+  local frecency = Frecency.new { db_root = dir.filename }
   frecency.picker = Picker.new(
     frecency.database,
     frecency.entry_maker,
@@ -89,345 +91,333 @@ local function with_fake_vim_ui_select(choice, callback)
 end
 
 describe("frecency", function()
-  local function test(db)
+  describe("register", function()
-    describe(db, function()
+    describe("when opening files", function()
-      describe("register", function()
+      with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
-        describe("when opening files", function()
+        local register = make_register(frecency, dir)
-          with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+        register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            local register = make_register(frecency, dir)
+        register("hoge2.txt", "2023-07-29T01:00:00+09:00")
-            register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge2.txt", "2023-07-29T01:00:00+09:00")
 
-            it("has valid records in DB", function()
+        it("has valid records in DB", function()
-              local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+          local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-              assert.are.same({
+          assert.are.same({
-                { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+            { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
-                { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
+            { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-              }, results)
+          }, results)
-            end)
+        end)
+      end)
+    end)
+
+    describe("when opening again", function()
+      with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+        local register = make_register(frecency, dir)
+        register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+        register("hoge2.txt", "2023-07-29T01:00:00+09:00")
+        register("hoge1.txt", "2023-07-29T02:00:00+09:00", true)
+
+        it("increases the score", function()
+          local results = finder:get_results(nil, "2023-07-29T03:00:00+09:00")
+          assert.are.same({
+            { count = 2, path = filepath(dir, "hoge1.txt"), score = 40 },
+            { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+          }, results)
+        end)
+      end)
+    end)
+
+    describe("when opening again but the same instance", function()
+      with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+        local register = make_register(frecency, dir)
+        register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+        register("hoge2.txt", "2023-07-29T01:00:00+09:00")
+        register("hoge1.txt", "2023-07-29T02:00:00+09:00")
+
+        it("does not increase the score", function()
+          local results = finder:get_results(nil, "2023-07-29T03:00:00+09:00")
+          assert.are.same({
+            { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+            { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
+          }, results)
+        end)
+      end)
+    end)
+
+    describe("when opening more than 10 times", function()
+      with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+        local register = make_register(frecency, dir)
+        register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+        register("hoge1.txt", "2023-07-29T00:01:00+09:00", true)
+
+        register("hoge2.txt", "2023-07-29T00:00:00+09:00")
+        register("hoge2.txt", "2023-07-29T00:01:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:02:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:03:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:04:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:05:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:06:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:07:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:08:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:09:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:10:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:11:00+09:00", true)
+
+        it("calculates score from the recent 10 times", function()
+          local results = finder:get_results(nil, "2023-07-29T00:12:00+09:00")
+          assert.are.same({
+            { count = 12, path = filepath(dir, "hoge2.txt"), score = 12 * (10 * 100) / 10 },
+            { count = 2, path = filepath(dir, "hoge1.txt"), score = 2 * (2 * 100) / 10 },
+          }, results)
+        end)
+      end)
+    end)
+  end)
+
+  describe("benchmark", function()
+    describe("after registered over >5000 files", function()
+      with_files({}, function(frecency, finder, dir)
+        with_fake_register(frecency, dir, function(register)
+          -- TODO: 6000 records is too many to use with native?
+          -- local file_count = 6000
+          local file_count = 600
+          if not os.getenv "CI" then
+            log.info "It works not on CI. Files is decreased into 10 count."
+            file_count = 10
+          end
+          local expected = {}
+          log.info(("making %d files and register them"):format(file_count))
+          for i = 1, file_count do
+            local file = ("hoge%08d.txt"):format(i)
+            table.insert(expected, { count = 1, path = filepath(dir, file), score = 10 })
+            -- HACK: disable log because it fails with too many logging
+            log.new({ level = "info" }, true)
+            register(file, "2023-07-29T00:00:00+09:00")
+            log.new({}, true)
+          end
+          local start = os.clock()
+          local results = finder:get_results(nil, "2023-07-29T00:01:00+09:00")
+          table.sort(results, function(a, b)
+            return a.path < b.path
+          end)
+          local elapsed = os.clock() - start
+          log.info(("it takes %f seconds in fetching all results"):format(elapsed))
+
+          it("returns appropriate latency (<1.0 second)", function()
+            assert.are.is_true(elapsed < 1.0)
+          end)
+
+          it("returns valid response", function()
+            assert.are.same(expected, results)
           end)
         end)
+      end)
+    end)
+  end)
 
-        describe("when opening again", function()
+  describe("validate_database", function()
-          with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+    describe("when no files are unlinked", function()
-            local register = make_register(frecency, dir)
+      with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
-            register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+        local register = make_register(frecency, dir)
-            register("hoge2.txt", "2023-07-29T01:00:00+09:00")
+        register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge1.txt", "2023-07-29T02:00:00+09:00", true)
+        register("hoge2.txt", "2023-07-29T00:01:00+09:00")
 
-            it("increases the score", function()
+        it("removes no entries", function()
-              local results = finder:get_results(nil, "2023-07-29T03:00:00+09:00")
+          local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-              assert.are.same({
+          assert.are.same({
-                { count = 2, path = filepath(dir, "hoge1.txt"), score = 40 },
+            { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
-                { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+            { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-              }, results)
+          }, results)
-            end)
-          end)
         end)
+      end)
+    end)
 
-        describe("when opening again but the same instance", function()
+    describe("when with not force", function()
-          with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+      describe("when files are unlinked but it is less than threshold", function()
-            local register = make_register(frecency, dir)
+        with_files({ "hoge1.txt", "hoge2.txt", "hoge3.txt", "hoge4.txt", "hoge5.txt" }, function(frecency, finder, dir)
-            register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+          local register = make_register(frecency, dir)
-            register("hoge2.txt", "2023-07-29T01:00:00+09:00")
+          register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge1.txt", "2023-07-29T02:00:00+09:00")
+          register("hoge2.txt", "2023-07-29T00:01:00+09:00")
+          register("hoge3.txt", "2023-07-29T00:02:00+09:00")
+          register("hoge4.txt", "2023-07-29T00:03:00+09:00")
+          register("hoge5.txt", "2023-07-29T00:04:00+09:00")
+          frecency.config.db_validate_threshold = 3
+          dir:joinpath("hoge1.txt"):rm()
+          dir:joinpath("hoge2.txt"):rm()
+          frecency:validate_database()
 
-            it("does not increase the score", function()
+          it("removes no entries", function()
-              local results = finder:get_results(nil, "2023-07-29T03:00:00+09:00")
+            local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-              assert.are.same({
+            table.sort(results, function(a, b)
-                { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+              return a.path < b.path
-                { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-              }, results)
-            end)
-          end)
-        end)
-
-        describe("when opening more than 10 times", function()
-          with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
-            local register = make_register(frecency, dir)
-            register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge1.txt", "2023-07-29T00:01:00+09:00", true)
-
-            register("hoge2.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge2.txt", "2023-07-29T00:01:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:02:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:03:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:04:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:05:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:06:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:07:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:08:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:09:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:10:00+09:00", true)
-            register("hoge2.txt", "2023-07-29T00:11:00+09:00", true)
-
-            it("calculates score from the recent 10 times", function()
-              local results = finder:get_results(nil, "2023-07-29T00:12:00+09:00")
-              assert.are.same({
-                { count = 12, path = filepath(dir, "hoge2.txt"), score = 12 * (10 * 100) / 10 },
-                { count = 2, path = filepath(dir, "hoge1.txt"), score = 2 * (2 * 100) / 10 },
-              }, results)
             end)
+            assert.are.same({
+              { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
+              { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+              { count = 1, path = filepath(dir, "hoge3.txt"), score = 10 },
+              { count = 1, path = filepath(dir, "hoge4.txt"), score = 10 },
+              { count = 1, path = filepath(dir, "hoge5.txt"), score = 10 },
+            }, results)
           end)
         end)
       end)
 
-      describe("benchmark", function()
+      describe("when files are unlinked and it is more than threshold", function()
-        describe("after registered over >5000 files", function()
+        describe('when the user response "yes"', function()
-          with_files({}, function(frecency, finder, dir)
+          with_files(
-            with_fake_register(frecency, dir, function(register)
+            { "hoge1.txt", "hoge2.txt", "hoge3.txt", "hoge4.txt", "hoge5.txt" },
-              -- TODO: 6000 records is too many to use with native?
+            function(frecency, finder, dir)
-              -- local file_count = 6000
-              local file_count = 600
-              if not os.getenv "CI" then
-                log.info "It works not on CI. Files is decreased into 10 count."
-                file_count = 10
-              end
-              local expected = {}
-              log.info(("making %d files and register them"):format(file_count))
-              for i = 1, file_count do
-                local file = ("hoge%08d.txt"):format(i)
-                table.insert(expected, { count = 1, path = filepath(dir, file), score = 10 })
-                -- HACK: disable log because it fails with too many logging
-                log.new({ level = "info" }, true)
-                register(file, "2023-07-29T00:00:00+09:00")
-                log.new({}, true)
-              end
-              local start = os.clock()
-              local results = finder:get_results(nil, "2023-07-29T00:01:00+09:00")
-              table.sort(results, function(a, b)
-                return a.path < b.path
-              end)
-              local elapsed = os.clock() - start
-              log.info(("it takes %f seconds in fetching all results"):format(elapsed))
-
-              it("returns appropriate latency (<1.0 second)", function()
-                assert.are.is_true(elapsed < 1.0)
-              end)
-
-              it("returns valid response", function()
-                assert.are.same(expected, results)
-              end)
-            end)
-          end)
-        end)
-      end)
-
-      describe("validate_database", function()
-        describe("when no files are unlinked", function()
-          with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
-            local register = make_register(frecency, dir)
-            register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge2.txt", "2023-07-29T00:01:00+09:00")
-
-            it("removes no entries", function()
-              local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-              assert.are.same({
-                { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
-                { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-              }, results)
-            end)
-          end)
-        end)
-
-        describe("when with not force", function()
-          describe("when files are unlinked but it is less than threshold", function()
-            with_files(
-              { "hoge1.txt", "hoge2.txt", "hoge3.txt", "hoge4.txt", "hoge5.txt" },
-              function(frecency, finder, dir)
-                local register = make_register(frecency, dir)
-                register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-                register("hoge2.txt", "2023-07-29T00:01:00+09:00")
-                register("hoge3.txt", "2023-07-29T00:02:00+09:00")
-                register("hoge4.txt", "2023-07-29T00:03:00+09:00")
-                register("hoge5.txt", "2023-07-29T00:04:00+09:00")
-                frecency.config.db_validate_threshold = 3
-                dir:joinpath("hoge1.txt"):rm()
-                dir:joinpath("hoge2.txt"):rm()
-                frecency:validate_database()
-
-                it("removes no entries", function()
-                  local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-                  table.sort(results, function(a, b)
-                    return a.path < b.path
-                  end)
-                  assert.are.same({
-                    { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-                    { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
-                    { count = 1, path = filepath(dir, "hoge3.txt"), score = 10 },
-                    { count = 1, path = filepath(dir, "hoge4.txt"), score = 10 },
-                    { count = 1, path = filepath(dir, "hoge5.txt"), score = 10 },
-                  }, results)
-                end)
-              end
-            )
-          end)
-
-          describe("when files are unlinked and it is more than threshold", function()
-            describe('when the user response "yes"', function()
-              with_files(
-                { "hoge1.txt", "hoge2.txt", "hoge3.txt", "hoge4.txt", "hoge5.txt" },
-                function(frecency, finder, dir)
-                  local register = make_register(frecency, dir)
-                  register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-                  register("hoge2.txt", "2023-07-29T00:01:00+09:00")
-                  register("hoge3.txt", "2023-07-29T00:02:00+09:00")
-                  register("hoge4.txt", "2023-07-29T00:03:00+09:00")
-                  register("hoge5.txt", "2023-07-29T00:04:00+09:00")
-                  frecency.config.db_validate_threshold = 3
-                  dir:joinpath("hoge1.txt"):rm()
-                  dir:joinpath("hoge2.txt"):rm()
-                  dir:joinpath("hoge3.txt"):rm()
-
-                  with_fake_vim_ui_select("y", function(called)
-                    frecency:validate_database()
-
-                    it("called vim.ui.select()", function()
-                      assert.are.same(1, called())
-                    end)
-                  end)
-
-                  it("removes entries", function()
-                    local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-                    table.sort(results, function(a, b)
-                      return a.path < b.path
-                    end)
-                    assert.are.same({
-                      { count = 1, path = filepath(dir, "hoge4.txt"), score = 10 },
-                      { count = 1, path = filepath(dir, "hoge5.txt"), score = 10 },
-                    }, results)
-                  end)
-                end
-              )
-            end)
-
-            describe('when the user response "no"', function()
-              with_files(
-                { "hoge1.txt", "hoge2.txt", "hoge3.txt", "hoge4.txt", "hoge5.txt" },
-                function(frecency, finder, dir)
-                  local register = make_register(frecency, dir)
-                  register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-                  register("hoge2.txt", "2023-07-29T00:01:00+09:00")
-                  register("hoge3.txt", "2023-07-29T00:02:00+09:00")
-                  register("hoge4.txt", "2023-07-29T00:03:00+09:00")
-                  register("hoge5.txt", "2023-07-29T00:04:00+09:00")
-                  frecency.config.db_validate_threshold = 3
-                  dir:joinpath("hoge1.txt"):rm()
-                  dir:joinpath("hoge2.txt"):rm()
-                  dir:joinpath("hoge3.txt"):rm()
-
-                  with_fake_vim_ui_select("n", function(called)
-                    frecency:validate_database()
-
-                    it("called vim.ui.select()", function()
-                      assert.are.same(1, called())
-                    end)
-                  end)
-
-                  it("removes no entries", function()
-                    local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
-                    table.sort(results, function(a, b)
-                      return a.path < b.path
-                    end)
-                    assert.are.same({
-                      { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-                      { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
-                      { count = 1, path = filepath(dir, "hoge3.txt"), score = 10 },
-                      { count = 1, path = filepath(dir, "hoge4.txt"), score = 10 },
-                      { count = 1, path = filepath(dir, "hoge5.txt"), score = 10 },
-                    }, results)
-                  end)
-                end
-              )
-            end)
-          end)
-        end)
-
-        describe("when with force", function()
-          describe("when db_safe_mode is true", function()
-            with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
               local register = make_register(frecency, dir)
               register("hoge1.txt", "2023-07-29T00:00:00+09:00")
               register("hoge2.txt", "2023-07-29T00:01:00+09:00")
+              register("hoge3.txt", "2023-07-29T00:02:00+09:00")
+              register("hoge4.txt", "2023-07-29T00:03:00+09:00")
+              register("hoge5.txt", "2023-07-29T00:04:00+09:00")
+              frecency.config.db_validate_threshold = 3
               dir:joinpath("hoge1.txt"):rm()
+              dir:joinpath("hoge2.txt"):rm()
+              dir:joinpath("hoge3.txt"):rm()
 
               with_fake_vim_ui_select("y", function(called)
-                frecency:validate_database(true)
+                frecency:validate_database()
 
                 it("called vim.ui.select()", function()
                   assert.are.same(1, called())
                 end)
               end)
 
-              it("needs confirmation for removing entries", function()
+              it("removes entries", function()
                 local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+                table.sort(results, function(a, b)
+                  return a.path < b.path
+                end)
                 assert.are.same({
-                  { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+                  { count = 1, path = filepath(dir, "hoge4.txt"), score = 10 },
+                  { count = 1, path = filepath(dir, "hoge5.txt"), score = 10 },
                 }, results)
               end)
-            end)
+            end
-          end)
+          )
+        end)
 
-          describe("when db_safe_mode is false", function()
+        describe('when the user response "no"', function()
-            with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+          with_files(
+            { "hoge1.txt", "hoge2.txt", "hoge3.txt", "hoge4.txt", "hoge5.txt" },
+            function(frecency, finder, dir)
               local register = make_register(frecency, dir)
               register("hoge1.txt", "2023-07-29T00:00:00+09:00")
               register("hoge2.txt", "2023-07-29T00:01:00+09:00")
+              register("hoge3.txt", "2023-07-29T00:02:00+09:00")
+              register("hoge4.txt", "2023-07-29T00:03:00+09:00")
+              register("hoge5.txt", "2023-07-29T00:04:00+09:00")
+              frecency.config.db_validate_threshold = 3
               dir:joinpath("hoge1.txt"):rm()
+              dir:joinpath("hoge2.txt"):rm()
+              dir:joinpath("hoge3.txt"):rm()
 
-              with_fake_vim_ui_select("y", function(called)
+              with_fake_vim_ui_select("n", function(called)
-                frecency.config.db_safe_mode = false
+                frecency:validate_database()
-                frecency:validate_database(true)
 
-                it("did not call vim.ui.select()", function()
+                it("called vim.ui.select()", function()
-                  assert.are.same(0, called())
+                  assert.are.same(1, called())
                 end)
               end)
 
-              it("needs no confirmation for removing entries", function()
+              it("removes no entries", function()
                 local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+                table.sort(results, function(a, b)
+                  return a.path < b.path
+                end)
                 assert.are.same({
+                  { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
                   { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+                  { count = 1, path = filepath(dir, "hoge3.txt"), score = 10 },
+                  { count = 1, path = filepath(dir, "hoge4.txt"), score = 10 },
+                  { count = 1, path = filepath(dir, "hoge5.txt"), score = 10 },
                 }, results)
               end)
+            end
+          )
+        end)
+      end)
+    end)
+
+    describe("when with force", function()
+      describe("when db_safe_mode is true", function()
+        with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+          local register = make_register(frecency, dir)
+          register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+          register("hoge2.txt", "2023-07-29T00:01:00+09:00")
+          dir:joinpath("hoge1.txt"):rm()
+
+          with_fake_vim_ui_select("y", function(called)
+            frecency:validate_database(true)
+
+            it("called vim.ui.select()", function()
+              assert.are.same(1, called())
             end)
           end)
+
+          it("needs confirmation for removing entries", function()
+            local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+            assert.are.same({
+              { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+            }, results)
+          end)
         end)
       end)
 
-      describe("delete", function()
+      describe("when db_safe_mode is false", function()
-        describe("when file exists", function()
+        with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
-          with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
+          local register = make_register(frecency, dir)
-            local register = make_register(frecency, dir)
+          register("hoge1.txt", "2023-07-29T00:00:00+09:00")
-            register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+          register("hoge2.txt", "2023-07-29T00:01:00+09:00")
-            register("hoge2.txt", "2023-07-29T00:01:00+09:00")
+          dir:joinpath("hoge1.txt"):rm()
 
-            it("deletes the file successfully", function()
+          with_fake_vim_ui_select("y", function(called)
-              local path = filepath(dir, "hoge2.txt")
+            frecency.config.db_safe_mode = false
-              local result
+            frecency:validate_database(true)
-              ---@diagnostic disable-next-line: duplicate-set-field
-              frecency.notify = function(self, fmt, ...)
-                vim.notify(self:message(fmt, ...))
-                result = true
-              end
-              frecency:delete(path)
-              assert.are.same(result, true)
-            end)
 
-            it("returns valid results", function()
+            it("did not call vim.ui.select()", function()
-              local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+              assert.are.same(0, called())
-              assert.are.same({
-                { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
-              }, results)
             end)
           end)
+
+          it("needs no confirmation for removing entries", function()
+            local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+            assert.are.same({
+              { count = 1, path = filepath(dir, "hoge2.txt"), score = 10 },
+            }, results)
+          end)
         end)
       end)
     end)
-  end
+  end)
 
-  use_sqlite = true
+  describe("delete", function()
-  test "sqlite"
+    describe("when file exists", function()
-  use_sqlite = false
+      with_files({ "hoge1.txt", "hoge2.txt" }, function(frecency, finder, dir)
-  test "native"
+        local register = make_register(frecency, dir)
+        register("hoge1.txt", "2023-07-29T00:00:00+09:00")
+        register("hoge2.txt", "2023-07-29T00:01:00+09:00")
+
+        it("deletes the file successfully", function()
+          local path = filepath(dir, "hoge2.txt")
+          local result
+          ---@diagnostic disable-next-line: duplicate-set-field
+          frecency.notify = function(self, fmt, ...)
+            vim.notify(self:message(fmt, ...))
+            result = true
+          end
+          frecency:delete(path)
+          assert.are.same(result, true)
+        end)
+
+        it("returns valid results", function()
+          local results = finder:get_results(nil, "2023-07-29T02:00:00+09:00")
+          assert.are.same({
+            { count = 1, path = filepath(dir, "hoge1.txt"), score = 10 },
+          }, results)
+        end)
+      end)
+    end)
+  end)
 end)

lua/frecency/tests/migrator_spec.lua

@@ -1,145 +0,0 @@
----@diagnostic disable: undefined-field
-local Migrator = require "frecency.migrator"
-local FS = require "frecency.fs"
-local Recency = require "frecency.recency"
-local Sqlite = require "frecency.database.sqlite"
-local Native = require "frecency.database.native"
-local util = require "frecency.tests.util"
-local wait = require "frecency.wait"
--- TODO: replace this with vim.system
-local Job = require "plenary.job"
-
----@param callback fun(migrator: FrecencyMigrator, sqlite: FrecencyDatabase): nil
----@return nil
-local function with(callback)
-  local dir, close = util.tmpdir()
-  local recency = Recency.new { max_count = 2 }
-  local fs = FS.new { ignore_patterns = {} }
-  local migrator = Migrator.new(fs, recency, dir.filename)
-  local sqlite = Sqlite.new(fs, { root = dir.filename })
-  callback(migrator, sqlite)
-  close()
-end
-
-local function strptime(iso8601)
-  local result = vim.fn.strptime("%FT%T%z", iso8601)
-  return result ~= 0 and result or nil
-end
-
--- NOTE: Windows has no strptime
-local function time_piece(iso8601)
-  local stdout, code =
-    Job:new({ "perl", "-MTime::Piece", "-e", "print Time::Piece->strptime('" .. iso8601 .. "', '%FT%T%z')->epoch" })
-      :sync(30000)
-  return code == 0 and tonumber(stdout[1]) or nil
-end
-
----@param source table<string,{ count: integer, timestamps: string[] }>
-local function v1_table(source)
-  local records = {}
-  for path, record in pairs(source) do
-    local timestamps = {}
-    for _, timestamp in ipairs(record.timestamps) do
-      local iso8601 = timestamp .. "+0000"
-      table.insert(timestamps, strptime(iso8601) or time_piece(iso8601))
-    end
-    records[path] = { count = record.count, timestamps = timestamps }
-  end
-  return { version = "v1", records = records }
-end
-
-describe("migrator", function()
-  describe("to_v1", function()
-    describe("when with simple database", function()
-      with(function(migrator, sqlite)
-        for _, path in ipairs { "hoge1.txt", "hoge2.txt" } do
-          sqlite:update(path, migrator.recency.config.max_count, "2023-08-21T00:00:00")
-        end
-        migrator:to_v1()
-        local native = Native.new(migrator.fs, { root = migrator.root })
-
-        it("has converted into a valid table", function()
-          assert.are.same(
-            native.table,
-            v1_table {
-              ["hoge1.txt"] = { count = 1, timestamps = { "2023-08-21T00:00:00" } },
-              ["hoge2.txt"] = { count = 1, timestamps = { "2023-08-21T00:00:00" } },
-            }
-          )
-        end)
-      end)
-    end)
-
-    describe("when with more large database", function()
-      with(function(migrator, sqlite)
-        for i, path in ipairs {
-          "hoge1.txt",
-          "hoge1.txt",
-          "hoge1.txt",
-          "hoge1.txt",
-          "hoge2.txt",
-          "hoge2.txt",
-          "hoge2.txt",
-          "hoge3.txt",
-          "hoge3.txt",
-          "hoge4.txt",
-        } do
-          sqlite:update(path, migrator.recency.config.max_count, ("2023-08-21T00:%02d:00"):format(i))
-        end
-        migrator:to_v1()
-        local native = Native.new(migrator.fs, { root = migrator.root })
-
-        it("has converted into a valid table", function()
-          assert.are.same(
-            native.table,
-            v1_table {
-              ["hoge1.txt"] = { count = 4, timestamps = { "2023-08-21T00:03:00", "2023-08-21T00:04:00" } },
-              ["hoge2.txt"] = { count = 3, timestamps = { "2023-08-21T00:06:00", "2023-08-21T00:07:00" } },
-              ["hoge3.txt"] = { count = 2, timestamps = { "2023-08-21T00:08:00", "2023-08-21T00:09:00" } },
-              ["hoge4.txt"] = { count = 1, timestamps = { "2023-08-21T00:10:00" } },
-            }
-          )
-        end)
-      end)
-    end)
-  end)
-
-  describe("to_sqlite", function()
-    with(function(migrator, sqlite)
-      local native = Native.new(migrator.fs, { root = migrator.root })
-      native.table = v1_table {
-        ["hoge1.txt"] = { count = 4, timestamps = { "2023-08-21T00:03:00", "2023-08-21T00:04:00" } },
-        ["hoge2.txt"] = { count = 3, timestamps = { "2023-08-21T00:06:00", "2023-08-21T00:07:00" } },
-        ["hoge3.txt"] = { count = 2, timestamps = { "2023-08-21T00:08:00", "2023-08-21T00:09:00" } },
-        ["hoge4.txt"] = { count = 1, timestamps = { "2023-08-21T00:10:00" } },
-      }
-      wait(function()
-        native:save()
-      end)
-      migrator:to_sqlite()
-      sqlite.sqlite.db:open()
-      local records = sqlite.sqlite.db:eval [[
-        select
-          f.path,
-          f.count,
-          datetime(strftime('%s', t.timestamp), 'unixepoch') datetime
-        from timestamps t
-        join files f
-          on f.id = t.file_id
-        order by path, datetime
-      ]]
-
-      it("has converted into a valid DB", function()
-        assert.are.same(records, {
-          { path = "hoge1.txt", count = 4, datetime = "2023-08-21 00:03:00" },
-          { path = "hoge1.txt", count = 4, datetime = "2023-08-21 00:04:00" },
-          { path = "hoge2.txt", count = 3, datetime = "2023-08-21 00:06:00" },
-          { path = "hoge2.txt", count = 3, datetime = "2023-08-21 00:07:00" },
-          { path = "hoge3.txt", count = 2, datetime = "2023-08-21 00:08:00" },
-          { path = "hoge3.txt", count = 2, datetime = "2023-08-21 00:09:00" },
-          { path = "hoge4.txt", count = 1, datetime = "2023-08-21 00:10:00" },
-        })
-      end)
-    end)
-  end)
-end)

lua/frecency/tests/minimal.lua

@@ -4,10 +4,6 @@ end
 if not vim.env.TELESCOPE_PATH then
   error "set $TELESCOPE_PATH to find telescope.nvim"
 end
-if not vim.env.SQLITE_PATH then
-  error "set $SQLITE_PATH to find telescope.nvim"
-end
 vim.opt.runtimepath:append(vim.env.PLENARY_PATH)
 vim.opt.runtimepath:append(vim.env.TELESCOPE_PATH)
-vim.opt.runtimepath:append(vim.env.SQLITE_PATH)
 vim.cmd.runtime "plugin/plenary.vim"

lua/frecency/types.lua

@@ -1,48 +1,5 @@
 ---@diagnostic disable: unused-local, missing-return
 
--- NOTE: types below are borrowed from sqlite.lua
-
----@class sqlite_db @Main sqlite.lua object.
----@field uri string: database uri. it can be an environment variable or an absolute path. default ":memory:"
----@field opts sqlite_opts: see https://www.sqlite.org/pragma.html |sqlite_opts|
----@field conn sqlite_blob: sqlite connection c object.
----@field db sqlite_db: reference to fallback to when overwriting |sqlite_db| methods (extended only).
-
----@class sqlite_query_update @Query fileds used when calling |sqlite:update| or |sqlite_tbl:update|
----@field where table: filter down values using key values.
----@field set table: key and value to updated.
-
----@class sqlite_query_select @Query fileds used when calling |sqlite:select| or |sqlite_tbl:get|
----@field where table? filter down values using key values.
----@field keys table? keys to include. (default all)
----@field join table? (TODO: support)
----@field order_by table? { asc = "key", dsc = {"key", "another_key"} }
----@field limit number? the number of result to limit by
----@field contains table? for sqlite glob ex. { title = "fix*" }
-
----@alias sqlite_query_delete table<string, any>
-
----@generic T
----@alias sqlite_map_func fun(self: sqlite_tbl, mapper: fun(entry: table): T?): T[]
-
----@class sqlite_tbl @Main sql table class
----@field db sqlite_db: sqlite.lua database object.
----@field name string: table name.
----@field mtime number: db last modified time.
----@field count fun(self: sqlite_tbl): integer
----@field insert fun(self: sqlite_tbl, rows: table<string, any>|table<string, any>[]): integer
----@field update fun(self: sqlite_tbl, specs: sqlite_query_update): boolean
----@field get fun(self: sqlite_tbl, query: sqlite_query_select): table[]
----@field remove fun(self: sqlite_tbl, where: sqlite_query_delete): boolean
----@field map sqlite_map_func
-
----@class sqlite_opts @Sqlite3 Options (TODO: add sqlite option fields and description)
----@class sqlite_blob @sqlite3 blob object
-
----@class sqlite_lib
----@field cast fun(source: integer, as: string): string
----@field julianday fun(timestring: string?): integer
-
 -- NOTE: types are borrowed from plenary.nvim
 
 ---@class PlenaryPath

lua/telescope/_extensions/frecency.lua

@@ -1,14 +1,8 @@
 local frecency = require "frecency"
-local sqlite = require "frecency.sqlite"
 
 return require("telescope").register_extension {
   setup = frecency.setup,
   health = function()
-    if sqlite.can_use then
-      vim.health.ok "sqlite.lua installed."
-    else
-      vim.health.info "sqlite.lua is required when use_sqlite = true"
-    end
     if vim.F.npcall(require, "nvim-web-devicons") then
       vim.health.ok "nvim-web-devicons installed."
     else