kristoferssolo/telescope-frecency.nvim
1
0
Fork 0
mirror of https://github.com/kristoferssolo/telescope-frecency.nvim.git synced 2025-10-21 20:10:38 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

feat!: make use_sqlite = false be the default (#148)

Browse Source 
* refactor: load lazily sqlite to get filename

* feat: add logic to start migration automatically

* feat: make use_sqlite = false be the default

* docs: update README for removing dependencies

* feat: migrate silently when use_sqlite = false

* chore: remove `SQLite3` from messages

* test: run again max to 5 times in Windows
This commit is contained in:
JINNOUCHI Yasushi 2023-10-10 18:17:08 +09:00 committed by GitHub
parent 4bdd9bafc7
commit daf59744f6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 175 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.github/workflows/ci.yml vendored
View File

@ -70,7 +70,12 @@ jobs:
TEST_DIR=lua/frecency/tests/
MINIMAL_LUA=${TEST_DIR}minimal.lua
NVIM=$(perl -e '$_ = $ENV{EXE}; s,\\,/,g; print')
$NVIM --headless --clean -u $MINIMAL_LUA -c "PlenaryBustedDirectory $TEST_DIR {minimal_init = '$MINIMAL_LUA', timeout = 180000, sequential = true}"
attempts=0
while ! $NVIM --headless --clean -u $MINIMAL_LUA -c "PlenaryBustedDirectory $TEST_DIR {minimal_init = '$MINIMAL_LUA', timeout = 180000, sequential = true}"; do
if (( attempts++ > 5 )); then
exit 1
fi
done
if: matrix.os == 'windows-latest'
- name: Type Check Code Base
uses: mrcjkb/lua-typecheck-action@v0.2.0

152
README.md
View File

@ -1,10 +1,19 @@
# telescope-frecency.nvim
A [telescope.nvim](https://github.com/nvim-telescope/telescope.nvim) extension that offers intelligent prioritization when selecting files from your editing history.
A [telescope.nvim][] extension that offers intelligent prioritization when
selecting files from your editing history.
Using an implementation of Mozilla's [Frecency algorithm](https://web.archive.org/web/20210421120120/https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/Frecency_algorithm) (used in [Firefox's address bar](https://support.mozilla.org/en-US/kb/address-bar-autocomplete-firefox)), files edited _frecently_ are given higher precedence in the list index.
[telescope.nvim]: https://github.com/nvim-telescope/telescope.nvim
As the extension learns your editing habits over time, the sorting of the list is dynamically altered to prioritize the files you're likely to need.
Using an implementation of Mozilla's [Frecency algorithm][] (used in [Firefox's
address bar][]), files edited _frecently_ are given higher precedence in the
list index.
[Frecency algorithm]: https://web.archive.org/web/20210421120120/https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/Frecency_algorithm
[Firefox's address bar]: https://support.mozilla.org/en-US/kb/address-bar-autocomplete-firefox
As the extension learns your editing habits over time, the sorting of the list
is dynamically altered to prioritize the files you're likely to need.
<img src="https://raw.githubusercontent.com/sunjon/images/master/gh_readme_telescope_frecency.png" alt="screenshot" width="800"/>
@ -12,11 +21,14 @@ As the extension learns your editing habits over time, the sorting of the list i
## Frecency: Sorting by 'frequency' _and_ 'recency'
'Frecency' is a score given to each unique file indexed in a file history database.
'Frecency' is a score given to each unique file indexed in a file history
database.
A timestamp is recorded once per session when a file is first loaded into a buffer.
A timestamp is recorded once per session when a file is first loaded into a
buffer.
The score is calculated using the age of the 10 (customizable) most recent timestamps and the total amount of times that the file has been loaded:
The score is calculated using the age of the 10 (customizable) most recent
timestamps and the total amount of times that the file has been loaded:
### Recency values (per timestamp)
@ -36,36 +48,46 @@ score = frequency * recency_score / max_number_of_timestamps
```
## What about files that are neither 'frequent' _or_ 'recent' ?
Frecency naturally works best for indexed files that have been given a reasonably high score.
Frecency naturally works best for indexed files that have been given a
reasonably high score.
New projects or rarely used files with generic names either don't get listed at all or can be buried under results with a higher score.
New projects or rarely used files with generic names either don't get listed at
all or can be buried under results with a higher score.
Frecency tackles this with *Workspace Filters*:
<img src="https://raw.githubusercontent.com/sunjon/images/master/frecency_workspace_folders.gif" alt="screenshot" width="800"/>
The workspace filter feature enables you to select from user defined _filter tags_ that map to a directory or collection of directories.
Filters are applied by entering `:workspace_tag:` anywhere in the query.
Filter name completion is available by pressing `<Tab>` after the first `:` character.
The workspace filter feature enables you to select from user defined _filter
tags_ that map to a directory or collection of directories. Filters are applied
by entering `:workspace_tag:` anywhere in the query. Filter name completion is
available by pressing `<Tab>` after the first `:` character.
When a filter is applied, results are reduced to entries whose path is a descendant of the workspace directory.
The indexed results are optionally augmented with a listing of _all_ files found in a recurssive search of the workspace directory.
Non-indexed files are given a score of zero and appear below the 'frecent' entries.
When a non-indexed file is opened, it gains a score value and is available in future 'frecent' search results.
When a filter is applied, results are reduced to entries whose path is a
descendant of the workspace directory. The indexed results are optionally
augmented with a listing of _all_ files found in a recurssive search of the
workspace directory. Non-indexed files are given a score of zero and appear
below the 'frecent' entries. When a non-indexed file is opened, it gains a
score value and is available in future 'frecent' search results.
If the active buffer (prior to the finder being launched) is attached to an LSP server, an automatic `LSP` tag is available, which maps to the workspace directories provided by the language server.
If the active buffer (prior to the finder being launched) is attached to an LSP
server, an automatic `LSP` tag is available, which maps to the workspace
directories provided by the language server.
## Requirements
- [telescope.nvim](https://github.com/nvim-telescope/telescope.nvim) (required)
- [sqlite.lua][] (required)
- [nvim-web-devicons](https://github.com/kyazdani42/nvim-web-devicons) (optional)
[sqlite.lua]: https://github.com/kkharji/sqlite.lua
**NOTE:** The former version of this plugin has used [SQLite3][] database to
store timestamps and file records. But the current build uses Lua native code
to store them, so you can now remove [sqlite.lua][] from dependencies. See
[*Remove dependency for sqlite.lua*][remove-sqlite] for the detail.
Timestamps and file records are stored in an [SQLite3](https://www.sqlite.org/index.html) database for persistence and speed.
This plugin uses `sqlite.lua` to perform the database transactions.
[SQLite3]: https://www.sqlite.org/index.html
[sqlite.lua]: https://github.com/kkharji/sqlite.lua
[remove-sqlite]: #user-content-remove-dependency-for-sqlitelua
## Installation
@ -77,7 +99,6 @@ use {
config = function()
require("telescope").load_extension "frecency"
end,
requires = { "kkharji/sqlite.lua" },
}
```
@ -89,11 +110,12 @@ use {
config = function()
require("telescope").load_extension "frecency"
end,
dependencies = { "kkharji/sqlite.lua" },
}
```
If no database is found when running Neovim with the plugin installed, a new one is created and entries from `shada` `v:oldfiles` are automatically imported.
If no database is found when running Neovim with the plugin installed, a new
one is created and entries from `shada` `v:oldfiles` are automatically
imported.
## Usage
@ -104,7 +126,7 @@ If no database is found when running Neovim with the plugin installed, a new one
or to map to a key:
```lua
vim.api.nvim_set_keymap("n", "<leader><leader>", "<Cmd>Telescope frecency<CR>")
vim.keymap.set("n", "<leader><leader>", "<Cmd>Telescope frecency<CR>")
```
Use a specific workspace tag:
@ -116,11 +138,12 @@ Use a specific workspace tag:
or
```lua
vim.api.nvim_set_keymap("n", "<leader><leader>", "<Cmd>Telescope frecency workspace=CWD<CR>")
vim.keymap.set("n", "<leader><leader>", "<Cmd>Telescope frecency workspace=CWD<CR>")
```
Filter tags are applied by typing the `:tag:` name (adding surrounding colons) in the finder query.
Entering `:<Tab>` will trigger omnicompletion for available tags.
Filter tags are applied by typing the `:tag:` name (adding surrounding colons)
in the finder query. Entering `:<Tab>` will trigger omnicompletion for
available tags.
## Configuration
@ -128,12 +151,14 @@ See [default configuration](https://github.com/nvim-telescope/telescope.nvim#tel
- `db_root` (default: `vim.fn.stdpath "data"`)
Path to parent directory of custom database location.
Defaults to `$XDG_DATA_HOME/nvim` if unset.
Path to parent directory of custom database location. Defaults to
`$XDG_DATA_HOME/nvim` if unset.
- `default_workspace` (default: `nil`)
Default workspace tag to filter by e.g. `'CWD'` to filter by default to the current directory. Can be overridden at query time by specifying another filter like `':*:'`.
Default workspace tag to filter by e.g. `'CWD'` to filter by default to the
current directory. Can be overridden at query time by specifying another
filter like `':*:'`.
- `disable_devicons` (default: `false`)
@ -141,17 +166,23 @@ See [default configuration](https://github.com/nvim-telescope/telescope.nvim#tel
- `ignore_patterns` (default: `{ "*.git/*", "*/tmp/*", "term://*" }`)
Patterns in this table control which files are indexed (and subsequently which you'll see in the finder results).
Patterns in this table control which files are indexed (and subsequently
which you'll see in the finder results).
- `max_timestamps` (default: `10`)
Set the max count of timestamps DB keeps when you open files. It ignores the value and use `10` if you set less than or equal to `0`.
Set the max count of timestamps DB keeps when you open files. It ignores the
value and use `10` if you set less than or equal to `0`.
**CAUTION** When you reduce the value of this option, it removes old timestamps when you open the file. It is reasonable to set this value more than or equal to the default value: `10`.
**CAUTION** When you reduce the value of this option, it removes old
timestamps when you open the file. It is reasonable to set this value more
than or equal to the default value: `10`.
- `show_filter_column` (default: `true`)
Show the path of the active filter before file paths. In default, it uses the tail of paths for `'LSP'` and `'CWD'` tags. You can configure this by setting a table for this option.
Show the path of the active filter before file paths. In default, it uses the
tail of paths for `'LSP'` and `'CWD'` tags. You can configure this by setting
a table for this option.
```lua
-- show the tail for "LSP", "CWD" and "FOO"
@ -160,21 +191,24 @@ See [default configuration](https://github.com/nvim-telescope/telescope.nvim#tel
- `show_scores` (default : `false`)
To see the scores generated by the algorithm in the results, set this to `true`.
To see the scores generated by the algorithm in the results, set this to
`true`.
- `show_unindexed` (default: `true`)
Determines if non-indexed files are included in workspace filter results.
- `use_sqlite` (default: `true`) ***experimental feature***
- `use_sqlite` (default: `false`)
Use [sqlite.lua] `true` or native code `false`. See [*Remove dependency for sqlite.lua*](#user-content-remove-dependency-for-sqlitelua) for the detail.
Use [sqlite.lua][] with `true` or native code with `false`. See [*Remove
dependency for sqlite.lua*][remove-sqlite] for the detail.
- `workspaces` (default: `{}`)
This table contains mappings of `workspace_tag` -> `workspace_directory`
The key corresponds to the `:tag_name` used to select the filter in queries.
The value corresponds to the top level directory by which results will be filtered.
This table contains mappings of `workspace_tag` -> `workspace_directory`. The
key corresponds to the `:tag_name` used to select the filter in queries. The
value corresponds to the top level directory by which results will be
filtered.
### Example Configuration:
@ -182,10 +216,10 @@ See [default configuration](https://github.com/nvim-telescope/telescope.nvim#tel
telescope.setup {
extensions = {
frecency = {
db_root = "home/my_username/path/to/db_root",
db_root = "/home/my_username/path/to/db_root",
show_scores = false,
show_unindexed = true,
ignore_patterns = {"*.git/*", "*/tmp/*"},
ignore_patterns = { "*.git/*", "*/tmp/*" },
disable_devicons = false,
workspaces = {
["conf"] = "/home/my_username/.config",
@ -202,19 +236,25 @@ telescope.setup {
### Location
The default location for the database is `$XDG_DATA_HOME/nvim` (eg `~/.local/share/nvim/` on linux).
This can be configured with the `db_root` config option.
The default location for the database is `$XDG_DATA_HOME/nvim` (eg
`~/.local/share/nvim/` on linux). This can be configured with the `db_root`
config option.
### Maintainance
By default, frecency will prune files that no longer exist from the database.
In certain workflows, switching branches in a repository, that behaviour might not be desired.
The following configuration control this behaviour:
In certain workflows, switching branches in a repository, that behaviour might
not be desired. The following configuration control this behaviour:
`db_safe_mode` - When this is enabled, the user will be prompted before any entries are removed from the database.
`auto_validate` - When this to false, stale entries will never be automatically removed.
<dl>
<dt><code>db_safe_mode</code></dt>
<dd>When this is enabled, the user will be prompted before any entries are removed from the database.</dd>
<dt><code>auto_validate</code></dt>
<dd>When this to false, stale entries will never be automatically removed.</dd>
</dl>
The command `FrecencyValidate` can be used to clean the database when `auto_validate` is disabled.
The command `FrecencyValidate` can be used to clean the database when
`auto_validate` is disabled.
```vim
" clean DB
@ -225,13 +265,19 @@ The command `FrecencyValidate` can be used to clean the database when `auto_vali
### Remove dependency for [sqlite.lua][]
***This is an experimental feature.***
The former version of this plugin has used SQLite3 library to store data. When
you upgrade from such version, Neovim will silently migrate DB and inform that
you can remove `sqlite.lua` from dependencies.
In default, it uses SQLite3 library to access the DB. When `use_sqlite` option is set to `false`, it stores the whole data and saves them with encoding by `string.dump()` Lua function.
| made by default | made by `sqlite.lua` |
|--|--|
| `~/.local/share/nvim/file_frecency.bin` | `~/.local/share/nvim/file_frecency.sqlite3` |
With this, we can remove the dependency for [sqlite.lua][] and obtain faster speed to open `:Telescope frecency`.
The DB file will be migrated into a filename above, and old file (SQLite3
version) will still remain. If you still want to use SQLite3 version, set
`use_sqlite = true`.
You can migrate from SQLite DB into native code by `:FrecencyMigrateDB` command. It converts data into native code, but does not delete the existent SQLite DB. You can use old SQLite logic by `use_sqlite = true` again.
Also you can explicitly migrate DB by calling `:FrecencyMigrateDB` command.
## Highlight Groups

1
lua/frecency/database.lua
View File

@ -8,6 +8,7 @@
---@class FrecencyDatabase
---@field config FrecencyDatabaseConfig
---@field filename string
---@field has_entry fun(): boolean
---@field new fun(fs: FrecencyFS, config: FrecencyDatabaseConfig): FrecencyDatabase
---@field protected fs FrecencyFS

1
lua/frecency/database/native.lua
View File

@ -7,7 +7,6 @@ local Path = require "plenary.path" --[[@as PlenaryPath]]
---@class FrecencyDatabaseNative: FrecencyDatabase
---@field version "v1"
---@field filename string
---@field file_lock FrecencyFileLock
---@field table FrecencyDatabaseNativeTable
local Native = {}

19
lua/frecency/database/sqlite.lua
View File

@ -1,5 +1,6 @@
local sqlite = require "frecency.sqlite"
local log = require "plenary.log"
local Path = require "plenary.path" --[[@as PlenaryPath]]
---@class FrecencySqliteDB: sqlite_db
---@field files sqlite_tbl
@ -25,13 +26,20 @@ local Sqlite = {}
---@param config FrecencyDatabaseConfig
---@return FrecencyDatabaseSqlite
Sqlite.new = function(fs, config)
local lib = sqlite.lib
local self = setmetatable(
{ config = config, buf_registered_flag_name = "telescope_frecency_registered", fs = fs },
{ __index = Sqlite }
)
self.sqlite = sqlite {
uri = self.config.root .. "/file_frecency.sqlite3",
self.filename = Path.new(self.config.root, "file_frecency.sqlite3").filename
self.sqlite = setmetatable({}, {
__index = function(this, key)
if not rawget(this, "instance") then
local lib = sqlite.lib
rawset(
this,
"instance",
sqlite {
uri = self.filename,
files = { id = true, count = { "integer", default = 1, required = true }, path = "string" },
timestamps = {
id = true,
@ -39,6 +47,11 @@ Sqlite.new = function(fs, config)
timestamp = { "real", default = lib.julianday "now" },
},
}
)
end
return rawget(this, "instance")[key]
end,
})
return self
end

67
lua/frecency/frecency.lua
View File

@ -33,7 +33,7 @@ local Frecency = {}
---@field show_filter_column boolean|string[]|nil default: true
---@field show_scores boolean? default: false
---@field show_unindexed boolean? default: true
---@field use_sqlite boolean? default: true
---@field use_sqlite boolean? default: false
---@field workspaces table<string, string>? default: {}
---@param opts FrecencyConfig?
@ -53,7 +53,7 @@ Frecency.new = function(opts)
show_filter_column = true,
show_scores = false,
show_unindexed = true,
use_sqlite = true,
use_sqlite = false,
workspaces = {},
}, opts or {})
local self = setmetatable({ buf_registered = {}, config = config }, { __index = Frecency })--[[@as Frecency]]
@ -66,6 +66,7 @@ Frecency.new = function(opts)
self:warn "use_sqlite = true, but sqlite module can not be found. It fallbacks to native code."
Database = Native
else
self:warn "SQLite mode is deprecated."
Database = Sqlite
end
self.database = Database.new(self.fs, { root = config.db_root })
@ -88,10 +89,7 @@ function Frecency:setup()
vim.api.nvim_set_hl(0, "TelescopeQueryFilter", { link = "WildMenu", default = true })
-- TODO: Should we schedule this after loading shada?
if not self.database:has_entry() then
self.database:insert_files(vim.v.oldfiles)
self:notify("Imported %d entries from oldfiles.", #vim.v.oldfiles)
end
self:assert_db_entries()
---@param cmd_info { bang: boolean }
vim.api.nvim_create_user_command("FrecencyValidate", function(cmd_info)
@ -142,6 +140,22 @@ function Frecency:complete(findstart, base)
return self.picker:complete(findstart, base)
end
---@private
---@return nil
function Frecency:assert_db_entries()
if self.database:has_entry() then
return
elseif not self.config.use_sqlite and sqlite_module.can_use then
local sqlite = Sqlite.new(self.fs, { root = self.config.db_root })
if sqlite:has_entry() then
self:migrate_database(false, true)
return
end
end
self.database:insert_files(vim.v.oldfiles)
self:notify("Imported %d entries from oldfiles.", #vim.v.oldfiles)
end
---@private
---@param force boolean?
---@return nil
@ -159,7 +173,7 @@ function Frecency:validate_database(force)
return
end
vim.ui.select({ "y", "n" }, {
prompt = self:message("remove %d entries from SQLite3 database?", #unlinked),
prompt = self:message("remove %d entries from database?", #unlinked),
---@param item "y"|"n"
---@return string
format_item = function(item)
@ -186,10 +200,28 @@ function Frecency:register(bufnr, datetime)
end
---@param to_sqlite boolean?
---@param silently boolean?
---@return nil
function Frecency:migrate_database(to_sqlite)
local prompt = to_sqlite and "migrate the DB into SQLite from native code?"
or "migrate the DB into native code from SQLite?"
function Frecency:migrate_database(to_sqlite, silently)
local function migrate()
if not sqlite_module.can_use then
self:error "sqlite.lua is unavailable"
elseif to_sqlite then
self.migrator:to_sqlite()
self:notify "Migration is finished successfully."
else
self.migrator:to_v1()
self:notify "Migration is finished successfully. You can remove sqlite.lua from dependencies."
end
end
if silently then
migrate()
return
end
local prompt = to_sqlite and "Migrate the DB into SQLite from native code?"
or "Migrate the DB into native code from SQLite?"
vim.ui.select({ "y", "n" }, {
prompt = prompt,
---@param item "y"|"n"
@ -198,20 +230,11 @@ function Frecency:migrate_database(to_sqlite)
return item == "y" and "Yes, Migrate it." or "No. Do nothing."
end,
}, function(item)
if item == "n" then
self:notify "migration aborted"
return
elseif to_sqlite then
if sqlite_module.can_use then
self.migrator:to_sqlite()
if item == "y" then
migrate()
else
self:error "sqlite.lua is unavailable"
return
self:notify "Migration aborted"
end
else
self.migrator:to_v1()
end
self:notify "migration finished successfully"
end)
end