Compare commits

...

11 次程式碼提交

作者 SHA1 備註 提交日期
46c52afe59 Release v0.6 2022-08-14 23:15:19 +02:00
431abfe7c0 USAGE.md: Update 2022-08-14 20:28:17 +02:00
47c19d121a gui: mainwindow: Add CTRL(+Shift+)Tab shortcut to switch between tabs 2022-08-14 20:25:44 +02:00
166c051cfb gui: mainwindow: Add CTRL+F and CTRL+W shortcuts
Add shortcuts to make entering queries more efficient

CTRL+F: Highlights rightmost filter, e. g. c:(word1 word2) would
highlight everything between (), so "word1 word" here. Alternatively,
highlights lone words, so p:(docs) word1, would highlight word1.

CTRL+W: Removes last filter or words.
2022-08-14 20:25:44 +02:00
47d0440ffb gui: mainwindow: Add checkbox to remove current database 2022-08-14 20:25:44 +02:00
d900d58f26 shared: migrations: Add 3.sql: Drop potentailly harmful trigger
In [1] it's stated that "If the values "inserted" into the text
columns as part of a 'delete' command are not the same as those
currently stored within the table, the results may be unpredictable."

It's to be assumed only inserting ftsid is unpredictable. We
have no way for a proper delete because files are not immutable
or may have been deleted.

For now the index will contain entries for files that don't exist.
They won't appear in search results as they won't be joined
in the query.

[1] https://www.sqlite.org/fts5.html#the_delete_command
2022-08-14 20:24:21 +02:00
eb58b8f770 gui: Clear previews always on new search results
If a first search generates previews and the next one does
not, old entries would still have been visible.

So we just clear them once we get results.
2022-08-06 10:10:45 +02:00
9a70a821bd gui: PreviewGeneratorPlainText: Show line numbers
Generate previews that show the line number and surrounding
lines (like grep -C) for context.
2022-08-06 10:01:24 +02:00
89bf65d9bb gui: PreviewGeneratorPlaintext: Add MAX_SNIPPETS const, remove redundant loop 2022-08-06 09:35:00 +02:00
ad06497b4b shared: Add LimitQueue which discards oldest entry once limit hit 2022-08-06 09:06:00 +02:00
00abc6bc1b gui, shared: Fix and simplify word extraction regexes
They did not work for chars like '-', causing errors.

We can actually just extract non-space chars for these cases.
2022-08-06 08:57:39 +02:00
共有 13 個文件被更改,包括 293 次插入27 次删除

查看文件

@ -1,8 +1,23 @@
# looqs: Release notes
## 2022-08-14 - v0.6
This release features multiple fixes and enhancements.
Bad news first: It drops a trivial trigger that appeared to work quite fine, but silently may cause "unpredictability" of the sqlite FTS5 index ( [9422a5b494](https://github.com/quitesimpleorg/looqs/commit/9422a5b494dabd0f1324dc2f92a34c3036137414) ). As a result, FTS queries may return weird and unexplainable results. This is not reasonably automatically recoverable by looqs. I strongly recommend creating a clean, new database. All previous versions are affected. To do that, go to "Settings" -> checking "Remove old database on save" -> "Save settings and restart". Alternatively, specify a new path to keep the old database.
CHANGES:
- GUI: Add line numbers and context lines to plaintext previews
- GUI: Fix case where previews for old queries would have still been visible if new query would not create previews
- GUI: Add CTRL + F, CTRL+W, CTRL+Tab, CTRL+Shift+Tab shortcuts (see user manual)
- GUI: Add checkbox in "Settings" tab allowing to delete database.
- General: Fix wrong regexes that caused query errors with chars like -
- General: Drop trigger sending incomplete sqlite fts5 deletion command, causing undefined index behaviour
## 2022-07-30 - v0.5.1
CHANGES:
- gui: Fix regression in implicit paths queries introduced in previous version
- gui: Fix regression in implicit paths queries introduced in previous version
## 2022-07-29 - v0.5
This release features multiple fixes and enhancements.

查看文件

@ -1,10 +1,9 @@
# looqs - Full-text search with previews for your files
looqs is a tool that creates a full-text search index for your files. It allows you to look at previews where your
search terms have been found, as shown in the screenshots below.
looqs is a tool that creates a full-text search index for your files. It allows you to look at previews where your search terms have been found, as shown in the screenshots below.
## Screenshots
### Preview
looqs allow you to look inside files. It marks what you have searched for.
looqs allows you to look inside files. It highlights what you have searched for.
![Screenshot looqs](https://garage.quitesimple.org/assets/looqs/orwell.png)
![Screenshot looqs search fstream](https://garage.quitesimple.org/assets/looqs/fstream_write.png)
@ -29,7 +28,7 @@ There is no need to write the long form of filters. There are also booleans avai
The screenshots in this section may occasionally be slightly outdated, but they are usually recent enough to get an overall impression of the current state of the GUI.
## Current status
Latest version: 2022-07-30, v0.5.1
Latest version: 2022-08-14, v0.6
Please see [Changelog](CHANGELOG.md) for a human readable list of changes.

查看文件

@ -9,6 +9,8 @@ looqs is still at an early stage and may exhibit some weirdness and contain bugs
## Current Limitations and things to know
You should be aware of the following:
- Lags are to be expected for networked mount points such as SMB and NFS etc.
- It may seem natural, but the GUI and CLI operate on the same database, so if you add files using the CLI, the GUI will know about them too.
- If a file is listed in the "Search results" tab, it does not imply that a preview will be available in the "Previews" tab, as looqs can search more file formats than it can generate previews for currently.
@ -26,6 +28,7 @@ Database default path: `$HOME/.local/share/quitesimple.org/looqs/looqs.sqlite`.
### First run
You will be presented with an empty list. Go to the **"Index"** tab, add some directories and click **"Start indexing"**.
### Indexing
For large directories the progress bar is essentially just decoration. As long as you see the counters
increase, everything is fine even if it seems the progress bar is stuck.
@ -33,9 +36,17 @@ The indexing can be stopped. If you run it again you do not start from scratch,
which files have been modified since they have been added to the index. Thus, files will
only be reprocessed when necessary. Note that cancellation itself may take a moment as files finish processing.
The counters increase in batches, therefore it's normal that it seems no progress is being made, particularly when processing lots of large documents. This aspect will be improved in a future version.
### Search
The text field at the top is where you type your query. It can be selected quickly using **CTRL + L**. Filters are available, see this document at the end. By default, both the full path and the content are searched. Path names take precedence, i. e. they will appear the top of the list.
**CTRL + F**: This is helpful shortcut if you want to perform several searches. Consider the following
query: "p:(docs) c:(invoice credit card)". Press CTRL+F to highlight 'invoice credit card'. This way
you can quickly perform content searches in paths containing 'docs'.
**CTRL + W**: Removes the last filter. If we take above's example "p:(docs) c:(invoice credit card)" again, then CTRL + W kills "c:(invoice credit card)".
### Configuring PDF viewer
It's most convenient if, when you click on a preview, the PDF reader opens the page you clicked. For that, looqs needs to know which viewer you want to launch.

查看文件

@ -341,6 +341,102 @@ bool MainWindow::indexerTabActive()
return ui->tabWidget->currentIndex() == 2;
}
void MainWindow::processShortcut(int key)
{
if(key == Qt::Key_Tab || key == Qt::Key_Backtab)
{
int tabIndex = ui->tabWidget->currentIndex();
if(key == Qt::Key_Tab)
{
++tabIndex;
}
if(key == Qt::Key_Backtab)
{
--tabIndex;
}
tabIndex = tabIndex % ui->tabWidget->count();
if(tabIndex < 0)
{
tabIndex = ui->tabWidget->count() - 1;
}
ui->tabWidget->setCurrentIndex(tabIndex);
}
if(key == Qt::Key_L)
{
ui->txtSearch->setFocus();
ui->txtSearch->selectAll();
}
if(key == Qt::Key_W)
{
ui->txtSearch->setFocus();
QString currentText = ui->txtSearch->text().trimmed();
int index = currentText.lastIndexOf(QRegularExpression("[\\s\\)]"));
if(index != -1)
{
bool isFilter = (index == currentText.length() - 1);
currentText.remove(index + 1, currentText.length() - index - 1);
if(isFilter)
{
index = currentText.lastIndexOf(' ', index);
if(index == -1)
{
index = 0;
}
currentText.remove(index, currentText.length());
}
if(currentText.length() > 0)
{
currentText += ' ';
}
ui->txtSearch->setText(currentText);
}
else
{
ui->txtSearch->clear();
}
}
if(key == Qt::Key_F)
{
ui->txtSearch->setFocus();
QString currentText = ui->txtSearch->text().trimmed();
int index = currentText.lastIndexOf(')');
if(index != -1)
{
bool isFilter = (index == currentText.length() - 1);
if(!isFilter)
{
ui->txtSearch->setSelection(index + 2, ui->txtSearch->text().length() - index - 1);
}
else
{
int begin = currentText.lastIndexOf('(', index - 1);
if(begin != -1)
{
ui->txtSearch->setSelection(begin + 1, index - begin - 1);
}
}
}
else
{
int spaceIndex = currentText.lastIndexOf(' ');
int colonIndex = currentText.lastIndexOf(':');
if(colonIndex > spaceIndex)
{
int target = currentText.indexOf(' ', colonIndex);
if(target == -1)
{
target = ui->txtSearch->text().size() - colonIndex;
}
ui->txtSearch->setSelection(colonIndex + 1, target - 1);
}
else
{
ui->txtSearch->setSelection(spaceIndex + 1, ui->txtSearch->text().size() - spaceIndex - 1);
}
}
}
}
void MainWindow::keyPressEvent(QKeyEvent *event)
{
bool quit =
@ -348,17 +444,15 @@ void MainWindow::keyPressEvent(QKeyEvent *event)
if(quit)
{
qApp->quit();
return;
}
if(event->modifiers() & Qt::ControlModifier)
{
if(event->key() == Qt::Key_L)
{
ui->txtSearch->setFocus();
ui->txtSearch->selectAll();
}
processShortcut(event->key());
return;
}
QWidget::keyPressEvent(event);
}
@ -408,6 +502,17 @@ void MainWindow::initSettingsTabs()
void MainWindow::saveSettings()
{
if(ui->chkRemoveOldDb->isChecked())
{
bool result = QFile::remove(Common::databasePath());
if(!result)
{
QMessageBox::critical(this, "Error removing database",
"Failed to remove old database. Settings not saved.");
return;
}
}
QSettings settings;
QString pdfViewerCmd = ui->txtSettingPdfPreviewerCmd->text();
@ -588,6 +693,8 @@ void MainWindow::lineEditReturnPressed()
void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
{
this->previewableSearchResults.clear();
qDeleteAll(ui->scrollAreaWidgetContents->children());
ui->treeResultsList->clear();
ui->comboPreviewFiles->clear();
ui->comboPreviewFiles->addItem("All previews");
@ -669,7 +776,7 @@ void MainWindow::makePreviews(int page)
processedPdfPreviews = 0;
QVector<QString> wordsToHighlight;
QRegularExpression extractor(R"#("([^"]*)"|((\p{L}|\p{N})+))#");
QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
for(const Token &token : this->contentSearchQuery.getTokens())
{
if(token.type == FILTER_CONTENT_CONTAINS)

查看文件

@ -63,7 +63,8 @@ class MainWindow : public QMainWindow
unsigned int currentPreviewGeneration = 1;
void initSettingsTabs();
int currentSelectedScale();
private slots:
void processShortcut(int key);
private slots:
void lineEditReturnPressed();
void treeSearchItemActivated(QTreeWidgetItem *item, int i);
void showSearchResultsContextMenu(const QPoint &point);

查看文件

@ -532,6 +532,13 @@
<item>
<widget class="QLineEdit" name="txtSettingDatabasePath"/>
</item>
<item>
<widget class="QCheckBox" name="chkRemoveOldDb">
<property name="text">
<string>Remove old database on save</string>
</property>
</widget>
</item>
</layout>
</widget>
</item>

查看文件

@ -2,6 +2,7 @@
#include "previewgeneratorplaintext.h"
#include "previewresultplaintext.h"
#include "../shared/limitqueue.h"
QString PreviewGeneratorPlainText::generatePreviewText(QString content, RenderConfig config, QString fileName)
{
@ -14,14 +15,13 @@ QString PreviewGeneratorPlainText::generatePreviewText(QString content, RenderCo
QHash<QString, int> countmap;
const unsigned int maxSnippets = 7;
unsigned int currentSnippets = 0;
for(QString &word : config.wordsToHighlight)
{
int lastPos = 0;
int index = content.indexOf(word, lastPos, Qt::CaseInsensitive);
while(index != -1 && currentSnippets < maxSnippets)
while(index != -1 && currentSnippets < MAX_SNIPPETS)
{
countmap[word] = countmap.value(word, 0) + 1;
@ -57,17 +57,14 @@ QString PreviewGeneratorPlainText::generatePreviewText(QString content, RenderCo
++i;
}
for(QString &word : config.wordsToHighlight)
{
resulText.replace(word, "<span style=\"background-color: yellow;\">" + word + "</span>", Qt::CaseInsensitive);
}
QString header = "<b>" + fileName + "</b> ";
for(QString &word : config.wordsToHighlight)
{
resulText.replace(word, "<span style=\"background-color: yellow;\">" + word + "</span>", Qt::CaseInsensitive);
header += word + ": " + QString::number(countmap[word]) + " ";
}
if(currentSnippets == maxSnippets)
if(currentSnippets == MAX_SNIPPETS)
{
header += "(truncated)";
}
@ -77,6 +74,82 @@ QString PreviewGeneratorPlainText::generatePreviewText(QString content, RenderCo
return header + resulText.replace("\n", "<br>").mid(0, 1000);
}
QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in, RenderConfig config, QString fileName)
{
QString resultText;
const unsigned int contextLinesCount = 2;
LimitQueue<QString> queue(contextLinesCount);
QString currentLine;
currentLine.reserve(512);
/* How many lines to read after a line with a match (like grep -A ) */
int justReadLinesCount = -1;
auto appendLine = [&resultText](int lineNumber, QString &line)
{ resultText.append(QString("<b>%1</b>%2<br>").arg(lineNumber).arg(line)); };
QHash<QString, int> countmap;
QString header = "<b>" + fileName + "</b> ";
unsigned int snippetsCount = 0;
unsigned int lineCount = 0;
while(in.readLineInto(&currentLine) && snippetsCount < MAX_SNIPPETS)
{
++lineCount;
bool matched = false;
if(justReadLinesCount > 0)
{
appendLine(lineCount, currentLine);
--justReadLinesCount;
continue;
}
if(justReadLinesCount == 0)
{
resultText += "---<br>";
justReadLinesCount = -1;
++snippetsCount;
}
for(QString &word : config.wordsToHighlight)
{
if(currentLine.contains(word, Qt::CaseInsensitive))
{
countmap[word] = countmap.value(word, 0) + 1;
matched = true;
currentLine.replace(word, "<span style=\"background-color: yellow;\">" + word + "</span>",
Qt::CaseInsensitive);
}
}
if(matched)
{
while(queue.size() > 0)
{
int queuedLineCount = lineCount - queue.size();
QString queuedLine = queue.dequeue();
appendLine(queuedLineCount, queuedLine);
}
appendLine(lineCount, currentLine);
justReadLinesCount = contextLinesCount;
}
else
{
queue.enqueue(currentLine);
}
}
for(QString &word : config.wordsToHighlight)
{
header += word + ": " + QString::number(countmap[word]) + " ";
}
if(snippetsCount == MAX_SNIPPETS)
{
header += "(truncated)";
}
header += "<hr>";
return header + resultText;
}
QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig config, QString documentPath,
unsigned int page)
{
@ -87,9 +160,7 @@ QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig c
return QSharedPointer<PreviewResultPlainText>(result);
}
QTextStream in(&file);
QString content = in.readAll();
QFileInfo info{documentPath};
result->setText(generatePreviewText(content, config, info.fileName()));
result->setText(generateLineBasedPreviewText(in, config, info.fileName()));
return QSharedPointer<PreviewResultPlainText>(result);
}

查看文件

@ -1,12 +1,17 @@
#ifndef PREVIEWGENERATORPLAINTEXT_H
#define PREVIEWGENERATORPLAINTEXT_H
#include <QTextStream>
#include "previewgenerator.h"
class PreviewGeneratorPlainText : public PreviewGenerator
{
protected:
const unsigned int MAX_SNIPPETS = 7;
public:
using PreviewGenerator::PreviewGenerator;
QString generatePreviewText(QString content, RenderConfig config, QString fileName);
QString generateLineBasedPreviewText(QTextStream &in, RenderConfig config, QString fileName);
QSharedPointer<PreviewResult> generate(RenderConfig config, QString documentPath, unsigned int page);
};

48
shared/limitqueue.h Normal file
查看文件

@ -0,0 +1,48 @@
#ifndef LIMITQUEUE_H
#define LIMITQUEUE_H
#include <QQueue>
template <class T> class LimitQueue
{
protected:
QQueue<T> queue;
unsigned int limit = 0;
public:
LimitQueue();
LimitQueue(unsigned int limit)
{
this->limit = limit;
}
void enqueue(const T &t)
{
if(queue.size() == limit)
{
queue.dequeue();
}
queue.enqueue(t);
}
int size()
{
return queue.size();
}
T dequeue()
{
return queue.dequeue();
}
void setLimit(unsigned int limit)
{
this->limit = limit;
}
void clear()
{
queue.clear();
}
};
#endif // LIMITQUEUE_H

查看文件

@ -180,9 +180,8 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
QStringList loneWords;
LooqsQuery result;
QRegularExpression rx(
"((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([\\p{L}\\p{N},])+)|(?<boolean>AND|OR)"
"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[\"\\p{L}\\p{N}]+))");
QRegularExpression rx("((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([^\\s])+)|(?<boolean>AND|OR)"
"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[^\\s]+))");
QRegularExpressionMatchIterator i = rx.globalMatch(expression);
auto previousWasBool = [&result] { return !result.tokens.empty() && ((result.tokens.last().type & BOOL) == BOOL); };
auto previousWas = [&result](TokenType t) { return !result.tokens.empty() && (result.tokens.last().type == t); };

1
shared/migrations/3.sql Normal file
查看文件

@ -0,0 +1 @@
DROP trigger content_ad;

查看文件

@ -2,5 +2,6 @@
<qresource prefix="/looqs-migrations">
<file>1.sql</file>
<file>2.sql</file>
<file>3.sql</file>
</qresource>
</RCC>

查看文件

@ -77,6 +77,7 @@ HEADERS += sqlitesearch.h \
filescanworker.h \
indexer.h \
indexsyncer.h \
limitqueue.h \
logger.h \
looqsgeneralexception.h \
looqsquery.h \