Porównaj commity

..

No commits in common. "595684c36457065b3786ff7a52a8e3e8a1abb389" and "20a1f8b2cd2687df95732ff32ebd6a497b48bdae" have entirely different histories.

49 zmienionych plików z 367 dodań i 1323 usunięć

Wyświetl plik

@ -1,24 +1,5 @@
# looqs: Release notes
## 2023-05-07 - v0.9
Highlights: Tag support. Also begin new index mode to only index metadata (currently only path + file size, more to come).
Note: Upgrading can take some time as new column indexes will be added
CHANGES:
- gui: Improve font rendering in previews
- gui: Allow indexing only metadata
- gui: Allow adding content for files which only had metadata indexed before
- gui: Allow assigning tags by right clicking on paths
- cli: "add" command: Implement --verbose (-v)
- cli: "add" command: Implement --no-content and --fill-content
- cli: Add "tag" command which allows managing tags for paths.
- search: Add "tag:()", "t:()" filters
- Minor improvements and refactorings under the hood
- Add packages: Ubuntu 23.04.
## 2022-11-19 - v0.8.1
CHANGES:

Wyświetl plik

@ -1,4 +1,4 @@
Copyright (c) 2018-2023: Albert Schwarzkopf <looqs at quitesimple period org>
Copyright (c) 2018-2022: Albert Schwarzkopf <looqs at quitesimple period org>
looqs is made available under the following license:

Wyświetl plik

@ -76,7 +76,7 @@ To build on Ubuntu and Debian, clone the repo and then run:
```
git submodule init
git submodule update
sudo apt install build-essential qtbase5-dev libqt5sql5-sqlite libpoppler-qt5-dev libuchardet-dev libquazip5-dev
sudo apt install build-essential qtbase5-dev libpoppler-qt5-dev libuchardet-dev libquazip5-dev
qmake
make
```
@ -97,9 +97,7 @@ The GUI is located in `gui/looqs-gui`, the binary for the CLI is in `cli/looqs`
## Packages
At this point, looqs is not in any official distro package repo, but I maintain some packages.
### Ubuntu 23.04, 22.10, 22.04
### Ubuntu 22.04, 22.10
Latest release can be installed using apt from the repo.
```
# First, obtain key, assume it's trusted.
@ -110,8 +108,6 @@ echo "deb [arch=amd64 signed-by=/usr/share/keyrings/repo.quitesimple.org.gpg] ht
sudo apt-get update
sudo apt-get install looqs
```
### Gentoo (EXPERIMENTAL)
Available in this overlay: https://github.com/quitesimpleorg/quitesimple-overlay
### Prebuilt tarball (distro-agnostic) (EXPERIMENTAL)
looqs is also distributed as a tarball containing prebuilt binaries and its library dependencies. The tarball is
@ -138,7 +134,7 @@ An AppImage may accompany the tarball in the future.
### Other distros
I appreciate help for others distros. If you create a package, let me know!
I'll probably add a package for voidlinux at some point and maybe will provide a Gentoo ebuild. However, I would appreciate help for others distros. If you create a package, let me know!
### Signature verification

Wyświetl plik

@ -165,8 +165,6 @@ A number of search filters are available.
| path.begins:(term) | pb:(term) | Filters path beginning with the specified term |
| contains:(terms) | c:(terms) | Full-text search, also understands quotes |
| limit:(integer) | - | Limits the number of results. The default is 1000. Say "limit:0" to see all results |
| tag:(tagname) | t:(tagname) | Filter for files that have been tagged with the corresponding tag |
Filters can be combined. The booleans AND and OR are supported. Negations can be applied too, except for c:(). Negations are specified with "!".
The AND boolean is implicit and thus entering it strictly optional.
@ -179,5 +177,11 @@ Examples:
|p:(notes) (pe:(odt) OR pe:(docx)) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|Finds files such as notes.docx, notes.odt but also any .docs and .odt when the path contains the string 'notes'|
|memcpy !(pe:(.c) OR pe:(.cpp))| Performs a FTS search for 'memcpy' but excludes .cpp and .c files.|
|c:("I think, therefore")|Performs a FTS search for the phrase "I think, therefore".|
|c:("invoice") Downloads|Equivalent to c:("invoice") p:("Downloads")|
|p:(Downloads) invoice|Equivalent to c:("invoice") p:("Downloads")|
|c:("invoice") Downloads|This query is equivalent to c:("invoice") p:("Downloads")|

Wyświetl plik

@ -15,7 +15,6 @@ DEFINES += QT_DEPRECATED_WARNINGS
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000 # disables all the APIs deprecated before Qt 6.0.0
SOURCES += \
commandtag.cpp \
main.cpp \
commandadd.cpp \
commanddelete.cpp \
@ -28,7 +27,6 @@ HEADERS += \
command.h \
commandadd.h \
commanddelete.h \
commandtag.h \
commandupdate.h \
commandsearch.h \
commandlist.h

Wyświetl plik

@ -2,6 +2,7 @@
#include <QThread>
#include <QDebug>
#include "command.h"
#include "looqsgeneralexception.h"
void Command::execute()
{

Wyświetl plik

@ -23,7 +23,7 @@ void CommandAdd::indexerFinished()
if(failedPathsCount > 0)
{
Logger::info() << "Failed paths: " << Qt::endl;
for(const QString &paths : result.failedPaths())
for(QString paths : result.failedPaths())
{
Logger::info() << paths << Qt::endl;
}
@ -41,36 +41,23 @@ int CommandAdd::handle(QStringList arguments)
{
QCommandLineParser parser;
parser.addOptions({{{"c", "continue"},
"Continue adding files, don't exit on first error. Exit code will be 0. If this option is not "
"given, looqs will "
"exit asap, but it's possible that a few files will still be processed."
"Continue adding files, don't exit on first error. If this option is not given, looqs will "
"exit asap, but it's possible that a few files will still be processed. "
"Set -t 1 to avoid this behavior, but processing will be slower. "},
{{"n", "no-content"}, "Only add paths to database. Do not index content"},
{{"v", "verbose"}, "Print paths of files being processed"},
{{"f", "fill-content"}, "Index content for files previously indexed with -n"},
{{"t", "threads"}, "Number of threads to use.", "threads"}});
parser.addHelpOption();
parser.addPositionalArgument("add", "Add paths to the index",
"add [paths...]. If no path is given, read from stdin, one path per line.");
parser.process(arguments);
this->keepGoing = parser.isSet("continue");
bool pathsOnly = parser.isSet("no-content");
bool fillContent = parser.isSet("fill-content");
bool verbose = parser.isSet("verbose");
if(parser.isSet("threads"))
{
QString threadsCount = parser.value("threads");
QThreadPool::globalInstance()->setMaxThreadCount(threadsCount.toInt());
}
if(pathsOnly && fillContent)
{
Logger::error() << "Invalid options: -n and -f cannot both be set";
return EXIT_FAILURE;
}
QStringList files = parser.positionalArguments();
if(files.length() == 0)
@ -84,47 +71,15 @@ int CommandAdd::handle(QStringList arguments)
}
}
FileSaverOptions fileSaverOptions;
fileSaverOptions.keepGoing = keepGoing;
fileSaverOptions.fillExistingContentless = fillContent;
fileSaverOptions.metadataOnly = pathsOnly;
fileSaverOptions.verbose = verbose;
indexer = new Indexer(*this->dbService);
indexer->setFileSaverOptions(fileSaverOptions);
indexer->setTargetPaths(files.toVector());
if(verbose)
{
indexer->setProgressReportThreshold(1);
}
indexer->setKeepGoing(keepGoing);
connect(indexer, &Indexer::pathsCountChanged, this,
[](int pathsCount) { Logger::info() << "Found paths: " << pathsCount << Qt::endl; });
connect(indexer, &Indexer::indexProgress, this,
[verbose, this](int pathsCount, unsigned int /*added*/, unsigned int /*skipped*/, unsigned int /*failed*/,
unsigned int /*totalCount*/)
{
Logger::info() << "Processed files: " << pathsCount << Qt::endl;
if(verbose)
{
IndexResult indexResult = indexer->getResult();
int newlyAdded = indexResult.results.count() - currentResult.results.count();
if(newlyAdded > 0)
{
int newOffset = indexResult.results.count() - newlyAdded;
for(int i = newOffset; i < indexResult.results.count(); i++)
{
auto result = indexResult.results.at(i);
Logger::info() << SaveFileResultToString(result.second) << result.first << Qt::endl;
}
}
this->currentResult = indexResult;
}
}
);
[](int pathsCount, unsigned int added, unsigned int skipped, unsigned int failed, unsigned int totalCount)
{ Logger::info() << "Processed files: " << pathsCount << Qt::endl; });
connect(indexer, &Indexer::finished, this, &CommandAdd::indexerFinished);
this->autoFinish = false;

Wyświetl plik

@ -13,8 +13,6 @@ class CommandAdd : public Command
bool keepGoing = true;
protected:
IndexResult currentResult;
public:
using Command::Command;

Wyświetl plik

@ -1,5 +1,6 @@
#include <QCommandLineParser>
#include "commandlist.h"
#include "databasefactory.h"
#include "logger.h"
int CommandList::handle(QStringList arguments)

Wyświetl plik

@ -1,5 +1,6 @@
#include <QCommandLineParser>
#include "commandsearch.h"
#include "databasefactory.h"
#include "logger.h"
int CommandSearch::handle(QStringList arguments)

Wyświetl plik

@ -1,153 +0,0 @@
#include <QCommandLineParser>
#include "commandtag.h"
#include "logger.h"
#include "tagmanager.h"
bool CommandTag::ensureAbsolutePaths(const QVector<QString> &paths, QVector<QString> &absolutePaths)
{
for(const QString &path : paths)
{
QFileInfo info{path};
if(!info.exists())
{
Logger::error() << "Can't add tag for file " + info.absoluteFilePath() + " because it does not exist"
<< Qt::endl;
return false;
}
QString absolutePath = info.absoluteFilePath();
if(!this->dbService->fileExistsInDatabase(absolutePath))
{
Logger::error() << "Only files that have been indexed can be tagged. File not in index: " + absolutePath
<< Qt::endl;
return false;
}
absolutePaths.append(absolutePath);
}
return true;
}
int CommandTag::handle(QStringList arguments)
{
QCommandLineParser parser;
parser.addPositionalArgument("add", "Adds a tag to a file",
"add [tag] [paths...]. Adds the tag to the specified paths");
parser.addPositionalArgument("remove", "Removes a path associated to a tag", "remove [tag] [path]");
parser.addPositionalArgument("delete", "Deletes a tag", "delete [tag]");
parser.addPositionalArgument("list", "Lists paths associated with a tag, or all tags", "list [tag]");
parser.addPositionalArgument("show", "Lists tags associated with a path", "show [path]");
parser.addHelpOption();
parser.parse(arguments);
QStringList args = parser.positionalArguments();
if(args.length() == 0)
{
parser.showHelp(EXIT_FAILURE);
return EXIT_FAILURE;
}
TagManager tagManager{*this->dbService};
QString cmd = args[0];
if(cmd == "add")
{
if(args.length() < 3)
{
Logger::error() << "Not enough arguments provided. 'add' requires a tag followed by at least one path"
<< Qt::endl;
return EXIT_FAILURE;
}
QString tag = args[1];
QVector<QString> paths = args.mid(2).toVector();
QVector<QString> absolutePaths;
if(!ensureAbsolutePaths(paths, absolutePaths))
{
return EXIT_FAILURE;
}
bool result = tagManager.addPathsToTag(tag, absolutePaths);
if(!result)
{
Logger::error() << "Failed to assign tags" << Qt::endl;
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
if(cmd == "list")
{
QString tag;
if(args.length() >= 2)
{
tag = args[1];
}
QVector<QString> entries;
if(tag.isEmpty())
{
entries = tagManager.getTags();
}
else
{
entries = tagManager.getPaths(tag);
}
for(const QString &entry : entries)
{
Logger::info() << entry << Qt::endl;
}
}
if(cmd == "remove")
{
if(args.length() < 3)
{
Logger::error() << "Not enough arguments provided. 'remove' requires a tag followed by at least one path"
<< Qt::endl;
return EXIT_FAILURE;
}
QString tag = args[1];
QVector<QString> paths = args.mid(2).toVector();
QVector<QString> absolutePaths;
if(!ensureAbsolutePaths(paths, absolutePaths))
{
return EXIT_FAILURE;
}
if(!tagManager.removePathsForTag(tag, absolutePaths))
{
Logger::error() << "Failed to remove path assignments" << Qt::endl;
return EXIT_FAILURE;
}
}
if(cmd == "delete")
{
if(args.length() != 2)
{
Logger::error() << "The 'delete' command requires the tag to delete" << Qt::endl;
return EXIT_FAILURE;
}
if(!tagManager.deleteTag(args[1]))
{
Logger::error() << "Failed to delete tag" << Qt::endl;
return EXIT_FAILURE;
}
}
if(cmd == "show")
{
if(args.length() != 2)
{
Logger::error() << "The 'show' command requires a path to show the assigned tags" << Qt::endl;
return EXIT_FAILURE;
}
QString path = args[1];
QVector<QString> absolutePaths;
if(!ensureAbsolutePaths({path}, absolutePaths))
{
return EXIT_FAILURE;
}
QVector<QString> tags = tagManager.getTags(absolutePaths.at(0));
for(const QString &entry : tags)
{
Logger::info() << entry << Qt::endl;
}
}
return EXIT_SUCCESS;
}

Wyświetl plik

@ -1,16 +0,0 @@
#ifndef COMMANDTAG_H
#define COMMANDTAG_H
#include "command.h"
class CommandTag : public Command
{
protected:
bool ensureAbsolutePaths(const QVector<QString> &paths, QVector<QString> &absolutePaths);
public:
using Command::Command;
int handle(QStringList arguments) override;
};
#endif // COMMANDTAG_H

Wyświetl plik

@ -38,13 +38,10 @@ int CommandUpdate::handle(QStringList arguments)
QThreadPool::globalInstance()->setMaxThreadCount(threadsCount.toInt());
}
bool hasErrors = false;
IndexSyncer *syncer = new IndexSyncer(*this->dbService);
FileSaverOptions fileOptions;
fileOptions.keepGoing = keepGoing;
fileOptions.verbose = verbose;
syncer->setFileSaverOptions(fileOptions);
syncer->setKeepGoing(keepGoing);
syncer->setVerbose(verbose);
syncer->setPattern(pattern);
syncer->setDryRun(dryRun);
syncer->setRemoveDeletedFromIndex(deleteMissing);
@ -63,7 +60,7 @@ int CommandUpdate::handle(QStringList arguments)
/* TODO: updated not printed, handled be verbose in FileSaver, but this can be improved */
}
connect(syncer, &IndexSyncer::finished, this,
[this, dryRun, keepGoing](unsigned int totalUpdated, unsigned int totalRemoved, unsigned int totalErrors)
[&](unsigned int totalUpdated, unsigned int totalRemoved, unsigned int totalErrors)
{
Logger::info() << "Syncing finished" << Qt::endl;
@ -75,7 +72,7 @@ int CommandUpdate::handle(QStringList arguments)
}
int retval = 0;
if(this->hasErrors && !keepGoing)
if(hasErrors && !keepGoing)
{
retval = 1;
}
@ -85,7 +82,7 @@ int CommandUpdate::handle(QStringList arguments)
[&](QString error)
{
Logger::error() << error << Qt::endl;
this->hasErrors = true;
hasErrors = true;
});
this->autoFinish = false;

Wyświetl plik

@ -4,9 +4,6 @@
#include "filesaver.h"
class CommandUpdate : public Command
{
protected:
bool hasErrors = false;
public:
using Command::Command;
int handle(QStringList arguments) override;

Wyświetl plik

@ -21,7 +21,6 @@
#include "commandupdate.h"
#include "commandsearch.h"
#include "commandlist.h"
#include "commandtag.h"
#include "databasefactory.h"
#include "logger.h"
#include "sandboxedprocessor.h"
@ -32,7 +31,7 @@
void printUsage(QString argv0)
{
qInfo() << "Usage:" << argv0 << "command";
qInfo() << "Valid commands: add, update, search, delete, tag, list. Each command has a --help option.";
qInfo() << "Valid commands: add, update, delete, search, list. Each command has a --help option.";
}
Command *commandFromName(QString name, SqliteDbService &dbService)
@ -57,10 +56,6 @@ Command *commandFromName(QString name, SqliteDbService &dbService)
{
return new CommandList(dbService);
}
if(name == "tag")
{
return new CommandTag(dbService);
}
return nullptr;
}

Wyświetl plik

@ -34,7 +34,6 @@ SOURCES += \
main.cpp \
mainwindow.cpp \
clicklabel.cpp \
previewcoordinator.cpp \
previewgenerator.cpp \
previewgeneratormapfunctor.cpp \
previewgeneratorodt.cpp \
@ -55,7 +54,6 @@ HEADERS += \
ipcserver.h \
mainwindow.h \
clicklabel.h \
previewcoordinator.h \
previewgenerator.h \
previewgeneratormapfunctor.h \
previewgeneratorodt.h \

Wyświetl plik

@ -28,7 +28,7 @@ void enableIpcSandbox()
policy->namespace_options = EXILE_UNSHARE_USER | EXILE_UNSHARE_MOUNT | EXILE_UNSHARE_NETWORK;
policy->no_new_privs = 1;
policy->drop_caps = 1;
policy->vow_promises = exile_vows_from_str("thread cpath rpath wpath unix stdio proc error");
policy->vow_promises = exile_vows_from_str("thread cpath rpath unix stdio proc error");
policy->mount_path_policies_to_chroot = 1;
QString ipcSocketPath = Common::ipcSocketPath();
@ -193,7 +193,7 @@ int main(int argc, char *argv[])
Logger::error() << error << Qt::endl;
QMessageBox::critical(nullptr, "Error during upgrade",
error);
exit(EXIT_FAILURE);
qApp->quit();
}
);

Wyświetl plik

@ -16,15 +16,14 @@
#include <QScreen>
#include <QProgressDialog>
#include <QDesktopWidget>
#include <QWidgetAction>
#include <QInputDialog>
#include "mainwindow.h"
#include "ui_mainwindow.h"
#include "clicklabel.h"
#include "../shared/sqlitesearch.h"
#include "../shared/looqsgeneralexception.h"
#include "../shared/common.h"
#include "ipcpreviewclient.h"
#include "previewgenerator.h"
#include "aboutdialog.h"
MainWindow::MainWindow(QWidget *parent, QString socketPath)
@ -33,7 +32,8 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
this->progressDialog.cancel(); // because constructing it shows it, quite weird
ui->setupUi(this);
setWindowTitle(QCoreApplication::applicationName());
this->ipcPreviewClient.moveToThread(&this->ipcClientThread);
this->ipcPreviewClient.setSocketPath(socketPath);
QSettings settings;
this->dbFactory = new DatabaseFactory(Common::databasePath());
@ -45,9 +45,6 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
indexer = new Indexer(*(this->dbService));
indexer->setParent(this);
tagManager = new TagManager(*(this->dbService));
connectSignals();
ui->treeResultsList->setContextMenuPolicy(Qt::ContextMenuPolicy::CustomContextMenu);
ui->tabWidget->setCurrentIndex(0);
@ -81,7 +78,7 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
ui->txtSearch->installEventFilter(this);
ui->scrollArea->viewport()->installEventFilter(this);
this->previewCoordinator.setSocketPath(socketPath);
this->ipcClientThread.start();
}
void MainWindow::addPathToIndex()
@ -153,7 +150,7 @@ void MainWindow::connectSignals()
connect(this->indexer, &Indexer::finished, this, &MainWindow::finishIndexing);
connect(ui->lstPaths->selectionModel(), &QItemSelectionModel::selectionChanged, this,
[&](const QItemSelection & /*selected*/, const QItemSelection & /*deselected*/)
[&](const QItemSelection &selected, const QItemSelection &deselected)
{ ui->btnDeletePath->setEnabled(this->ui->lstPaths->selectedItems().count() > 0); });
connect(ui->btnDeletePath, &QPushButton::clicked, this, [&] { qDeleteAll(ui->lstPaths->selectedItems()); });
@ -173,29 +170,30 @@ void MainWindow::connectSignals()
}
});
connect(ui->menuAboutAction, &QAction::triggered, this,
[this](bool /*checked*/)
[this](bool checked)
{
AboutDialog aboutDialog(this);
aboutDialog.exec();
});
connect(ui->menuAboutQtAction, &QAction::triggered, this,
[this](bool /*checked*/) { QMessageBox::aboutQt(this, "About Qt"); });
[this](bool checked) { QMessageBox::aboutQt(this, "About Qt"); });
connect(ui->menuSyncIndexAction, &QAction::triggered, this, &MainWindow::startIndexSync);
connect(ui->menuOpenUserManualAction, &QAction::triggered, this,
[]() { QDesktopServices::openUrl(Common::userManualUrl()); });
[this]() { QDesktopServices::openUrl(Common::userManualUrl()); });
connect(
indexSyncer, &IndexSyncer::finished, this,
[&](unsigned int totalUpdated, unsigned int totalDeleted, unsigned int totalErrored)
{
this->progressDialog.cancel();
connect(indexSyncer, &IndexSyncer::finished, this,
[&](unsigned int totalUpdated, unsigned int totalDeleted, unsigned int totalErrored)
{
this->progressDialog.cancel();
QMessageBox::information(
this, "Syncing finished",
QString("Syncing finished\n\nTotal updated: %1\nTotal deleted: %2\nTotal errors: %3\n")
.arg(QString::number(totalUpdated), QString::number(totalDeleted), QString::number(totalErrored)));
});
QMessageBox::information(
this, "Syncing finished",
QString("Syncing finished\n\nTotal updated: %1\nTotal deleted: %2\nTotal errors: %3\n")
.arg(QString::number(totalUpdated))
.arg(QString::number(totalDeleted))
.arg(QString::number(totalErrored)));
});
connect(this, &MainWindow::beginIndexSync, indexSyncer, &IndexSyncer::sync);
connect(&this->progressDialog, &QProgressDialog::canceled, indexSyncer, &IndexSyncer::cancel);
connect(ui->btnSaveSettings, &QPushButton::clicked, this, &MainWindow::saveSettings);
@ -210,9 +208,9 @@ void MainWindow::connectSignals()
}
},
Qt::QueuedConnection);
connect(&previewCoordinator, &PreviewCoordinator::previewReady, this, &MainWindow::previewReceived,
connect(&ipcPreviewClient, &IPCPreviewClient::previewReceived, this, &MainWindow::previewReceived,
Qt::QueuedConnection);
connect(&previewCoordinator, &PreviewCoordinator::completedGeneration, this,
connect(&ipcPreviewClient, &IPCPreviewClient::finished, this,
[&]
{
this->ui->previewProcessBar->setValue(this->ui->previewProcessBar->maximum());
@ -220,24 +218,22 @@ void MainWindow::connectSignals()
this->ui->comboPreviewFiles->setEnabled(true);
ui->txtSearch->setEnabled(true);
});
connect(&previewCoordinator, &PreviewCoordinator::error, this,
connect(&ipcPreviewClient, &IPCPreviewClient::error, this,
[this](QString msg)
{
qCritical() << msg << Qt::endl;
QMessageBox::critical(this, "IPC error", msg);
});
connect(ui->radioMetadataOnly, &QRadioButton::toggled, this,
[this](bool toggled)
{
if(toggled)
{
this->ui->chkFillContentForContentless->setChecked(false);
};
});
connect(this, &MainWindow::startIpcPreviews, &ipcPreviewClient, &IPCPreviewClient::startGeneration,
Qt::QueuedConnection);
connect(this, &MainWindow::stopIpcPreviews, &ipcPreviewClient, &IPCPreviewClient::stopGeneration,
Qt::QueuedConnection);
}
void MainWindow::exportFailedPaths()
{
QString filename =
QString("/tmp/looqs_indexresult_failed_%1").arg(QDateTime::currentDateTime().toString("yyyy-MM-dd_hhmmss"));
QFile outFile(filename);
@ -270,11 +266,8 @@ void MainWindow::startIndexSync()
progressDialog.setValue(0);
progressDialog.open();
FileSaverOptions options;
options.keepGoing = true;
options.verbose = false;
indexSyncer->setFileSaverOptions(options);
indexSyncer->setKeepGoing(true);
indexSyncer->setVerbose(false);
indexSyncer->setDryRun(false);
indexSyncer->setRemoveDeletedFromIndex(true);
@ -293,7 +286,6 @@ void MainWindow::startIndexing()
if(this->indexer->isRunning())
{
ui->btnStartIndexing->setEnabled(false);
ui->btnStartIndexing->setText("Start indexing");
this->indexer->requestCancellation();
return;
@ -303,8 +295,6 @@ void MainWindow::startIndexing()
ui->resultsTab->setEnabled(false);
ui->settingsTab->setEnabled(false);
ui->txtPathScanAdd->setEnabled(false);
ui->btnAddPath->setEnabled(false);
ui->btnChoosePath->setEnabled(false);
ui->txtSearch->setEnabled(false);
ui->previewProcessBar->setValue(0);
ui->previewProcessBar->setVisible(true);
@ -321,15 +311,6 @@ void MainWindow::startIndexing()
this->indexer->setTargetPaths(paths);
QString ignorePatterns = ui->txtIgnorePatterns->text();
this->indexer->setIgnorePattern(ignorePatterns.split(";"));
FileSaverOptions options;
options.fillExistingContentless =
ui->chkFillContentForContentless->isEnabled() && ui->chkFillContentForContentless->isChecked();
options.metadataOnly = ui->radioMetadataOnly->isChecked();
options.verbose = false;
options.keepGoing = true;
this->indexer->setFileSaverOptions(options);
this->indexer->beginIndexing();
QSettings settings;
settings.setValue("indexPaths", pathSettingsValue);
@ -352,8 +333,6 @@ void MainWindow::finishIndexing()
ui->resultsTab->setEnabled(true);
ui->settingsTab->setEnabled(true);
ui->txtPathScanAdd->setEnabled(true);
ui->btnAddPath->setEnabled(true);
ui->btnChoosePath->setEnabled(true);
ui->txtSearch->setEnabled(true);
if(result.erroredPaths > 0)
{
@ -361,7 +340,7 @@ void MainWindow::finishIndexing()
}
}
void MainWindow::comboScaleChanged(int /*i*/)
void MainWindow::comboScaleChanged(int i)
{
QSettings scaleSetting;
scaleSetting.setValue("currentScale", ui->comboScale->currentText());
@ -407,8 +386,7 @@ void MainWindow::processShortcut(int key)
{
ui->txtSearch->setFocus();
QString currentText = ui->txtSearch->text().trimmed();
static QRegularExpression separatorRegex("[\\s\\)]");
int index = currentText.lastIndexOf(separatorRegex);
int index = currentText.lastIndexOf(QRegularExpression("[\\s\\)]"));
if(index != -1)
{
bool isFilter = (index == currentText.length() - 1);
@ -654,17 +632,13 @@ void MainWindow::saveSettings()
qApp->quit();
}
void MainWindow::previewReceived()
void MainWindow::previewReceived(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration)
{
this->ui->previewProcessBar->setValue(this->ui->previewProcessBar->value() + 1);
QBoxLayout *layout = static_cast<QBoxLayout *>(ui->scrollAreaWidgetContents->layout());
int index = layout->count();
if(index > 0)
if(previewGeneration < this->currentPreviewGeneration)
{
--index;
return;
}
QSharedPointer<PreviewResult> preview = this->previewCoordinator.resultAt(index);
this->ui->previewProcessBar->setValue(this->ui->previewProcessBar->value() + 1);
if(!preview.isNull() && preview->hasPreview())
{
QString docPath = preview->getDocumentPath();
@ -687,8 +661,8 @@ void MainWindow::previewReceived()
{
QFileInfo fileInfo{docPath};
QMenu menu("labeRightClick", this);
createSearchResultMenu(menu, fileInfo);
menu.addAction("Copy page number", this,
createSearchResutlMenu(menu, fileInfo);
menu.addAction("Copy page number",
[previewPage] { QGuiApplication::clipboard()->setText(QString::number(previewPage)); });
menu.exec(QCursor::pos());
};
@ -710,7 +684,24 @@ void MainWindow::previewReceived()
previewWidget->setLayout(previewLayout);
layout->insertWidget(index, previewWidget);
QBoxLayout *layout = static_cast<QBoxLayout *>(ui->scrollAreaWidgetContents->layout());
int pos = previewOrder[docPath + QString::number(previewPage)];
if(pos <= layout->count())
{
layout->insertWidget(pos, previewWidget);
for(auto it = previewWidgetOrderCache.constKeyValueBegin();
it != previewWidgetOrderCache.constKeyValueEnd(); it++)
{
if(it->first <= layout->count())
{
layout->insertWidget(it->first, it->second);
}
}
}
else
{
previewWidgetOrderCache[pos] = previewWidget;
}
}
}
@ -827,6 +818,7 @@ void MainWindow::lineEditReturnPressed()
void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
{
this->previewableSearchResults.clear();
qDeleteAll(ui->scrollAreaWidgetContents->children());
ui->treeResultsList->clear();
@ -835,8 +827,6 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
ui->comboPreviewFiles->setVisible(true);
ui->lblTotalPreviewPagesCount->setText("");
this->previewCoordinator.init(results);
bool hasDeleted = false;
QHash<QString, bool> seenMap;
for(const SearchResult &result : results)
@ -857,29 +847,34 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
item->setText(3, this->locale().formattedDataSize(result.fileData.size));
}
bool exists = pathInfo.exists();
if(!exists)
if(exists)
{
if(result.wasContentSearch)
{
if(!pathInfo.suffix().contains("htm")) // hack until we can preview them properly...
{
if(PreviewGenerator::get(pathInfo) != nullptr)
{
this->previewableSearchResults.append(result);
if(!seenMap.contains(result.fileData.absPath))
{
ui->comboPreviewFiles->addItem(result.fileData.absPath);
}
}
}
}
}
else
{
hasDeleted = true;
}
seenMap[absPath] = true;
}
seenMap.clear();
for(const SearchResult &result : this->previewCoordinator.getPreviewableSearchResults())
{
const QString &absPath = result.fileData.absPath;
if(!seenMap.contains(absPath))
{
ui->comboPreviewFiles->addItem(absPath);
}
seenMap[absPath] = true;
}
ui->treeResultsList->resizeColumnToContents(0);
ui->treeResultsList->resizeColumnToContents(1);
ui->treeResultsList->resizeColumnToContents(2);
previewDirty = this->previewCoordinator.previewableCount() > 0;
previewDirty = !this->previewableSearchResults.empty();
ui->spinPreviewPage->setValue(1);
@ -888,10 +883,8 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
makePreviews(1);
}
ui->tabWidget->setTabEnabled(1, previewDirty);
QString statusText = "Results: " + QString::number(results.size()) + " files";
statusText += ", previewable: " + QString::number(this->previewCoordinator.previewableCount());
statusText += ", previewable: " + QString::number(this->previewableSearchResults.count());
if(hasDeleted)
{
statusText += " WARNING: Some files are inaccessible. No preview available for those. Index may be out of sync";
@ -908,7 +901,7 @@ int MainWindow::currentSelectedScale()
void MainWindow::makePreviews(int page)
{
if(this->previewCoordinator.previewableCount() == 0)
if(this->previewableSearchResults.empty())
{
return;
}
@ -925,10 +918,11 @@ void MainWindow::makePreviews(int page)
ui->scrollAreaWidgetContents->setLayout(new QVBoxLayout());
ui->scrollAreaWidgetContents->layout()->setAlignment(Qt::AlignCenter);
}
ui->previewProcessBar->setMaximum(this->previewCoordinator.previewableCount());
ui->previewProcessBar->setMaximum(this->previewableSearchResults.size());
processedPdfPreviews = 0;
QVector<QString> wordsToHighlight;
static QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
for(const Token &token : this->contentSearchQuery.getTokens())
{
if(token.type == FILTER_CONTENT_CONTAINS)
@ -960,8 +954,12 @@ void MainWindow::makePreviews(int page)
renderConfig.scaleY = QGuiApplication::primaryScreen()->physicalDotsPerInchY() * (currentScale / 100.);
renderConfig.wordsToHighlight = wordsToHighlight;
this->previewOrder.clear();
this->previewWidgetOrderCache.clear();
int previewPos = 0;
QVector<RenderTarget> targets;
for(const SearchResult &sr : this->previewCoordinator.getPreviewableSearchResults())
for(SearchResult &sr : this->previewableSearchResults)
{
if(ui->comboPreviewFiles->currentIndex() != 0)
{
@ -973,8 +971,11 @@ void MainWindow::makePreviews(int page)
RenderTarget renderTarget;
renderTarget.path = sr.fileData.absPath;
renderTarget.page = (int)sr.page;
targets.append(renderTarget);
int pos = previewPos - beginOffset;
this->previewOrder[renderTarget.path + QString::number(renderTarget.page)] = pos;
++previewPos;
}
int numpages = ceil(static_cast<double>(targets.size()) / previewsPerPage);
ui->spinPreviewPage->setMaximum(numpages);
@ -984,12 +985,12 @@ void MainWindow::makePreviews(int page)
ui->previewProcessBar->setMaximum(targets.count());
ui->previewProcessBar->setMinimum(0);
ui->previewProcessBar->setValue(0);
ui->previewProcessBar->setVisible(this->previewCoordinator.previewableCount() > 0);
ui->previewProcessBar->setVisible(this->previewableSearchResults.size() > 0);
++this->currentPreviewGeneration;
this->ui->spinPreviewPage->setEnabled(false);
this->ui->comboPreviewFiles->setEnabled(false);
this->ui->txtSearch->setEnabled(false);
this->previewCoordinator.startGeneration(renderConfig, targets);
emit startIpcPreviews(renderConfig, targets);
}
void MainWindow::handleSearchError(QString error)
@ -997,87 +998,27 @@ void MainWindow::handleSearchError(QString error)
ui->lblSearchResults->setText("Error:" + error);
}
void MainWindow::createSearchResultMenu(QMenu &menu, const QFileInfo &fileInfo)
void MainWindow::createSearchResutlMenu(QMenu &menu, const QFileInfo &fileInfo)
{
menu.addAction("Copy filename to clipboard", this,
menu.addAction("Copy filename to clipboard",
[&fileInfo] { QGuiApplication::clipboard()->setText(fileInfo.fileName()); });
menu.addAction("Copy full path to clipboard", this,
menu.addAction("Copy full path to clipboard",
[&fileInfo] { QGuiApplication::clipboard()->setText(fileInfo.absoluteFilePath()); });
menu.addAction("Open containing folder", this, [this, &fileInfo] { this->openFile(fileInfo.absolutePath()); });
menu.addAction("Open containing folder", [this, &fileInfo] { this->openFile(fileInfo.absolutePath()); });
auto previewables = this->previewCoordinator.getPreviewableSearchResults();
auto result =
std::find_if(previewables.begin(), previewables.end(),
[&fileInfo](SearchResult &a) { return fileInfo.absoluteFilePath() == a.fileData.absPath; });
std::find_if(this->previewableSearchResults.begin(), this->previewableSearchResults.end(),
[this, &fileInfo](SearchResult &a) { return fileInfo.absoluteFilePath() == a.fileData.absPath; });
if(result != previewables.end())
if(result != this->previewableSearchResults.end())
{
menu.addAction("Show previews for this file", this,
menu.addAction("Show previews for this file",
[this, &fileInfo]
{
ui->tabWidget->setCurrentIndex(1);
this->ui->comboPreviewFiles->setCurrentText(fileInfo.absoluteFilePath());
});
}
QMenu *tagMenu = menu.addMenu("Tag file with: ");
QVector<QString> allTags = this->dbService->getTags();
QHash<QString, bool> fileTags;
QString path = fileInfo.absoluteFilePath();
for(const QString &fileTag : this->dbService->getTagsForPath(path))
{
fileTags[fileTag] = true;
}
for(const QString &tag : allTags)
{
QCheckBox *checkBox = new QCheckBox(tagMenu);
QWidgetAction *checkableAction = new QWidgetAction(tagMenu);
checkableAction->setDefaultWidget(checkBox);
checkBox->setText(tag);
checkBox->setChecked(fileTags.contains(tag));
tagMenu->addAction(checkableAction);
connect(checkBox, &QCheckBox::stateChanged, this,
[this, checkBox, path]
{
QVector<QString> currentTags = this->dbService->getTagsForPath(path);
QString checkBoxText = checkBox->text();
if(checkBox->isChecked())
{
if(!this->tagManager->addTagsToPath(path, {checkBoxText}))
{
QMessageBox::critical(this, "Error while adding tag",
"An error occured while trying to add the tag");
}
}
else
{
if(!this->tagManager->removeTagsForPath(path, {checkBoxText}))
{
QMessageBox::critical(this, "Error while removing tag",
"An error occured while trying to remove the tag");
}
}
});
}
tagMenu->addAction("Add new tags", this,
[this, path]
{
bool ok;
QString text =
QInputDialog::getText(this, tr("Enter new tags"), tr("New tags (comma separated):"),
QLineEdit::Normal, "", &ok);
if(ok && !this->tagManager->addTagsToPath(path, text, ','))
{
QMessageBox::critical(this, "Error while trying to add tags",
"An error occured while trying to add tags");
}
});
}
void MainWindow::openDocument(QString path, int num)
@ -1107,7 +1048,7 @@ void MainWindow::openFile(QString path)
QDesktopServices::openUrl(QUrl::fromLocalFile(path));
}
void MainWindow::treeSearchItemActivated(QTreeWidgetItem *item, int /*i*/)
void MainWindow::treeSearchItemActivated(QTreeWidgetItem *item, int i)
{
openFile(item->text(1));
}
@ -1121,22 +1062,22 @@ void MainWindow::showSearchResultsContextMenu(const QPoint &point)
}
QFileInfo pathinfo(item->text(1));
QMenu menu("SearchResults", this);
createSearchResultMenu(menu, pathinfo);
createSearchResutlMenu(menu, pathinfo);
menu.exec(QCursor::pos());
}
MainWindow::~MainWindow()
{
syncerThread.terminate();
ipcClientThread.terminate();
delete this->indexSyncer;
delete this->dbService;
delete this->dbFactory;
delete this->indexer;
delete this->tagManager;
delete ui;
}
void MainWindow::closeEvent(QCloseEvent * /*event*/)
void MainWindow::closeEvent(QCloseEvent *event)
{
QStringList list = this->searchHistory.toList();
QSettings settings;

Wyświetl plik

@ -12,9 +12,8 @@
#include <QProgressDialog>
#include "../shared/looqsquery.h"
#include "../shared/indexsyncer.h"
#include "previewcoordinator.h"
#include "ipcpreviewclient.h"
#include "indexer.h"
#include "tagmanager.h"
namespace Ui
{
class MainWindow;
@ -28,9 +27,8 @@ class MainWindow : public QMainWindow
DatabaseFactory *dbFactory;
SqliteDbService *dbService;
Ui::MainWindow *ui;
PreviewCoordinator previewCoordinator;
IPCPreviewClient ipcPreviewClient;
QThread ipcClientThread;
QThread syncerThread;
Indexer *indexer;
IndexSyncer *indexSyncer;
@ -38,15 +36,18 @@ class MainWindow : public QMainWindow
QFileIconProvider iconProvider;
QSqlDatabase db;
QFutureWatcher<QVector<SearchResult>> searchWatcher;
QVector<SearchResult> previewableSearchResults;
LooqsQuery contentSearchQuery;
QVector<QString> searchHistory;
TagManager *tagManager;
int currentSearchHistoryIndex = 0;
QString currentSavedSearchText;
QHash<QString, int> previewOrder; /* Quick lookup for the order a preview should have */
QMap<int, QWidget *>
previewWidgetOrderCache /* Saves those that arrived out of order to be inserted later at the correct pos */;
bool previewDirty = false;
int previewsPerPage = 20;
unsigned int processedPdfPreviews = 0;
unsigned int currentPreviewGeneration = 1;
void connectSignals();
void makePreviews(int page);
@ -55,20 +56,20 @@ class MainWindow : public QMainWindow
void keyPressEvent(QKeyEvent *event) override;
void handleSearchResults(const QVector<SearchResult> &results);
void handleSearchError(QString error);
void createSearchResultMenu(QMenu &menu, const QFileInfo &fileInfo);
void createSearchResutlMenu(QMenu &menu, const QFileInfo &fileInfo);
void openDocument(QString path, int num);
void openFile(QString path);
void initSettingsTabs();
int currentSelectedScale();
void processShortcut(int key);
bool eventFilter(QObject *object, QEvent *event) override;
bool eventFilter(QObject *object, QEvent *event);
private slots:
void lineEditReturnPressed();
void treeSearchItemActivated(QTreeWidgetItem *item, int i);
void showSearchResultsContextMenu(const QPoint &point);
void tabChanged();
void previewReceived();
void previewReceived(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration);
void comboScaleChanged(int i);
void spinPreviewPageValueChanged(int val);
void startIndexing();

Wyświetl plik

@ -18,13 +18,16 @@
<item>
<widget class="QLineEdit" name="txtSearch"/>
</item>
<item>
<layout class="QHBoxLayout" name="horizontalLayout_3"/>
</item>
<item>
<widget class="QTabWidget" name="tabWidget">
<property name="tabPosition">
<enum>QTabWidget::South</enum>
</property>
<property name="currentIndex">
<number>2</number>
<number>1</number>
</property>
<widget class="QWidget" name="resultsTab">
<attribute name="title">
@ -79,7 +82,7 @@
<x>0</x>
<y>0</y>
<width>1244</width>
<height>641</height>
<height>633</height>
</rect>
</property>
<layout class="QHBoxLayout" name="horizontalLayout"/>
@ -192,6 +195,62 @@
</attribute>
<layout class="QGridLayout" name="gridLayout">
<item row="6" column="0">
<widget class="QLineEdit" name="txtIgnorePatterns"/>
</item>
<item row="11" column="0">
<widget class="QPushButton" name="btnStartIndexing">
<property name="text">
<string>Start indexing</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QGroupBox" name="groupBoxPaths">
<property name="title">
<string>Add paths to scan</string>
</property>
<layout class="QGridLayout" name="gridLayout_2">
<item row="1" column="0">
<widget class="QLineEdit" name="txtPathScanAdd"/>
</item>
<item row="3" column="0" colspan="5">
<widget class="QListWidget" name="lstPaths"/>
</item>
<item row="1" column="3">
<widget class="QToolButton" name="btnDeletePath">
<property name="enabled">
<bool>false</bool>
</property>
<property name="text">
<string>Delete</string>
</property>
</widget>
</item>
<item row="1" column="1">
<widget class="QPushButton" name="btnChoosePath">
<property name="text">
<string>...</string>
</property>
</widget>
</item>
<item row="1" column="2">
<widget class="QPushButton" name="btnAddPath">
<property name="text">
<string>Add</string>
</property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="5" column="0">
<widget class="QLabel" name="label">
<property name="text">
<string>Ignore patterns, separated by ';'. Example: *.js;*Downloads*</string>
</property>
</widget>
</item>
<item row="9" column="0">
<widget class="QGroupBox" name="groupBoxIndexProgress">
<property name="contextMenuPolicy">
<enum>Qt::PreventContextMenu</enum>
@ -393,108 +452,6 @@
</layout>
</widget>
</item>
<item row="2" column="0">
<widget class="QGroupBox" name="groupBoxIndexOptions">
<property name="title">
<string>Index options</string>
</property>
<layout class="QVBoxLayout" name="verticalLayout_11">
<item>
<widget class="QLabel" name="label">
<property name="text">
<string>Ignore patterns, separated by ';'. Example: *.js;*Downloads*:</string>
</property>
</widget>
</item>
<item>
<widget class="QLineEdit" name="txtIgnorePatterns"/>
</item>
<item>
<widget class="Line" name="line">
<property name="orientation">
<enum>Qt::Horizontal</enum>
</property>
</widget>
</item>
<item>
<widget class="QRadioButton" name="radioIndexEverything">
<property name="text">
<string>Index everything (metadata + file content)</string>
</property>
<property name="checked">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="chkFillContentForContentless">
<property name="enabled">
<bool>true</bool>
</property>
<property name="text">
<string>Index content for files previously indexed without content</string>
</property>
<property name="checked">
<bool>false</bool>
</property>
</widget>
</item>
<item>
<widget class="QRadioButton" name="radioMetadataOnly">
<property name="text">
<string>Index metadata only, don't process content of files</string>
</property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="8" column="0">
<widget class="QPushButton" name="btnStartIndexing">
<property name="text">
<string>Start indexing</string>
</property>
</widget>
</item>
<item row="1" column="0">
<widget class="QGroupBox" name="groupBoxPaths">
<property name="title">
<string>Add paths to scan</string>
</property>
<layout class="QGridLayout" name="gridLayout_2">
<item row="1" column="0">
<widget class="QLineEdit" name="txtPathScanAdd"/>
</item>
<item row="3" column="0" colspan="5">
<widget class="QListWidget" name="lstPaths"/>
</item>
<item row="1" column="3">
<widget class="QToolButton" name="btnDeletePath">
<property name="enabled">
<bool>false</bool>
</property>
<property name="text">
<string>Delete</string>
</property>
</widget>
</item>
<item row="1" column="1">
<widget class="QPushButton" name="btnChoosePath">
<property name="text">
<string>...</string>
</property>
</widget>
</item>
<item row="1" column="2">
<widget class="QPushButton" name="btnAddPath">
<property name="text">
<string>Add</string>
</property>
</widget>
</item>
</layout>
</widget>
</item>
</layout>
</widget>
<widget class="QWidget" name="settingsTab">
@ -744,22 +701,5 @@
</widget>
<layoutdefault spacing="6" margin="11"/>
<resources/>
<connections>
<connection>
<sender>radioIndexEverything</sender>
<signal>toggled(bool)</signal>
<receiver>chkFillContentForContentless</receiver>
<slot>setEnabled(bool)</slot>
<hints>
<hint type="sourcelabel">
<x>639</x>
<y>464</y>
</hint>
<hint type="destinationlabel">
<x>639</x>
<y>497</y>
</hint>
</hints>
</connection>
</connections>
<connections/>
</ui>

Wyświetl plik

@ -1,97 +0,0 @@
#include "previewcoordinator.h"
#include <QFileInfo>
PreviewCoordinator::PreviewCoordinator()
{
this->ipcPreviewClient.moveToThread(&this->ipcClientThread);
connect(&ipcPreviewClient, &IPCPreviewClient::previewReceived, this, &PreviewCoordinator::handleReceivedPreview,
Qt::QueuedConnection);
connect(&ipcPreviewClient, &IPCPreviewClient::finished, this, [&] { emit completedGeneration(); });
connect(this, &PreviewCoordinator::ipcStartGeneration, &ipcPreviewClient, &IPCPreviewClient::startGeneration,
Qt::QueuedConnection);
this->ipcClientThread.start();
}
void PreviewCoordinator::init(const QVector<SearchResult> &searchResults)
{
this->previewableSearchResults.clear();
for(const SearchResult &result : searchResults)
{
if(result.wasContentSearch)
{
QString path = result.fileData.absPath;
// HACK until we can preview them properly
if(path.endsWith(".html") || path.endsWith(".htm"))
{
continue;
}
QFileInfo info{path};
if(info.exists())
{
this->previewableSearchResults.append(result);
}
}
}
}
void PreviewCoordinator::setSocketPath(QString socketPath)
{
this->socketPath = socketPath;
this->ipcPreviewClient.setSocketPath(socketPath);
}
int PreviewCoordinator::previewableCount() const
{
return this->previewableSearchResults.count();
}
QSharedPointer<PreviewResult> PreviewCoordinator::resultAt(int index)
{
if(this->previewResults.size() > index)
{
return {this->previewResults[index]};
}
return {nullptr};
}
const QVector<SearchResult> &PreviewCoordinator::getPreviewableSearchResults() const
{
return this->previewableSearchResults;
}
void PreviewCoordinator::handleReceivedPreview(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration)
{
if(previewGeneration < this->currentPreviewGeneration)
{
return;
}
if(!preview.isNull() && preview->hasPreview())
{
QString docPath = preview->getDocumentPath();
auto previewPage = preview->getPage();
int pos = previewOrder[docPath + QString::number(previewPage)];
this->previewResults[pos] = preview;
emit previewReady();
}
}
void PreviewCoordinator::startGeneration(RenderConfig config, const QVector<RenderTarget> &targets)
{
++this->currentPreviewGeneration;
this->previewOrder.clear();
this->previewResults.clear();
this->previewResults.resize(targets.size());
this->previewResults.fill(nullptr);
int i = 0;
for(const RenderTarget &target : targets)
{
this->previewOrder[target.path + QString::number(target.page)] = i++;
}
emit ipcStartGeneration(config, targets);
}

Wyświetl plik

@ -1,48 +0,0 @@
#ifndef PREVIEWCOORDINATOR_H
#define PREVIEWCOORDINATOR_H
#include <QVector>
#include <QObject>
#include <QThread>
#include "searchresult.h"
#include "previewresult.h"
#include "ipcpreviewclient.h"
#include "rendertarget.h"
class PreviewCoordinator : public QObject
{
Q_OBJECT
private:
QThread ipcClientThread;
IPCPreviewClient ipcPreviewClient;
QString socketPath;
QVector<QSharedPointer<PreviewResult>> previewResults;
QVector<SearchResult> previewableSearchResults;
unsigned int currentPreviewGeneration = 1;
/* Quick lookup table for the order a preview should have */
QHash<QString, int> previewOrder;
public:
PreviewCoordinator();
void init(const QVector<SearchResult> &searchResults);
int previewableCount() const;
const QVector<SearchResult> &getPreviewableSearchResults() const;
QSharedPointer<PreviewResult> resultAt(int index);
void setSocketPath(QString socketPath);
public slots:
void startGeneration(RenderConfig config, const QVector<RenderTarget> &targets);
void handleReceivedPreview(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration);
signals:
void previewReady();
void completedGeneration();
void error(QString);
void ipcStartGeneration(RenderConfig config, const QVector<RenderTarget> &targets);
};
#endif // PREVIEWCOORDINATOR_H

Wyświetl plik

@ -24,7 +24,7 @@ QSharedPointer<PreviewResult> PreviewGeneratorOdt::generate(RenderConfig config,
throw LooqsGeneralException("Error while reading content.xml of " + documentPath);
}
TagStripperProcessor tsp;
QString content = tsp.process(entireContent).constFirst().content;
QString content = tsp.process(entireContent).first().content;
PreviewGeneratorPlainText plainTextGenerator;
result->setText(plainTextGenerator.generatePreviewText(content, config, info.fileName()));

Wyświetl plik

@ -20,8 +20,6 @@ Poppler::Document *PreviewGeneratorPdf::document(QString path)
return nullptr;
}
result->setRenderHint(Poppler::Document::TextAntialiasing);
result->setRenderHint(Poppler::Document::TextHinting);
result->setRenderHint(Poppler::Document::TextSlightHinting);
locker.relock();
documentcache.insert(path, result);

Wyświetl plik

@ -195,7 +195,7 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
int totalWordsA = 0;
int differentWordsB = 0;
int totalWordsB = 0;
for(int count : qAsConst(a.wordCountMap))
for(int count : a.wordCountMap.values())
{
if(count > 0)
{
@ -203,7 +203,7 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
}
totalWordsA += count;
}
for(int count : qAsConst(b.wordCountMap))
for(int count : b.wordCountMap.values())
{
if(count > 0)
{
@ -246,21 +246,17 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
totalWordCountMap[it->first] = totalWordCountMap.value(it->first, 0) + it->second;
}
}
if(!resultText.isEmpty())
if(isTruncated)
{
if(isTruncated)
{
header += "(truncated) ";
}
for(QString &word : config.wordsToHighlight)
{
header += word + ": " + QString::number(totalWordCountMap[word]) + " ";
}
header += "<hr>";
resultText = header + resultText;
header += "(truncated) ";
}
return resultText;
for(QString &word : config.wordsToHighlight)
{
header += word + ": " + QString::number(totalWordCountMap[word]) + " ";
}
header += "<hr>";
return header + resultText;
}
QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig config, QString documentPath,

Wyświetl plik

@ -25,22 +25,10 @@ SaveFileResult FileSaver::addFile(QString path)
QString absPath = info.absoluteFilePath();
auto mtime = info.lastModified().toSecsSinceEpoch();
bool exists = false;
if(this->fileSaverOptions.fillExistingContentless)
{
exists = this->dbService->fileExistsInDatabase(absPath, mtime, 'c');
}
else
{
exists = this->dbService->fileExistsInDatabase(absPath, mtime);
}
if(exists)
if(this->dbService->fileExistsInDatabase(absPath, mtime))
{
return SKIPPED;
}
return saveFile(info);
}
@ -50,17 +38,18 @@ SaveFileResult FileSaver::updateFile(QString path)
return saveFile(info);
}
int FileSaver::addFiles(const QVector<QString> paths)
int FileSaver::addFiles(const QVector<QString> paths, bool keepGoing, bool verbose)
{
return processFiles(paths, std::bind(&FileSaver::addFile, this, std::placeholders::_1));
return processFiles(paths, std::bind(&FileSaver::addFile, this, std::placeholders::_1), keepGoing, verbose);
}
int FileSaver::updateFiles(const QVector<QString> paths)
int FileSaver::updateFiles(const QVector<QString> paths, bool keepGoing, bool verbose)
{
return processFiles(paths, std::bind(&FileSaver::updateFile, this, std::placeholders::_1));
return processFiles(paths, std::bind(&FileSaver::updateFile, this, std::placeholders::_1), keepGoing, verbose);
}
int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc)
int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc,
bool keepGoing, bool verbose)
{
std::atomic<bool> terminate{false};
std::atomic<int> processedCount{0};
@ -71,7 +60,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
{
return;
}
if(this->fileSaverOptions.verbose)
if(verbose)
{
Logger::info() << "Processing " << path << Qt::endl;
}
@ -79,7 +68,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
if(result == DBFAIL || result == PROCESSFAIL)
{
Logger::error() << "Failed to process " << path << Qt::endl;
if(!this->fileSaverOptions.keepGoing)
if(!keepGoing)
{
terminate = true;
}
@ -87,7 +76,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
else
{
++processedCount;
if(this->fileSaverOptions.verbose)
if(verbose)
{
if(result == SKIPPED)
{
@ -131,26 +120,11 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
{
if(canonicalPath.startsWith(excludedPath))
{
if(this->fileSaverOptions.verbose)
{
Logger::info() << "Skipped due to excluded path";
}
return SKIPPED;
}
}
bool mustFillContent = this->fileSaverOptions.fillExistingContentless;
if(!mustFillContent)
{
mustFillContent = !this->fileSaverOptions.metadataOnly;
if(mustFillContent)
{
auto filetype = this->dbService->queryFileType(fileInfo.absolutePath());
mustFillContent = !filetype.has_value() || filetype.value() == 'c';
}
}
if(fileInfo.size() > 0 && mustFillContent)
if(fileInfo.size() > 0)
{
QProcess process;
QStringList args;
@ -185,7 +159,7 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
}
}
}
SaveFileResult result = this->dbService->saveFile(fileInfo, pageData, this->fileSaverOptions.metadataOnly);
SaveFileResult result = this->dbService->saveFile(fileInfo, pageData);
if(result == OK && processorReturnCode == OK_WASEMPTY)
{
return OK_WASEMPTY;

Wyświetl plik

@ -2,7 +2,6 @@
#define FILESAVER_H
#include <QSqlDatabase>
#include <QFileInfo>
#include "filesaveroptions.h"
#include "pagedata.h"
#include "filedata.h"
#include "sqlitedbservice.h"
@ -12,21 +11,16 @@ class FileSaver
private:
SqliteDbService *dbService;
QStringList excludedPaths = Common::excludedPaths();
FileSaverOptions fileSaverOptions;
public:
FileSaver(SqliteDbService &dbService);
SaveFileResult addFile(QString path);
SaveFileResult updateFile(QString path);
SaveFileResult saveFile(const QFileInfo &fileInfo);
int processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc);
int addFiles(const QVector<QString> paths);
int updateFiles(const QVector<QString> paths);
void setFileSaverOptions(FileSaverOptions options)
{
this->fileSaverOptions = options;
}
int processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc,
bool keepGoing, bool verbose);
int addFiles(const QVector<QString> paths, bool keepGoing, bool verbose);
int updateFiles(const QVector<QString> paths, bool keepGoing, bool verbose);
};
#endif // FILESAVER_H

Wyświetl plik

@ -1,14 +0,0 @@
#ifndef FILESAVEROPTIONS_H
#define FILESAVEROPTIONS_H
class FileSaverOptions
{
public:
bool verbose = false;
bool keepGoing = false;
bool metadataOnly = false;
/* Whether those previously explicitly without content should be filled */
bool fillExistingContentless = false;
};
#endif // FILESAVEROPTIONS_H

Wyświetl plik

@ -12,7 +12,6 @@ FileScanWorker::FileScanWorker(SqliteDbService &db, ConcurrentQueue<QString> &qu
void FileScanWorker::run()
{
FileSaver saver{*this->dbService};
saver.setFileSaverOptions(this->fileSaverOptions);
auto paths = queue->dequeue(batchsize);
for(QString &path : paths)
{
@ -21,18 +20,11 @@ void FileScanWorker::run()
{
sfr = saver.addFile(path);
}
catch(LooqsGeneralException &e)
{
Logger::error() << e.message << Qt::endl;
sfr = PROCESSFAIL;
}
catch(std::exception &e)
{
Logger::error() << e.what() << Qt::endl;
Logger::error() << e.what();
sfr = PROCESSFAIL; // well...
}
emit result({path, sfr});
if(stopToken->load(std::memory_order_relaxed)) // TODO: relaxed should suffice here, but recheck
{
@ -42,8 +34,3 @@ void FileScanWorker::run()
}
emit finished();
}
void FileScanWorker::setFileSaverOptions(FileSaverOptions options)
{
this->fileSaverOptions = options;
}

Wyświetl plik

@ -15,14 +15,12 @@ class FileScanWorker : public QObject, public QRunnable
protected:
SqliteDbService *dbService;
ConcurrentQueue<QString> *queue;
FileSaverOptions fileSaverOptions;
int batchsize;
std::atomic<bool> *stopToken;
public:
FileScanWorker(SqliteDbService &db, ConcurrentQueue<QString> &queue, int batchsize, std::atomic<bool> &stopToken);
void run() override;
void setFileSaverOptions(FileSaverOptions options);
signals:
void result(FileScanResult);
void finished();

Wyświetl plik

@ -73,6 +73,16 @@ void Indexer::setTargetPaths(QVector<QString> pathsToScan)
this->pathsToScan = pathsToScan;
}
void Indexer::setVerbose(bool verbose)
{
this->verbose = verbose;
}
void Indexer::setKeepGoing(bool keepGoing)
{
this->keepGoing = keepGoing;
}
void Indexer::requestCancellation()
{
this->dirScanner->cancel();
@ -98,7 +108,6 @@ void Indexer::launchWorker(ConcurrentQueue<QString> &queue, int batchsize)
FileScanWorker *runnable = new FileScanWorker(*this->db, queue, batchsize, this->workerCancellationToken);
connect(runnable, &FileScanWorker::result, this, &Indexer::processFileScanResult);
connect(runnable, &FileScanWorker::finished, this, &Indexer::processFinishedWorker);
runnable->setFileSaverOptions(this->fileSaverOptions);
++this->runningWorkers;
QThreadPool::globalInstance()->start(runnable);
}
@ -111,6 +120,24 @@ void Indexer::dirScanProgress(int current, int total)
void Indexer::processFileScanResult(FileScanResult result)
{
if(isErrorSaveFileResult(result.second))
{
this->currentIndexResult.results.append(result);
if(!keepGoing)
{
this->requestCancellation();
emit finished();
return;
}
}
else
{
if(verbose)
{
this->currentIndexResult.results.append(result);
}
}
/* TODO: OK_WASEMPTY might need a special list */
if(result.second == OK || result.second == OK_WASEMPTY)
{
@ -125,24 +152,6 @@ void Indexer::processFileScanResult(FileScanResult result)
++this->currentIndexResult.erroredPaths;
}
if(isErrorSaveFileResult(result.second))
{
this->currentIndexResult.results.append(result);
if(!this->fileSaverOptions.keepGoing)
{
this->requestCancellation();
emit finished();
return;
}
}
else
{
if(this->fileSaverOptions.verbose)
{
this->currentIndexResult.results.append(result);
}
}
QTime currentTime = QTime::currentTime();
if(currentScanProcessedCount++ == progressReportThreshold || this->lastProgressReportTime.secsTo(currentTime) >= 10)
{
@ -166,13 +175,3 @@ void Indexer::processFinishedWorker()
emit finished();
}
}
void Indexer::setFileSaverOptions(FileSaverOptions options)
{
this->fileSaverOptions = options;
}
void Indexer::setProgressReportThreshold(int threshold)
{
this->progressReportThreshold = threshold;
}

Wyświetl plik

@ -52,7 +52,8 @@ class Indexer : public QObject
{
Q_OBJECT
protected:
FileSaverOptions fileSaverOptions;
bool verbose = false;
bool keepGoing = true;
SqliteDbService *db;
int progressReportThreshold = 50;
@ -79,10 +80,8 @@ class Indexer : public QObject
void beginIndexing();
void setIgnorePattern(QStringList ignorePattern);
void setTargetPaths(QVector<QString> pathsToScan);
void setFileSaverOptions(FileSaverOptions options);
void setProgressReportThreshold(int threshold);
void setVerbose(bool verbose);
void setKeepGoing(bool keepGoing);
void requestCancellation();

Wyświetl plik

@ -7,16 +7,21 @@ IndexSyncer::IndexSyncer(SqliteDbService &dbService)
this->dbService = &dbService;
}
void IndexSyncer::setFileSaverOptions(FileSaverOptions options)
{
fileSaverOptions = options;
}
void IndexSyncer::setDryRun(bool dryRun)
{
this->dryRun = dryRun;
}
void IndexSyncer::setVerbose(bool verbose)
{
this->verbose = verbose;
}
void IndexSyncer::setKeepGoing(bool keepGoing)
{
this->keepGoing = keepGoing;
}
void IndexSyncer::setRemoveDeletedFromIndex(bool removeDeletedFromIndex)
{
this->removeDeletedFromIndex = removeDeletedFromIndex;
@ -30,7 +35,7 @@ void IndexSyncer::setPattern(QString pattern)
void IndexSyncer::sync()
{
this->stopToken.store(false, std::memory_order_relaxed);
FileSaver saver(*this->dbService);
QVector<FileData> files;
int offset = 0;
int limit = 10000;
@ -82,7 +87,7 @@ void IndexSyncer::sync()
if(!this->dbService->deleteFile(fileData.absPath))
{
emit error("Error: Failed to delete " + fileData.absPath + " from the index");
if(!this->fileSaverOptions.keepGoing)
if(!this->keepGoing)
{
emit finished(totalUpdatesFilesCount, totalDeletedFilesCount, totalErroredFilesCount);
return;
@ -99,15 +104,13 @@ void IndexSyncer::sync()
}
}
FileSaver saver(*this->dbService);
saver.setFileSaverOptions(this->fileSaverOptions);
unsigned int updatedFilesCount = saver.updateFiles(filePathsToUpdate);
unsigned int updatedFilesCount = saver.updateFiles(filePathsToUpdate, keepGoing, verbose);
unsigned int shouldHaveUpdatedCount = static_cast<unsigned int>(filePathsToUpdate.size());
if(updatedFilesCount != shouldHaveUpdatedCount)
{
totalErroredFilesCount += (shouldHaveUpdatedCount - updatedFilesCount);
if(!this->fileSaverOptions.keepGoing)
if(!keepGoing)
{
QString errorMsg = QString("Failed to update all files selected for updating in this batch. Updated") +
updatedFilesCount + "out of" + shouldHaveUpdatedCount + "selected for updating";

Wyświetl plik

@ -1,15 +1,16 @@
#ifndef INDEXSYNCER_H
#define INDEXSYNCER_H
#include "sqlitedbservice.h"
#include "filesaveroptions.h"
class IndexSyncer : public QObject
{
Q_OBJECT
private:
SqliteDbService *dbService = nullptr;
FileSaverOptions fileSaverOptions;
bool keepGoing = true;
bool removeDeletedFromIndex = true;
bool dryRun = false;
bool verbose = false;
QString pattern;
std::atomic<bool> stopToken{false};
@ -17,12 +18,12 @@ class IndexSyncer : public QObject
public:
IndexSyncer(SqliteDbService &dbService);
void setFileSaverOptions(FileSaverOptions options);
public slots:
void sync();
void cancel();
void setDryRun(bool dryRun);
void setVerbose(bool verbose);
void setKeepGoing(bool keepGoing);
void setRemoveDeletedFromIndex(bool removeDeletedFromIndex);
void setPattern(QString pattern);

Wyświetl plik

@ -7,7 +7,6 @@
#include <optional>
#include <algorithm>
#include "looqsquery.h"
#include "looqsgeneralexception.h"
const QVector<Token> &LooqsQuery::getTokens() const
{
@ -181,9 +180,8 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
QStringList loneWords;
LooqsQuery result;
static QRegularExpression rx(
"((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([^\\s])+)|(?<boolean>AND|OR)"
"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[^\\s]+))");
QRegularExpression rx("((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([^\\s])+)|(?<boolean>AND|OR)"
"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[^\\s]+))");
QRegularExpressionMatchIterator i = rx.globalMatch(expression);
auto previousWasBool = [&result] { return !result.tokens.empty() && ((result.tokens.last().type & BOOL) == BOOL); };
auto previousWas = [&result](TokenType t) { return !result.tokens.empty() && (result.tokens.last().type == t); };
@ -285,10 +283,6 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
{
tokenType = FILTER_CONTENT_PAGE;
}
else if(filtername == "t" || filtername == "tag")
{
tokenType = FILTER_TAG_ASSIGNED;
}
// TODO: given this is not really a "filter", this feels slightly misplaced here
else if(filtername == "sort")
{

Wyświetl plik

@ -2,6 +2,7 @@
#define LOOQSQUERY_H
#include <QString>
#include <QVector>
#include "looqsgeneralexception.h"
#include "token.h"
/* Fields that can be queried or sorted */
enum QueryField
@ -45,7 +46,7 @@ class LooqsQuery
void addToken(Token t);
void updateTokensMask()
{
for(const Token &t : qAsConst(tokens))
for(const Token &t : tokens)
{
this->tokensMask |= t.type;
}
@ -91,6 +92,14 @@ class LooqsQuery
this->sortConditions = sortConditions;
updateTokensMask();
}
LooqsQuery(const LooqsQuery &o)
{
this->tokens = o.tokens;
this->sortConditions = o.sortConditions;
this->tokensMask = o.tokensMask;
this->limit = o.limit;
}
};
#endif // LOOQSQUERY_H

Wyświetl plik

@ -1,6 +0,0 @@
CREATE TABLE tag(id integer PRIMARY KEY, name varchar(128) UNIQUE);
CREATE TABLE filetag(fileid integer, tagid integer);
CREATE INDEX filetag_fileid ON filetag(fileid);
CREATE INDEX tag_id ON tag(id);
CREATE INDEX file_path ON file ( path );
UPDATE file SET filetype='c' WHERE filetype='f';

Wyświetl plik

@ -4,6 +4,5 @@
<file>2.sql</file>
<file>3.sql</file>
<file>4.sql</file>
<file>5.sql</file>
</qresource>
</RCC>

Wyświetl plik

@ -10,7 +10,7 @@ class NothingProcessor : public Processor
NothingProcessor();
public:
QVector<PageData> process(const QByteArray & /*data*/) const override
QVector<PageData> process(const QByteArray &data) const override
{
return {};
}

Wyświetl plik

@ -3,7 +3,7 @@
#include "odtprocessor.h"
#include "tagstripperprocessor.h"
QVector<PageData> OdtProcessor::process(const QByteArray & /*data*/) const
QVector<PageData> OdtProcessor::process(const QByteArray &data) const
{
throw LooqsGeneralException("Not implemented yet");
}

Wyświetl plik

@ -1,3 +1,5 @@
#include "paralleldirscanner.h"
#include <QRunnable>
#include <QMutex>
#include <QDirIterator>
@ -5,7 +7,7 @@
#include <QThreadPool>
#include <functional>
#include "dirscanworker.h"
#include "paralleldirscanner.h"
#include "logger.h"
ParallelDirScanner::ParallelDirScanner()
{

Wyświetl plik

@ -60,7 +60,6 @@ SOURCES += sqlitesearch.cpp \
processor.cpp \
sandboxedprocessor.cpp \
sqlitedbservice.cpp \
tagmanager.cpp \
tagstripperprocessor.cpp \
utils.cpp \
../submodules/exile.h/exile.c \
@ -75,7 +74,6 @@ HEADERS += sqlitesearch.h \
encodingdetector.h \
filedata.h \
filesaver.h \
filesaveroptions.h \
filescanworker.h \
indexer.h \
indexsyncer.h \
@ -94,7 +92,6 @@ HEADERS += sqlitesearch.h \
savefileresult.h \
searchresult.h \
sqlitedbservice.h \
tagmanager.h \
tagstripperprocessor.h \
token.h \
common.h \

Wyświetl plik

@ -2,10 +2,25 @@
#include <QFileInfo>
#include <QDateTime>
#include <QSqlError>
#include "looqsgeneralexception.h"
#include "sqlitedbservice.h"
#include "filedata.h"
#include "logger.h"
bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime)
{
auto query = QSqlQuery(dbFactory->forCurrentThread());
query.prepare("SELECT 1 FROM file WHERE path = ? and mtime = ?");
query.addBindValue(path);
query.addBindValue(mtime);
if(!query.exec())
{
throw LooqsGeneralException("Error while trying to query for file existance: " + query.lastError().text());
}
if(!query.next())
{
return false;
}
return query.value(0).toBool();
}
QVector<SearchResult> SqliteDbService::search(const LooqsQuery &query)
{
@ -14,29 +29,20 @@ QVector<SearchResult> SqliteDbService::search(const LooqsQuery &query)
return searcher.search(query);
}
std::optional<QChar> SqliteDbService::queryFileType(QString absPath)
{
auto query = exec("SELECT filetype FROM file WHERE path = ?", {absPath});
if(!query.next())
{
return {};
}
return query.value(0).toChar();
}
bool SqliteDbService::fileExistsInDatabase(QString path)
{
return execBool("SELECT 1 FROM file WHERE path = ?", {path});
}
bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime)
{
return execBool("SELECT 1 FROM file WHERE path = ? AND mtime = ?", {path, mtime});
}
bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime, QChar fileType)
{
return execBool("SELECT 1 FROM file WHERE path = ? AND mtime = ? AND filetype = ?", {path, mtime, fileType});
auto query = QSqlQuery(dbFactory->forCurrentThread());
query.prepare("SELECT 1 FROM file WHERE path = ?");
query.addBindValue(path);
if(!query.exec())
{
throw LooqsGeneralException("Error while trying to query for file existance: " + query.lastError().text());
}
if(!query.next())
{
return false;
}
return query.value(0).toBool();
}
SqliteDbService::SqliteDbService(DatabaseFactory &dbFactory)
@ -104,117 +110,6 @@ unsigned int SqliteDbService::getFiles(QVector<FileData> &results, QString wildC
return processedRows;
}
QVector<QString> SqliteDbService::getTags()
{
QVector<QString> result;
auto query = QSqlQuery(dbFactory->forCurrentThread());
query.prepare("SELECT name FROM tag ORDER by name ASC");
query.setForwardOnly(true);
if(!query.exec())
{
throw LooqsGeneralException("Error while trying to retrieve tags from database: " + query.lastError().text());
}
while(query.next())
{
QString tagname = query.value(0).toString();
result.append(tagname);
}
return result;
}
QVector<QString> SqliteDbService::getTagsForPath(QString path)
{
QVector<QString> result;
auto query = QSqlQuery(dbFactory->forCurrentThread());
query.prepare("SELECT name FROM tag INNER JOIN filetag ON tag.id = filetag.tagid INNER JOIN file ON filetag.fileid "
"= file.id WHERE file.path = ? ORDER BY name ASC");
query.addBindValue(path);
query.setForwardOnly(true);
if(!query.exec())
{
throw LooqsGeneralException("Error while trying to retrieve tags from database: " + query.lastError().text());
}
while(query.next())
{
QString tagname = query.value(0).toString();
result.append(tagname);
}
return result;
}
QVector<QString> SqliteDbService::getPathsForTag(QString tag)
{
QVector<QString> result;
auto query = QSqlQuery(dbFactory->forCurrentThread());
query.prepare(
"SELECT file.path FROM tag INNER JOIN filetag ON tag.id = filetag.tagid INNER JOIN file ON filetag.fileid "
"= file.id WHERE tag.name = ?");
query.addBindValue(tag.toLower());
query.setForwardOnly(true);
if(!query.exec())
{
throw LooqsGeneralException("Error while trying to retrieve paths from database: " + query.lastError().text());
}
while(query.next())
{
QString path = query.value(0).toString();
result.append(path);
}
return result;
}
bool SqliteDbService::setTags(QString path, const QSet<QString> &tags)
{
QSqlDatabase db = dbFactory->forCurrentThread();
if(!db.transaction())
{
Logger::error() << "Failed to open transaction for " << path << " : " << db.lastError() << Qt::endl;
return false;
}
QSqlQuery deletionQuery = QSqlQuery(db);
deletionQuery.prepare("DELETE FROM filetag WHERE fileid = (SELECT id FROM file WHERE path = ?)");
deletionQuery.addBindValue(path);
if(!deletionQuery.exec())
{
db.rollback();
Logger::error() << "Failed to delete existing tags " << deletionQuery.lastError() << Qt::endl;
return false;
}
for(const QString &tag : tags)
{
QSqlQuery tagQuery = QSqlQuery(db);
tagQuery.prepare("INSERT OR IGNORE INTO tag (name) VALUES(?)");
tagQuery.addBindValue(tag.toLower());
if(!tagQuery.exec())
{
db.rollback();
Logger::error() << "Failed to insert tag " << tagQuery.lastError() << Qt::endl;
return false;
}
QSqlQuery fileTagQuery(db);
fileTagQuery.prepare(
"INSERT INTO filetag(fileid, tagid) VALUES((SELECT id FROM file WHERE path = ?), (SELECT id "
"FROM tag WHERE name = ?))");
fileTagQuery.bindValue(0, path);
fileTagQuery.bindValue(1, tag);
if(!fileTagQuery.exec())
{
db.rollback();
Logger::error() << "Failed to assign tag to file" << Qt::endl;
return false;
}
}
if(!db.commit())
{
db.rollback();
Logger::error() << "Failed to commit transaction when saving tags" << Qt::endl;
return false;
}
return true;
}
bool SqliteDbService::insertToFTS(bool useTrigrams, QSqlDatabase &db, int fileid, QVector<PageData> &pageData)
{
QString ftsInsertStatement;
@ -253,40 +148,11 @@ bool SqliteDbService::insertToFTS(bool useTrigrams, QSqlDatabase &db, int fileid
return true;
}
QSqlQuery SqliteDbService::exec(QString querystr, std::initializer_list<QVariant> args)
{
auto query = QSqlQuery(dbFactory->forCurrentThread());
query.prepare(querystr);
for(const QVariant &v : args)
{
query.addBindValue(v);
}
if(!query.exec())
{
throw LooqsGeneralException("Error while exec(): " + query.lastError().text() + " for query: " + querystr);
}
return query;
}
bool SqliteDbService::execBool(QString querystr, std::initializer_list<QVariant> args)
{
auto query = exec(querystr, args);
if(!query.next())
{
return false;
}
return query.value(0).toBool();
}
SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &pageData, bool pathsOnly)
SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &pageData)
{
QString absPath = fileInfo.absoluteFilePath();
auto mtime = fileInfo.lastModified().toSecsSinceEpoch();
QChar fileType = fileInfo.isDir() ? 'd' : 'c';
if(pathsOnly)
{
fileType = 'f';
}
QChar fileType = fileInfo.isDir() ? 'd' : 'f';
QSqlDatabase db = dbFactory->forCurrentThread();
QSqlQuery delQuery(db);
@ -320,23 +186,19 @@ SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &
return DBFAIL;
}
if(!pathsOnly)
int lastid = inserterQuery.lastInsertId().toInt();
if(!insertToFTS(false, db, lastid, pageData))
{
int lastid = inserterQuery.lastInsertId().toInt();
if(!insertToFTS(false, db, lastid, pageData))
{
db.rollback();
Logger::error() << "Failed to insert data to FTS index " << Qt::endl;
return DBFAIL;
}
if(!insertToFTS(true, db, lastid, pageData))
{
db.rollback();
Logger::error() << "Failed to insert data to FTS index " << Qt::endl;
return DBFAIL;
}
db.rollback();
Logger::error() << "Failed to insert data to FTS index " << Qt::endl;
return DBFAIL;
}
if(!insertToFTS(true, db, lastid, pageData))
{
db.rollback();
Logger::error() << "Failed to insert data to FTS index " << Qt::endl;
return DBFAIL;
}
if(!db.commit())
{
db.rollback();
@ -345,123 +207,3 @@ SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &
}
return OK;
}
bool SqliteDbService::addTag(QString tag, QString path)
{
QVector<QString> paths;
paths.append(path);
return addTag(tag, paths);
}
bool SqliteDbService::addTag(QString tag, const QVector<QString> &paths)
{
QSqlDatabase db = dbFactory->forCurrentThread();
QSqlQuery tagQuery(db);
QSqlQuery fileTagQuery(db);
tag = tag.toLower();
tagQuery.prepare("INSERT OR IGNORE INTO tag (name) VALUES(?)");
tagQuery.addBindValue(tag);
fileTagQuery.prepare("INSERT INTO filetag(fileid, tagid) VALUES((SELECT id FROM file WHERE path = ?), (SELECT id "
"FROM tag WHERE name = ?))");
fileTagQuery.bindValue(1, tag);
if(!db.transaction())
{
Logger::error() << "Failed to open transaction to add paths for tag " << tag << " : " << db.lastError()
<< Qt::endl;
return false;
}
if(!tagQuery.exec())
{
db.rollback();
Logger::error() << "Failed INSERT query" << tagQuery.lastError() << Qt::endl;
return false;
}
for(const QString &path : paths)
{
fileTagQuery.bindValue(0, path);
if(!fileTagQuery.exec())
{
db.rollback();
Logger::error() << "Failed to add paths to tag" << Qt::endl;
return false;
}
}
if(!db.commit())
{
db.rollback();
Logger::error() << "Failed to commit tag insertion transaction" << db.lastError() << Qt::endl;
return false;
}
return true;
}
bool SqliteDbService::removePathsForTag(QString tag, const QVector<QString> &paths)
{
QSqlDatabase db = dbFactory->forCurrentThread();
QSqlQuery tagQuery(db);
QSqlQuery fileTagQuery(db);
tag = tag.toLower();
fileTagQuery.prepare(
"DELETE FROM filetag WHERE fileid = (SELECT id FROM file WHERE path = ?) AND tagid = (SELECT id "
"FROM tag WHERE name = ?)");
fileTagQuery.bindValue(1, tag);
for(const QString &path : paths)
{
fileTagQuery.bindValue(0, path);
if(!fileTagQuery.exec())
{
Logger::error() << "An error occured while trying to remove paths from tag assignment" << Qt::endl;
return false;
}
}
return true;
}
bool SqliteDbService::deleteTag(QString tag)
{
QSqlDatabase db = dbFactory->forCurrentThread();
if(!db.transaction())
{
Logger::error() << "Failed to open transaction while trying to delete tag " << tag << " : " << db.lastError()
<< Qt::endl;
return false;
}
tag = tag.toLower();
QSqlQuery assignmentDeleteQuery(db);
assignmentDeleteQuery.prepare("DELETE FROM filetag WHERE tagid = (SELECT id FROM tag WHERE name = ?)");
assignmentDeleteQuery.addBindValue(tag);
if(!assignmentDeleteQuery.exec())
{
db.rollback();
Logger::error() << "Error while trying to delete tag: " << db.lastError() << Qt::endl;
return false;
}
QSqlQuery deleteTagQuery(db);
deleteTagQuery.prepare("DELETE FROM tag WHERE name = ?");
deleteTagQuery.addBindValue(tag);
if(!deleteTagQuery.exec())
{
db.rollback();
Logger::error() << "Error while trying to delete tag: " << db.lastError() << Qt::endl;
return false;
}
if(!db.commit())
{
db.rollback();
Logger::error() << "Error while trying to delete tag: " << db.lastError() << Qt::endl;
return false;
}
return true;
}

Wyświetl plik

@ -1,8 +1,6 @@
#ifndef SQLITEDBSERVICE_H
#define SQLITEDBSERVICE_H
#include <QFileInfo>
#include <optional>
#include "databasefactory.h"
#include "utils.h"
#include "pagedata.h"
@ -17,31 +15,14 @@ class SqliteDbService
DatabaseFactory *dbFactory = nullptr;
bool insertToFTS(bool useTrigrams, QSqlDatabase &db, int fileid, QVector<PageData> &pageData);
QSqlQuery exec(QString query, std::initializer_list<QVariant> args);
bool execBool(QString querystr, std::initializer_list<QVariant> args);
public:
SqliteDbService(DatabaseFactory &dbFactory);
SaveFileResult saveFile(QFileInfo fileInfo, QVector<PageData> &pageData, bool pathsOnly);
SaveFileResult saveFile(QFileInfo fileInfo, QVector<PageData> &pageData);
unsigned int getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit);
bool deleteFile(QString path);
bool fileExistsInDatabase(QString path);
bool fileExistsInDatabase(QString path, qint64 mtime);
bool fileExistsInDatabase(QString path, qint64 mtime, QChar filetype);
unsigned int getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit);
bool addTag(QString tag, QString path);
bool addTag(QString tag, const QVector<QString> &paths);
QVector<QString> getTags();
QVector<QString> getTagsForPath(QString path);
QVector<QString> getPathsForTag(QString path);
bool setTags(QString path, const QSet<QString> &tags);
bool removePathsForTag(QString tag, const QVector<QString> &paths);
bool deleteTag(QString tag);
QVector<SearchResult> search(const LooqsQuery &query);
std::optional<QChar> queryFileType(QString absPath);
};
#endif // SQLITEDBSERVICE_H

Wyświetl plik

@ -69,7 +69,7 @@ QString SqliteSearch::createSortSql(const QVector<SortCondition> sortConditions)
QString SqliteSearch::escapeFtsArgument(QString ftsArg)
{
QString result;
static QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
QRegularExpressionMatchIterator i = extractor.globalMatch(ftsArg);
while(i.hasNext())
{
@ -143,17 +143,13 @@ QPair<QString, QVector<QString>> SqliteSearch::createSql(const Token &token)
{
return {" fts MATCH ? ", {escapeFtsArgument(value)}};
}
if(token.type == FILTER_TAG_ASSIGNED)
{
return {" file.id IN (SELECT fileid FROM filetag WHERE tagid = (SELECT id FROM tag WHERE name = ?)) ",
{value.toLower()}};
}
throw LooqsGeneralException("Unknown token passed (should not happen)");
}
QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
{
QString whereSql;
QString joinSql;
QVector<QString> bindValues;
bool isContentSearch = (query.getTokensMask() & FILTER_CONTENT) == FILTER_CONTENT;
if(query.getTokens().isEmpty())
@ -184,18 +180,18 @@ QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
}
QString whereSqlTrigram = whereSql;
whereSqlTrigram.replace("fts MATCH", "fts_trigram MATCH"); // A bit dirty...
prepSql = "SELECT DISTINCT path, page, mtime, size, filetype FROM ("
"SELECT file.path AS path, content.page AS page, file.mtime AS mtime, file.size AS size, "
"file.filetype AS filetype, 0 AS prio, fts.rank AS rank FROM file INNER JOIN content ON file.id = "
"content.fileid "
"INNER JOIN fts ON content.ftsid = fts.ROWID WHERE 1=1 AND " +
whereSql +
"UNION ALL SELECT file.path AS path, content.page AS page, file.mtime AS mtime, file.size AS size, "
"file.filetype AS filetype, 1 as prio, fts_trigram.rank AS rank FROM file INNER JOIN content ON "
"file.id = "
"content.fileid " +
"INNER JOIN fts_trigram ON content.fts_trigramid = fts_trigram.ROWID WHERE 1=1 AND " +
whereSqlTrigram + " ) " + sortSql;
prepSql =
"SELECT DISTINCT path, page, mtime, size, filetype FROM ("
"SELECT file.path AS path, content.page AS page, file.mtime AS mtime, file.size AS size, "
"file.filetype AS filetype, 0 AS prio, fts.rank AS rank FROM file INNER JOIN content ON file.id = "
"content.fileid "
"INNER JOIN fts ON content.ftsid = fts.ROWID WHERE 1=1 AND " +
whereSql +
"UNION ALL SELECT file.path AS path, content.page AS page, file.mtime AS mtime, file.size AS size, "
"file.filetype AS filetype, 1 as prio, fts_trigram.rank AS rank FROM file INNER JOIN content ON file.id = "
"content.fileid " +
"INNER JOIN fts_trigram ON content.fts_trigramid = fts_trigram.ROWID WHERE 1=1 AND " + whereSqlTrigram +
" ) " + sortSql;
++bindIterations;
}
else

Wyświetl plik

@ -1,66 +0,0 @@
#include "tagmanager.h"
TagManager::TagManager(SqliteDbService &dbService)
{
this->dbService = &dbService;
}
bool TagManager::addTagsToPath(QString path, const QSet<QString> &tags)
{
QVector<QString> currentTags = this->dbService->getTagsForPath(path);
for(const QString &tag : tags)
{
currentTags.append(tag.toLower());
}
QSet<QString> newTags{currentTags.begin(), currentTags.end()};
return this->dbService->setTags(path, newTags);
}
bool TagManager::removeTagsForPath(QString path, const QSet<QString> &tags)
{
QVector<QString> currentTags = this->dbService->getTagsForPath(path);
for(const QString &tag : tags)
{
currentTags.removeAll(tag);
}
QSet<QString> newTags{currentTags.begin(), currentTags.end()};
return this->dbService->setTags(path, newTags);
}
bool TagManager::removePathsForTag(QString tag, const QVector<QString> &paths)
{
return this->dbService->removePathsForTag(tag, paths);
}
bool TagManager::deleteTag(QString tag)
{
return this->dbService->deleteTag(tag);
}
QVector<QString> TagManager::getTags(QString path)
{
return this->dbService->getTagsForPath(path);
}
QVector<QString> TagManager::getTags()
{
return this->dbService->getTags();
}
QVector<QString> TagManager::getPaths(QString tag)
{
return this->dbService->getPathsForTag(tag);
}
bool TagManager::addTagsToPath(QString path, QString tagstring, QChar delim)
{
auto splitted = tagstring.split(delim);
return addTagsToPath(path, QSet<QString>{splitted.begin(), splitted.end()});
}
bool TagManager::addPathsToTag(QString tag, const QVector<QString> &paths)
{
return this->dbService->addTag(tag, paths);
}

Wyświetl plik

@ -1,28 +0,0 @@
#ifndef TAGMANAGER_H
#define TAGMANAGER_H
#include "sqlitedbservice.h"
class TagManager
{
private:
SqliteDbService *dbService = nullptr;
bool ensurePathOkay(QString inpath);
public:
TagManager(SqliteDbService &dbService);
bool addTagsToPath(QString path, const QSet<QString> &tags);
bool addTagsToPath(QString path, QString tagstring, QChar delim);
bool addPathsToTag(QString tag, const QVector<QString> &paths);
bool removeTagsForPath(QString path, const QSet<QString> &tags);
bool removePathsForTag(QString tag, const QVector<QString> &paths);
bool deleteTag(QString tag);
QVector<QString> getTags(QString path);
QVector<QString> getTags();
QVector<QString> getPaths(QString tag);
};
#endif // TAGMANAGER_H

Wyświetl plik

@ -19,8 +19,7 @@ enum TokenType
FILTER_PATH_SIZE,
FILTER_PATH_ENDS,
FILTER_PATH_STARTS,
FILTER_TAG_ASSIGNED,
FILTER_CONTENT = 512, /* Everything below here is content search (except LIMIT) */
FILTER_CONTENT = 512,
FILTER_CONTENT_CONTAINS,
FILTER_CONTENT_PAGE,
LIMIT = 1024

@ -1 +1 @@
Subproject commit 44b9a17becf6882e1b3728cbf885ae9e5a6717af
Subproject commit 769f729dc51f2feb8bc3cbb2a48ed91ff2d56bf3