Update docs

Apparently poppler or something needs open() with write
flags to render pdfs with proper fonts.

Landlock guards file system write access, so this is fine.

CHANGELOG.md

@@ -1,5 +1,24 @@
 # looqs: Release notes
 
+## 2023-05-07 - v0.9
+Highlights: Tag support. Also begin new index mode to only index metadata (currently only path + file size, more to come).
+
+Note: Upgrading can take some time as new column indexes will be added
+
+CHANGES:
+
+ - gui: Improve font rendering in previews
+ - gui: Allow indexing only metadata
+ - gui: Allow adding content for files which only had metadata indexed before
+ - gui: Allow assigning tags by right clicking on paths
+ - cli: "add" command: Implement --verbose (-v)
+ - cli: "add" command: Implement --no-content and --fill-content
+ - cli: Add "tag" command which allows managing tags for paths.
+ - search: Add "tag:()", "t:()" filters
+ - Minor improvements and refactorings under the hood
+ - Add packages: Ubuntu 23.04.
+
+
 ## 2022-11-19 - v0.8.1
 
 CHANGES:

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2018-2022: Albert Schwarzkopf <looqs at quitesimple period org>
+Copyright (c) 2018-2023: Albert Schwarzkopf <looqs at quitesimple period org>
 
 looqs is made available under the following license: 
 

README.md

@@ -76,7 +76,7 @@ To build on Ubuntu and Debian, clone the repo and then run:
 ```
 git submodule init
 git submodule update
-sudo apt install build-essential qtbase5-dev libpoppler-qt5-dev libuchardet-dev libquazip5-dev
+sudo apt install build-essential qtbase5-dev libqt5sql5-sqlite libpoppler-qt5-dev libuchardet-dev libquazip5-dev
 qmake
 make
 ```
@@ -97,7 +97,9 @@ The GUI is located in `gui/looqs-gui`, the binary for the CLI is in `cli/looqs`
 ## Packages
 At this point, looqs is not in any official distro package repo, but I maintain some packages.
 
-### Ubuntu 22.04, 22.10
+
+
+### Ubuntu 23.04, 22.10, 22.04
 Latest release can be installed using apt from the repo.
 ```
 # First, obtain key, assume it's trusted.
@@ -108,6 +110,8 @@ echo "deb [arch=amd64 signed-by=/usr/share/keyrings/repo.quitesimple.org.gpg] ht
 sudo apt-get update
 sudo apt-get install looqs
 ```
+### Gentoo (EXPERIMENTAL)
+Available in this overlay: https://github.com/quitesimpleorg/quitesimple-overlay
 
 ### Prebuilt tarball (distro-agnostic) (EXPERIMENTAL)
 looqs is also distributed as a tarball containing prebuilt binaries and its library dependencies. The tarball is
@@ -134,7 +138,7 @@ An AppImage may accompany the tarball in the future.
 
 
 ### Other distros
-I'll probably add a package for voidlinux at some point and maybe will provide a Gentoo ebuild. However, I would appreciate help for others distros. If you create a package, let me know!
+I appreciate help for others distros. If you create a package, let me know!
 
 
 ### Signature verification

USAGE.md

@@ -165,6 +165,8 @@ A number of search filters are available.
 | path.begins:(term) | pb:(term) |  Filters path beginning with the specified term |
 | contains:(terms) | c:(terms) | Full-text search, also understands quotes |
 | limit:(integer) | - | Limits the number of results. The default is 1000. Say "limit:0" to see all results |
+| tag:(tagname) | t:(tagname) | Filter for files that have been tagged with the corresponding tag |
+
 Filters can be combined. The booleans AND and OR are supported. Negations can be applied too, except for c:(). Negations are specified with "!".
 The AND boolean is implicit and thus entering it strictly optional.
 
@@ -177,11 +179,5 @@ Examples:
 |p:(notes) (pe:(odt) OR pe:(docx))          |Finds files such as notes.docx, notes.odt but also any .docs and .odt when the path contains the string 'notes'|
 |memcpy !(pe:(.c) OR pe:(.cpp))| Performs a FTS search for 'memcpy' but excludes .cpp and .c files.|
 |c:("I think, therefore")|Performs a FTS search for the phrase "I think, therefore".|
-|c:("invoice") Downloads|This query is equivalent to c:("invoice") p:("Downloads")|
+|c:("invoice") Downloads|Equivalent to c:("invoice") p:("Downloads")|
-
+|p:(Downloads) invoice|Equivalent to c:("invoice") p:("Downloads")|
-
-
-
-
-
-

cli/cli.pro

@@ -15,6 +15,7 @@ DEFINES += QT_DEPRECATED_WARNINGS
 # You can also select to disable deprecated APIs only up to a certain version of Qt.
 #DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0
 SOURCES += \
+    commandtag.cpp \
         main.cpp \
     commandadd.cpp \
     commanddelete.cpp \
@@ -27,6 +28,7 @@ HEADERS += \
     command.h \
     commandadd.h \
     commanddelete.h \
+    commandtag.h \
     commandupdate.h \
     commandsearch.h \
     commandlist.h

cli/command.cpp

@@ -2,7 +2,6 @@
 #include <QThread>
 #include <QDebug>
 #include "command.h"
-#include "looqsgeneralexception.h"
 
 void Command::execute()
 {

cli/commandadd.cpp

@@ -23,7 +23,7 @@ void CommandAdd::indexerFinished()
 	if(failedPathsCount > 0)
 	{
 		Logger::info() << "Failed paths: " << Qt::endl;
-		for(QString paths : result.failedPaths())
+		for(const QString &paths : result.failedPaths())
 		{
 			Logger::info() << paths << Qt::endl;
 		}
@@ -41,23 +41,36 @@ int CommandAdd::handle(QStringList arguments)
 {
 	QCommandLineParser parser;
 	parser.addOptions({{{"c", "continue"},
-						"Continue adding files, don't exit on first error. If this option is not given, looqs will "
+						"Continue adding files, don't exit on first error. Exit code will be 0. If this option is not "
-						"exit asap, but it's possible that a few files will still be processed. "
+						"given, looqs will "
+						"exit asap, but it's possible that a few files will still be processed."
 						"Set -t 1 to avoid this behavior, but processing will be slower. "},
+					   {{"n", "no-content"}, "Only add paths to database. Do not index content"},
+					   {{"v", "verbose"}, "Print paths of files being processed"},
+					   {{"f", "fill-content"}, "Index content for files previously indexed with -n"},
 					   {{"t", "threads"}, "Number of threads to use.", "threads"}});
-
 	parser.addHelpOption();
 	parser.addPositionalArgument("add", "Add paths to the index",
 								 "add [paths...]. If no path is given, read from stdin, one path per line.");
 
 	parser.process(arguments);
 	this->keepGoing = parser.isSet("continue");
+	bool pathsOnly = parser.isSet("no-content");
+	bool fillContent = parser.isSet("fill-content");
+	bool verbose = parser.isSet("verbose");
+
 	if(parser.isSet("threads"))
 	{
 		QString threadsCount = parser.value("threads");
 		QThreadPool::globalInstance()->setMaxThreadCount(threadsCount.toInt());
 	}
 
+	if(pathsOnly && fillContent)
+	{
+		Logger::error() << "Invalid options: -n and -f cannot both be set";
+		return EXIT_FAILURE;
+	}
+
 	QStringList files = parser.positionalArguments();
 
 	if(files.length() == 0)
@@ -71,15 +84,47 @@ int CommandAdd::handle(QStringList arguments)
 		}
 	}
 
+	FileSaverOptions fileSaverOptions;
+	fileSaverOptions.keepGoing = keepGoing;
+	fileSaverOptions.fillExistingContentless = fillContent;
+	fileSaverOptions.metadataOnly = pathsOnly;
+	fileSaverOptions.verbose = verbose;
+
 	indexer = new Indexer(*this->dbService);
+	indexer->setFileSaverOptions(fileSaverOptions);
+
 	indexer->setTargetPaths(files.toVector());
-	indexer->setKeepGoing(keepGoing);
+
+	if(verbose)
+	{
+		indexer->setProgressReportThreshold(1);
+	}
 
 	connect(indexer, &Indexer::pathsCountChanged, this,
 			[](int pathsCount) { Logger::info() << "Found paths: " << pathsCount << Qt::endl; });
 	connect(indexer, &Indexer::indexProgress, this,
-			[](int pathsCount, unsigned int added, unsigned int skipped, unsigned int failed, unsigned int totalCount)
+			[verbose, this](int pathsCount, unsigned int /*added*/, unsigned int /*skipped*/, unsigned int /*failed*/,
-			{ Logger::info() << "Processed files: " << pathsCount << Qt::endl; });
+							unsigned int /*totalCount*/)
+			{
+				Logger::info() << "Processed files: " << pathsCount << Qt::endl;
+				if(verbose)
+				{
+					IndexResult indexResult = indexer->getResult();
+					int newlyAdded = indexResult.results.count() - currentResult.results.count();
+					if(newlyAdded > 0)
+					{
+						int newOffset = indexResult.results.count() - newlyAdded;
+						for(int i = newOffset; i < indexResult.results.count(); i++)
+						{
+							auto result = indexResult.results.at(i);
+							Logger::info() << SaveFileResultToString(result.second) << result.first << Qt::endl;
+						}
+					}
+					this->currentResult = indexResult;
+				}
+			}
+
+	);
 	connect(indexer, &Indexer::finished, this, &CommandAdd::indexerFinished);
 
 	this->autoFinish = false;

cli/commandadd.h

@@ -13,6 +13,8 @@ class CommandAdd : public Command
 	bool keepGoing = true;
 
   protected:
+	IndexResult currentResult;
+
   public:
 	using Command::Command;
 

cli/commandlist.cpp

@@ -1,6 +1,5 @@
 #include <QCommandLineParser>
 #include "commandlist.h"
-#include "databasefactory.h"
 #include "logger.h"
 
 int CommandList::handle(QStringList arguments)

cli/commandsearch.cpp

@@ -1,6 +1,5 @@
 #include <QCommandLineParser>
 #include "commandsearch.h"
-#include "databasefactory.h"
 #include "logger.h"
 
 int CommandSearch::handle(QStringList arguments)

cli/commandtag.cpp

@@ -0,0 +1,153 @@
+#include <QCommandLineParser>
+#include "commandtag.h"
+#include "logger.h"
+#include "tagmanager.h"
+
+bool CommandTag::ensureAbsolutePaths(const QVector<QString> &paths, QVector<QString> &absolutePaths)
+{
+	for(const QString &path : paths)
+	{
+		QFileInfo info{path};
+		if(!info.exists())
+		{
+			Logger::error() << "Can't add tag for file  " + info.absoluteFilePath() + " because it does not exist"
+							<< Qt::endl;
+			return false;
+		}
+		QString absolutePath = info.absoluteFilePath();
+		if(!this->dbService->fileExistsInDatabase(absolutePath))
+		{
+			Logger::error() << "Only files that have been indexed can be tagged. File not in index: " + absolutePath
+							<< Qt::endl;
+			return false;
+		}
+		absolutePaths.append(absolutePath);
+	}
+	return true;
+}
+
+int CommandTag::handle(QStringList arguments)
+{
+	QCommandLineParser parser;
+	parser.addPositionalArgument("add", "Adds a tag to a file",
+								 "add [tag] [paths...]. Adds the tag to the specified paths");
+	parser.addPositionalArgument("remove", "Removes a path associated to a tag", "remove [tag] [path]");
+	parser.addPositionalArgument("delete", "Deletes a tag", "delete [tag]");
+	parser.addPositionalArgument("list", "Lists paths associated with a tag, or all tags", "list [tag]");
+	parser.addPositionalArgument("show", "Lists tags associated with a path", "show [path]");
+
+	parser.addHelpOption();
+
+	parser.parse(arguments);
+
+	QStringList args = parser.positionalArguments();
+	if(args.length() == 0)
+	{
+		parser.showHelp(EXIT_FAILURE);
+		return EXIT_FAILURE;
+	}
+	TagManager tagManager{*this->dbService};
+	QString cmd = args[0];
+	if(cmd == "add")
+	{
+		if(args.length() < 3)
+		{
+			Logger::error() << "Not enough arguments provided. 'add' requires a tag followed by at least one path"
+							<< Qt::endl;
+			return EXIT_FAILURE;
+		}
+		QString tag = args[1];
+		QVector<QString> paths = args.mid(2).toVector();
+
+		QVector<QString> absolutePaths;
+		if(!ensureAbsolutePaths(paths, absolutePaths))
+		{
+			return EXIT_FAILURE;
+		}
+		bool result = tagManager.addPathsToTag(tag, absolutePaths);
+		if(!result)
+		{
+			Logger::error() << "Failed to assign tags" << Qt::endl;
+			return EXIT_FAILURE;
+		}
+		return EXIT_SUCCESS;
+	}
+	if(cmd == "list")
+	{
+
+		QString tag;
+		if(args.length() >= 2)
+		{
+			tag = args[1];
+		}
+		QVector<QString> entries;
+		if(tag.isEmpty())
+		{
+			entries = tagManager.getTags();
+		}
+		else
+		{
+			entries = tagManager.getPaths(tag);
+		}
+		for(const QString &entry : entries)
+		{
+			Logger::info() << entry << Qt::endl;
+		}
+	}
+	if(cmd == "remove")
+	{
+		if(args.length() < 3)
+		{
+			Logger::error() << "Not enough arguments provided. 'remove' requires a tag followed by at least one path"
+							<< Qt::endl;
+			return EXIT_FAILURE;
+		}
+		QString tag = args[1];
+		QVector<QString> paths = args.mid(2).toVector();
+
+		QVector<QString> absolutePaths;
+		if(!ensureAbsolutePaths(paths, absolutePaths))
+		{
+			return EXIT_FAILURE;
+		}
+
+		if(!tagManager.removePathsForTag(tag, absolutePaths))
+		{
+			Logger::error() << "Failed to remove path assignments" << Qt::endl;
+			return EXIT_FAILURE;
+		}
+	}
+	if(cmd == "delete")
+	{
+		if(args.length() != 2)
+		{
+			Logger::error() << "The 'delete' command requires the tag to delete" << Qt::endl;
+			return EXIT_FAILURE;
+		}
+		if(!tagManager.deleteTag(args[1]))
+		{
+			Logger::error() << "Failed to delete tag" << Qt::endl;
+			return EXIT_FAILURE;
+		}
+	}
+	if(cmd == "show")
+	{
+		if(args.length() != 2)
+		{
+			Logger::error() << "The 'show' command requires a path to show the assigned tags" << Qt::endl;
+			return EXIT_FAILURE;
+		}
+		QString path = args[1];
+		QVector<QString> absolutePaths;
+		if(!ensureAbsolutePaths({path}, absolutePaths))
+		{
+			return EXIT_FAILURE;
+		}
+		QVector<QString> tags = tagManager.getTags(absolutePaths.at(0));
+		for(const QString &entry : tags)
+		{
+			Logger::info() << entry << Qt::endl;
+		}
+	}
+	return EXIT_SUCCESS;
+}

cli/commandtag.h

@@ -0,0 +1,16 @@
+#ifndef COMMANDTAG_H
+#define COMMANDTAG_H
+#include "command.h"
+
+class CommandTag : public Command
+{
+  protected:
+	bool ensureAbsolutePaths(const QVector<QString> &paths, QVector<QString> &absolutePaths);
+
+  public:
+	using Command::Command;
+
+	int handle(QStringList arguments) override;
+};
+
+#endif // COMMANDTAG_H

cli/commandupdate.cpp

@@ -38,10 +38,13 @@ int CommandUpdate::handle(QStringList arguments)
 		QThreadPool::globalInstance()->setMaxThreadCount(threadsCount.toInt());
 	}
 
-	bool hasErrors = false;
 	IndexSyncer *syncer = new IndexSyncer(*this->dbService);
-	syncer->setKeepGoing(keepGoing);
+
-	syncer->setVerbose(verbose);
+	FileSaverOptions fileOptions;
+	fileOptions.keepGoing = keepGoing;
+	fileOptions.verbose = verbose;
+
+	syncer->setFileSaverOptions(fileOptions);
 	syncer->setPattern(pattern);
 	syncer->setDryRun(dryRun);
 	syncer->setRemoveDeletedFromIndex(deleteMissing);
@@ -60,7 +63,7 @@ int CommandUpdate::handle(QStringList arguments)
 		/* TODO: updated not printed, handled be verbose in FileSaver, but this can be improved */
 	}
 	connect(syncer, &IndexSyncer::finished, this,
-			[&](unsigned int totalUpdated, unsigned int totalRemoved, unsigned int totalErrors)
+			[this, dryRun, keepGoing](unsigned int totalUpdated, unsigned int totalRemoved, unsigned int totalErrors)
 			{
 				Logger::info() << "Syncing finished" << Qt::endl;
 
@@ -72,7 +75,7 @@ int CommandUpdate::handle(QStringList arguments)
 				}
 
 				int retval = 0;
-				if(hasErrors && !keepGoing)
+				if(this->hasErrors && !keepGoing)
 				{
 					retval = 1;
 				}
@@ -82,7 +85,7 @@ int CommandUpdate::handle(QStringList arguments)
 			[&](QString error)
 			{
 				Logger::error() << error << Qt::endl;
-				hasErrors = true;
+				this->hasErrors = true;
 			});
 
 	this->autoFinish = false;

cli/commandupdate.h

@@ -4,6 +4,9 @@
 #include "filesaver.h"
 class CommandUpdate : public Command
 {
+  protected:
+	bool hasErrors = false;
+
   public:
 	using Command::Command;
 	int handle(QStringList arguments) override;

cli/main.cpp

@@ -21,6 +21,7 @@
 #include "commandupdate.h"
 #include "commandsearch.h"
 #include "commandlist.h"
+#include "commandtag.h"
 #include "databasefactory.h"
 #include "logger.h"
 #include "sandboxedprocessor.h"
@@ -31,7 +32,7 @@
 void printUsage(QString argv0)
 {
 	qInfo() << "Usage:" << argv0 << "command";
-	qInfo() << "Valid commands: add, update, delete, search, list. Each command has a --help option.";
+	qInfo() << "Valid commands: add, update, search, delete, tag, list. Each command has a --help option.";
 }
 
 Command *commandFromName(QString name, SqliteDbService &dbService)
@@ -56,6 +57,10 @@ Command *commandFromName(QString name, SqliteDbService &dbService)
 	{
 		return new CommandList(dbService);
 	}
+	if(name == "tag")
+	{
+		return new CommandTag(dbService);
+	}
 
 	return nullptr;
 }

gui/gui.pro

@@ -34,6 +34,7 @@ SOURCES += \
         main.cpp \
         mainwindow.cpp \
       clicklabel.cpp \
+    previewcoordinator.cpp \
     previewgenerator.cpp \
     previewgeneratormapfunctor.cpp \
     previewgeneratorodt.cpp \
@@ -54,6 +55,7 @@ HEADERS += \
     ipcserver.h \
         mainwindow.h \
     clicklabel.h \
+    previewcoordinator.h \
     previewgenerator.h \
     previewgeneratormapfunctor.h \
     previewgeneratorodt.h \

gui/main.cpp

@@ -28,7 +28,7 @@ void enableIpcSandbox()
 	policy->namespace_options = EXILE_UNSHARE_USER | EXILE_UNSHARE_MOUNT | EXILE_UNSHARE_NETWORK;
 	policy->no_new_privs = 1;
 	policy->drop_caps = 1;
-	policy->vow_promises = exile_vows_from_str("thread cpath rpath unix stdio proc error");
+	policy->vow_promises = exile_vows_from_str("thread cpath rpath wpath unix stdio proc error");
 	policy->mount_path_policies_to_chroot = 1;
 
 	QString ipcSocketPath = Common::ipcSocketPath();
@@ -193,7 +193,7 @@ int main(int argc, char *argv[])
 															   Logger::error() << error << Qt::endl;
 															   QMessageBox::critical(nullptr, "Error during upgrade",
 																					 error);
-															   qApp->quit();
+															   exit(EXIT_FAILURE);
 														   }
 
 								 );

gui/mainwindow.cpp

@@ -16,14 +16,15 @@
 #include <QScreen>
 #include <QProgressDialog>
 #include <QDesktopWidget>
+#include <QWidgetAction>
+#include <QInputDialog>
+
 #include "mainwindow.h"
 #include "ui_mainwindow.h"
 #include "clicklabel.h"
 #include "../shared/sqlitesearch.h"
 #include "../shared/looqsgeneralexception.h"
 #include "../shared/common.h"
-#include "ipcpreviewclient.h"
-#include "previewgenerator.h"
 #include "aboutdialog.h"
 
 MainWindow::MainWindow(QWidget *parent, QString socketPath)
@@ -32,8 +33,7 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
 	this->progressDialog.cancel(); // because constructing it shows it, quite weird
 	ui->setupUi(this);
 	setWindowTitle(QCoreApplication::applicationName());
-	this->ipcPreviewClient.moveToThread(&this->ipcClientThread);
+
-	this->ipcPreviewClient.setSocketPath(socketPath);
 	QSettings settings;
 
 	this->dbFactory = new DatabaseFactory(Common::databasePath());
@@ -45,6 +45,9 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
 
 	indexer = new Indexer(*(this->dbService));
 	indexer->setParent(this);
+
+	tagManager = new TagManager(*(this->dbService));
+
 	connectSignals();
 	ui->treeResultsList->setContextMenuPolicy(Qt::ContextMenuPolicy::CustomContextMenu);
 	ui->tabWidget->setCurrentIndex(0);
@@ -78,7 +81,7 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
 	ui->txtSearch->installEventFilter(this);
 	ui->scrollArea->viewport()->installEventFilter(this);
 
-	this->ipcClientThread.start();
+	this->previewCoordinator.setSocketPath(socketPath);
 }
 
 void MainWindow::addPathToIndex()
@@ -150,7 +153,7 @@ void MainWindow::connectSignals()
 	connect(this->indexer, &Indexer::finished, this, &MainWindow::finishIndexing);
 
 	connect(ui->lstPaths->selectionModel(), &QItemSelectionModel::selectionChanged, this,
-			[&](const QItemSelection &selected, const QItemSelection &deselected)
+			[&](const QItemSelection & /*selected*/, const QItemSelection & /*deselected*/)
 			{ ui->btnDeletePath->setEnabled(this->ui->lstPaths->selectedItems().count() > 0); });
 
 	connect(ui->btnDeletePath, &QPushButton::clicked, this, [&] { qDeleteAll(ui->lstPaths->selectedItems()); });
@@ -170,19 +173,20 @@ void MainWindow::connectSignals()
 				}
 			});
 	connect(ui->menuAboutAction, &QAction::triggered, this,
-			[this](bool checked)
+			[this](bool /*checked*/)
 			{
 				AboutDialog aboutDialog(this);
 
 				aboutDialog.exec();
 			});
 	connect(ui->menuAboutQtAction, &QAction::triggered, this,
-			[this](bool checked) { QMessageBox::aboutQt(this, "About Qt"); });
+			[this](bool /*checked*/) { QMessageBox::aboutQt(this, "About Qt"); });
 	connect(ui->menuSyncIndexAction, &QAction::triggered, this, &MainWindow::startIndexSync);
 	connect(ui->menuOpenUserManualAction, &QAction::triggered, this,
-			[this]() { QDesktopServices::openUrl(Common::userManualUrl()); });
+			[]() { QDesktopServices::openUrl(Common::userManualUrl()); });
 
-	connect(indexSyncer, &IndexSyncer::finished, this,
+	connect(
+		indexSyncer, &IndexSyncer::finished, this,
 		[&](unsigned int totalUpdated, unsigned int totalDeleted, unsigned int totalErrored)
 		{
 			this->progressDialog.cancel();
@@ -190,9 +194,7 @@ void MainWindow::connectSignals()
 			QMessageBox::information(
 				this, "Syncing finished",
 				QString("Syncing finished\n\nTotal updated: %1\nTotal deleted: %2\nTotal errors: %3\n")
-						.arg(QString::number(totalUpdated))
+					.arg(QString::number(totalUpdated), QString::number(totalDeleted), QString::number(totalErrored)));
-						.arg(QString::number(totalDeleted))
-						.arg(QString::number(totalErrored)));
 		});
 	connect(this, &MainWindow::beginIndexSync, indexSyncer, &IndexSyncer::sync);
 	connect(&this->progressDialog, &QProgressDialog::canceled, indexSyncer, &IndexSyncer::cancel);
@@ -208,9 +210,9 @@ void MainWindow::connectSignals()
 			}
 		},
 		Qt::QueuedConnection);
-	connect(&ipcPreviewClient, &IPCPreviewClient::previewReceived, this, &MainWindow::previewReceived,
+	connect(&previewCoordinator, &PreviewCoordinator::previewReady, this, &MainWindow::previewReceived,
 			Qt::QueuedConnection);
-	connect(&ipcPreviewClient, &IPCPreviewClient::finished, this,
+	connect(&previewCoordinator, &PreviewCoordinator::completedGeneration, this,
 			[&]
 			{
 				this->ui->previewProcessBar->setValue(this->ui->previewProcessBar->maximum());
@@ -218,22 +220,24 @@ void MainWindow::connectSignals()
 				this->ui->comboPreviewFiles->setEnabled(true);
 				ui->txtSearch->setEnabled(true);
 			});
-	connect(&ipcPreviewClient, &IPCPreviewClient::error, this,
+	connect(&previewCoordinator, &PreviewCoordinator::error, this,
 			[this](QString msg)
 			{
 				qCritical() << msg << Qt::endl;
 				QMessageBox::critical(this, "IPC error", msg);
 			});
-
+	connect(ui->radioMetadataOnly, &QRadioButton::toggled, this,
-	connect(this, &MainWindow::startIpcPreviews, &ipcPreviewClient, &IPCPreviewClient::startGeneration,
+			[this](bool toggled)
-			Qt::QueuedConnection);
+			{
-	connect(this, &MainWindow::stopIpcPreviews, &ipcPreviewClient, &IPCPreviewClient::stopGeneration,
+				if(toggled)
-			Qt::QueuedConnection);
+				{
+					this->ui->chkFillContentForContentless->setChecked(false);
+				};
+			});
 }
 
 void MainWindow::exportFailedPaths()
 {
-
 	QString filename =
 		QString("/tmp/looqs_indexresult_failed_%1").arg(QDateTime::currentDateTime().toString("yyyy-MM-dd_hhmmss"));
 	QFile outFile(filename);
@@ -266,8 +270,11 @@ void MainWindow::startIndexSync()
 	progressDialog.setValue(0);
 	progressDialog.open();
 
-	indexSyncer->setKeepGoing(true);
+	FileSaverOptions options;
-	indexSyncer->setVerbose(false);
+	options.keepGoing = true;
+	options.verbose = false;
+
+	indexSyncer->setFileSaverOptions(options);
 	indexSyncer->setDryRun(false);
 	indexSyncer->setRemoveDeletedFromIndex(true);
 
@@ -286,6 +293,7 @@ void MainWindow::startIndexing()
 	if(this->indexer->isRunning())
 	{
 		ui->btnStartIndexing->setEnabled(false);
+
 		ui->btnStartIndexing->setText("Start indexing");
 		this->indexer->requestCancellation();
 		return;
@@ -295,6 +303,8 @@ void MainWindow::startIndexing()
 	ui->resultsTab->setEnabled(false);
 	ui->settingsTab->setEnabled(false);
 	ui->txtPathScanAdd->setEnabled(false);
+	ui->btnAddPath->setEnabled(false);
+	ui->btnChoosePath->setEnabled(false);
 	ui->txtSearch->setEnabled(false);
 	ui->previewProcessBar->setValue(0);
 	ui->previewProcessBar->setVisible(true);
@@ -311,6 +321,15 @@ void MainWindow::startIndexing()
 	this->indexer->setTargetPaths(paths);
 	QString ignorePatterns = ui->txtIgnorePatterns->text();
 	this->indexer->setIgnorePattern(ignorePatterns.split(";"));
+
+	FileSaverOptions options;
+	options.fillExistingContentless =
+		ui->chkFillContentForContentless->isEnabled() && ui->chkFillContentForContentless->isChecked();
+	options.metadataOnly = ui->radioMetadataOnly->isChecked();
+	options.verbose = false;
+	options.keepGoing = true;
+
+	this->indexer->setFileSaverOptions(options);
 	this->indexer->beginIndexing();
 	QSettings settings;
 	settings.setValue("indexPaths", pathSettingsValue);
@@ -333,6 +352,8 @@ void MainWindow::finishIndexing()
 	ui->resultsTab->setEnabled(true);
 	ui->settingsTab->setEnabled(true);
 	ui->txtPathScanAdd->setEnabled(true);
+	ui->btnAddPath->setEnabled(true);
+	ui->btnChoosePath->setEnabled(true);
 	ui->txtSearch->setEnabled(true);
 	if(result.erroredPaths > 0)
 	{
@@ -340,7 +361,7 @@ void MainWindow::finishIndexing()
 	}
 }
 
-void MainWindow::comboScaleChanged(int i)
+void MainWindow::comboScaleChanged(int /*i*/)
 {
 	QSettings scaleSetting;
 	scaleSetting.setValue("currentScale", ui->comboScale->currentText());
@@ -386,7 +407,8 @@ void MainWindow::processShortcut(int key)
 	{
 		ui->txtSearch->setFocus();
 		QString currentText = ui->txtSearch->text().trimmed();
-		int index = currentText.lastIndexOf(QRegularExpression("[\\s\\)]"));
+		static QRegularExpression separatorRegex("[\\s\\)]");
+		int index = currentText.lastIndexOf(separatorRegex);
 		if(index != -1)
 		{
 			bool isFilter = (index == currentText.length() - 1);
@@ -632,13 +654,17 @@ void MainWindow::saveSettings()
 	qApp->quit();
 }
 
-void MainWindow::previewReceived(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration)
+void MainWindow::previewReceived()
 {
-	if(previewGeneration < this->currentPreviewGeneration)
-	{
-		return;
-	}
 	this->ui->previewProcessBar->setValue(this->ui->previewProcessBar->value() + 1);
+	QBoxLayout *layout = static_cast<QBoxLayout *>(ui->scrollAreaWidgetContents->layout());
+	int index = layout->count();
+	if(index > 0)
+	{
+		--index;
+	}
+	QSharedPointer<PreviewResult> preview = this->previewCoordinator.resultAt(index);
+
 	if(!preview.isNull() && preview->hasPreview())
 	{
 		QString docPath = preview->getDocumentPath();
@@ -661,8 +687,8 @@ void MainWindow::previewReceived(QSharedPointer<PreviewResult> preview, unsigned
 		{
 			QFileInfo fileInfo{docPath};
 			QMenu menu("labeRightClick", this);
-			createSearchResutlMenu(menu, fileInfo);
+			createSearchResultMenu(menu, fileInfo);
-			menu.addAction("Copy page number",
+			menu.addAction("Copy page number", this,
 						   [previewPage] { QGuiApplication::clipboard()->setText(QString::number(previewPage)); });
 			menu.exec(QCursor::pos());
 		};
@@ -684,24 +710,7 @@ void MainWindow::previewReceived(QSharedPointer<PreviewResult> preview, unsigned
 
 		previewWidget->setLayout(previewLayout);
 
-		QBoxLayout *layout = static_cast<QBoxLayout *>(ui->scrollAreaWidgetContents->layout());
+		layout->insertWidget(index, previewWidget);
-		int pos = previewOrder[docPath + QString::number(previewPage)];
-		if(pos <= layout->count())
-		{
-			layout->insertWidget(pos, previewWidget);
-			for(auto it = previewWidgetOrderCache.constKeyValueBegin();
-				it != previewWidgetOrderCache.constKeyValueEnd(); it++)
-			{
-				if(it->first <= layout->count())
-				{
-					layout->insertWidget(it->first, it->second);
-				}
-			}
-		}
-		else
-		{
-			previewWidgetOrderCache[pos] = previewWidget;
-		}
 	}
 }
 
@@ -818,7 +827,6 @@ void MainWindow::lineEditReturnPressed()
 
 void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 {
-	this->previewableSearchResults.clear();
 	qDeleteAll(ui->scrollAreaWidgetContents->children());
 
 	ui->treeResultsList->clear();
@@ -827,6 +835,8 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 	ui->comboPreviewFiles->setVisible(true);
 	ui->lblTotalPreviewPagesCount->setText("");
 
+	this->previewCoordinator.init(results);
+
 	bool hasDeleted = false;
 	QHash<QString, bool> seenMap;
 	for(const SearchResult &result : results)
@@ -847,34 +857,29 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 			item->setText(3, this->locale().formattedDataSize(result.fileData.size));
 		}
 		bool exists = pathInfo.exists();
-		if(exists)
+		if(!exists)
-		{
-			if(result.wasContentSearch)
-			{
-				if(!pathInfo.suffix().contains("htm")) // hack until we can preview them properly...
-				{
-					if(PreviewGenerator::get(pathInfo) != nullptr)
-					{
-						this->previewableSearchResults.append(result);
-						if(!seenMap.contains(result.fileData.absPath))
-						{
-							ui->comboPreviewFiles->addItem(result.fileData.absPath);
-						}
-					}
-				}
-			}
-		}
-		else
 		{
 			hasDeleted = true;
 		}
 		seenMap[absPath] = true;
 	}
 
+	seenMap.clear();
+	for(const SearchResult &result : this->previewCoordinator.getPreviewableSearchResults())
+	{
+		const QString &absPath = result.fileData.absPath;
+		if(!seenMap.contains(absPath))
+		{
+			ui->comboPreviewFiles->addItem(absPath);
+		}
+		seenMap[absPath] = true;
+	}
+
 	ui->treeResultsList->resizeColumnToContents(0);
 	ui->treeResultsList->resizeColumnToContents(1);
 	ui->treeResultsList->resizeColumnToContents(2);
-	previewDirty = !this->previewableSearchResults.empty();
+
+	previewDirty = this->previewCoordinator.previewableCount() > 0;
 
 	ui->spinPreviewPage->setValue(1);
 
@@ -883,8 +888,10 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 		makePreviews(1);
 	}
 
+	ui->tabWidget->setTabEnabled(1, previewDirty);
+
 	QString statusText = "Results: " + QString::number(results.size()) + " files";
-	statusText += ", previewable: " + QString::number(this->previewableSearchResults.count());
+	statusText += ", previewable: " + QString::number(this->previewCoordinator.previewableCount());
 	if(hasDeleted)
 	{
 		statusText += " WARNING: Some files are inaccessible. No preview available for those. Index may be out of sync";
@@ -901,7 +908,7 @@ int MainWindow::currentSelectedScale()
 
 void MainWindow::makePreviews(int page)
 {
-	if(this->previewableSearchResults.empty())
+	if(this->previewCoordinator.previewableCount() == 0)
 	{
 		return;
 	}
@@ -918,11 +925,10 @@ void MainWindow::makePreviews(int page)
 		ui->scrollAreaWidgetContents->setLayout(new QVBoxLayout());
 		ui->scrollAreaWidgetContents->layout()->setAlignment(Qt::AlignCenter);
 	}
-	ui->previewProcessBar->setMaximum(this->previewableSearchResults.size());
+	ui->previewProcessBar->setMaximum(this->previewCoordinator.previewableCount());
-	processedPdfPreviews = 0;
 
 	QVector<QString> wordsToHighlight;
-	QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
+	static QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
 	for(const Token &token : this->contentSearchQuery.getTokens())
 	{
 		if(token.type == FILTER_CONTENT_CONTAINS)
@@ -954,12 +960,8 @@ void MainWindow::makePreviews(int page)
 	renderConfig.scaleY = QGuiApplication::primaryScreen()->physicalDotsPerInchY() * (currentScale / 100.);
 	renderConfig.wordsToHighlight = wordsToHighlight;
 
-	this->previewOrder.clear();
-	this->previewWidgetOrderCache.clear();
-
-	int previewPos = 0;
 	QVector<RenderTarget> targets;
-	for(SearchResult &sr : this->previewableSearchResults)
+	for(const SearchResult &sr : this->previewCoordinator.getPreviewableSearchResults())
 	{
 		if(ui->comboPreviewFiles->currentIndex() != 0)
 		{
@@ -971,11 +973,8 @@ void MainWindow::makePreviews(int page)
 		RenderTarget renderTarget;
 		renderTarget.path = sr.fileData.absPath;
 		renderTarget.page = (int)sr.page;
-		targets.append(renderTarget);
 
-		int pos = previewPos - beginOffset;
+		targets.append(renderTarget);
-		this->previewOrder[renderTarget.path + QString::number(renderTarget.page)] = pos;
-		++previewPos;
 	}
 	int numpages = ceil(static_cast<double>(targets.size()) / previewsPerPage);
 	ui->spinPreviewPage->setMaximum(numpages);
@@ -985,12 +984,12 @@ void MainWindow::makePreviews(int page)
 	ui->previewProcessBar->setMaximum(targets.count());
 	ui->previewProcessBar->setMinimum(0);
 	ui->previewProcessBar->setValue(0);
-	ui->previewProcessBar->setVisible(this->previewableSearchResults.size() > 0);
+	ui->previewProcessBar->setVisible(this->previewCoordinator.previewableCount() > 0);
-	++this->currentPreviewGeneration;
 	this->ui->spinPreviewPage->setEnabled(false);
 	this->ui->comboPreviewFiles->setEnabled(false);
 	this->ui->txtSearch->setEnabled(false);
-	emit startIpcPreviews(renderConfig, targets);
+
+	this->previewCoordinator.startGeneration(renderConfig, targets);
 }
 
 void MainWindow::handleSearchError(QString error)
@@ -998,27 +997,87 @@ void MainWindow::handleSearchError(QString error)
 	ui->lblSearchResults->setText("Error:" + error);
 }
 
-void MainWindow::createSearchResutlMenu(QMenu &menu, const QFileInfo &fileInfo)
+void MainWindow::createSearchResultMenu(QMenu &menu, const QFileInfo &fileInfo)
 {
-	menu.addAction("Copy filename to clipboard",
+	menu.addAction("Copy filename to clipboard", this,
 				   [&fileInfo] { QGuiApplication::clipboard()->setText(fileInfo.fileName()); });
-	menu.addAction("Copy full path to clipboard",
+	menu.addAction("Copy full path to clipboard", this,
 				   [&fileInfo] { QGuiApplication::clipboard()->setText(fileInfo.absoluteFilePath()); });
-	menu.addAction("Open containing folder", [this, &fileInfo] { this->openFile(fileInfo.absolutePath()); });
+	menu.addAction("Open containing folder", this, [this, &fileInfo] { this->openFile(fileInfo.absolutePath()); });
 
+	auto previewables = this->previewCoordinator.getPreviewableSearchResults();
 	auto result =
-		std::find_if(this->previewableSearchResults.begin(), this->previewableSearchResults.end(),
+		std::find_if(previewables.begin(), previewables.end(),
-					 [this, &fileInfo](SearchResult &a) { return fileInfo.absoluteFilePath() == a.fileData.absPath; });
+					 [&fileInfo](SearchResult &a) { return fileInfo.absoluteFilePath() == a.fileData.absPath; });
 
-	if(result != this->previewableSearchResults.end())
+	if(result != previewables.end())
 	{
-		menu.addAction("Show previews for this file",
+		menu.addAction("Show previews for this file", this,
 					   [this, &fileInfo]
 					   {
 						   ui->tabWidget->setCurrentIndex(1);
 						   this->ui->comboPreviewFiles->setCurrentText(fileInfo.absoluteFilePath());
 					   });
 	}
+
+	QMenu *tagMenu = menu.addMenu("Tag file with: ");
+	QVector<QString> allTags = this->dbService->getTags();
+	QHash<QString, bool> fileTags;
+
+	QString path = fileInfo.absoluteFilePath();
+
+	for(const QString &fileTag : this->dbService->getTagsForPath(path))
+	{
+		fileTags[fileTag] = true;
+	}
+
+	for(const QString &tag : allTags)
+	{
+		QCheckBox *checkBox = new QCheckBox(tagMenu);
+		QWidgetAction *checkableAction = new QWidgetAction(tagMenu);
+		checkableAction->setDefaultWidget(checkBox);
+		checkBox->setText(tag);
+		checkBox->setChecked(fileTags.contains(tag));
+		tagMenu->addAction(checkableAction);
+
+		connect(checkBox, &QCheckBox::stateChanged, this,
+				[this, checkBox, path]
+				{
+					QVector<QString> currentTags = this->dbService->getTagsForPath(path);
+					QString checkBoxText = checkBox->text();
+					if(checkBox->isChecked())
+					{
+						if(!this->tagManager->addTagsToPath(path, {checkBoxText}))
+						{
+							QMessageBox::critical(this, "Error while adding tag",
+												  "An error occured while trying to add the tag");
+						}
+					}
+					else
+					{
+						if(!this->tagManager->removeTagsForPath(path, {checkBoxText}))
+						{
+							QMessageBox::critical(this, "Error while removing tag",
+												  "An error occured while trying to remove the tag");
+						}
+					}
+				});
+	}
+
+	tagMenu->addAction("Add new tags", this,
+					   [this, path]
+					   {
+						   bool ok;
+						   QString text =
+							   QInputDialog::getText(this, tr("Enter new tags"), tr("New tags (comma separated):"),
+													 QLineEdit::Normal, "", &ok);
+
+						   if(ok && !this->tagManager->addTagsToPath(path, text, ','))
+						   {
+							   QMessageBox::critical(this, "Error while trying to add tags",
+													 "An error occured while trying to add tags");
+						   }
+					   });
 }
 
 void MainWindow::openDocument(QString path, int num)
@@ -1048,7 +1107,7 @@ void MainWindow::openFile(QString path)
 	QDesktopServices::openUrl(QUrl::fromLocalFile(path));
 }
 
-void MainWindow::treeSearchItemActivated(QTreeWidgetItem *item, int i)
+void MainWindow::treeSearchItemActivated(QTreeWidgetItem *item, int /*i*/)
 {
 	openFile(item->text(1));
 }
@@ -1062,22 +1121,22 @@ void MainWindow::showSearchResultsContextMenu(const QPoint &point)
 	}
 	QFileInfo pathinfo(item->text(1));
 	QMenu menu("SearchResults", this);
-	createSearchResutlMenu(menu, pathinfo);
+	createSearchResultMenu(menu, pathinfo);
 	menu.exec(QCursor::pos());
 }
 
 MainWindow::~MainWindow()
 {
 	syncerThread.terminate();
-	ipcClientThread.terminate();
 	delete this->indexSyncer;
 	delete this->dbService;
 	delete this->dbFactory;
 	delete this->indexer;
+	delete this->tagManager;
 	delete ui;
 }
 
-void MainWindow::closeEvent(QCloseEvent *event)
+void MainWindow::closeEvent(QCloseEvent * /*event*/)
 {
 	QStringList list = this->searchHistory.toList();
 	QSettings settings;

gui/mainwindow.h

@@ -12,8 +12,9 @@
 #include <QProgressDialog>
 #include "../shared/looqsquery.h"
 #include "../shared/indexsyncer.h"
-#include "ipcpreviewclient.h"
+#include "previewcoordinator.h"
 #include "indexer.h"
+#include "tagmanager.h"
 namespace Ui
 {
 class MainWindow;
@@ -27,8 +28,9 @@ class MainWindow : public QMainWindow
 	DatabaseFactory *dbFactory;
 	SqliteDbService *dbService;
 	Ui::MainWindow *ui;
-	IPCPreviewClient ipcPreviewClient;
+
-	QThread ipcClientThread;
+	PreviewCoordinator previewCoordinator;
+
 	QThread syncerThread;
 	Indexer *indexer;
 	IndexSyncer *indexSyncer;
@@ -36,18 +38,15 @@ class MainWindow : public QMainWindow
 	QFileIconProvider iconProvider;
 	QSqlDatabase db;
 	QFutureWatcher<QVector<SearchResult>> searchWatcher;
-	QVector<SearchResult> previewableSearchResults;
 	LooqsQuery contentSearchQuery;
 	QVector<QString> searchHistory;
+
+	TagManager *tagManager;
+
 	int currentSearchHistoryIndex = 0;
 	QString currentSavedSearchText;
-	QHash<QString, int> previewOrder; /* Quick lookup for the order a preview should have */
-	QMap<int, QWidget *>
-		previewWidgetOrderCache /* Saves those that arrived out of order to be inserted later at the correct pos */;
 	bool previewDirty = false;
 	int previewsPerPage = 20;
-	unsigned int processedPdfPreviews = 0;
-	unsigned int currentPreviewGeneration = 1;
 
 	void connectSignals();
 	void makePreviews(int page);
@@ -56,20 +55,20 @@ class MainWindow : public QMainWindow
 	void keyPressEvent(QKeyEvent *event) override;
 	void handleSearchResults(const QVector<SearchResult> &results);
 	void handleSearchError(QString error);
-	void createSearchResutlMenu(QMenu &menu, const QFileInfo &fileInfo);
+	void createSearchResultMenu(QMenu &menu, const QFileInfo &fileInfo);
 	void openDocument(QString path, int num);
 	void openFile(QString path);
 	void initSettingsTabs();
 	int currentSelectedScale();
 	void processShortcut(int key);
-	bool eventFilter(QObject *object, QEvent *event);
+	bool eventFilter(QObject *object, QEvent *event) override;
 
   private slots:
 	void lineEditReturnPressed();
 	void treeSearchItemActivated(QTreeWidgetItem *item, int i);
 	void showSearchResultsContextMenu(const QPoint &point);
 	void tabChanged();
-	void previewReceived(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration);
+	void previewReceived();
 	void comboScaleChanged(int i);
 	void spinPreviewPageValueChanged(int val);
 	void startIndexing();

gui/mainwindow.ui

@@ -18,16 +18,13 @@
     <item>
      <widget class="QLineEdit" name="txtSearch"/>
     </item>
-    <item>
-     <layout class="QHBoxLayout" name="horizontalLayout_3"/>
-    </item>
     <item>
      <widget class="QTabWidget" name="tabWidget">
       <property name="tabPosition">
        <enum>QTabWidget::South</enum>
       </property>
       <property name="currentIndex">
-       <number>1</number>
+       <number>2</number>
       </property>
       <widget class="QWidget" name="resultsTab">
        <attribute name="title">
@@ -82,7 +79,7 @@
              <x>0</x>
              <y>0</y>
              <width>1244</width>
-             <height>633</height>
+             <height>641</height>
             </rect>
            </property>
            <layout class="QHBoxLayout" name="horizontalLayout"/>
@@ -195,62 +192,6 @@
        </attribute>
        <layout class="QGridLayout" name="gridLayout">
         <item row="6" column="0">
-         <widget class="QLineEdit" name="txtIgnorePatterns"/>
-        </item>
-        <item row="11" column="0">
-         <widget class="QPushButton" name="btnStartIndexing">
-          <property name="text">
-           <string>Start indexing</string>
-          </property>
-         </widget>
-        </item>
-        <item row="1" column="0">
-         <widget class="QGroupBox" name="groupBoxPaths">
-          <property name="title">
-           <string>Add paths to scan</string>
-          </property>
-          <layout class="QGridLayout" name="gridLayout_2">
-           <item row="1" column="0">
-            <widget class="QLineEdit" name="txtPathScanAdd"/>
-           </item>
-           <item row="3" column="0" colspan="5">
-            <widget class="QListWidget" name="lstPaths"/>
-           </item>
-           <item row="1" column="3">
-            <widget class="QToolButton" name="btnDeletePath">
-             <property name="enabled">
-              <bool>false</bool>
-             </property>
-             <property name="text">
-              <string>Delete</string>
-             </property>
-            </widget>
-           </item>
-           <item row="1" column="1">
-            <widget class="QPushButton" name="btnChoosePath">
-             <property name="text">
-              <string>...</string>
-             </property>
-            </widget>
-           </item>
-           <item row="1" column="2">
-            <widget class="QPushButton" name="btnAddPath">
-             <property name="text">
-              <string>Add</string>
-             </property>
-            </widget>
-           </item>
-          </layout>
-         </widget>
-        </item>
-        <item row="5" column="0">
-         <widget class="QLabel" name="label">
-          <property name="text">
-           <string>Ignore patterns, separated by ';'. Example: *.js;*Downloads*</string>
-          </property>
-         </widget>
-        </item>
-        <item row="9" column="0">
          <widget class="QGroupBox" name="groupBoxIndexProgress">
           <property name="contextMenuPolicy">
            <enum>Qt::PreventContextMenu</enum>
@@ -452,6 +393,108 @@
           </layout>
          </widget>
         </item>
+        <item row="2" column="0">
+         <widget class="QGroupBox" name="groupBoxIndexOptions">
+          <property name="title">
+           <string>Index options</string>
+          </property>
+          <layout class="QVBoxLayout" name="verticalLayout_11">
+           <item>
+            <widget class="QLabel" name="label">
+             <property name="text">
+              <string>Ignore patterns, separated by ';'. Example: *.js;*Downloads*:</string>
+             </property>
+            </widget>
+           </item>
+           <item>
+            <widget class="QLineEdit" name="txtIgnorePatterns"/>
+           </item>
+           <item>
+            <widget class="Line" name="line">
+             <property name="orientation">
+              <enum>Qt::Horizontal</enum>
+             </property>
+            </widget>
+           </item>
+           <item>
+            <widget class="QRadioButton" name="radioIndexEverything">
+             <property name="text">
+              <string>Index everything (metadata + file content)</string>
+             </property>
+             <property name="checked">
+              <bool>true</bool>
+             </property>
+            </widget>
+           </item>
+           <item>
+            <widget class="QCheckBox" name="chkFillContentForContentless">
+             <property name="enabled">
+              <bool>true</bool>
+             </property>
+             <property name="text">
+              <string>Index content for files previously indexed without content</string>
+             </property>
+             <property name="checked">
+              <bool>false</bool>
+             </property>
+            </widget>
+           </item>
+           <item>
+            <widget class="QRadioButton" name="radioMetadataOnly">
+             <property name="text">
+              <string>Index metadata only, don't process content of files</string>
+             </property>
+            </widget>
+           </item>
+          </layout>
+         </widget>
+        </item>
+        <item row="8" column="0">
+         <widget class="QPushButton" name="btnStartIndexing">
+          <property name="text">
+           <string>Start indexing</string>
+          </property>
+         </widget>
+        </item>
+        <item row="1" column="0">
+         <widget class="QGroupBox" name="groupBoxPaths">
+          <property name="title">
+           <string>Add paths to scan</string>
+          </property>
+          <layout class="QGridLayout" name="gridLayout_2">
+           <item row="1" column="0">
+            <widget class="QLineEdit" name="txtPathScanAdd"/>
+           </item>
+           <item row="3" column="0" colspan="5">
+            <widget class="QListWidget" name="lstPaths"/>
+           </item>
+           <item row="1" column="3">
+            <widget class="QToolButton" name="btnDeletePath">
+             <property name="enabled">
+              <bool>false</bool>
+             </property>
+             <property name="text">
+              <string>Delete</string>
+             </property>
+            </widget>
+           </item>
+           <item row="1" column="1">
+            <widget class="QPushButton" name="btnChoosePath">
+             <property name="text">
+              <string>...</string>
+             </property>
+            </widget>
+           </item>
+           <item row="1" column="2">
+            <widget class="QPushButton" name="btnAddPath">
+             <property name="text">
+              <string>Add</string>
+             </property>
+            </widget>
+           </item>
+          </layout>
+         </widget>
+        </item>
        </layout>
       </widget>
       <widget class="QWidget" name="settingsTab">
@@ -701,5 +744,22 @@
  </widget>
  <layoutdefault spacing="6" margin="11"/>
  <resources/>
- <connections/>
+ <connections>
+  <connection>
+   <sender>radioIndexEverything</sender>
+   <signal>toggled(bool)</signal>
+   <receiver>chkFillContentForContentless</receiver>
+   <slot>setEnabled(bool)</slot>
+   <hints>
+    <hint type="sourcelabel">
+     <x>639</x>
+     <y>464</y>
+    </hint>
+    <hint type="destinationlabel">
+     <x>639</x>
+     <y>497</y>
+    </hint>
+   </hints>
+  </connection>
+ </connections>
 </ui>

gui/previewcoordinator.cpp

@@ -0,0 +1,97 @@
+#include "previewcoordinator.h"
+#include <QFileInfo>
+
+PreviewCoordinator::PreviewCoordinator()
+{
+	this->ipcPreviewClient.moveToThread(&this->ipcClientThread);
+
+	connect(&ipcPreviewClient, &IPCPreviewClient::previewReceived, this, &PreviewCoordinator::handleReceivedPreview,
+			Qt::QueuedConnection);
+	connect(&ipcPreviewClient, &IPCPreviewClient::finished, this, [&] { emit completedGeneration(); });
+	connect(this, &PreviewCoordinator::ipcStartGeneration, &ipcPreviewClient, &IPCPreviewClient::startGeneration,
+			Qt::QueuedConnection);
+
+	this->ipcClientThread.start();
+}
+
+void PreviewCoordinator::init(const QVector<SearchResult> &searchResults)
+{
+	this->previewableSearchResults.clear();
+	for(const SearchResult &result : searchResults)
+	{
+		if(result.wasContentSearch)
+		{
+			QString path = result.fileData.absPath;
+			// HACK until we can preview them properly
+			if(path.endsWith(".html") || path.endsWith(".htm"))
+			{
+				continue;
+			}
+			QFileInfo info{path};
+			if(info.exists())
+			{
+				this->previewableSearchResults.append(result);
+			}
+		}
+	}
+}
+
+void PreviewCoordinator::setSocketPath(QString socketPath)
+{
+	this->socketPath = socketPath;
+	this->ipcPreviewClient.setSocketPath(socketPath);
+}
+
+int PreviewCoordinator::previewableCount() const
+{
+	return this->previewableSearchResults.count();
+}
+
+QSharedPointer<PreviewResult> PreviewCoordinator::resultAt(int index)
+{
+	if(this->previewResults.size() > index)
+	{
+		return {this->previewResults[index]};
+	}
+	return {nullptr};
+}
+
+const QVector<SearchResult> &PreviewCoordinator::getPreviewableSearchResults() const
+{
+	return this->previewableSearchResults;
+}
+
+void PreviewCoordinator::handleReceivedPreview(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration)
+{
+	if(previewGeneration < this->currentPreviewGeneration)
+	{
+		return;
+	}
+	if(!preview.isNull() && preview->hasPreview())
+	{
+		QString docPath = preview->getDocumentPath();
+		auto previewPage = preview->getPage();
+		int pos = previewOrder[docPath + QString::number(previewPage)];
+		this->previewResults[pos] = preview;
+		emit previewReady();
+	}
+}
+
+void PreviewCoordinator::startGeneration(RenderConfig config, const QVector<RenderTarget> &targets)
+{
+	++this->currentPreviewGeneration;
+
+	this->previewOrder.clear();
+	this->previewResults.clear();
+
+	this->previewResults.resize(targets.size());
+	this->previewResults.fill(nullptr);
+
+	int i = 0;
+	for(const RenderTarget &target : targets)
+	{
+		this->previewOrder[target.path + QString::number(target.page)] = i++;
+	}
+
+	emit ipcStartGeneration(config, targets);
+}

gui/previewcoordinator.h

@@ -0,0 +1,48 @@
+#ifndef PREVIEWCOORDINATOR_H
+#define PREVIEWCOORDINATOR_H
+#include <QVector>
+#include <QObject>
+#include <QThread>
+#include "searchresult.h"
+#include "previewresult.h"
+#include "ipcpreviewclient.h"
+#include "rendertarget.h"
+class PreviewCoordinator : public QObject
+{
+	Q_OBJECT
+  private:
+	QThread ipcClientThread;
+	IPCPreviewClient ipcPreviewClient;
+	QString socketPath;
+
+	QVector<QSharedPointer<PreviewResult>> previewResults;
+	QVector<SearchResult> previewableSearchResults;
+
+	unsigned int currentPreviewGeneration = 1;
+
+	/* Quick lookup table for the order a preview should have */
+	QHash<QString, int> previewOrder;
+
+  public:
+	PreviewCoordinator();
+
+	void init(const QVector<SearchResult> &searchResults);
+
+	int previewableCount() const;
+	const QVector<SearchResult> &getPreviewableSearchResults() const;
+
+	QSharedPointer<PreviewResult> resultAt(int index);
+
+	void setSocketPath(QString socketPath);
+  public slots:
+	void startGeneration(RenderConfig config, const QVector<RenderTarget> &targets);
+	void handleReceivedPreview(QSharedPointer<PreviewResult> preview, unsigned int previewGeneration);
+
+  signals:
+	void previewReady();
+	void completedGeneration();
+	void error(QString);
+	void ipcStartGeneration(RenderConfig config, const QVector<RenderTarget> &targets);
+};
+
+#endif // PREVIEWCOORDINATOR_H

gui/previewgeneratorodt.cpp

@@ -24,7 +24,7 @@ QSharedPointer<PreviewResult> PreviewGeneratorOdt::generate(RenderConfig config,
 		throw LooqsGeneralException("Error while reading content.xml of " + documentPath);
 	}
 	TagStripperProcessor tsp;
-	QString content = tsp.process(entireContent).first().content;
+	QString content = tsp.process(entireContent).constFirst().content;
 
 	PreviewGeneratorPlainText plainTextGenerator;
 	result->setText(plainTextGenerator.generatePreviewText(content, config, info.fileName()));

gui/previewgeneratorpdf.cpp

@@ -20,6 +20,8 @@ Poppler::Document *PreviewGeneratorPdf::document(QString path)
 		return nullptr;
 	}
 	result->setRenderHint(Poppler::Document::TextAntialiasing);
+	result->setRenderHint(Poppler::Document::TextHinting);
+	result->setRenderHint(Poppler::Document::TextSlightHinting);
 
 	locker.relock();
 	documentcache.insert(path, result);

gui/previewgeneratorplaintext.cpp

@@ -195,7 +195,7 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
 				  int totalWordsA = 0;
 				  int differentWordsB = 0;
 				  int totalWordsB = 0;
-				  for(int count : a.wordCountMap.values())
+				  for(int count : qAsConst(a.wordCountMap))
 				  {
 					  if(count > 0)
 					  {
@@ -203,7 +203,7 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
 					  }
 					  totalWordsA += count;
 				  }
-				  for(int count : b.wordCountMap.values())
+				  for(int count : qAsConst(b.wordCountMap))
 				  {
 					  if(count > 0)
 					  {
@@ -246,6 +246,8 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
 			totalWordCountMap[it->first] = totalWordCountMap.value(it->first, 0) + it->second;
 		}
 	}
+	if(!resultText.isEmpty())
+	{
 		if(isTruncated)
 		{
 			header += "(truncated) ";
@@ -256,7 +258,9 @@ QString PreviewGeneratorPlainText::generateLineBasedPreviewText(QTextStream &in,
 		}
 		header += "<hr>";
 
-	return header + resultText;
+		resultText = header + resultText;
+	}
+	return resultText;
 }
 
 QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig config, QString documentPath,

shared/filesaver.cpp

@@ -25,10 +25,22 @@ SaveFileResult FileSaver::addFile(QString path)
 	QString absPath = info.absoluteFilePath();
 
 	auto mtime = info.lastModified().toSecsSinceEpoch();
-	if(this->dbService->fileExistsInDatabase(absPath, mtime))
+
+	bool exists = false;
+	if(this->fileSaverOptions.fillExistingContentless)
+	{
+		exists = this->dbService->fileExistsInDatabase(absPath, mtime, 'c');
+	}
+	else
+	{
+		exists = this->dbService->fileExistsInDatabase(absPath, mtime);
+	}
+
+	if(exists)
 	{
 		return SKIPPED;
 	}
+
 	return saveFile(info);
 }
 
@@ -38,18 +50,17 @@ SaveFileResult FileSaver::updateFile(QString path)
 	return saveFile(info);
 }
 
-int FileSaver::addFiles(const QVector<QString> paths, bool keepGoing, bool verbose)
+int FileSaver::addFiles(const QVector<QString> paths)
 {
-	return processFiles(paths, std::bind(&FileSaver::addFile, this, std::placeholders::_1), keepGoing, verbose);
+	return processFiles(paths, std::bind(&FileSaver::addFile, this, std::placeholders::_1));
 }
 
-int FileSaver::updateFiles(const QVector<QString> paths, bool keepGoing, bool verbose)
+int FileSaver::updateFiles(const QVector<QString> paths)
 {
-	return processFiles(paths, std::bind(&FileSaver::updateFile, this, std::placeholders::_1), keepGoing, verbose);
+	return processFiles(paths, std::bind(&FileSaver::updateFile, this, std::placeholders::_1));
 }
 
-int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc,
+int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc)
-							bool keepGoing, bool verbose)
 {
 	std::atomic<bool> terminate{false};
 	std::atomic<int> processedCount{0};
@@ -60,7 +71,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
 								  {
 									  return;
 								  }
-								  if(verbose)
+								  if(this->fileSaverOptions.verbose)
 								  {
 									  Logger::info() << "Processing " << path << Qt::endl;
 								  }
@@ -68,7 +79,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
 								  if(result == DBFAIL || result == PROCESSFAIL)
 								  {
 									  Logger::error() << "Failed to process " << path << Qt::endl;
-									  if(!keepGoing)
+									  if(!this->fileSaverOptions.keepGoing)
 									  {
 										  terminate = true;
 									  }
@@ -76,7 +87,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
 								  else
 								  {
 									  ++processedCount;
-									  if(verbose)
+									  if(this->fileSaverOptions.verbose)
 									  {
 										  if(result == SKIPPED)
 										  {
@@ -120,11 +131,26 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
 		{
 			if(canonicalPath.startsWith(excludedPath))
 			{
+				if(this->fileSaverOptions.verbose)
+				{
+					Logger::info() << "Skipped due to excluded path";
+				}
 				return SKIPPED;
 			}
 		}
 
-		if(fileInfo.size() > 0)
+		bool mustFillContent = this->fileSaverOptions.fillExistingContentless;
+		if(!mustFillContent)
+		{
+			mustFillContent = !this->fileSaverOptions.metadataOnly;
+			if(mustFillContent)
+			{
+				auto filetype = this->dbService->queryFileType(fileInfo.absolutePath());
+				mustFillContent = !filetype.has_value() || filetype.value() == 'c';
+			}
+		}
+
+		if(fileInfo.size() > 0 && mustFillContent)
 		{
 			QProcess process;
 			QStringList args;
@@ -159,7 +185,7 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
 			}
 		}
 	}
-	SaveFileResult result = this->dbService->saveFile(fileInfo, pageData);
+	SaveFileResult result = this->dbService->saveFile(fileInfo, pageData, this->fileSaverOptions.metadataOnly);
 	if(result == OK && processorReturnCode == OK_WASEMPTY)
 	{
 		return OK_WASEMPTY;

shared/filesaver.h

@@ -2,6 +2,7 @@
 #define FILESAVER_H
 #include <QSqlDatabase>
 #include <QFileInfo>
+#include "filesaveroptions.h"
 #include "pagedata.h"
 #include "filedata.h"
 #include "sqlitedbservice.h"
@@ -11,16 +12,21 @@ class FileSaver
   private:
 	SqliteDbService *dbService;
 	QStringList excludedPaths = Common::excludedPaths();
+	FileSaverOptions fileSaverOptions;
 
   public:
 	FileSaver(SqliteDbService &dbService);
 	SaveFileResult addFile(QString path);
 	SaveFileResult updateFile(QString path);
 	SaveFileResult saveFile(const QFileInfo &fileInfo);
-	int processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc,
+	int processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc);
-					 bool keepGoing, bool verbose);
+	int addFiles(const QVector<QString> paths);
-	int addFiles(const QVector<QString> paths, bool keepGoing, bool verbose);
+	int updateFiles(const QVector<QString> paths);
-	int updateFiles(const QVector<QString> paths, bool keepGoing, bool verbose);
+
+	void setFileSaverOptions(FileSaverOptions options)
+	{
+		this->fileSaverOptions = options;
+	}
 };
 
 #endif // FILESAVER_H

shared/filesaveroptions.h

@@ -0,0 +1,14 @@
+#ifndef FILESAVEROPTIONS_H
+#define FILESAVEROPTIONS_H
+
+class FileSaverOptions
+{
+  public:
+	bool verbose = false;
+	bool keepGoing = false;
+	bool metadataOnly = false;
+	/* Whether those previously explicitly without content should be filled */
+	bool fillExistingContentless = false;
+};
+
+#endif // FILESAVEROPTIONS_H

shared/filescanworker.cpp

@@ -12,6 +12,7 @@ FileScanWorker::FileScanWorker(SqliteDbService &db, ConcurrentQueue<QString> &qu
 void FileScanWorker::run()
 {
 	FileSaver saver{*this->dbService};
+	saver.setFileSaverOptions(this->fileSaverOptions);
 	auto paths = queue->dequeue(batchsize);
 	for(QString &path : paths)
 	{
@@ -20,11 +21,18 @@ void FileScanWorker::run()
 		{
 			sfr = saver.addFile(path);
 		}
+		catch(LooqsGeneralException &e)
+		{
+			Logger::error() << e.message << Qt::endl;
+			sfr = PROCESSFAIL;
+		}
+
 		catch(std::exception &e)
 		{
-			Logger::error() << e.what();
+			Logger::error() << e.what() << Qt::endl;
 			sfr = PROCESSFAIL; // well...
 		}
+
 		emit result({path, sfr});
 		if(stopToken->load(std::memory_order_relaxed)) // TODO: relaxed should suffice here, but recheck
 		{
@@ -34,3 +42,8 @@ void FileScanWorker::run()
 	}
 	emit finished();
 }
+
+void FileScanWorker::setFileSaverOptions(FileSaverOptions options)
+{
+	this->fileSaverOptions = options;
+}

shared/filescanworker.h

@@ -15,12 +15,14 @@ class FileScanWorker : public QObject, public QRunnable
   protected:
 	SqliteDbService *dbService;
 	ConcurrentQueue<QString> *queue;
+	FileSaverOptions fileSaverOptions;
 	int batchsize;
 	std::atomic<bool> *stopToken;
 
   public:
 	FileScanWorker(SqliteDbService &db, ConcurrentQueue<QString> &queue, int batchsize, std::atomic<bool> &stopToken);
 	void run() override;
+	void setFileSaverOptions(FileSaverOptions options);
   signals:
 	void result(FileScanResult);
 	void finished();

shared/indexer.cpp

@@ -73,16 +73,6 @@ void Indexer::setTargetPaths(QVector<QString> pathsToScan)
 	this->pathsToScan = pathsToScan;
 }
 
-void Indexer::setVerbose(bool verbose)
-{
-	this->verbose = verbose;
-}
-
-void Indexer::setKeepGoing(bool keepGoing)
-{
-	this->keepGoing = keepGoing;
-}
-
 void Indexer::requestCancellation()
 {
 	this->dirScanner->cancel();
@@ -108,6 +98,7 @@ void Indexer::launchWorker(ConcurrentQueue<QString> &queue, int batchsize)
 	FileScanWorker *runnable = new FileScanWorker(*this->db, queue, batchsize, this->workerCancellationToken);
 	connect(runnable, &FileScanWorker::result, this, &Indexer::processFileScanResult);
 	connect(runnable, &FileScanWorker::finished, this, &Indexer::processFinishedWorker);
+	runnable->setFileSaverOptions(this->fileSaverOptions);
 	++this->runningWorkers;
 	QThreadPool::globalInstance()->start(runnable);
 }
@@ -120,24 +111,6 @@ void Indexer::dirScanProgress(int current, int total)
 
 void Indexer::processFileScanResult(FileScanResult result)
 {
-	if(isErrorSaveFileResult(result.second))
-	{
-		this->currentIndexResult.results.append(result);
-		if(!keepGoing)
-		{
-			this->requestCancellation();
-			emit finished();
-			return;
-		}
-	}
-	else
-	{
-		if(verbose)
-		{
-			this->currentIndexResult.results.append(result);
-		}
-	}
-
 	/* TODO: OK_WASEMPTY might need a special list */
 	if(result.second == OK || result.second == OK_WASEMPTY)
 	{
@@ -152,6 +125,24 @@ void Indexer::processFileScanResult(FileScanResult result)
 		++this->currentIndexResult.erroredPaths;
 	}
 
+	if(isErrorSaveFileResult(result.second))
+	{
+		this->currentIndexResult.results.append(result);
+		if(!this->fileSaverOptions.keepGoing)
+		{
+			this->requestCancellation();
+			emit finished();
+			return;
+		}
+	}
+	else
+	{
+		if(this->fileSaverOptions.verbose)
+		{
+			this->currentIndexResult.results.append(result);
+		}
+	}
+
 	QTime currentTime = QTime::currentTime();
 	if(currentScanProcessedCount++ == progressReportThreshold || this->lastProgressReportTime.secsTo(currentTime) >= 10)
 	{
@@ -175,3 +166,13 @@ void Indexer::processFinishedWorker()
 		emit finished();
 	}
 }
+
+void Indexer::setFileSaverOptions(FileSaverOptions options)
+{
+	this->fileSaverOptions = options;
+}
+
+void Indexer::setProgressReportThreshold(int threshold)
+{
+	this->progressReportThreshold = threshold;
+}

shared/indexer.h

@@ -52,8 +52,7 @@ class Indexer : public QObject
 {
 	Q_OBJECT
   protected:
-	bool verbose = false;
+	FileSaverOptions fileSaverOptions;
-	bool keepGoing = true;
 	SqliteDbService *db;
 
 	int progressReportThreshold = 50;
@@ -80,8 +79,10 @@ class Indexer : public QObject
 	void beginIndexing();
 	void setIgnorePattern(QStringList ignorePattern);
 	void setTargetPaths(QVector<QString> pathsToScan);
-	void setVerbose(bool verbose);
+
-	void setKeepGoing(bool keepGoing);
+	void setFileSaverOptions(FileSaverOptions options);
+
+	void setProgressReportThreshold(int threshold);
 
 	void requestCancellation();
 

shared/indexsyncer.cpp

@@ -7,21 +7,16 @@ IndexSyncer::IndexSyncer(SqliteDbService &dbService)
 	this->dbService = &dbService;
 }
 
+void IndexSyncer::setFileSaverOptions(FileSaverOptions options)
+{
+	fileSaverOptions = options;
+}
+
 void IndexSyncer::setDryRun(bool dryRun)
 {
 	this->dryRun = dryRun;
 }
 
-void IndexSyncer::setVerbose(bool verbose)
-{
-	this->verbose = verbose;
-}
-
-void IndexSyncer::setKeepGoing(bool keepGoing)
-{
-	this->keepGoing = keepGoing;
-}
-
 void IndexSyncer::setRemoveDeletedFromIndex(bool removeDeletedFromIndex)
 {
 	this->removeDeletedFromIndex = removeDeletedFromIndex;
@@ -35,7 +30,7 @@ void IndexSyncer::setPattern(QString pattern)
 void IndexSyncer::sync()
 {
 	this->stopToken.store(false, std::memory_order_relaxed);
-	FileSaver saver(*this->dbService);
+
 	QVector<FileData> files;
 	int offset = 0;
 	int limit = 10000;
@@ -87,7 +82,7 @@ void IndexSyncer::sync()
 						if(!this->dbService->deleteFile(fileData.absPath))
 						{
 							emit error("Error: Failed to delete " + fileData.absPath + " from the index");
-							if(!this->keepGoing)
+							if(!this->fileSaverOptions.keepGoing)
 							{
 								emit finished(totalUpdatesFilesCount, totalDeletedFilesCount, totalErroredFilesCount);
 								return;
@@ -104,13 +99,15 @@ void IndexSyncer::sync()
 			}
 		}
 
-		unsigned int updatedFilesCount = saver.updateFiles(filePathsToUpdate, keepGoing, verbose);
+		FileSaver saver(*this->dbService);
+		saver.setFileSaverOptions(this->fileSaverOptions);
+		unsigned int updatedFilesCount = saver.updateFiles(filePathsToUpdate);
 		unsigned int shouldHaveUpdatedCount = static_cast<unsigned int>(filePathsToUpdate.size());
 		if(updatedFilesCount != shouldHaveUpdatedCount)
 		{
 
 			totalErroredFilesCount += (shouldHaveUpdatedCount - updatedFilesCount);
-			if(!keepGoing)
+			if(!this->fileSaverOptions.keepGoing)
 			{
 				QString errorMsg = QString("Failed to update all files selected for updating in this batch. Updated") +
 								   updatedFilesCount + "out of" + shouldHaveUpdatedCount + "selected for updating";

shared/indexsyncer.h

@@ -1,16 +1,15 @@
 #ifndef INDEXSYNCER_H
 #define INDEXSYNCER_H
 #include "sqlitedbservice.h"
-
+#include "filesaveroptions.h"
 class IndexSyncer : public QObject
 {
 	Q_OBJECT
   private:
 	SqliteDbService *dbService = nullptr;
-	bool keepGoing = true;
+	FileSaverOptions fileSaverOptions;
 	bool removeDeletedFromIndex = true;
 	bool dryRun = false;
-	bool verbose = false;
 	QString pattern;
 
 	std::atomic<bool> stopToken{false};
@@ -18,12 +17,12 @@ class IndexSyncer : public QObject
   public:
 	IndexSyncer(SqliteDbService &dbService);
 
+	void setFileSaverOptions(FileSaverOptions options);
+
   public slots:
 	void sync();
 	void cancel();
 	void setDryRun(bool dryRun);
-	void setVerbose(bool verbose);
-	void setKeepGoing(bool keepGoing);
 	void setRemoveDeletedFromIndex(bool removeDeletedFromIndex);
 	void setPattern(QString pattern);
 

shared/looqsquery.cpp

@@ -7,6 +7,7 @@
 #include <optional>
 #include <algorithm>
 #include "looqsquery.h"
+#include "looqsgeneralexception.h"
 
 const QVector<Token> &LooqsQuery::getTokens() const
 {
@@ -180,7 +181,8 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
 
 	QStringList loneWords;
 	LooqsQuery result;
-	QRegularExpression rx("((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([^\\s])+)|(?<boolean>AND|OR)"
+	static QRegularExpression rx(
+		"((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([^\\s])+)|(?<boolean>AND|OR)"
 		"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[^\\s]+))");
 	QRegularExpressionMatchIterator i = rx.globalMatch(expression);
 	auto previousWasBool = [&result] { return !result.tokens.empty() && ((result.tokens.last().type & BOOL) == BOOL); };
@@ -283,6 +285,10 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
 			{
 				tokenType = FILTER_CONTENT_PAGE;
 			}
+			else if(filtername == "t" || filtername == "tag")
+			{
+				tokenType = FILTER_TAG_ASSIGNED;
+			}
 			// TODO: given this is not really a "filter", this feels slightly misplaced here
 			else if(filtername == "sort")
 			{

shared/looqsquery.h

@@ -2,7 +2,6 @@
 #define LOOQSQUERY_H
 #include <QString>
 #include <QVector>
-#include "looqsgeneralexception.h"
 #include "token.h"
 /* Fields that can be queried or sorted */
 enum QueryField
@@ -46,7 +45,7 @@ class LooqsQuery
 	void addToken(Token t);
 	void updateTokensMask()
 	{
-		for(const Token &t : tokens)
+		for(const Token &t : qAsConst(tokens))
 		{
 			this->tokensMask |= t.type;
 		}
@@ -92,14 +91,6 @@ class LooqsQuery
 		this->sortConditions = sortConditions;
 		updateTokensMask();
 	}
-
-	LooqsQuery(const LooqsQuery &o)
-	{
-		this->tokens = o.tokens;
-		this->sortConditions = o.sortConditions;
-		this->tokensMask = o.tokensMask;
-		this->limit = o.limit;
-	}
 };
 
 #endif // LOOQSQUERY_H

shared/migrations/5.sql

@@ -0,0 +1,6 @@
+CREATE TABLE tag(id integer PRIMARY KEY, name varchar(128) UNIQUE);
+CREATE TABLE filetag(fileid integer, tagid integer);
+CREATE INDEX filetag_fileid ON filetag(fileid);
+CREATE INDEX tag_id ON tag(id);
+CREATE INDEX file_path ON file ( path );
+UPDATE file SET filetype='c' WHERE filetype='f';

shared/migrations/migrations.qrc

@@ -4,5 +4,6 @@
         <file>2.sql</file>
         <file>3.sql</file>
         <file>4.sql</file>
+        <file>5.sql</file>
     </qresource>
 </RCC>

shared/nothingprocessor.h

@@ -10,7 +10,7 @@ class NothingProcessor : public Processor
 	NothingProcessor();
 
   public:
-	QVector<PageData> process(const QByteArray &data) const override
+	QVector<PageData> process(const QByteArray & /*data*/) const override
 	{
 		return {};
 	}

shared/odtprocessor.cpp

@@ -3,7 +3,7 @@
 #include "odtprocessor.h"
 #include "tagstripperprocessor.h"
 
-QVector<PageData> OdtProcessor::process(const QByteArray &data) const
+QVector<PageData> OdtProcessor::process(const QByteArray & /*data*/) const
 {
 	throw LooqsGeneralException("Not implemented yet");
 }

shared/paralleldirscanner.cpp

@@ -1,5 +1,3 @@
-#include "paralleldirscanner.h"
-
 #include <QRunnable>
 #include <QMutex>
 #include <QDirIterator>
@@ -7,7 +5,7 @@
 #include <QThreadPool>
 #include <functional>
 #include "dirscanworker.h"
-#include "logger.h"
+#include "paralleldirscanner.h"
 
 ParallelDirScanner::ParallelDirScanner()
 {

shared/shared.pro

@@ -60,6 +60,7 @@ SOURCES += sqlitesearch.cpp \
     processor.cpp \
     sandboxedprocessor.cpp \
     sqlitedbservice.cpp \
+    tagmanager.cpp \
     tagstripperprocessor.cpp \
     utils.cpp \
     ../submodules/exile.h/exile.c \
@@ -74,6 +75,7 @@ HEADERS += sqlitesearch.h \
     encodingdetector.h \
     filedata.h \
     filesaver.h \
+    filesaveroptions.h \
     filescanworker.h \
     indexer.h \
     indexsyncer.h \
@@ -92,6 +94,7 @@ HEADERS += sqlitesearch.h \
     savefileresult.h \
     searchresult.h \
     sqlitedbservice.h \
+    tagmanager.h \
     tagstripperprocessor.h \
     token.h \
     common.h \

shared/sqlitedbservice.cpp

@@ -2,25 +2,10 @@
 #include <QFileInfo>
 #include <QDateTime>
 #include <QSqlError>
+#include "looqsgeneralexception.h"
 #include "sqlitedbservice.h"
 #include "filedata.h"
 #include "logger.h"
-bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime)
-{
-	auto query = QSqlQuery(dbFactory->forCurrentThread());
-	query.prepare("SELECT 1 FROM file WHERE path = ? and mtime = ?");
-	query.addBindValue(path);
-	query.addBindValue(mtime);
-	if(!query.exec())
-	{
-		throw LooqsGeneralException("Error while trying to query for file existance: " + query.lastError().text());
-	}
-	if(!query.next())
-	{
-		return false;
-	}
-	return query.value(0).toBool();
-}
 
 QVector<SearchResult> SqliteDbService::search(const LooqsQuery &query)
 {
@@ -29,20 +14,29 @@ QVector<SearchResult> SqliteDbService::search(const LooqsQuery &query)
 	return searcher.search(query);
 }
 
-bool SqliteDbService::fileExistsInDatabase(QString path)
+std::optional<QChar> SqliteDbService::queryFileType(QString absPath)
 {
-	auto query = QSqlQuery(dbFactory->forCurrentThread());
+	auto query = exec("SELECT filetype FROM file WHERE path = ?", {absPath});
-	query.prepare("SELECT 1 FROM file WHERE path = ?");
-	query.addBindValue(path);
-	if(!query.exec())
-	{
-		throw LooqsGeneralException("Error while trying to query for file existance: " + query.lastError().text());
-	}
 	if(!query.next())
 	{
-		return false;
+		return {};
 	}
-	return query.value(0).toBool();
+	return query.value(0).toChar();
+}
+
+bool SqliteDbService::fileExistsInDatabase(QString path)
+{
+	return execBool("SELECT 1 FROM file WHERE path = ?", {path});
+}
+
+bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime)
+{
+	return execBool("SELECT 1 FROM file WHERE path = ? AND mtime = ?", {path, mtime});
+}
+
+bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime, QChar fileType)
+{
+	return execBool("SELECT 1 FROM file WHERE path = ? AND mtime = ? AND filetype = ?", {path, mtime, fileType});
 }
 
 SqliteDbService::SqliteDbService(DatabaseFactory &dbFactory)
@@ -110,6 +104,117 @@ unsigned int SqliteDbService::getFiles(QVector<FileData> &results, QString wildC
 	return processedRows;
 }
 
+QVector<QString> SqliteDbService::getTags()
+{
+	QVector<QString> result;
+	auto query = QSqlQuery(dbFactory->forCurrentThread());
+	query.prepare("SELECT name FROM tag ORDER by name ASC");
+	query.setForwardOnly(true);
+	if(!query.exec())
+	{
+		throw LooqsGeneralException("Error while trying to retrieve tags from database: " + query.lastError().text());
+	}
+	while(query.next())
+	{
+		QString tagname = query.value(0).toString();
+		result.append(tagname);
+	}
+	return result;
+}
+
+QVector<QString> SqliteDbService::getTagsForPath(QString path)
+{
+	QVector<QString> result;
+	auto query = QSqlQuery(dbFactory->forCurrentThread());
+	query.prepare("SELECT name FROM tag INNER JOIN filetag ON tag.id = filetag.tagid INNER JOIN file ON filetag.fileid "
+				  "= file.id WHERE file.path = ? ORDER BY name ASC");
+	query.addBindValue(path);
+	query.setForwardOnly(true);
+	if(!query.exec())
+	{
+		throw LooqsGeneralException("Error while trying to retrieve tags from database: " + query.lastError().text());
+	}
+	while(query.next())
+	{
+		QString tagname = query.value(0).toString();
+		result.append(tagname);
+	}
+	return result;
+}
+
+QVector<QString> SqliteDbService::getPathsForTag(QString tag)
+{
+	QVector<QString> result;
+	auto query = QSqlQuery(dbFactory->forCurrentThread());
+	query.prepare(
+		"SELECT file.path FROM tag INNER JOIN filetag ON tag.id = filetag.tagid INNER JOIN file ON filetag.fileid "
+		"= file.id WHERE tag.name = ?");
+	query.addBindValue(tag.toLower());
+	query.setForwardOnly(true);
+	if(!query.exec())
+	{
+		throw LooqsGeneralException("Error while trying to retrieve paths from database: " + query.lastError().text());
+	}
+	while(query.next())
+	{
+		QString path = query.value(0).toString();
+		result.append(path);
+	}
+	return result;
+}
+
+bool SqliteDbService::setTags(QString path, const QSet<QString> &tags)
+{
+	QSqlDatabase db = dbFactory->forCurrentThread();
+	if(!db.transaction())
+	{
+		Logger::error() << "Failed to open transaction for " << path << " : " << db.lastError() << Qt::endl;
+		return false;
+	}
+
+	QSqlQuery deletionQuery = QSqlQuery(db);
+	deletionQuery.prepare("DELETE FROM filetag WHERE fileid = (SELECT id FROM file WHERE path = ?)");
+	deletionQuery.addBindValue(path);
+	if(!deletionQuery.exec())
+	{
+		db.rollback();
+		Logger::error() << "Failed to delete existing tags " << deletionQuery.lastError() << Qt::endl;
+		return false;
+	}
+
+	for(const QString &tag : tags)
+	{
+		QSqlQuery tagQuery = QSqlQuery(db);
+		tagQuery.prepare("INSERT OR IGNORE INTO tag (name) VALUES(?)");
+		tagQuery.addBindValue(tag.toLower());
+		if(!tagQuery.exec())
+		{
+			db.rollback();
+			Logger::error() << "Failed to insert tag " << tagQuery.lastError() << Qt::endl;
+			return false;
+		}
+		QSqlQuery fileTagQuery(db);
+		fileTagQuery.prepare(
+			"INSERT INTO filetag(fileid, tagid) VALUES((SELECT id FROM file WHERE path = ?), (SELECT id "
+			"FROM tag WHERE name = ?))");
+		fileTagQuery.bindValue(0, path);
+		fileTagQuery.bindValue(1, tag);
+		if(!fileTagQuery.exec())
+		{
+			db.rollback();
+			Logger::error() << "Failed to assign tag to file" << Qt::endl;
+			return false;
+		}
+	}
+	if(!db.commit())
+	{
+		db.rollback();
+		Logger::error() << "Failed to commit transaction when saving tags" << Qt::endl;
+		return false;
+	}
+	return true;
+}
+
 bool SqliteDbService::insertToFTS(bool useTrigrams, QSqlDatabase &db, int fileid, QVector<PageData> &pageData)
 {
 	QString ftsInsertStatement;
@@ -148,11 +253,40 @@ bool SqliteDbService::insertToFTS(bool useTrigrams, QSqlDatabase &db, int fileid
 	return true;
 }
 
-SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &pageData)
+QSqlQuery SqliteDbService::exec(QString querystr, std::initializer_list<QVariant> args)
+{
+	auto query = QSqlQuery(dbFactory->forCurrentThread());
+	query.prepare(querystr);
+	for(const QVariant &v : args)
+	{
+		query.addBindValue(v);
+	}
+	if(!query.exec())
+	{
+		throw LooqsGeneralException("Error while exec(): " + query.lastError().text() + " for query: " + querystr);
+	}
+	return query;
+}
+
+bool SqliteDbService::execBool(QString querystr, std::initializer_list<QVariant> args)
+{
+	auto query = exec(querystr, args);
+	if(!query.next())
+	{
+		return false;
+	}
+	return query.value(0).toBool();
+}
+
+SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &pageData, bool pathsOnly)
 {
 	QString absPath = fileInfo.absoluteFilePath();
 	auto mtime = fileInfo.lastModified().toSecsSinceEpoch();
-	QChar fileType = fileInfo.isDir() ? 'd' : 'f';
+	QChar fileType = fileInfo.isDir() ? 'd' : 'c';
+	if(pathsOnly)
+	{
+		fileType = 'f';
+	}
 
 	QSqlDatabase db = dbFactory->forCurrentThread();
 	QSqlQuery delQuery(db);
@@ -186,6 +320,8 @@ SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &
 		return DBFAIL;
 	}
 
+	if(!pathsOnly)
+	{
 		int lastid = inserterQuery.lastInsertId().toInt();
 		if(!insertToFTS(false, db, lastid, pageData))
 		{
@@ -199,6 +335,8 @@ SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &
 			Logger::error() << "Failed to insert data to FTS index " << Qt::endl;
 			return DBFAIL;
 		}
+	}
+
 	if(!db.commit())
 	{
 		db.rollback();
@@ -207,3 +345,123 @@ SaveFileResult SqliteDbService::saveFile(QFileInfo fileInfo, QVector<PageData> &
 	}
 	return OK;
 }
+
+bool SqliteDbService::addTag(QString tag, QString path)
+{
+	QVector<QString> paths;
+	paths.append(path);
+	return addTag(tag, paths);
+}
+
+bool SqliteDbService::addTag(QString tag, const QVector<QString> &paths)
+{
+	QSqlDatabase db = dbFactory->forCurrentThread();
+	QSqlQuery tagQuery(db);
+	QSqlQuery fileTagQuery(db);
+
+	tag = tag.toLower();
+
+	tagQuery.prepare("INSERT OR IGNORE INTO tag (name) VALUES(?)");
+	tagQuery.addBindValue(tag);
+
+	fileTagQuery.prepare("INSERT INTO filetag(fileid, tagid) VALUES((SELECT id FROM file WHERE path = ?), (SELECT id "
+						 "FROM tag WHERE name = ?))");
+	fileTagQuery.bindValue(1, tag);
+	if(!db.transaction())
+	{
+		Logger::error() << "Failed to open transaction to add paths for tag " << tag << " : " << db.lastError()
+						<< Qt::endl;
+		return false;
+	}
+	if(!tagQuery.exec())
+	{
+		db.rollback();
+		Logger::error() << "Failed INSERT query" << tagQuery.lastError() << Qt::endl;
+		return false;
+	}
+
+	for(const QString &path : paths)
+	{
+		fileTagQuery.bindValue(0, path);
+		if(!fileTagQuery.exec())
+		{
+			db.rollback();
+			Logger::error() << "Failed to add paths to tag" << Qt::endl;
+			return false;
+		}
+	}
+
+	if(!db.commit())
+	{
+		db.rollback();
+		Logger::error() << "Failed to commit tag insertion transaction" << db.lastError() << Qt::endl;
+		return false;
+	}
+
+	return true;
+}
+
+bool SqliteDbService::removePathsForTag(QString tag, const QVector<QString> &paths)
+{
+	QSqlDatabase db = dbFactory->forCurrentThread();
+	QSqlQuery tagQuery(db);
+	QSqlQuery fileTagQuery(db);
+
+	tag = tag.toLower();
+
+	fileTagQuery.prepare(
+		"DELETE FROM filetag WHERE fileid = (SELECT id FROM file WHERE path = ?) AND tagid = (SELECT id "
+		"FROM tag WHERE name = ?)");
+
+	fileTagQuery.bindValue(1, tag);
+	for(const QString &path : paths)
+	{
+		fileTagQuery.bindValue(0, path);
+		if(!fileTagQuery.exec())
+		{
+			Logger::error() << "An error occured while trying to remove paths from tag assignment" << Qt::endl;
+			return false;
+		}
+	}
+	return true;
+}
+
+bool SqliteDbService::deleteTag(QString tag)
+{
+	QSqlDatabase db = dbFactory->forCurrentThread();
+	if(!db.transaction())
+	{
+		Logger::error() << "Failed to open transaction while trying to delete tag " << tag << " : " << db.lastError()
+						<< Qt::endl;
+		return false;
+	}
+
+	tag = tag.toLower();
+	QSqlQuery assignmentDeleteQuery(db);
+	assignmentDeleteQuery.prepare("DELETE FROM filetag WHERE tagid = (SELECT id FROM tag WHERE name = ?)");
+	assignmentDeleteQuery.addBindValue(tag);
+	if(!assignmentDeleteQuery.exec())
+	{
+		db.rollback();
+		Logger::error() << "Error while trying to delete tag: " << db.lastError() << Qt::endl;
+		return false;
+	}
+
+	QSqlQuery deleteTagQuery(db);
+	deleteTagQuery.prepare("DELETE FROM tag WHERE name = ?");
+	deleteTagQuery.addBindValue(tag);
+	if(!deleteTagQuery.exec())
+	{
+		db.rollback();
+		Logger::error() << "Error while trying to delete tag: " << db.lastError() << Qt::endl;
+		return false;
+	}
+
+	if(!db.commit())
+	{
+		db.rollback();
+		Logger::error() << "Error while trying to delete tag: " << db.lastError() << Qt::endl;
+		return false;
+	}
+	return true;
+}

shared/sqlitedbservice.h

@@ -1,6 +1,8 @@
 #ifndef SQLITEDBSERVICE_H
 #define SQLITEDBSERVICE_H
 #include <QFileInfo>
+#include <optional>
+
 #include "databasefactory.h"
 #include "utils.h"
 #include "pagedata.h"
@@ -15,14 +17,31 @@ class SqliteDbService
 	DatabaseFactory *dbFactory = nullptr;
 	bool insertToFTS(bool useTrigrams, QSqlDatabase &db, int fileid, QVector<PageData> &pageData);
 
+	QSqlQuery exec(QString query, std::initializer_list<QVariant> args);
+	bool execBool(QString querystr, std::initializer_list<QVariant> args);
+
   public:
 	SqliteDbService(DatabaseFactory &dbFactory);
-	SaveFileResult saveFile(QFileInfo fileInfo, QVector<PageData> &pageData);
+	SaveFileResult saveFile(QFileInfo fileInfo, QVector<PageData> &pageData, bool pathsOnly);
-	unsigned int getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit);
+
 	bool deleteFile(QString path);
 	bool fileExistsInDatabase(QString path);
 	bool fileExistsInDatabase(QString path, qint64 mtime);
+	bool fileExistsInDatabase(QString path, qint64 mtime, QChar filetype);
+	unsigned int getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit);
+
+	bool addTag(QString tag, QString path);
+	bool addTag(QString tag, const QVector<QString> &paths);
+	QVector<QString> getTags();
+	QVector<QString> getTagsForPath(QString path);
+	QVector<QString> getPathsForTag(QString path);
+	bool setTags(QString path, const QSet<QString> &tags);
+	bool removePathsForTag(QString tag, const QVector<QString> &paths);
+	bool deleteTag(QString tag);
+
 	QVector<SearchResult> search(const LooqsQuery &query);
+
+	std::optional<QChar> queryFileType(QString absPath);
 };
 
 #endif // SQLITEDBSERVICE_H

shared/sqlitesearch.cpp

@@ -69,7 +69,7 @@ QString SqliteSearch::createSortSql(const QVector<SortCondition> sortConditions)
 QString SqliteSearch::escapeFtsArgument(QString ftsArg)
 {
 	QString result;
-	QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
+	static QRegularExpression extractor(R"#("([^"]*)"|([^\s]+))#");
 	QRegularExpressionMatchIterator i = extractor.globalMatch(ftsArg);
 	while(i.hasNext())
 	{
@@ -143,13 +143,17 @@ QPair<QString, QVector<QString>> SqliteSearch::createSql(const Token &token)
 	{
 		return {" fts MATCH ? ", {escapeFtsArgument(value)}};
 	}
+	if(token.type == FILTER_TAG_ASSIGNED)
+	{
+		return {" file.id IN (SELECT fileid FROM filetag WHERE tagid = (SELECT id FROM tag WHERE name = ?)) ",
+				{value.toLower()}};
+	}
 	throw LooqsGeneralException("Unknown token passed (should not happen)");
 }
 
 QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
 {
 	QString whereSql;
-	QString joinSql;
 	QVector<QString> bindValues;
 	bool isContentSearch = (query.getTokensMask() & FILTER_CONTENT) == FILTER_CONTENT;
 	if(query.getTokens().isEmpty())
@@ -180,18 +184,18 @@ QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
 		}
 		QString whereSqlTrigram = whereSql;
 		whereSqlTrigram.replace("fts MATCH", "fts_trigram MATCH"); // A bit dirty...
-		prepSql =
+		prepSql = "SELECT DISTINCT path, page, mtime, size, filetype FROM ("
-			"SELECT DISTINCT path, page, mtime, size, filetype FROM ("
 				  "SELECT file.path AS path,  content.page AS page, file.mtime AS mtime, file.size AS size, "
 				  "file.filetype AS filetype, 0 AS prio, fts.rank AS rank FROM file INNER JOIN content ON file.id = "
 				  "content.fileid "
 				  "INNER JOIN fts ON content.ftsid = fts.ROWID WHERE 1=1 AND " +
 				  whereSql +
 				  "UNION ALL SELECT file.path AS path,  content.page AS page, file.mtime AS mtime, file.size AS size, "
-			"file.filetype AS filetype, 1 as prio, fts_trigram.rank AS rank FROM file INNER JOIN content ON file.id = "
+				  "file.filetype AS filetype, 1 as prio, fts_trigram.rank AS rank FROM file INNER JOIN content ON "
+				  "file.id = "
 				  "content.fileid " +
-			"INNER JOIN fts_trigram ON content.fts_trigramid = fts_trigram.ROWID WHERE 1=1 AND " + whereSqlTrigram +
+				  "INNER JOIN fts_trigram ON content.fts_trigramid = fts_trigram.ROWID WHERE 1=1 AND " +
-			" ) " + sortSql;
+				  whereSqlTrigram + " ) " + sortSql;
 		++bindIterations;
 	}
 	else

shared/tagmanager.cpp

@@ -0,0 +1,66 @@
+#include "tagmanager.h"
+
+TagManager::TagManager(SqliteDbService &dbService)
+{
+	this->dbService = &dbService;
+}
+
+bool TagManager::addTagsToPath(QString path, const QSet<QString> &tags)
+{
+	QVector<QString> currentTags = this->dbService->getTagsForPath(path);
+	for(const QString &tag : tags)
+	{
+		currentTags.append(tag.toLower());
+	}
+
+	QSet<QString> newTags{currentTags.begin(), currentTags.end()};
+	return this->dbService->setTags(path, newTags);
+}
+
+bool TagManager::removeTagsForPath(QString path, const QSet<QString> &tags)
+{
+	QVector<QString> currentTags = this->dbService->getTagsForPath(path);
+	for(const QString &tag : tags)
+	{
+		currentTags.removeAll(tag);
+	}
+	QSet<QString> newTags{currentTags.begin(), currentTags.end()};
+	return this->dbService->setTags(path, newTags);
+}
+
+bool TagManager::removePathsForTag(QString tag, const QVector<QString> &paths)
+{
+	return this->dbService->removePathsForTag(tag, paths);
+}
+
+bool TagManager::deleteTag(QString tag)
+{
+	return this->dbService->deleteTag(tag);
+}
+
+QVector<QString> TagManager::getTags(QString path)
+{
+	return this->dbService->getTagsForPath(path);
+}
+
+QVector<QString> TagManager::getTags()
+{
+	return this->dbService->getTags();
+}
+
+QVector<QString> TagManager::getPaths(QString tag)
+{
+	return this->dbService->getPathsForTag(tag);
+}
+
+bool TagManager::addTagsToPath(QString path, QString tagstring, QChar delim)
+{
+	auto splitted = tagstring.split(delim);
+
+	return addTagsToPath(path, QSet<QString>{splitted.begin(), splitted.end()});
+}
+
+bool TagManager::addPathsToTag(QString tag, const QVector<QString> &paths)
+{
+	return this->dbService->addTag(tag, paths);
+}

shared/tagmanager.h

@@ -0,0 +1,28 @@
+#ifndef TAGMANAGER_H
+#define TAGMANAGER_H
+#include "sqlitedbservice.h"
+
+class TagManager
+{
+  private:
+	SqliteDbService *dbService = nullptr;
+	bool ensurePathOkay(QString inpath);
+
+  public:
+	TagManager(SqliteDbService &dbService);
+
+	bool addTagsToPath(QString path, const QSet<QString> &tags);
+	bool addTagsToPath(QString path, QString tagstring, QChar delim);
+
+	bool addPathsToTag(QString tag, const QVector<QString> &paths);
+	bool removeTagsForPath(QString path, const QSet<QString> &tags);
+
+	bool removePathsForTag(QString tag, const QVector<QString> &paths);
+	bool deleteTag(QString tag);
+
+	QVector<QString> getTags(QString path);
+	QVector<QString> getTags();
+	QVector<QString> getPaths(QString tag);
+};
+
+#endif // TAGMANAGER_H

shared/token.h

@@ -19,7 +19,8 @@ enum TokenType
 	FILTER_PATH_SIZE,
 	FILTER_PATH_ENDS,
 	FILTER_PATH_STARTS,
-	FILTER_CONTENT = 512,
+	FILTER_TAG_ASSIGNED,
+	FILTER_CONTENT = 512, /* Everything below here is content search (except LIMIT) */
 	FILTER_CONTENT_CONTAINS,
 	FILTER_CONTENT_PAGE,
 	LIMIT = 1024

submodules/exile.h

@@ -1 +1 @@
-Subproject commit 769f729dc51f2feb8bc3cbb2a48ed91ff2d56bf3
+Subproject commit 44b9a17becf6882e1b3728cbf885ae9e5a6717af