cli: Exit explicitly on unknown command

The indexer is quite slow with unshared network namespaces. It appears something in
Qt needs it as IPC or whatever. Seeing also dbus-related errors:

Issue: #33

So disable it for now.

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "submodules/exile.h"]
+	path = submodules/exile.h
+	url = https://gitea.quitesimple.org/crtxcr/exile.h

README.md

@@ -1,30 +1,56 @@
-# looqs - Looks for files. And looks inside them
-looqs creates a full text search for your files. It allows you to look at previews where your
-search terms have been found. 
+# looqs - FTS for the Linux desktop with previews for search results
+looqs creates a full text search index for your files. It allows you to look at previews where your
+search terms have been found, as shown in the screenshots below.
 
-Currently, this allows you search all indexed pdfs and take a look at the pages side by side in an instant.
 
 ## Screenshots
-Coming soon™
+### List
+![Screenshot looqs results](https://garage.quitesimple.org/assets/looqs/opearting_systems_looqs.png)
+
+### Preview
+![Screenshot looqs](https://garage.quitesimple.org/assets/looqs/orwell.png)
+![Screenshot looqs search fstream](https://garage.quitesimple.org/assets/looqs/fstream_write.png)
 
 
-## Goals
+## Current status
+Last version: 2022-0X-XX, v0.1
+
+Please see [Changelog](CHANGELOG.md) for a human readable list of changes. 
+
+
+## Goals and principles
  * **Find & Preview**. Instead of merely telling you where your search phrase has been found, it should also render the corresponding portion/pages of the documents and highlight the searched words.
- * **No daemons**. As other solutions are prone to have annoying daemons running that eat system resources away, this solution should make do without daemons if possible.
- * **Easy setup**. Similiarly, there should be no need for heavy-weight databases. Instead, this solution tries to squeeze out the most from simple approaches. In particular, it relies on sqlite. 
+ * **No daemons**. As some other desktop search projects are prone to have annoying daemons running that eat system resources away, this solution should make do without daemons where possible.
+ * **Easy setup**. Similiarly, there should be no need for heavy-weight databases. Instead, this solution tries to squeeze out the most from simple approaches. In particular, it relies on sqlite.
  * **GUI & CLI**. Provide CLI interfaces and GUI interfaces
+ * **Sandboxing**. As reading and rendering lots of formats naturally opens the door for security bugs, those tasks are offloaded to small, sandboxed sub-processes to mitigate the effect of exploited vulnerabilities.
+
+## Supported platforms
+Linux (on amd64) is currently the main focus. Currently, I don't plan on supporting anything else and the sandboxing architecture does not make it likely. I suppose a version without sandboxing might be conceivable for other platforms, but I have no plans or resources to actively target anything but Linux at this point.
+
+### Licence
+GPLv3. 
+
+### Contributing
+Fow now, github issues and pull-requests are preferred, but you can also just email
+your patches or issues to : looqs at quitesimple.org
 
 
 ## Build
-### Ubuntu 21.04
+
+### Ubuntu 21.10/22.04
 ```
+git submodule init
+git submodule update
 sudo apt install build-essential qtbase5-dev libpoppler-qt5-dev libuchardet-dev libquazip5-dev
 qmake
 make
 ```
 
+
+
 ## Documentation
-Coming soon™
+Please see [Usage.md](USAGE.md) for the user manual. 
 
 ## Packages
 Coming soon™

TODO

@@ -1,33 +0,0 @@
-general database classes instead of sqlite specific code 
-database: set a number of paths to default index. will require an indexer code though
-sandboxing!
-allow from GUI to ues commandlin, e. g. "/add ..." would be the same as "qss add ..." in termial
-
-PdfPreview: Use some kind of paging instead of memory limit
-Consider injecting Logger (default stdout/stderr) to classes instead of using Logger::info()/Logger::error()::
-sync/share GUI and CLI data-structures. Better to share codebase in general
-- cli: tagging
-- cli: command line parser: wrong position of [options] 
-- ability to set the WHERE condition (allow editing the SQL query)
-- Basic OCR in images (screenshots)
-- Index .ebup
-- Preview for: .txt, source code files, .ebup, images
-- index: DVD menus, media metadata in audio/files etc?
-- Stats: Number of results found
-- PdfPreview: Files per file, or directory (so basically filter the 
-results)
-- Hide PdfPreview Tab if there are no pdfs in the results
--Tagging: add an "addtag" utility, to assign tags to folder and files
--gui: Search expclitly for tags, list by tags, remove tags, add tags
--gui: Filter already found results. For example, "show only folders",
-or just search with those results.
--split each tab into own class, not everything in mainwindow.cpp?
-Menu:
--Delete from index
--Delete from fs
--Reindex
-Filter:
-type:d 
-type:file
-mtime:
-tag:

cli/cli.pro

@@ -14,57 +14,29 @@ DEFINES += QT_DEPRECATED_WARNINGS
 # In order to do so, uncomment the following line.
 # You can also select to disable deprecated APIs only up to a certain version of Qt.
 #DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0
-LIBS += -luchardet -lpoppler-qt5 -lquazip5
 SOURCES += \
         main.cpp \
-    encodingdetector.cpp \
-    processor.cpp \
-    pdfprocessor.cpp \
-    defaulttextprocessor.cpp \
     commandadd.cpp \
-    tagstripperprocessor.cpp \
-    nothingprocessor.cpp \
-    odtprocessor.cpp \
-    utils.cpp \
-    odsprocessor.cpp \
     commanddelete.cpp \
     commandupdate.cpp \
-    filesaver.cpp \
-    databasefactory.cpp \
-    sqlitedbservice.cpp \
-    logger.cpp \
     commandsearch.cpp \
-    commandlist.cpp
+    commandlist.cpp \
+    command.cpp
 
 HEADERS += \
-    encodingdetector.h \
-    processor.h \
-    pagedata.h \
-    pdfprocessor.h \
-    defaulttextprocessor.h \
     command.h \
     commandadd.h \
-    tagstripperprocessor.h \
-    nothingprocessor.h \
-    odtprocessor.h \
-    utils.h \
-    odsprocessor.h \
     commanddelete.h \
     commandupdate.h \
-    filesaver.h \
-    databasefactory.h \
-    sqlitedbservice.h \
-    logger.h \
     commandsearch.h \
     commandlist.h
-INCLUDEPATH += /usr/include/poppler/qt5/ /usr/include/quazip5
-
-
 
 win32:CONFIG(release, debug|release): LIBS += -L$$OUT_PWD/../shared/release/ -lshared
 else:win32:CONFIG(debug, debug|release): LIBS += -L$$OUT_PWD/../shared/debug/ -lshared
 else:unix: LIBS += -L$$OUT_PWD/../shared/ -lshared
 
+LIBS += -luchardet -lpoppler-qt5 -lquazip5
+
 INCLUDEPATH += $$PWD/../shared
 DEPENDPATH += $$PWD/../shared
 

cli/command.cpp

@@ -3,3 +3,12 @@
 #include <QDebug>
 #include "command.h"
 #include "looqsgeneralexception.h"
+
+void Command::execute()
+{
+	int result = handle(arguments);
+	if(autoFinish)
+	{
+		emit finishedCmd(result);
+	}
+}

cli/command.h

@@ -1,26 +1,39 @@
 #ifndef COMMAND_H
 #define COMMAND_H
 #include <QStringList>
-#include <QSqlDatabase>
-#include <QSqlQuery>
 #include <QThreadStorage>
 #include <QVariant>
+#include <QMutex>
+#include <QWaitCondition>
 #include "utils.h"
 #include "sqlitedbservice.h"
-class Command
+class Command : public QObject
 {
+	Q_OBJECT
+  signals:
+	void finishedCmd(int retval);
+
   protected:
 	SqliteDbService *dbService;
 	QString dbConnectionString;
+	QStringList arguments;
+
+	bool autoFinish = true;
 
   public:
 	Command(SqliteDbService &dbService)
 	{
 		this->dbService = &dbService;
 	}
-
+	void setArguments(QStringList arguments)
+	{
+		this->arguments = arguments;
+	}
 	virtual int handle(QStringList arguments) = 0;
 	virtual ~Command(){};
+
+  public slots:
+	void execute();
 };
 
 #endif // COMMAND_H

cli/commandadd.cpp

@@ -1,7 +1,5 @@
 #include <QFileInfo>
 #include <QDebug>
-#include <QSqlQuery>
-#include <QSqlError>
 #include <QDateTime>
 #include <QMap>
 #include <QTextStream>
@@ -13,6 +11,28 @@
 #include "commandadd.h"
 #include "logger.h"
 
+void CommandAdd::indexerFinished()
+{
+	IndexResult result = indexer->getResult();
+
+	Logger::info() << "Total: " << result.total() << Qt::endl;
+	Logger::info() << "Added: " << result.addedPaths << Qt::endl;
+	Logger::info() << "Skipped: " << result.skippedPaths << Qt::endl;
+	auto failedPathsCount = result.erroredPaths;
+	Logger::info() << "Failed: " << failedPathsCount << Qt::endl;
+	if(failedPathsCount > 0)
+	{
+		Logger::info() << "Failed paths: " << Qt::endl;
+		for(QString paths : result.failedPaths())
+		{
+			Logger::info() << paths << Qt::endl;
+		}
+	}
+
+	/* TODO maybe not 0 if keepGoing not given */
+	emit finishedCmd(0);
+}
+
 int CommandAdd::handle(QStringList arguments)
 {
 	QCommandLineParser parser;
@@ -53,15 +73,21 @@ int CommandAdd::handle(QStringList arguments)
 		}
 	}
 
-	FileSaver saver(*this->dbService);
-	int numFilesCount = files.size();
-	int processedFilesCount = saver.addFiles(files.toVector(), keepGoing, verbose);
-	if(processedFilesCount != numFilesCount)
-	{
-		Logger::error() << "Errors occured while trying to add files to the database. Processed " << processedFilesCount
-						<< "out of" << numFilesCount << "files" << Qt::endl;
-		return 1;
-	}
+	indexer = new Indexer(*this->dbService);
+	indexer->setTargetPaths(files.toVector());
+
+	connect(indexer, &Indexer::pathsCountChanged, this,
+			[](int pathsCount) { Logger::info() << "Found paths: " << pathsCount << Qt::endl; });
+
+	connect(indexer, &Indexer::indexProgress, this,
+			[](int pathsCount, unsigned int added, unsigned int skipped, unsigned int failed, unsigned int totalCount)
+			{ Logger::info() << "Processed files: " << pathsCount << Qt::endl; });
+	connect(indexer, &Indexer::finished, this, &CommandAdd::indexerFinished);
+
+	/* TODO: keepGoing, verbose */
+
+	this->autoFinish = false;
+	indexer->beginIndexing();
 
 	return 0;
 }

cli/commandadd.h

@@ -3,15 +3,21 @@
 #include <QMutex>
 #include "command.h"
 #include "filesaver.h"
+#include "indexer.h"
+
 class CommandAdd : public Command
 {
   private:
 	SaveFileResult addFile(QString path);
+	Indexer *indexer;
 
+  protected:
   public:
 	using Command::Command;
 
 	int handle(QStringList arguments) override;
+  private slots:
+	void indexerFinished();
 };
 
 #endif // COMMANDADD_H

cli/commanddelete.cpp

@@ -3,7 +3,6 @@
 #include <QFileInfo>
 #include <QDebug>
 #include <QRegularExpression>
-#include <QSqlError>
 #include "commanddelete.h"
 #include "logger.h"
 

cli/commandlist.cpp

@@ -24,7 +24,7 @@ int CommandList::handle(QStringList arguments)
 
 	QStringList files = parser.positionalArguments();
 	QString queryStrings = files.join(' ');
-	auto results = dbService->search(LooqsQuery::build(queryStrings));
+	auto results = dbService->search(LooqsQuery::build(queryStrings, TokenType::FILTER_PATH_CONTAINS, false));
 
 	for(SearchResult &result : results)
 	{

cli/commandsearch.cpp

@@ -16,7 +16,7 @@ int CommandSearch::handle(QStringList arguments)
 
 	QStringList files = parser.positionalArguments();
 	QString queryStrings = files.join(' ');
-	LooqsQuery query = LooqsQuery::build(queryStrings);
+	LooqsQuery query = LooqsQuery::build(queryStrings, TokenType::FILTER_PATH_CONTAINS, false);
 	bool reverse = parser.isSet("reverse");
 	if(reverse)
 	{

cli/commandupdate.cpp

@@ -2,7 +2,6 @@
 #include <QFileInfo>
 #include <QDateTime>
 #include <QThreadPool>
-#include <QtConcurrentRun>
 #include "commandupdate.h"
 #include "logger.h"
 

cli/main.cpp

@@ -5,13 +5,12 @@
 #include <QDataStream>
 #include <QDebug>
 #include <QProcessEnvironment>
-#include <QSqlDatabase>
-#include <QSqlQuery>
-#include <QSqlError>
 #include <QMap>
 #include <QDebug>
 #include <QSettings>
 #include <functional>
+#include <QTimer>
+
 #include <exception>
 #include "encodingdetector.h"
 #include "pdfprocessor.h"
@@ -24,7 +23,9 @@
 #include "commandsearch.h"
 #include "databasefactory.h"
 #include "logger.h"
+#include "sandboxedprocessor.h"
 #include "../shared/common.h"
+#include "../shared/filescanworker.h"
 
 void printUsage(QString argv0)
 {
@@ -59,6 +60,7 @@ int main(int argc, char *argv[])
 	QCoreApplication app(argc, argv);
 	QStringList args = app.arguments();
 	QString argv0 = args.takeFirst();
+
 	if(args.length() < 1)
 	{
 		printUsage(argv0);
@@ -74,26 +76,45 @@ int main(int argc, char *argv[])
 		Logger::error() << "Error: " << e.message;
 		return 1;
 	}
+	qRegisterMetaType<PageData>();
+	qRegisterMetaType<FileScanResult>("FileScanResult");
 
 	QString connectionString = Common::databasePath();
 	DatabaseFactory dbFactory(connectionString);
 	SqliteDbService dbService(dbFactory);
 	QString commandName = args.first();
+	if(commandName == "process")
+	{
+		if(args.length() < 1)
+		{
+			qDebug() << "Filename is required";
+			return 1;
+		}
+
+		QString file = args.at(1);
+		SandboxedProcessor processor(file);
+		return processor.process();
+	}
 	Command *cmd = commandFromName(commandName, dbService);
 	if(cmd != nullptr)
 	{
 		try
 		{
-			return cmd->handle(args);
+			QObject::connect(cmd, &Command::finishedCmd, [](int retval) { QCoreApplication::exit(retval); });
+			cmd->setArguments(args);
+			QTimer::singleShot(0, cmd, &Command::execute);
 		}
 		catch(const LooqsGeneralException &e)
 		{
-			Logger::error() << "Exception caught, message: " << e.message << Qt::endl;
+			Logger::error() << "Exception caught, message:" << e.message << Qt::endl;
+			return 1;
 		}
 	}
 	else
 	{
-		Logger::error() << "Unknown command " << commandName << Qt::endl;
+		Logger::error() << "Unknown command:" << commandName << Qt::endl;
+		return 1;
 	}
-	return 1;
+
+	return app.exec();
 }

gui/gui.pro

@@ -4,7 +4,7 @@
 #
 #-------------------------------------------------
 
-QT       += core concurrent gui
+QT       += core concurrent gui network
 
 greaterThan(QT_MAJOR_VERSION, 4): QT += widgets
 CONFIG += c++17
@@ -23,29 +23,49 @@ DEFINES += QT_DEPRECATED_WARNINGS
 #DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0
 
 SOURCES += \
+    ipcclient.cpp \
+    ipcserver.cpp \
         main.cpp \
         mainwindow.cpp \
-      pdfworker.cpp \
-      pdfpreview.cpp \
-      clicklabel.cpp
+      clicklabel.cpp \
+    previewgenerator.cpp \
+    previewgeneratormapfunctor.cpp \
+    previewgeneratorpdf.cpp \
+    previewgeneratorplaintext.cpp \
+    previewresult.cpp \
+    previewresultpdf.cpp \
+    previewresultplaintext.cpp \
+    previewworker.cpp
 
 HEADERS += \
+    ipc.h \
+    ipcclient.h \
+    ipcserver.h \
         mainwindow.h \
-    pdfworker.h \
-    pdfpreview.h \
-    clicklabel.h
+    clicklabel.h \
+    previewgenerator.h \
+    previewgeneratormapfunctor.h \
+    previewgeneratorpdf.h \
+    previewgeneratorplaintext.h \
+    previewresult.h \
+    previewresultpdf.h \
+    previewresultplaintext.h \
+    previewworker.h \
+    renderconfig.h
 
 FORMS += \
         mainwindow.ui
 
 INCLUDEPATH += /usr/include/poppler/qt5/
-LIBS += -lpoppler-qt5
+
 QT += widgets sql
 
 win32:CONFIG(release, debug|release): LIBS += -L$$OUT_PWD/../shared/release/ -lshared
 else:win32:CONFIG(debug, debug|release): LIBS += -L$$OUT_PWD/../shared/debug/ -lshared
 else:unix: LIBS += -L$$OUT_PWD/../shared/ -lshared
 
+LIBS += -luchardet -lpoppler-qt5 -lquazip5
+
 INCLUDEPATH += $$PWD/../shared
 DEPENDPATH += $$PWD/../shared
 

gui/ipc.h

@@ -0,0 +1,10 @@
+#ifndef IPC_H
+#define IPC_H
+
+enum IPCCommand
+{
+	DocOpen,
+	FileOpen,
+	AddFile,
+};
+#endif // IPC_H

gui/ipcclient.cpp

@@ -0,0 +1,27 @@
+#include <QDataStream>
+#include "ipcclient.h"
+
+IPCClient::IPCClient(QString socketPath)
+{
+	this->socketPath = socketPath;
+}
+
+bool IPCClient::sendCommand(IPCCommand command, QStringList args)
+{
+	bool result = false;
+	QLocalSocket socket;
+	socket.connectToServer(socketPath);
+	if(socket.isOpen() && socket.isWritable())
+	{
+		QDataStream stream(&socket);
+		stream << command;
+		stream << args;
+		socket.flush();
+		result = true;
+	}
+	else
+	{
+		qDebug() << "Not connected to IPC server";
+	}
+	return result;
+}

gui/ipcclient.h

@@ -0,0 +1,18 @@
+#ifndef IPCCLIENT_H
+#define IPCCLIENT_H
+#include <QLocalSocket>
+#include <QString>
+#include <QStringList>
+#include "ipc.h"
+
+class IPCClient
+{
+  private:
+	QString socketPath;
+
+  public:
+	IPCClient(QString socketPath);
+	bool sendCommand(IPCCommand command, QStringList args);
+};
+
+#endif // IPCCLIENT_H

gui/ipcserver.cpp

@@ -0,0 +1,108 @@
+#include <QFile>
+#include <QDesktopServices>
+#include <QSettings>
+#include <QProcess>
+#include <QUrl>
+#include <QLocalSocket>
+#include <QDataStream>
+#include "ipcserver.h"
+#include "common.h"
+#include "databasefactory.h"
+#include "../shared/logger.h"
+
+IpcServer::IpcServer()
+{
+	this->dbFactory = QSharedPointer<DatabaseFactory>(new DatabaseFactory(Common::databasePath()));
+	this->dbService = QSharedPointer<SqliteDbService>(new SqliteDbService(*this->dbFactory.get()));
+	this->fileSaver = QSharedPointer<FileSaver>(new FileSaver(*this->dbService.get()));
+	connect(&this->spawningServer, &QLocalServer::newConnection, this, &IpcServer::spawnerNewConnection);
+}
+
+bool IpcServer::startSpawner(QString socketPath)
+{
+	QFile::remove(socketPath);
+	return this->spawningServer.listen(socketPath);
+}
+
+bool IpcServer::docOpen(QString path, int pagenum)
+{
+
+	QSettings settings;
+	QString command = settings.value("pdfviewer").toString();
+	if(path.endsWith(".pdf") && command != "" && command.contains("%p") && command.contains("%f"))
+	{
+		QStringList splitted = command.split(" ");
+		if(splitted.size() > 1)
+		{
+			QString cmd = splitted[0];
+			QStringList args = splitted.mid(1);
+			args.replaceInStrings("%f", path);
+			args.replaceInStrings("%p", QString::number(pagenum));
+
+			QProcess::startDetached(cmd, args);
+		}
+	}
+	else
+	{
+		QDesktopServices::openUrl(QUrl::fromLocalFile(path));
+	}
+	return true;
+}
+
+bool IpcServer::fileOpen(QString path)
+{
+	return QDesktopServices::openUrl(QUrl::fromLocalFile(path));
+}
+
+SaveFileResult IpcServer::addFile(QString file)
+{
+	try
+	{
+		return this->fileSaver->addFile(file);
+	}
+	catch(std::exception &e)
+	{
+		Logger::error() << e.what() << Qt::endl;
+		return PROCESSFAIL;
+	}
+}
+
+void IpcServer::spawnerNewConnection()
+{
+	QScopedPointer<QLocalSocket> socket{this->spawningServer.nextPendingConnection()};
+	if(!socket.isNull())
+	{
+		if(!socket->waitForReadyRead())
+		{
+			return;
+		}
+		QDataStream stream(socket.get());
+		IPCCommand command;
+		QStringList args;
+		stream >> command;
+		stream >> args;
+		if(args.size() < 1)
+		{
+			stream << "invalid";
+			return;
+		}
+		if(command == DocOpen)
+		{
+			if(args.size() < 2)
+			{
+				stream << "invalid";
+				return;
+			}
+			docOpen(args[0], args[1].toInt());
+		}
+		if(command == FileOpen)
+		{
+			if(args.size() < 1)
+			{
+				stream << "invalid";
+				return;
+			}
+			fileOpen(args[0]);
+		}
+	}
+}

gui/ipcserver.h

@@ -0,0 +1,26 @@
+#ifndef IPCSERVER_H
+#define IPCSERVER_H
+#include <QString>
+#include <QLocalServer>
+#include "ipc.h"
+#include "filesaver.h"
+class IpcServer : public QObject
+{
+	Q_OBJECT
+  private:
+	QSharedPointer<DatabaseFactory> dbFactory;
+	QSharedPointer<SqliteDbService> dbService;
+	QSharedPointer<FileSaver> fileSaver;
+	QLocalServer spawningServer;
+	bool docOpen(QString path, int pagenum);
+	bool fileOpen(QString path);
+	SaveFileResult addFile(QString file);
+  private slots:
+	void spawnerNewConnection();
+
+  public:
+	IpcServer();
+	bool startSpawner(QString socketPath);
+};
+
+#endif // IPCSERVER_H

gui/main.cpp

@@ -1,18 +1,153 @@
 #include <QApplication>
 #include <QSettings>
 #include <QMessageBox>
+#include <QStandardPaths>
+#include <QProcess>
+#include <QDir>
+#include <QCommandLineParser>
+
 #include "mainwindow.h"
 #include "searchresult.h"
-#include "pdfpreview.h"
+#include "previewresultpdf.h"
 #include "../shared/common.h"
+#include "../shared/sandboxedprocessor.h"
+#include "../submodules/exile.h/exile.h"
+#include "ipcserver.h"
 
+void enableSandbox(QString socketPath)
+{
+	struct exile_policy *policy = exile_init_policy();
+	if(policy == NULL)
+	{
+		qCritical() << "Failed to init policy for sandbox";
+		exit(EXIT_FAILURE);
+	}
+	QDir dir;
+	dir.mkpath(QStandardPaths::writableLocation(QStandardPaths::AppLocalDataLocation));
+	dir.mkpath(QStandardPaths::writableLocation(QStandardPaths::CacheLocation));
+
+	std::string appDataLocation = QStandardPaths::writableLocation(QStandardPaths::AppLocalDataLocation).toStdString();
+	std::string cacheDataLocation = QStandardPaths::writableLocation(QStandardPaths::CacheLocation).toStdString();
+	std::string configDataLocation = QStandardPaths::writableLocation(QStandardPaths::ConfigLocation).toStdString();
+
+	std::string sockPath = socketPath.toStdString();
+	std::string dbPath = QFileInfo(Common::databasePath()).absolutePath().toStdString();
+	std::string mySelf = QFileInfo("/proc/self/exe").symLinkTarget().toStdString();
+	policy->namespace_options = EXILE_UNSHARE_USER;
+	if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ, "/") != 0)
+	{
+		qCritical() << "Failed to append a path to the path policy";
+		exit(EXIT_FAILURE);
+	}
+
+	if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ | EXILE_FS_ALLOW_ALL_WRITE,
+								  appDataLocation.c_str()) != 0)
+	{
+		qCritical() << "Failed to append appDataLocation path to the path policy";
+		exit(EXIT_FAILURE);
+	}
+	if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ | EXILE_FS_ALLOW_ALL_WRITE,
+								  cacheDataLocation.c_str()) != 0)
+	{
+		qCritical() << "Failed to append cacheDataLocation path to the path policy";
+		exit(EXIT_FAILURE);
+	}
+	if(exile_append_path_policies(policy,
+								  EXILE_FS_ALLOW_ALL_READ | EXILE_FS_ALLOW_REMOVE_FILE | EXILE_FS_ALLOW_ALL_WRITE,
+								  dbPath.c_str()) != 0)
+	{
+		qCritical() << "Failed to append dbPath path to the path policy";
+		exit(EXIT_FAILURE);
+	}
+	if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ | EXILE_FS_ALLOW_EXEC, mySelf.c_str(), "/lib64",
+								  "/lib") != 0)
+	{
+		qCritical() << "Failed to append mySelf path to the path policy";
+		exit(EXIT_FAILURE);
+	}
+	if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ | EXILE_FS_ALLOW_ALL_WRITE,
+								  configDataLocation.c_str()) != 0)
+	{
+		qCritical() << "Failed to append configDataLocation path to the path policy";
+		exit(EXIT_FAILURE);
+	}
+	int ret = exile_enable_policy(policy);
+	if(ret != 0)
+	{
+		qDebug() << "Failed to establish sandbox";
+		exit(EXIT_FAILURE);
+	}
+	exile_free_policy(policy);
+}
 int main(int argc, char *argv[])
 {
+	QString socketPath = "/tmp/looqs-spawner";
+	if(argc > 1)
+	{
+		QString arg = argv[1];
+		if(arg == "ipc")
+		{
+			Common::setupAppInfo();
+			QApplication a(argc, argv);
+
+			IpcServer *ipcserver = new IpcServer();
+			qDebug() << "Launching IPC Server";
+			if(!ipcserver->startSpawner(socketPath))
+			{
+				qCritical() << "Error failed to spawn";
+				return 1;
+			}
+			qDebug() << "Launched IPC Server";
+			return a.exec();
+		}
+		if(arg == "process")
+		{
+			Common::setupAppInfo();
+			QApplication a(argc, argv);
+
+			QStringList args = a.arguments();
+			if(args.length() < 1)
+			{
+				qDebug() << "Filename is required";
+				return 1;
+			}
+
+			QString file = args.at(1);
+			SandboxedProcessor processor(file);
+			return processor.process();
+		}
+	}
+	QProcess process;
+	QStringList args;
+	args << "ipc";
+	if(!process.startDetached("/proc/self/exe", args))
+	{
+		QString errorMsg = "Failed to start IPC server";
+		qDebug() << errorMsg;
+		QMessageBox::critical(nullptr, "Error", errorMsg);
+	}
 	Common::setupAppInfo();
-	QApplication a(argc, argv);
+	QCommandLineParser parser;
+	parser.addOption({{"s", "no-sandbox"}, "Disable sandboxing"});
+	QStringList appArgs;
+	for(int i = 0; i < argc; i++)
+	{
+		appArgs.append(argv[i]);
+	}
+	parser.parse(appArgs);
+
 	try
 	{
 		Common::ensureConfigured();
+		if(!parser.isSet("no-sandbox"))
+		{
+			enableSandbox(socketPath);
+			qInfo() << "Sandbox: on";
+		}
+		else
+		{
+			qInfo() << "Sandbox: off";
+		}
 	}
 	catch(LooqsGeneralException &e)
 	{
@@ -20,10 +155,16 @@ int main(int argc, char *argv[])
 		QMessageBox::critical(nullptr, "Error", e.message);
 		return 1;
 	}
+	// Keep this post sandbox, afterwards does not work (suspect due to threads, but unconfirmed)
+	QApplication a(argc, argv);
+
 	qRegisterMetaType<QVector<SearchResult>>("QVector<SearchResult>");
-	qRegisterMetaType<QVector<PdfPreview>>("QVector<PdfPreview>");
-	qRegisterMetaType<PdfPreview>("PdfPreview");
-	MainWindow w;
+	qRegisterMetaType<QVector<PreviewResultPdf>>("QVector<PreviewResultPdf>");
+	qRegisterMetaType<PreviewResultPdf>("PreviewResultPdf");
+	qRegisterMetaType<FileScanResult>("FileScanResult");
+
+	IPCClient client{socketPath};
+	MainWindow w{0, client};
 	w.showMaximized();
 
 	return a.exec();

gui/mainwindow.cpp

@@ -11,6 +11,7 @@
 #include <QProcess>
 #include <QComboBox>
 #include <QtConcurrent/QtConcurrent>
+#include <QMessageBox>
 #include "mainwindow.h"
 #include "ui_mainwindow.h"
 #include "clicklabel.h"
@@ -18,30 +19,51 @@
 #include "../shared/looqsgeneralexception.h"
 #include "../shared/common.h"
 
-MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), ui(new Ui::MainWindow)
+MainWindow::MainWindow(QWidget *parent, IPCClient &client) : QMainWindow(parent), ui(new Ui::MainWindow)
 {
 	ui->setupUi(this);
 	setWindowTitle(QCoreApplication::applicationName());
+	this->ipcClient = &client;
 	QSettings settings;
 
-	db = QSqlDatabase::addDatabase("QSQLITE");
-	db.setDatabaseName(Common::databasePath());
-	if(!db.open())
-	{
-		qDebug() << "failed to open database";
-		throw std::runtime_error("Failed to open database");
-	}
+	this->dbFactory = new DatabaseFactory(Common::databasePath());
+
+	db = this->dbFactory->forCurrentThread();
+	this->dbService = new SqliteDbService(*this->dbFactory);
+
+	indexer = new Indexer(*(this->dbService));
+	indexer->setParent(this);
 	connectSignals();
 	ui->treeResultsList->setContextMenuPolicy(Qt::ContextMenuPolicy::CustomContextMenu);
 	ui->tabWidget->setCurrentIndex(0);
 	ui->statusBar->addWidget(ui->lblSearchResults);
-	ui->statusBar->addWidget(ui->pdfProcessBar);
-	ui->pdfProcessBar->hide();
+	ui->statusBar->addWidget(ui->previewProcessBar);
+	ui->previewProcessBar->hide();
 	ui->comboScale->setCurrentText(settings.value("currentScale").toString());
-	pdfPreviewsPerPage = settings.value("pdfPreviewsPerPage", 20).toInt();
-	ui->spinPdfPreviewPage->setMinimum(1);
+	previewsPerPage = settings.value("previewsPerPage", 20).toInt();
+	ui->spinPreviewPage->setMinimum(1);
+
+	QStringList indexPaths = settings.value("indexPaths").toStringList();
+	ui->lstPaths->addItems(indexPaths);
 }
 
+void MainWindow::addPathToIndex()
+{
+	QString path = this->ui->txtPathScanAdd->text();
+	QFileInfo fileInfo{path};
+	if(!fileInfo.exists(path))
+	{
+		QMessageBox::critical(this, "Invalid path", "Path does not seem to exist");
+		return;
+	}
+	if(!fileInfo.isReadable())
+	{
+		QMessageBox::critical(this, "Invalid path", "Path cannot be read");
+		return;
+	}
+	this->ui->lstPaths->addItem(path);
+	this->ui->txtPathScanAdd->clear();
+}
 void MainWindow::connectSignals()
 {
 	connect(ui->txtSearch, &QLineEdit::returnPressed, this, &MainWindow::lineEditReturnPressed);
@@ -59,36 +81,127 @@ void MainWindow::connectSignals()
 					handleSearchError(e.message);
 				}
 			});
+	connect(&previewWorkerWatcher, &QFutureWatcher<QSharedPointer<PreviewResult>>::resultReadyAt, this,
+			[&](int index) { previewReceived(previewWorkerWatcher.resultAt(index)); });
+	connect(&previewWorkerWatcher, &QFutureWatcher<QSharedPointer<PreviewResult>>::progressValueChanged,
+			ui->previewProcessBar, &QProgressBar::setValue);
+	connect(&previewWorkerWatcher, &QFutureWatcher<QSharedPointer<PreviewResult>>::started, this,
+			[&] { ui->indexerTab->setEnabled(false); });
+
+	connect(&previewWorkerWatcher, &QFutureWatcher<QSharedPointer<PreviewResult>>::finished, this,
+			[&] { ui->indexerTab->setEnabled(true); });
 
-	connect(&pdfWorkerWatcher, &QFutureWatcher<PdfPreview>::resultReadyAt, this,
-			[&](int index) { pdfPreviewReceived(pdfWorkerWatcher.resultAt(index)); });
-	connect(&pdfWorkerWatcher, &QFutureWatcher<PdfPreview>::progressValueChanged, ui->pdfProcessBar,
-			&QProgressBar::setValue);
 	connect(ui->treeResultsList, &QTreeWidget::itemActivated, this, &MainWindow::treeSearchItemActivated);
 	connect(ui->treeResultsList, &QTreeWidget::customContextMenuRequested, this,
 			&MainWindow::showSearchResultsContextMenu);
 	connect(ui->tabWidget, &QTabWidget::currentChanged, this, &MainWindow::tabChanged);
 	connect(ui->comboScale, qOverload<int>(&QComboBox::currentIndexChanged), this, &MainWindow::comboScaleChanged);
-	connect(ui->spinPdfPreviewPage, qOverload<int>(&QSpinBox::valueChanged), this,
-			&MainWindow::spinPdfPreviewPageValueChanged);
+	connect(ui->spinPreviewPage, qOverload<int>(&QSpinBox::valueChanged), this,
+			&MainWindow::spinPreviewPageValueChanged);
+
+	connect(ui->btnAddPath, &QPushButton::clicked, this, &MainWindow::addPathToIndex);
+	connect(ui->txtPathScanAdd, &QLineEdit::returnPressed, this, &MainWindow::addPathToIndex);
+	connect(ui->btnStartIndexing, &QPushButton::clicked, this, &MainWindow::startIndexing);
+
+	connect(this->indexer, &Indexer::pathsCountChanged, this,
+			[&](int number)
+			{
+				ui->lblSearchResults->setText("Found paths: " + QString::number(number));
+				ui->lblPathsFoundValue->setText(QString::number(number));
+				ui->previewProcessBar->setMaximum(number);
+			});
+	connect(this->indexer, &Indexer::indexProgress, this,
+
+			[&](int number, unsigned int added, unsigned int skipped, unsigned int failed, unsigned int totalCount)
+			{
+				ui->lblSearchResults->setText("Processed " + QString::number(number) + " files");
+				ui->previewProcessBar->setValue(number);
+				ui->previewProcessBar->setMaximum(totalCount);
+				ui->lblAddedValue->setText(QString::number(added));
+				ui->lblSkippedValue->setText(QString::number(skipped));
+				ui->lblFailedValue->setText(QString::number(failed));
+			});
+
+	connect(this->indexer, &Indexer::finished, this, &MainWindow::finishIndexing);
+
+	connect(ui->lstPaths->selectionModel(), &QItemSelectionModel::selectionChanged, this,
+			[&](const QItemSelection &selected, const QItemSelection &deselected)
+			{ ui->btnDeletePath->setEnabled(this->ui->lstPaths->selectedItems().count() > 0); });
+
+	connect(ui->btnDeletePath, &QPushButton::clicked, this, [&] { qDeleteAll(ui->lstPaths->selectedItems()); });
 }
 
-void MainWindow::spinPdfPreviewPageValueChanged(int val)
+void MainWindow::spinPreviewPageValueChanged(int val)
 {
-	makePdfPreview(val);
+	makePreviews(val);
+}
+
+void MainWindow::startIndexing()
+{
+	if(this->indexer->isRunning())
+	{
+		ui->btnStartIndexing->setEnabled(false);
+		ui->btnStartIndexing->setText("Start indexing");
+		this->indexer->requestCancellation();
+		return;
+	}
+
+	ui->previewsTab->setEnabled(false);
+	ui->resultsTab->setEnabled(false);
+	ui->txtPathScanAdd->setEnabled(false);
+	ui->txtSearch->setEnabled(false);
+	ui->previewProcessBar->setValue(0);
+	ui->previewProcessBar->setVisible(true);
+
+	QVector<QString> paths;
+	QStringList pathSettingsValue;
+	for(int i = 0; i < ui->lstPaths->count(); i++)
+	{
+		QString path = ui->lstPaths->item(i)->text();
+		paths.append(path);
+		pathSettingsValue.append(path);
+	}
+	this->indexer->setTargetPaths(paths);
+	this->indexer->beginIndexing();
+	QSettings settings;
+	settings.setValue("indexPaths", pathSettingsValue);
+	ui->btnStartIndexing->setText("Stop indexing");
+}
+
+void MainWindow::finishIndexing()
+{
+	IndexResult result = this->indexer->getResult();
+
+	ui->lblSearchResults->setText("Indexing finished");
+	ui->previewProcessBar->setValue(ui->previewProcessBar->maximum());
+	ui->lblFailedValue->setText(QString::number(result.erroredPaths));
+	ui->lblSkippedValue->setText(QString::number(result.skippedPaths));
+	ui->lblAddedValue->setText(QString::number(result.addedPaths));
+	ui->btnStartIndexing->setEnabled(true);
+	ui->btnStartIndexing->setText("Start indexing");
+	ui->previewsTab->setEnabled(true);
+	ui->resultsTab->setEnabled(true);
+	ui->txtPathScanAdd->setEnabled(true);
+	ui->txtSearch->setEnabled(true);
 }
 
 void MainWindow::comboScaleChanged(int i)
 {
 	QSettings scaleSetting;
 	scaleSetting.setValue("currentScale", ui->comboScale->currentText());
-	makePdfPreview(ui->spinPdfPreviewPage->value());
+	makePreviews(ui->spinPreviewPage->value());
 }
-bool MainWindow::pdfTabActive()
+
+bool MainWindow::previewTabActive()
 {
 	return ui->tabWidget->currentIndex() == 1;
 }
 
+bool MainWindow::indexerTabActive()
+{
+	return ui->tabWidget->currentIndex() == 2;
+}
+
 void MainWindow::keyPressEvent(QKeyEvent *event)
 {
 	bool quit =
@@ -112,53 +225,36 @@ void MainWindow::keyPressEvent(QKeyEvent *event)
 
 void MainWindow::tabChanged()
 {
-	if(pdfTabActive())
+	if(ui->tabWidget->currentIndex() == 0)
 	{
-		if(pdfDirty)
-		{
-			makePdfPreview(ui->spinPdfPreviewPage->value());
-		}
-		ui->pdfProcessBar->show();
+		ui->previewProcessBar->hide();
 	}
 	else
 	{
-		ui->pdfProcessBar->hide();
+		if(ui->previewProcessBar->value() > 0)
+		{
+			ui->previewProcessBar->show();
+		}
+	}
+	if(previewTabActive())
+	{
+		if(previewDirty)
+		{
+			makePreviews(ui->spinPreviewPage->value());
+		}
 	}
 }
 
-void MainWindow::pdfPreviewReceived(PdfPreview preview)
+void MainWindow::previewReceived(QSharedPointer<PreviewResult> preview)
 {
-	if(preview.hasPreviewImage())
+	if(preview->hasPreview())
 	{
-		ClickLabel *label = new ClickLabel();
-		QString docPath = preview.documentPath;
-		auto previewPage = preview.page;
-		label->setPixmap(QPixmap::fromImage(preview.previewImage));
-		label->setToolTip(preview.documentPath);
-		ui->scrollAreaWidgetContents->layout()->addWidget(label);
-		connect(label, &ClickLabel::leftClick,
-				[docPath, previewPage]()
-				{
-					QSettings settings;
-					QString command = settings.value("pdfviewer").toString();
-					if(command != "" && command.contains("%p") && command.contains("%f"))
-					{
-						QStringList splitted = command.split(" ");
-						if(splitted.size() > 1)
-						{
-							QString cmd = splitted[0];
-							QStringList args = splitted.mid(1);
-							args.replaceInStrings("%f", docPath);
-							args.replaceInStrings("%p", QString::number(previewPage));
+		QString docPath = preview->getDocumentPath();
+		auto previewPage = preview->getPage();
 
-							QProcess::startDetached(cmd, args);
-						}
-					}
-					else
-					{
-						QDesktopServices::openUrl(QUrl::fromLocalFile(docPath));
-					}
-				});
+		ClickLabel *label = dynamic_cast<ClickLabel *>(preview->createPreviewWidget());
+		ui->scrollAreaWidgetContents->layout()->addWidget(label);
+		connect(label, &ClickLabel::leftClick, [this, docPath, previewPage]() { ipcDocOpen(docPath, previewPage); });
 		connect(label, &ClickLabel::rightClick,
 				[this, docPath, previewPage]()
 				{
@@ -181,21 +277,37 @@ void MainWindow::lineEditReturnPressed()
 		return;
 	}
 	// TODO: validate q;
+	ui->treeResultsList->clear();
 	ui->lblSearchResults->setText("Searching...");
 	this->ui->txtSearch->setEnabled(false);
 	QFuture<QVector<SearchResult>> searchFuture = QtConcurrent::run(
 		[&, q]()
 		{
 			SqliteSearch searcher(db);
-			this->currentQuery = LooqsQuery::build(q);
-			return searcher.search(this->currentQuery);
+			QVector<SearchResult> results;
+			this->contentSearchQuery = LooqsQuery::build(q, TokenType::FILTER_CONTENT_CONTAINS, true);
+
+			/* We can have a path search in contentsearch too (if given explicitly), so no need to do it twice.
+				Make sure path results are listed first. */
+			bool addContentSearch = this->contentSearchQuery.hasContentSearch();
+			bool addPathSearch = !this->contentSearchQuery.hasPathSearch() || !addContentSearch;
+			if(addPathSearch)
+			{
+				LooqsQuery filesQuery = LooqsQuery::build(q, TokenType::FILTER_PATH_CONTAINS, false);
+				results.append(searcher.search(filesQuery));
+			}
+			if(addContentSearch)
+			{
+				results.append(searcher.search(this->contentSearchQuery));
+			}
+			return results;
 		});
 	searchWatcher.setFuture(searchFuture);
 }
 
 void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 {
-	this->pdfSearchResults.clear();
+	this->previewableSearchResults.clear();
 	ui->treeResultsList->clear();
 
 	bool hasDeleted = false;
@@ -215,9 +327,9 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 		bool exists = pathInfo.exists();
 		if(exists)
 		{
-			if(result.fileData.absPath.endsWith(".pdf"))
+			if(PreviewGenerator::get(pathInfo) != nullptr)
 			{
-				this->pdfSearchResults.append(result);
+				this->previewableSearchResults.append(result);
 			}
 		}
 		else
@@ -227,15 +339,15 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 	}
 	ui->treeResultsList->resizeColumnToContents(0);
 	ui->treeResultsList->resizeColumnToContents(1);
-	pdfDirty = !this->pdfSearchResults.empty();
+	previewDirty = !this->previewableSearchResults.empty();
 
-	int numpages = ceil(static_cast<double>(this->pdfSearchResults.size()) / pdfPreviewsPerPage);
-	ui->spinPdfPreviewPage->setMinimum(1);
-	ui->spinPdfPreviewPage->setMaximum(numpages);
-	ui->spinPdfPreviewPage->setValue(1);
-	if(pdfTabActive() && pdfDirty)
+	int numpages = ceil(static_cast<double>(this->previewableSearchResults.size()) / previewsPerPage);
+	ui->spinPreviewPage->setMinimum(1);
+	ui->spinPreviewPage->setMaximum(numpages);
+	ui->spinPreviewPage->setValue(1);
+	if(previewTabActive() && previewDirty)
 	{
-		makePdfPreview(1);
+		makePreviews(1);
 	}
 
 	QString statusText = "Results: " + QString::number(results.size()) + " files";
@@ -246,25 +358,25 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
 	ui->lblSearchResults->setText(statusText);
 }
 
-void MainWindow::makePdfPreview(int page)
+void MainWindow::makePreviews(int page)
 {
 
-	this->pdfWorkerWatcher.cancel();
-	this->pdfWorkerWatcher.waitForFinished();
+	this->previewWorkerWatcher.cancel();
+	this->previewWorkerWatcher.waitForFinished();
 
 	QCoreApplication::processEvents(); // Maybe not necessary anymore, depends on whether it's possible that a slot is
 									   // still to be fired.
 	qDeleteAll(ui->scrollAreaWidgetContents->children());
 
 	ui->scrollAreaWidgetContents->setLayout(new QHBoxLayout());
-	ui->pdfProcessBar->setMaximum(this->pdfSearchResults.size());
+	ui->previewProcessBar->setMaximum(this->previewableSearchResults.size());
 	processedPdfPreviews = 0;
 	QString scaleText = ui->comboScale->currentText();
 	scaleText.chop(1);
 
 	QVector<QString> wordsToHighlight;
 	QRegularExpression extractor(R"#("([^"]*)"|(\w+))#");
-	for(const Token &token : this->currentQuery.getTokens())
+	for(const Token &token : this->contentSearchQuery.getTokens())
 	{
 		if(token.type == FILTER_CONTENT_CONTAINS)
 		{
@@ -281,13 +393,14 @@ void MainWindow::makePdfPreview(int page)
 			}
 		}
 	}
-	PdfWorker worker;
-	int end = pdfPreviewsPerPage;
-	int begin = page * pdfPreviewsPerPage - pdfPreviewsPerPage;
-	this->pdfWorkerWatcher.setFuture(
-		worker.generatePreviews(this->pdfSearchResults.mid(begin, end), wordsToHighlight, scaleText.toInt() / 100.));
-	ui->pdfProcessBar->setMaximum(this->pdfWorkerWatcher.progressMaximum());
-	ui->pdfProcessBar->setMinimum(this->pdfWorkerWatcher.progressMinimum());
+	PreviewWorker worker;
+	int end = previewsPerPage;
+	int begin = page * previewsPerPage - previewsPerPage;
+	this->previewWorkerWatcher.setFuture(worker.generatePreviews(this->previewableSearchResults.mid(begin, end),
+																 wordsToHighlight, scaleText.toInt() / 100.));
+	ui->previewProcessBar->setMaximum(this->previewWorkerWatcher.progressMaximum());
+	ui->previewProcessBar->setMinimum(this->previewWorkerWatcher.progressMinimum());
+	ui->previewProcessBar->setVisible(this->previewableSearchResults.size() > 0);
 }
 
 void MainWindow::handleSearchError(QString error)
@@ -301,17 +414,27 @@ void MainWindow::createSearchResutlMenu(QMenu &menu, const QFileInfo &fileInfo)
 				   [&fileInfo] { QGuiApplication::clipboard()->setText(fileInfo.fileName()); });
 	menu.addAction("Copy full path to clipboard",
 				   [&fileInfo] { QGuiApplication::clipboard()->setText(fileInfo.absoluteFilePath()); });
-	menu.addAction("Open containing folder",
-				   [&fileInfo]
-				   {
-					   QString dir = fileInfo.absolutePath();
-					   QDesktopServices::openUrl(QUrl::fromLocalFile(dir));
-				   });
+	menu.addAction("Open containing folder", [this, &fileInfo] { this->ipcFileOpen(fileInfo.absolutePath()); });
+}
+
+void MainWindow::ipcDocOpen(QString path, int num)
+{
+	QStringList args;
+	args << path;
+	args << QString::number(num);
+	this->ipcClient->sendCommand(DocOpen, args);
+}
+
+void MainWindow::ipcFileOpen(QString path)
+{
+	QStringList args;
+	args << path;
+	this->ipcClient->sendCommand(FileOpen, args);
 }
 
 void MainWindow::treeSearchItemActivated(QTreeWidgetItem *item, int i)
 {
-	QDesktopServices::openUrl(QUrl::fromLocalFile(item->text(1)));
+	ipcFileOpen(item->text(1));
 }
 
 void MainWindow::showSearchResultsContextMenu(const QPoint &point)

gui/mainwindow.h

@@ -8,8 +8,11 @@
 #include <QKeyEvent>
 #include <QFutureWatcher>
 #include <QSqlDatabase>
-#include "pdfworker.h"
+#include <QLocalSocket>
+#include "previewworker.h"
 #include "../shared/looqsquery.h"
+#include "ipcclient.h"
+#include "indexer.h"
 namespace Ui
 {
 class MainWindow;
@@ -20,39 +23,50 @@ class MainWindow : public QMainWindow
 	Q_OBJECT
 
   public:
-	explicit MainWindow(QWidget *parent = 0);
+	explicit MainWindow(QWidget *parent, IPCClient &client);
 	~MainWindow();
   signals:
 	void beginSearch(const QString &query);
 	void startPdfPreviewGeneration(QVector<SearchResult> paths, double scalefactor);
 
   private:
+	DatabaseFactory *dbFactory;
+	SqliteDbService *dbService;
 	Ui::MainWindow *ui;
+	IPCClient *ipcClient;
+	Indexer *indexer;
 	QFileIconProvider iconProvider;
-	bool pdfDirty;
+	bool previewDirty;
 	QSqlDatabase db;
 	QFutureWatcher<QVector<SearchResult>> searchWatcher;
-	QFutureWatcher<PdfPreview> pdfWorkerWatcher;
+	QFutureWatcher<QSharedPointer<PreviewResult>> previewWorkerWatcher;
 	void add(QString path, unsigned int page);
-	QVector<SearchResult> pdfSearchResults;
+	QVector<SearchResult> previewableSearchResults;
 	void connectSignals();
-	void makePdfPreview(int page);
-	bool pdfTabActive();
+	void makePreviews(int page);
+	bool previewTabActive();
+	bool indexerTabActive();
 	void keyPressEvent(QKeyEvent *event) override;
 	unsigned int processedPdfPreviews;
 	void handleSearchResults(const QVector<SearchResult> &results);
 	void handleSearchError(QString error);
-	LooqsQuery currentQuery;
-	int pdfPreviewsPerPage;
+	LooqsQuery contentSearchQuery;
+	int previewsPerPage;
 	void createSearchResutlMenu(QMenu &menu, const QFileInfo &fileInfo);
+	void ipcDocOpen(QString path, int num);
+	void ipcFileOpen(QString path);
+
   private slots:
 	void lineEditReturnPressed();
 	void treeSearchItemActivated(QTreeWidgetItem *item, int i);
 	void showSearchResultsContextMenu(const QPoint &point);
 	void tabChanged();
-	void pdfPreviewReceived(PdfPreview preview);
+	void previewReceived(QSharedPointer<PreviewResult> preview);
 	void comboScaleChanged(int i);
-	void spinPdfPreviewPageValueChanged(int val);
+	void spinPreviewPageValueChanged(int val);
+	void startIndexing();
+	void finishIndexing();
+	void addPathToIndex();
 };
 
 #endif // MAINWINDOW_H

gui/mainwindow.ui

@@ -7,7 +7,7 @@
     <x>0</x>
     <y>0</y>
     <width>1221</width>
-    <height>614</height>
+    <height>674</height>
    </rect>
   </property>
   <property name="windowTitle">
@@ -27,7 +27,7 @@
        <enum>QTabWidget::South</enum>
       </property>
       <property name="currentIndex">
-       <number>1</number>
+       <number>2</number>
       </property>
       <widget class="QWidget" name="resultsTab">
        <attribute name="title">
@@ -60,9 +60,9 @@
         </item>
        </layout>
       </widget>
-      <widget class="QWidget" name="pdfPreviewTab">
+      <widget class="QWidget" name="previewsTab">
        <attribute name="title">
-        <string>PDF-Preview</string>
+        <string>Previews</string>
        </attribute>
        <layout class="QVBoxLayout" name="verticalLayout_2" stretch="1,0">
         <item>
@@ -81,8 +81,8 @@
             <rect>
              <x>0</x>
              <y>0</y>
-             <width>1179</width>
-             <height>370</height>
+             <width>1185</width>
+             <height>419</height>
             </rect>
            </property>
            <layout class="QHBoxLayout" name="horizontalLayout"/>
@@ -143,7 +143,7 @@
            </widget>
           </item>
           <item>
-           <widget class="QSpinBox" name="spinPdfPreviewPage">
+           <widget class="QSpinBox" name="spinPreviewPage">
             <property name="buttonSymbols">
              <enum>QAbstractSpinBox::PlusMinus</enum>
             </property>
@@ -172,6 +172,171 @@
         </item>
        </layout>
       </widget>
+      <widget class="QWidget" name="indexerTab">
+       <attribute name="title">
+        <string>Index</string>
+       </attribute>
+       <layout class="QGridLayout" name="gridLayout">
+        <item row="1" column="0">
+         <widget class="QGroupBox" name="groupBoxPaths">
+          <property name="title">
+           <string>Add paths to scan</string>
+          </property>
+          <layout class="QGridLayout" name="gridLayout_2">
+           <item row="1" column="0">
+            <widget class="QLineEdit" name="txtPathScanAdd"/>
+           </item>
+           <item row="3" column="0" colspan="5">
+            <widget class="QListWidget" name="lstPaths"/>
+           </item>
+           <item row="1" column="3">
+            <widget class="QToolButton" name="btnDeletePath">
+             <property name="enabled">
+              <bool>false</bool>
+             </property>
+             <property name="text">
+              <string>Delete</string>
+             </property>
+            </widget>
+           </item>
+           <item row="1" column="1">
+            <widget class="QPushButton" name="btnChoosePath">
+             <property name="text">
+              <string>...</string>
+             </property>
+            </widget>
+           </item>
+           <item row="1" column="2">
+            <widget class="QPushButton" name="btnAddPath">
+             <property name="text">
+              <string>Add</string>
+             </property>
+            </widget>
+           </item>
+          </layout>
+         </widget>
+        </item>
+        <item row="5" column="0">
+         <widget class="QLabel" name="label">
+          <property name="text">
+           <string>Ignore patterns, separated by ';'. Example: *.js;*Downloads*</string>
+          </property>
+         </widget>
+        </item>
+        <item row="9" column="0">
+         <widget class="QGroupBox" name="groupBoxIndexProgress">
+          <property name="contextMenuPolicy">
+           <enum>Qt::PreventContextMenu</enum>
+          </property>
+          <property name="title">
+           <string>Index Progress</string>
+          </property>
+          <layout class="QVBoxLayout" name="verticalLayout_4">
+           <item>
+            <layout class="QHBoxLayout" name="horizontalLayout_7">
+             <item>
+              <widget class="QLabel" name="lblPathsFound">
+               <property name="text">
+                <string>Paths found:</string>
+               </property>
+              </widget>
+             </item>
+             <item>
+              <widget class="QLabel" name="lblPathsFoundValue">
+               <property name="text">
+                <string/>
+               </property>
+              </widget>
+             </item>
+            </layout>
+           </item>
+           <item>
+            <layout class="QHBoxLayout" name="horizontalLayout_4">
+             <item>
+              <widget class="QLabel" name="lblAdded">
+               <property name="font">
+                <font>
+                 <weight>50</weight>
+                 <bold>false</bold>
+                </font>
+               </property>
+               <property name="text">
+                <string>Added:</string>
+               </property>
+              </widget>
+             </item>
+             <item>
+              <widget class="QLabel" name="lblAddedValue">
+               <property name="text">
+                <string/>
+               </property>
+              </widget>
+             </item>
+            </layout>
+           </item>
+           <item>
+            <layout class="QHBoxLayout" name="horizontalLayout_5">
+             <item>
+              <widget class="QLabel" name="lblSkipped">
+               <property name="font">
+                <font>
+                 <weight>50</weight>
+                 <bold>false</bold>
+                </font>
+               </property>
+               <property name="text">
+                <string>Skipped:</string>
+               </property>
+              </widget>
+             </item>
+             <item>
+              <widget class="QLabel" name="lblSkippedValue">
+               <property name="text">
+                <string/>
+               </property>
+              </widget>
+             </item>
+            </layout>
+           </item>
+           <item>
+            <layout class="QHBoxLayout" name="horizontalLayout_6">
+             <item>
+              <widget class="QLabel" name="lblFailed">
+               <property name="font">
+                <font>
+                 <weight>50</weight>
+                 <bold>false</bold>
+                </font>
+               </property>
+               <property name="text">
+                <string>Failed:</string>
+               </property>
+              </widget>
+             </item>
+             <item>
+              <widget class="QLabel" name="lblFailedValue">
+               <property name="text">
+                <string/>
+               </property>
+              </widget>
+             </item>
+            </layout>
+           </item>
+          </layout>
+         </widget>
+        </item>
+        <item row="6" column="0">
+         <widget class="QLineEdit" name="lineEdit"/>
+        </item>
+        <item row="10" column="0">
+         <widget class="QPushButton" name="btnStartIndexing">
+          <property name="text">
+           <string>Start indexing</string>
+          </property>
+         </widget>
+        </item>
+       </layout>
+      </widget>
      </widget>
     </item>
     <item>
@@ -182,7 +347,7 @@
      </widget>
     </item>
     <item>
-     <widget class="QProgressBar" name="pdfProcessBar">
+     <widget class="QProgressBar" name="previewProcessBar">
       <property name="value">
        <number>24</number>
       </property>
@@ -190,24 +355,6 @@
     </item>
    </layout>
   </widget>
-  <widget class="QMenuBar" name="menuBar">
-   <property name="geometry">
-    <rect>
-     <x>0</x>
-     <y>0</y>
-     <width>1221</width>
-     <height>20</height>
-    </rect>
-   </property>
-  </widget>
-  <widget class="QToolBar" name="mainToolBar">
-   <attribute name="toolBarArea">
-    <enum>TopToolBarArea</enum>
-   </attribute>
-   <attribute name="toolBarBreak">
-    <bool>false</bool>
-   </attribute>
-  </widget>
   <widget class="QStatusBar" name="statusBar"/>
  </widget>
  <layoutdefault spacing="6" margin="11"/>

gui/pdfpreview.cpp

@@ -1,5 +0,0 @@
-#include "pdfpreview.h"
-
-PdfPreview::PdfPreview()
-{
-}

gui/pdfpreview.h

@@ -1,19 +0,0 @@
-#ifndef PDFPREVIEW_H
-#define PDFPREVIEW_H
-#include <QImage>
-
-class PdfPreview
-{
-  public:
-	PdfPreview();
-	QImage previewImage;
-	QString documentPath;
-	unsigned int page;
-
-	bool hasPreviewImage()
-	{
-		return !previewImage.isNull();
-	}
-};
-
-#endif // PDFPREVIEW_H

gui/pdfworker.cpp

@@ -1,108 +0,0 @@
-
-#include <QApplication>
-#include <QScreen>
-#include <QDebug>
-#include <QScopedPointer>
-#include <QMutexLocker>
-#include <QtConcurrent/QtConcurrent>
-#include <QtConcurrent/QtConcurrentMap>
-#include <QPainter>
-#include <atomic>
-#include "pdfworker.h"
-
-static QMutex cacheMutex;
-struct Renderer
-{
-
-	typedef PdfPreview result_type;
-	double scaleX;
-	double scaleY;
-	QHash<QString, Poppler::Document *> documentcache;
-	QVector<QString> wordsToHighlight;
-	Renderer(double scaleX, double scaleY, QVector<QString> wordsToHighlight)
-	{
-		this->scaleX = scaleX;
-		this->scaleY = scaleY;
-		this->wordsToHighlight = wordsToHighlight;
-	}
-
-	~Renderer()
-	{
-		qDeleteAll(documentcache);
-	}
-
-	Poppler::Document *document(QString path)
-	{
-		if(documentcache.contains(path))
-		{
-			return documentcache.value(path);
-		}
-		Poppler::Document *result = Poppler::Document::load(path);
-		if(result == nullptr)
-		{
-			// TODO: some kind of user feedback would be nice
-			return nullptr;
-		}
-		result->setRenderHint(Poppler::Document::TextAntialiasing);
-		QMutexLocker locker(&cacheMutex);
-		documentcache.insert(path, result);
-		locker.unlock();
-		return result;
-	}
-
-	PdfPreview operator()(const PdfPreview &preview)
-	{
-		PdfPreview result = preview;
-		Poppler::Document *doc = document(preview.documentPath);
-		if(doc == nullptr)
-		{
-			return preview;
-		}
-		if(doc->isLocked())
-		{
-			return preview;
-		}
-		int p = (int)preview.page - 1;
-		if(p < 0)
-		{
-			p = 0;
-		}
-		Poppler::Page *pdfPage = doc->page(p);
-		QImage img = pdfPage->renderToImage(scaleX, scaleY);
-		for(QString &word : wordsToHighlight)
-		{
-			QList<QRectF> rects = pdfPage->search(word, Poppler::Page::SearchFlag::IgnoreCase);
-			for(QRectF &rect : rects)
-			{
-				QPainter painter(&img);
-				painter.scale(scaleX / 72.0, scaleY / 72.0);
-				painter.fillRect(rect, QColor(255, 255, 0, 64));
-			}
-		}
-		result.previewImage = img;
-		return result;
-	}
-};
-
-QFuture<PdfPreview> PdfWorker::generatePreviews(const QVector<SearchResult> paths, QVector<QString> wordsToHighlight,
-												double scalefactor)
-{
-	QVector<PdfPreview> previews;
-
-	for(const SearchResult &sr : paths)
-	{
-		for(int page : sr.pages)
-		{
-			PdfPreview p;
-			p.documentPath = sr.fileData.absPath;
-			p.page = page;
-			previews.append(p);
-		}
-	}
-
-	double scaleX = QGuiApplication::primaryScreen()->physicalDotsPerInchX() * scalefactor;
-	double scaleY = QGuiApplication::primaryScreen()->physicalDotsPerInchY() * scalefactor;
-
-	QSettings setting;
-	return QtConcurrent::mapped(previews, Renderer(scaleX, scaleY, wordsToHighlight));
-}

gui/pdfworker.h

@@ -1,23 +0,0 @@
-#ifndef PDFWORKER_H
-#define PDFWORKER_H
-#include <QObject>
-#include <QImage>
-#include <QHash>
-#include <QThread>
-#include <QMutex>
-#include <QWaitCondition>
-#include <QMutex>
-#include <QFuture>
-#include <poppler-qt5.h>
-#include "pdfpreview.h"
-#include "searchresult.h"
-
-class PdfWorker : public QObject
-{
-	Q_OBJECT
-  public:
-	QFuture<PdfPreview> generatePreviews(const QVector<SearchResult> paths, QVector<QString> wordsToHighlight,
-										 double scalefactor);
-};
-
-#endif // PDFWORKER_H

gui/previewgenerator.cpp

@@ -0,0 +1,15 @@
+#include "previewgenerator.h"
+#include "previewgeneratorpdf.h"
+#include "previewgeneratorplaintext.h"
+
+static PreviewGenerator *plainTextGenerator = new PreviewGeneratorPlainText();
+
+static QMap<QString, PreviewGenerator *> generators{
+	{"pdf", new PreviewGeneratorPdf()}, {"txt", plainTextGenerator},  {"md", plainTextGenerator},
+	{"py", plainTextGenerator},			{"java", plainTextGenerator}, {"js", plainTextGenerator},
+	{"cpp", plainTextGenerator},		{"c", plainTextGenerator},	  {"sql", plainTextGenerator}};
+
+PreviewGenerator *PreviewGenerator::get(QFileInfo &info)
+{
+	return generators.value(info.suffix(), nullptr);
+}

gui/previewgenerator.h

@@ -0,0 +1,20 @@
+#ifndef PREVIEWGENERATOR_H
+#define PREVIEWGENERATOR_H
+#include <QVector>
+#include <QSharedPointer>
+#include <QFileInfo>
+#include "previewresult.h"
+#include "renderconfig.h"
+
+class PreviewGenerator
+{
+  public:
+	virtual PreviewResult *generate(RenderConfig config, QString documentPath, unsigned int page) = 0;
+	virtual ~PreviewGenerator()
+	{
+	}
+
+	static PreviewGenerator *get(QFileInfo &info);
+};
+
+#endif // PREVIEWGENERATOR_H

gui/previewgeneratormapfunctor.cpp

@@ -0,0 +1,25 @@
+#include "previewgeneratormapfunctor.h"
+#include "previewgeneratorpdf.h"
+
+PreviewGeneratorMapFunctor::PreviewGeneratorMapFunctor()
+{
+}
+
+void PreviewGeneratorMapFunctor::setRenderConfig(RenderConfig config)
+{
+	this->renderConfig = config;
+}
+
+QSharedPointer<PreviewResult> PreviewGeneratorMapFunctor::operator()(const QSharedPointer<PreviewResult> &renderResult)
+{
+	QFileInfo info{renderResult->getDocumentPath()};
+	PreviewGenerator *previewGenerator = PreviewGenerator::get(info);
+	if(previewGenerator == nullptr)
+	{
+		return QSharedPointer<PreviewResult>();
+	}
+	auto preview =
+		previewGenerator->generate(this->renderConfig, renderResult->getDocumentPath(), renderResult->getPage());
+
+	return QSharedPointer<PreviewResult>(preview);
+}

gui/previewgeneratormapfunctor.h

@@ -0,0 +1,28 @@
+#ifndef PREVIEWGENERATORMAPFUNCTOR_H
+#define PREVIEWGENERATORMAPFUNCTOR_H
+
+#include "renderconfig.h"
+#include "previewgenerator.h"
+
+class PreviewGeneratorMapFunctor
+{
+
+  private:
+	enum GeneratorIndex
+	{
+		PDF = 0,
+		LAST_DUMMY
+	};
+	RenderConfig renderConfig;
+
+  public:
+	typedef QSharedPointer<PreviewResult> result_type;
+
+	PreviewGeneratorMapFunctor();
+
+	void setRenderConfig(RenderConfig config);
+
+	QSharedPointer<PreviewResult> operator()(const QSharedPointer<PreviewResult> &renderResult);
+};
+
+#endif // PREVIEWGENERATORMAPFUNCTOR_H

gui/previewgeneratorpdf.cpp

@@ -0,0 +1,59 @@
+#include <QMutexLocker>
+#include <QPainter>
+
+#include "previewgeneratorpdf.h"
+
+static QMutex cacheMutex;
+
+Poppler::Document *PreviewGeneratorPdf::document(QString path)
+{
+	if(documentcache.contains(path))
+	{
+		return documentcache.value(path);
+	}
+	Poppler::Document *result = Poppler::Document::load(path);
+	if(result == nullptr)
+	{
+		// TODO: some kind of user feedback would be nice
+		return nullptr;
+	}
+	result->setRenderHint(Poppler::Document::TextAntialiasing);
+	QMutexLocker locker(&cacheMutex);
+	documentcache.insert(path, result);
+	locker.unlock();
+	return result;
+}
+
+PreviewResult *PreviewGeneratorPdf::generate(RenderConfig config, QString documentPath, unsigned int page)
+{
+	PreviewResultPdf *result = new PreviewResultPdf(documentPath, page);
+
+	Poppler::Document *doc = document(documentPath);
+	if(doc == nullptr)
+	{
+		return result;
+	}
+	if(doc->isLocked())
+	{
+		return result;
+	}
+	int p = (int)page - 1;
+	if(p < 0)
+	{
+		p = 0;
+	}
+	Poppler::Page *pdfPage = doc->page(p);
+	QImage img = pdfPage->renderToImage(config.scaleX, config.scaleY);
+	for(QString &word : config.wordsToHighlight)
+	{
+		QList<QRectF> rects = pdfPage->search(word, Poppler::Page::SearchFlag::IgnoreCase);
+		for(QRectF &rect : rects)
+		{
+			QPainter painter(&img);
+			painter.scale(config.scaleX / 72.0, config.scaleY / 72.0);
+			painter.fillRect(rect, QColor(255, 255, 0, 64));
+		}
+	}
+	result->previewImage = img;
+	return result;
+}

gui/previewgeneratorpdf.h

@@ -0,0 +1,24 @@
+#ifndef PREVIEWGENERATORPDF_H
+#define PREVIEWGENERATORPDF_H
+#include <poppler-qt5.h>
+#include "previewgenerator.h"
+#include "previewresultpdf.h"
+
+class PreviewGeneratorPdf : public PreviewGenerator
+{
+  protected:
+	QHash<QString, Poppler::Document *> documentcache;
+	Poppler::Document *document(QString path);
+
+  public:
+	using PreviewGenerator::PreviewGenerator;
+
+	PreviewResult *generate(RenderConfig config, QString documentPath, unsigned int page);
+
+	~PreviewGeneratorPdf()
+	{
+		qDeleteAll(documentcache);
+	}
+};
+
+#endif // PREVIEWGENERATORPDF_H

gui/previewgeneratorplaintext.cpp

@@ -0,0 +1,81 @@
+#include <QTextStream>
+
+#include "previewgeneratorplaintext.h"
+#include "previewresultplaintext.h"
+
+PreviewResult *PreviewGeneratorPlainText::generate(RenderConfig config, QString documentPath, unsigned int page)
+{
+	PreviewResultPlainText *result = new PreviewResultPlainText(documentPath, page);
+	QFile file(documentPath);
+	if(!file.open(QFile::ReadOnly | QFile::Text))
+	{
+		return result;
+	}
+	QTextStream in(&file);
+
+	QString resulText = "";
+	QString content = in.readAll();
+	QMap<int, QString> snippet;
+
+	int coveredRange = 0;
+
+	int lastWordPos = 0;
+	QHash<QString, int> countmap;
+
+	for(QString &word : config.wordsToHighlight)
+	{
+
+		int lastPos = 0;
+		int index = content.indexOf(word, lastPos, Qt::CaseInsensitive);
+		while(index != -1)
+		{
+			countmap[word] = countmap.value(word, 0) + 1;
+
+			if(index >= lastWordPos && index <= coveredRange)
+			{
+				break;
+			}
+			int begin = index - 50;
+			if(begin < 0)
+			{
+				begin = 0;
+			}
+			int after = index + 50;
+			if(after > content.size())
+			{
+				after = content.size();
+			}
+
+			snippet[index] = "...<br>" + content.mid(begin, after) + "...<br>";
+			coveredRange = after;
+			lastPos = index;
+
+			index = content.indexOf(word, lastPos + 1, Qt::CaseInsensitive);
+		}
+		lastWordPos = lastPos;
+	}
+
+	auto i = snippet.constBegin();
+	while(i != snippet.constEnd())
+	{
+		resulText.append(i.value());
+		++i;
+	}
+
+	for(QString &word : config.wordsToHighlight)
+	{
+		resulText.replace(word, "<span style=\"background-color: yellow;\">" + word + "</span>", Qt::CaseInsensitive);
+	}
+
+	QFileInfo info{documentPath};
+
+	QString header = "<b>" + info.fileName() + "</b> ";
+	for(QString &word : config.wordsToHighlight)
+	{
+		header += word + ": " + QString::number(countmap[word]) + " ";
+	}
+	header += "<hr>";
+
+	result->setText(header + resulText.replace("\n", "<br>"));
+	return result;
+}

gui/previewgeneratorplaintext.h

@@ -0,0 +1,12 @@
+#ifndef PREVIEWGENERATORPLAINTEXT_H
+#define PREVIEWGENERATORPLAINTEXT_H
+#include "previewgenerator.h"
+
+class PreviewGeneratorPlainText : public PreviewGenerator
+{
+  public:
+	using PreviewGenerator::PreviewGenerator;
+	PreviewResult *generate(RenderConfig config, QString documentPath, unsigned int page);
+};
+
+#endif // PREVIEWGENERATORPLAINTEXT_H

gui/previewresult.cpp

@@ -0,0 +1,35 @@
+#include "previewresult.h"
+
+PreviewResult::PreviewResult()
+{
+}
+
+PreviewResult::~PreviewResult()
+{
+}
+
+QWidget *PreviewResult::createPreviewWidget()
+{
+	return nullptr;
+}
+
+bool PreviewResult::hasPreview()
+{
+	return false;
+}
+
+PreviewResult::PreviewResult(QString documentPath, unsigned int page)
+{
+	this->documentPath = documentPath;
+	this->page = page;
+}
+
+QString PreviewResult::getDocumentPath() const
+{
+	return this->documentPath;
+}
+
+unsigned int PreviewResult::getPage() const
+{
+	return this->page;
+}

gui/previewresult.h

@@ -0,0 +1,22 @@
+#ifndef PREVIEWRESULT_H
+#define PREVIEWRESULT_H
+#include "clicklabel.h"
+
+class PreviewResult
+{
+  protected:
+	QString documentPath;
+	unsigned int page;
+
+  public:
+	PreviewResult();
+	PreviewResult(QString documentPath, unsigned int page);
+	PreviewResult(const PreviewResult &o) = default;
+	virtual ~PreviewResult();
+	virtual QWidget *createPreviewWidget();
+	virtual bool hasPreview();
+	QString getDocumentPath() const;
+	unsigned int getPage() const;
+};
+
+#endif // PREVIEWRESULT_H

gui/previewresultpdf.cpp

@@ -0,0 +1,21 @@
+#include "previewresultpdf.h"
+
+PreviewResultPdf::PreviewResultPdf(const PreviewResult &o)
+{
+	this->documentPath = o.getDocumentPath();
+	this->page = o.getPage();
+}
+
+QWidget *PreviewResultPdf::createPreviewWidget()
+{
+	ClickLabel *label = new ClickLabel();
+	label->setPixmap(QPixmap::fromImage(previewImage));
+	label->setToolTip(getDocumentPath());
+	return label;
+}
+
+bool PreviewResultPdf::hasPreview()
+{
+	bool result = !this->previewImage.isNull();
+	return result;
+}

gui/previewresultpdf.h

@@ -0,0 +1,17 @@
+#ifndef PREVIEWRESULTPDF_H
+#define PREVIEWRESULTPDF_H
+#include <QImage>
+#include "previewresult.h"
+
+class PreviewResultPdf : public PreviewResult
+{
+  public:
+	using PreviewResult::PreviewResult;
+	PreviewResultPdf(const PreviewResult &o);
+	QImage previewImage;
+
+	QWidget *createPreviewWidget() override;
+	bool hasPreview() override;
+};
+
+#endif // PREVIEWRESULTPDF_H

gui/previewresultplaintext.cpp

@@ -0,0 +1,30 @@
+#include "previewresultplaintext.h"
+
+PreviewResultPlainText::PreviewResultPlainText(const PreviewResult &o)
+{
+	this->documentPath = o.getDocumentPath();
+	this->page = o.getPage();
+}
+
+QWidget *PreviewResultPlainText::createPreviewWidget()
+{
+
+	ClickLabel *label = new ClickLabel();
+	label->setText(this->text);
+	label->setToolTip(getDocumentPath());
+	label->setStyleSheet("border: 1px solid black");
+	label->setMaximumWidth(768);
+	label->setMaximumHeight(512);
+	label->setTextFormat(Qt::RichText);
+	return label;
+}
+
+bool PreviewResultPlainText::hasPreview()
+{
+	return !text.isEmpty();
+}
+
+void PreviewResultPlainText::setText(QString text)
+{
+	this->text = text;
+}

gui/previewresultplaintext.h

@@ -0,0 +1,20 @@
+#ifndef PREVIEWRESULTPLAINTEXT_H
+#define PREVIEWRESULTPLAINTEXT_H
+#include "previewresult.h"
+
+class PreviewResultPlainText : public PreviewResult
+{
+  private:
+	QString text;
+
+  public:
+	using PreviewResult::PreviewResult;
+	PreviewResultPlainText(const PreviewResult &o);
+
+	QWidget *createPreviewWidget() override;
+	bool hasPreview() override;
+
+	void setText(QString text);
+};
+
+#endif // PREVIEWRESULTPLAINTEXT_H

gui/previewworker.cpp

@@ -0,0 +1,39 @@
+#include <QApplication>
+#include <QScreen>
+#include <QScopedPointer>
+#include <QMutexLocker>
+#include <QtConcurrent/QtConcurrent>
+#include <QtConcurrent/QtConcurrentMap>
+#include <atomic>
+#include "previewworker.h"
+
+PreviewWorker::PreviewWorker()
+{
+}
+
+QFuture<QSharedPointer<PreviewResult>> PreviewWorker::generatePreviews(const QVector<SearchResult> paths,
+																	   QVector<QString> wordsToHighlight,
+																	   double scalefactor)
+{
+	QVector<QSharedPointer<PreviewResult>> previews;
+
+	for(const SearchResult &sr : paths)
+	{
+		for(unsigned int page : sr.pages)
+		{
+			QSharedPointer<PreviewResult> ptr =
+				QSharedPointer<PreviewResult>(new PreviewResult{sr.fileData.absPath, page});
+			previews.append(ptr);
+		}
+	}
+
+	RenderConfig renderConfig;
+	renderConfig.scaleX = QGuiApplication::primaryScreen()->physicalDotsPerInchX() * scalefactor;
+	renderConfig.scaleY = QGuiApplication::primaryScreen()->physicalDotsPerInchY() * scalefactor;
+	renderConfig.wordsToHighlight = wordsToHighlight;
+
+	auto mapFunctor = new PreviewGeneratorMapFunctor();
+	mapFunctor->setRenderConfig(renderConfig);
+
+	return QtConcurrent::mapped(previews, *mapFunctor);
+}

gui/previewworker.h

@@ -0,0 +1,29 @@
+#ifndef PREVIEWWORKER_H
+#define PREVIEWWORKER_H
+#include <QObject>
+#include <QImage>
+#include <QHash>
+#include <QThread>
+#include <QMutex>
+#include <QWaitCondition>
+#include <QMutex>
+#include <QFuture>
+#include "previewresultpdf.h"
+#include "searchresult.h"
+#include "previewgenerator.h"
+#include "previewworker.h"
+#include "previewgeneratorpdf.h"
+#include "previewgeneratormapfunctor.h"
+
+class PreviewWorker : public QObject
+{
+	Q_OBJECT
+  public:
+	PreviewWorker();
+	QSharedPointer<PreviewGenerator> createGenerator(QString path);
+
+	QFuture<QSharedPointer<PreviewResult>> generatePreviews(const QVector<SearchResult> paths,
+															QVector<QString> wordsToHighlight, double scalefactor);
+};
+
+#endif // PREVIEWWORKER_H

gui/renderconfig.h

@@ -0,0 +1,12 @@
+#ifndef RENDERCONFIG_H
+#define RENDERCONFIG_H
+#include <QVector>
+
+struct RenderConfig
+{
+	double scaleX = 50 / 100.;
+	double scaleY = scaleX;
+	QVector<QString> wordsToHighlight;
+};
+
+#endif // RENDERCONFIG_H

looqs.desktop

@@ -0,0 +1,9 @@
+[Desktop Entry]
+Name=looqs
+Exec=/usr/bin/looqs-gui
+Terminal=false
+Type=Application
+Icon=looqs
+StartupWMClass=looqs
+Comment=FTS desktop search with previews
+Categories=Qt;Utility;

shared/common.cpp

@@ -9,13 +9,17 @@
 #include <QDebug>
 #include "looqsgeneralexception.h"
 #include "common.h"
+#include "dbmigrator.h"
+#include "databasefactory.h"
+#include "logger.h"
 
 #define SETTINGS_KEY_DBPATH "dbpath"
 #define SETTINGS_KEY_FIRSTRUN "firstrun"
+#define SETTINGS_KEY_IPCSOCKETPATH "ipcsocketpath"
 
 inline void initResources()
 {
-	Q_INIT_RESOURCE(create);
+	Q_INIT_RESOURCE(migrations);
 }
 
 bool Common::initSqliteDatabase(QString path)
@@ -28,28 +32,9 @@ bool Common::initSqliteDatabase(QString path)
 		return false;
 	}
 	initResources();
-	QFile file(":./create.sql");
-	if(!file.open(QIODevice::ReadOnly))
-	{
-		qDebug() << "Failed to load SQL creation script from embedded resource";
-		return false;
-	}
-	QTextStream stream(&file);
-	db.transaction();
-	while(!stream.atEnd())
-	{
-		QString sql = stream.readLine();
-		QSqlQuery sqlQuery;
-		if(!sqlQuery.exec(sql))
-		{
-			qDebug() << "Failed to execute sql statement while initializing database: " << sqlQuery.lastError();
-			db.rollback();
-			return false;
-		}
-	}
-	db.commit();
+	DBMigrator migrator{db};
+	migrator.performMigrations();
 	db.close();
-	file.close();
 	return true;
 }
 
@@ -84,6 +69,21 @@ void Common::ensureConfigured()
 		{
 			throw LooqsGeneralException("Database " + dbpath + " was not found");
 		}
+		DatabaseFactory factory{dbpath};
+		auto db = factory.forCurrentThread();
+		DBMigrator migrator{db};
+		if(migrator.migrationNeeded())
+		{
+			QFile out;
+			out.open(stderr, QIODevice::WriteOnly);
+			Logger migrationLogger{&out};
+			migrationLogger << "Database is being upgraded, please be patient..." << Qt::endl;
+			QObject::connect(&migrator, &DBMigrator::migrationDone,
+							 [&migrationLogger](uint32_t migration)
+							 { migrationLogger << "Progress: Successfully migrated to: " << migration << Qt::endl; });
+			migrator.performMigrations();
+			migrationLogger << "Database upgraded successfully" << Qt::endl;
+		}
 	}
 }
 
@@ -96,7 +96,7 @@ void Common::setupAppInfo()
 
 QString Common::databasePath()
 {
-	QString env = QProcessEnvironment::systemEnvironment().value("QSS_DB_OVERRIDE");
+	QString env = QProcessEnvironment::systemEnvironment().value("LOOQS_DB_OVERRIDE");
 	if(env == "")
 	{
 		QSettings settings;
@@ -104,3 +104,9 @@ QString Common::databasePath()
 	}
 	return env;
 }
+
+QString Common::ipcSocketPath()
+{
+	QSettings settings;
+	return settings.value(SETTINGS_KEY_IPCSOCKETPATH, "/tmp/looqs-spawner").toString();
+}

shared/common.h

@@ -6,6 +6,7 @@ namespace Common
 {
 void setupAppInfo();
 QString databasePath();
+QString ipcSocketPath();
 bool initSqliteDatabase(QString path);
 void ensureConfigured();
 } // namespace Common

shared/concurrentqueue.cpp

@@ -0,0 +1 @@
+#include "concurrentqueue.h"

shared/concurrentqueue.h

@@ -0,0 +1,66 @@
+#ifndef CONCURRENTQUEUE_H
+#define CONCURRENTQUEUE_H
+#include <QList>
+#include <QMutex>
+#include <QSemaphore>
+#define QUEUE_SIZE 10000
+template <class T> class ConcurrentQueue : protected QList<T>
+{
+  protected:
+	QMutex mutex;
+
+	QSemaphore avail{QUEUE_SIZE};
+
+  public:
+	void enqueue(const T &t)
+	{
+		avail.acquire(1);
+		QMutexLocker locker(&mutex);
+		QList<T>::append(t);
+	}
+
+	QVector<T> dequeue(int batchsize)
+	{
+		avail.release(batchsize);
+		// TODO: this sucks
+		QVector<T> result;
+		QMutexLocker locker(&mutex);
+		for(int i = 0; i < batchsize; i++)
+		{
+			result.append(QList<T>::takeFirst());
+		}
+		return result;
+	}
+
+	void enqueue(const QVector<T> &t)
+	{
+		QList<T> tmp(t.begin(), t.end());
+		avail.acquire(t.size());
+		QMutexLocker locker(&mutex);
+		QList<T>::append(tmp);
+	}
+
+	unsigned int remaining()
+	{
+		return QUEUE_SIZE - avail.available();
+	}
+
+	void clear()
+	{
+		QMutexLocker locker(&mutex);
+		QList<T>::clear();
+		avail.release(QUEUE_SIZE);
+	}
+
+	bool dequeue(T &result)
+	{
+		QMutexLocker locker(&mutex);
+		if(QList<T>::isEmpty())
+			return false;
+		avail.release(1);
+		result = QList<T>::takeFirst();
+		return true;
+	}
+};
+
+#endif // CONCURRENTQUEUE_H

shared/create.qrc

@@ -1,5 +0,0 @@
-<!DOCTYPE RCC><RCC version="1.0">
-<qresource>
-    <file>create.sql</file>
-</qresource>
-</RCC>

shared/databasefactory.cpp

@@ -1,6 +1,7 @@
 #include <QThread>
 #include "databasefactory.h"
 #include "logger.h"
+#include "looqsgeneralexception.h"
 DatabaseFactory::DatabaseFactory(QString connectionString)
 {
 	this->connectionString = connectionString;
@@ -11,7 +12,7 @@ static QThreadStorage<QSqlDatabase> dbStore;
 QSqlDatabase DatabaseFactory::createNew()
 {
 	static int counter = 0;
-	QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE", "QSS" + QString::number(counter++));
+	QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE", "LOOQS" + QString::number(counter++));
 	db.setDatabaseName(this->connectionString);
 	if(!db.open())
 	{
@@ -28,7 +29,7 @@ QSqlDatabase DatabaseFactory::forCurrentThread()
 		return dbStore.localData();
 	}
 	QSqlDatabase db =
-		QSqlDatabase::addDatabase("QSQLITE", "QSS" + QString::number((quint64)QThread::currentThread(), 16));
+		QSqlDatabase::addDatabase("QSQLITE", "LOOQS" + QString::number((quint64)QThread::currentThread(), 16));
 	db.setDatabaseName(this->connectionString);
 	if(!db.open())
 	{

shared/databasefactory.h

@@ -2,7 +2,6 @@
 #define DATABASEFACTORY_H
 #include <QSqlDatabase>
 #include <QThreadStorage>
-#include "utils.h"
 class DatabaseFactory
 {
   private:

shared/dbmigrator.cpp

@@ -0,0 +1,85 @@
+#include <QDirIterator>
+#include <QSqlQuery>
+#include <QSqlError>
+#include <QTextStream>
+#include <QDebug>
+#include "dbmigrator.h"
+#include "looqsgeneralexception.h"
+
+DBMigrator::DBMigrator(QSqlDatabase &db)
+{
+	Q_INIT_RESOURCE(migrations);
+	this->db = &db;
+}
+
+DBMigrator::~DBMigrator()
+{
+	Q_CLEANUP_RESOURCE(migrations);
+}
+
+QStringList DBMigrator::getMigrationFilenames()
+{
+	QStringList result;
+	QDirIterator it(":/looqs-migrations/");
+	while(it.hasNext())
+	{
+		result.append(it.next());
+	}
+	return result;
+}
+
+uint32_t DBMigrator::currentRevision()
+{
+	QSqlQuery dbquery(*db);
+	dbquery.exec("PRAGMA user_version;");
+	if(!dbquery.next())
+	{
+		throw new LooqsGeneralException("Failed to query current db revision");
+	}
+	uint32_t result = dbquery.value(0).toUInt();
+	return result;
+}
+
+bool DBMigrator::migrationNeeded()
+{
+	QStringList migrations = getMigrationFilenames();
+	uint32_t currentRev = currentRevision();
+
+	return currentRev < static_cast<uint32_t>(migrations.size());
+}
+
+void DBMigrator::performMigrations()
+{
+	QStringList migrations = getMigrationFilenames();
+	uint32_t currentRev = currentRevision();
+	uint32_t targetRev = (migrations.size());
+
+	for(uint32_t i = currentRev + 1; i <= targetRev; i++)
+	{
+		QString fileName = QString(":/looqs-migrations/%1.sql").arg(i);
+		QFile file{fileName};
+		if(!file.open(QIODevice::ReadOnly))
+		{
+			throw LooqsGeneralException("Migration: Failed to find required revision file");
+		}
+		QTextStream stream(&file);
+		db->transaction();
+		while(!stream.atEnd())
+		{
+			QString sql = stream.readLine();
+			QSqlQuery sqlQuery{*db};
+			if(!sqlQuery.exec(sql))
+			{
+
+				db->rollback();
+				throw LooqsGeneralException("Failed to execute sql statement while initializing database: " +
+											sqlQuery.lastError().text());
+			}
+		}
+		QSqlQuery updateVersion{*db};
+		updateVersion.exec(QString("PRAGMA user_version=%1;").arg(i));
+		db->commit();
+		emit migrationDone(i);
+	}
+	emit done();
+}

shared/dbmigrator.h

@@ -0,0 +1,24 @@
+#ifndef DBMIGRATOR_H
+#define DBMIGRATOR_H
+#include <QStringList>
+#include <QSqlDatabase>
+#include <QObject>
+class DBMigrator : public QObject
+{
+	Q_OBJECT
+  private:
+	QSqlDatabase *db;
+
+  public:
+	DBMigrator(QSqlDatabase &db);
+	~DBMigrator();
+	uint32_t currentRevision();
+	void performMigrations();
+	QStringList getMigrationFilenames();
+	bool migrationNeeded();
+  signals:
+	void migrationDone(uint32_t);
+	void done();
+};
+
+#endif // DBMIGRATOR_H

shared/dirscanworker.cpp

@@ -0,0 +1,53 @@
+#include <QThread>
+#include "dirscanworker.h"
+#include "logger.h"
+DirScanWorker::DirScanWorker(ConcurrentQueue<QString> &queue, ConcurrentQueue<QString> &resultQueue,
+							 QStringList ignorePattern, unsigned int progressReportThreshold,
+							 std::atomic<bool> &stopToken)
+{
+	this->queue = &queue;
+	this->resultQueue = &resultQueue;
+	this->ignorePattern = ignorePattern;
+	this->progressReportThreshold = progressReportThreshold;
+	this->stopToken = &stopToken;
+	setAutoDelete(false);
+}
+
+void DirScanWorker::run()
+{
+	unsigned int currentProgress = 0;
+	QString path;
+	/* TODO: if we have e. g. only one path, then only one thread will scan this path.
+		 *
+		 * Thus, we must resubmit to the queue directories so other threads can help
+		the current one (requires a new logic for threads in ParallelDirScanner). Alterantively,
+		start new DirScanWorkers ourselves here... */
+	while(queue->dequeue(path))
+	{
+		QDirIterator iterator(path, ignorePattern, QDir::Files, QDirIterator::Subdirectories);
+		while(iterator.hasNext())
+		{
+			this->results.append(iterator.next());
+			++currentProgress;
+			if(currentProgress == progressReportThreshold)
+			{
+				if(this->stopToken->load(std::memory_order_relaxed))
+				{
+					Logger::info() << "Received cancel request" << Qt::endl;
+					this->results.clear();
+					emit finished();
+					return;
+				}
+
+				this->resultQueue->enqueue(this->results);
+				emit progress(results.length());
+				currentProgress = 0;
+				this->results.clear();
+			}
+		}
+	}
+	this->resultQueue->enqueue(this->results);
+	emit progress(results.length());
+	this->results.clear();
+	emit finished();
+}

shared/dirscanworker.h

@@ -0,0 +1,31 @@
+#ifndef DIRSCANWORKER_H
+#define DIRSCANWORKER_H
+#include <QObject>
+#include <QRunnable>
+#include <QDirIterator>
+#include "concurrentqueue.h"
+class DirScanWorker : public QObject, public QRunnable
+{
+	Q_OBJECT
+  protected:
+	unsigned int progressReportThreshold = 1000;
+	ConcurrentQueue<QString> *queue = nullptr;
+	ConcurrentQueue<QString> *resultQueue = nullptr;
+
+	QStringList ignorePattern;
+	QVector<QString> results;
+
+	std::atomic<bool> *stopToken;
+
+  public:
+	DirScanWorker(ConcurrentQueue<QString> &queue, ConcurrentQueue<QString> &resultQueue, QStringList ignorePattern,
+				  unsigned int progressReportThreshold, std::atomic<bool> &stopToken);
+
+	void run() override;
+
+  signals:
+	void progress(unsigned int);
+	void finished();
+};
+
+#endif // DIRSCANWORKER_H

shared/filesaver.cpp

@@ -1,11 +1,11 @@
 #include <QSqlError>
 #include <QDateTime>
 #include <QtConcurrentMap>
+#include <QProcess>
 #include <functional>
 #include "filesaver.h"
 #include "processor.h"
 #include "pdfprocessor.h"
-#include "commandadd.h"
 #include "defaulttextprocessor.h"
 #include "tagstripperprocessor.h"
 #include "nothingprocessor.h"
@@ -13,18 +13,6 @@
 #include "odsprocessor.h"
 #include "utils.h"
 #include "logger.h"
-static DefaultTextProcessor *defaultTextProcessor = new DefaultTextProcessor();
-static TagStripperProcessor *tagStripperProcessor = new TagStripperProcessor();
-static NothingProcessor *nothingProcessor = new NothingProcessor();
-static OdtProcessor *odtProcessor = new OdtProcessor();
-static OdsProcessor *odsProcessor = new OdsProcessor();
-
-static QMap<QString, Processor *> processors{
-	{"pdf", new PdfProcessor()},	{"txt", defaultTextProcessor}, {"md", defaultTextProcessor},
-	{"py", defaultTextProcessor},	{"xml", nothingProcessor},	   {"html", tagStripperProcessor},
-	{"java", defaultTextProcessor}, {"js", defaultTextProcessor},  {"cpp", defaultTextProcessor},
-	{"c", defaultTextProcessor},	{"sql", defaultTextProcessor}, {"odt", odtProcessor},
-	{"ods", odsProcessor}};
 
 FileSaver::FileSaver(SqliteDbService &dbService)
 {
@@ -106,32 +94,53 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
 
 SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
 {
-	Processor *processor = processors.value(fileInfo.suffix(), nothingProcessor);
 	QVector<PageData> pageData;
 	QString absPath = fileInfo.absoluteFilePath();
 
+	int status = -1;
+
+	if(!fileInfo.exists())
+	{
+		return NOTFOUND;
+	}
+
 	if(fileInfo.isFile())
 	{
-		try
+		QProcess process;
+		QStringList args;
+		args << "process" << absPath;
+		process.setProcessChannelMode(QProcess::ForwardedErrorChannel);
+		process.start("/proc/self/exe", args);
+		process.waitForStarted();
+		process.waitForFinished();
+
+		/* TODO: This is suboptimal as it eats lots of mem
+		 * but avoids a weird QDataStream/QProcess behaviour
+		 * where it thinks the process has ended when it has not...
+		 *
+		 * Also, there seem to be issues with reads not being blocked, so
+		 * the only reliable way appears to be waiting until the process
+		 * finishes.
+		 */
+		QDataStream in(process.readAllStandardOutput());
+		while(!in.atEnd())
 		{
-			if(processor->PREFERED_DATA_SOURCE == FILEPATH)
-			{
-				pageData = processor->process(absPath);
-			}
-			else
-			{
-				pageData = processor->process(Utils::readFile(absPath));
-			}
+			PageData pd;
+			in >> pd;
+			pageData.append(pd);
 		}
-		catch(LooqsGeneralException &e)
+		status = process.exitCode();
+		if(status != 0 && status != NOTHING_PROCESSED)
 		{
-			Logger::error() << "Error while processing" << absPath << ":" << e.message << Qt::endl;
+			Logger::error() << "FileSaver::saveFile(): Error while processing" << absPath << ":"
+							<< "Exit code " << status << Qt::endl;
+
 			return PROCESSFAIL;
 		}
 	}
 
 	// Could happen if a file corrupted for example
-	if(pageData.isEmpty() && processor != nothingProcessor)
+	if(pageData.isEmpty() && status != NOTHING_PROCESSED)
 	{
 		Logger::error() << "Could not get any content for " << absPath << Qt::endl;
 	}

shared/filesaver.h

@@ -2,7 +2,6 @@
 #define FILESAVER_H
 #include <QSqlDatabase>
 #include <QFileInfo>
-#include "command.h"
 #include "pagedata.h"
 #include "filedata.h"
 #include "sqlitedbservice.h"
@@ -12,12 +11,10 @@ class FileSaver
   private:
 	SqliteDbService *dbService;
 
-  protected:
-	SaveFileResult addFile(QString path);
-	SaveFileResult updateFile(QString path);
-
   public:
 	FileSaver(SqliteDbService &dbService);
+	SaveFileResult addFile(QString path);
+	SaveFileResult updateFile(QString path);
 	SaveFileResult saveFile(const QFileInfo &fileInfo);
 	int processFiles(const QVector<QString> paths, std::function<SaveFileResult(QString path)> saverFunc,
 					 bool keepGoing, bool verbose);

shared/filescanworker.cpp

@@ -0,0 +1,36 @@
+#include "filescanworker.h"
+#include "logger.h"
+FileScanWorker::FileScanWorker(SqliteDbService &db, ConcurrentQueue<QString> &queue, int batchsize,
+							   std::atomic<bool> &stopToken)
+{
+	this->dbService = &db;
+	this->queue = &queue;
+	this->batchsize = batchsize;
+	this->stopToken = &stopToken;
+}
+
+void FileScanWorker::run()
+{
+	FileSaver saver{*this->dbService};
+	auto paths = queue->dequeue(batchsize);
+	for(QString &path : paths)
+	{
+		SaveFileResult sfr;
+		try
+		{
+			sfr = saver.addFile(path);
+		}
+		catch(std::exception &e)
+		{
+			Logger::error() << e.what();
+			sfr = PROCESSFAIL; // well...
+		}
+		emit result({path, sfr});
+		if(stopToken->load(std::memory_order_relaxed)) // TODO: relaxed should suffice here, but recheck
+		{
+			emit finished();
+			return;
+		}
+	}
+	emit finished();
+}

shared/filescanworker.h

@@ -0,0 +1,29 @@
+#ifndef FILESCANWORKER_H
+#define FILESCANWORKER_H
+#include <QString>
+#include <QObject>
+#include <QtConcurrent>
+#include <utility>
+#include "paralleldirscanner.h"
+#include "filesaver.h"
+
+typedef std::pair<QString, SaveFileResult> FileScanResult;
+
+class FileScanWorker : public QObject, public QRunnable
+{
+	Q_OBJECT
+  protected:
+	SqliteDbService *dbService;
+	ConcurrentQueue<QString> *queue;
+	int batchsize;
+	std::atomic<bool> *stopToken;
+
+  public:
+	FileScanWorker(SqliteDbService &db, ConcurrentQueue<QString> &queue, int batchsize, std::atomic<bool> &stopToken);
+	void run() override;
+  signals:
+	void result(FileScanResult);
+	void finished();
+};
+
+#endif // FILESCANWORKER_H

shared/indexer.cpp

@@ -0,0 +1,132 @@
+#include "indexer.h"
+#include "logger.h"
+
+Indexer::Indexer(SqliteDbService &db)
+{
+	dirScanner = QSharedPointer<ParallelDirScanner>(new ParallelDirScanner());
+	connect(dirScanner.data(), &ParallelDirScanner::scanComplete, this, &Indexer::dirScanFinished);
+	connect(dirScanner.data(), &ParallelDirScanner::progress, this, &Indexer::dirScanProgress);
+	this->db = &db;
+}
+
+void Indexer::beginIndexing()
+{
+	this->runningWorkers = 0;
+	this->currentScanProcessedCount = 0;
+	this->currentIndexResult = IndexResult();
+	this->currentIndexResult.begin = QDateTime::currentDateTime();
+	QVector<QString> dirs;
+
+	for(QString &path : this->pathsToScan)
+	{
+		QFileInfo info{path};
+		if(info.isDir())
+		{
+			dirs.append(path);
+		}
+		else
+		{
+			this->filePathTargetsQueue.enqueue(path);
+		}
+	}
+	this->dirScanner->setPaths(dirs);
+	this->dirScanner->setIgnorePatterns(this->ignorePattern);
+
+	this->dirScanner->scan();
+
+	this->workerCancellationToken.store(false, std::memory_order_seq_cst);
+	launchWorker(this->filePathTargetsQueue, this->filePathTargetsQueue.remaining());
+}
+
+void Indexer::setIgnorePattern(QStringList ignorePattern)
+{
+	this->ignorePattern = ignorePattern;
+}
+
+void Indexer::setTargetPaths(QVector<QString> pathsToScan)
+{
+	this->pathsToScan = pathsToScan;
+}
+
+void Indexer::requestCancellation()
+{
+	this->dirScanner->cancel();
+	this->workerCancellationToken.store(true, std::memory_order_release);
+}
+
+IndexResult Indexer::getResult()
+{
+	return this->currentIndexResult;
+}
+
+void Indexer::dirScanFinished()
+{
+	Logger::info() << "Dir scan finished";
+	if(!isRunning())
+	{
+		emit finished();
+	}
+}
+
+void Indexer::launchWorker(ConcurrentQueue<QString> &queue, int batchsize)
+{
+	FileScanWorker *runnable = new FileScanWorker(*this->db, queue, batchsize, this->workerCancellationToken);
+	connect(runnable, &FileScanWorker::result, this, &Indexer::processFileScanResult);
+	connect(runnable, &FileScanWorker::finished, this, &Indexer::processFinishedWorker);
+	++this->runningWorkers;
+	QThreadPool::globalInstance()->start(runnable);
+}
+
+void Indexer::dirScanProgress(int current, int total)
+{
+	launchWorker(this->dirScanner->getResults(), current);
+	emit pathsCountChanged(total);
+}
+
+void Indexer::processFileScanResult(FileScanResult result)
+{
+	if(verbose)
+	{
+		this->currentIndexResult.results.append(result);
+	}
+	else
+	{
+		if(result.second == DBFAIL || result.second == PROCESSFAIL || result.second == NOTFOUND)
+		{
+			this->currentIndexResult.results.append(result);
+		}
+	}
+	if(result.second == OK)
+	{
+		++this->currentIndexResult.addedPaths;
+	}
+	else if(result.second == SKIPPED)
+	{
+		++this->currentIndexResult.skippedPaths;
+	}
+	else
+	{
+		++this->currentIndexResult.erroredPaths;
+	}
+
+	if(currentScanProcessedCount++ == progressReportThreshold)
+	{
+		emit indexProgress(this->currentIndexResult.total(), this->currentIndexResult.addedPaths,
+						   this->currentIndexResult.skippedPaths, this->currentIndexResult.erroredPaths,
+						   this->dirScanner->pathCount());
+		currentScanProcessedCount = 0;
+	}
+}
+
+bool Indexer::isRunning()
+{
+	return this->runningWorkers > 0 || this->dirScanner->isRunning();
+}
+void Indexer::processFinishedWorker()
+{
+	--this->runningWorkers;
+	if(!isRunning())
+	{
+		emit finished();
+	}
+}

shared/indexer.h

@@ -0,0 +1,90 @@
+#ifndef INDEXER_H
+#define INDEXER_H
+#include <QVector>
+#include <QObject>
+#include "sqlitedbservice.h"
+#include "paralleldirscanner.h"
+#include "filescanworker.h"
+
+class IndexResult
+{
+  public:
+	QDateTime begin;
+	QDateTime end;
+	QVector<FileScanResult> results;
+
+	unsigned int addedPaths = 0;
+	unsigned int skippedPaths = 0;
+	unsigned int erroredPaths = 0;
+
+	unsigned int total()
+	{
+		return addedPaths + skippedPaths + erroredPaths;
+	}
+
+	QVector<QString> failedPaths() const
+	{
+		QVector<QString> result;
+		std::for_each(results.begin(), results.end(),
+					  [&result](FileScanResult res)
+					  {
+						  if(res.second == DBFAIL || res.second == PROCESSFAIL || res.second == NOTFOUND)
+						  {
+							  result.append(res.first);
+						  }
+					  });
+		return result;
+	}
+};
+
+class Indexer : public QObject
+{
+	Q_OBJECT
+  protected:
+	bool verbose = false;
+	bool keepGoing = true;
+	SqliteDbService *db;
+
+	int progressReportThreshold = 50;
+	int currentScanProcessedCount = 0;
+	int runningWorkers = 0;
+
+	QVector<QString> pathsToScan;
+	QSharedPointer<ParallelDirScanner> dirScanner;
+
+	QStringList ignorePattern;
+
+	/* Those path pointing to files not directories */
+	ConcurrentQueue<QString> filePathTargetsQueue;
+
+	std::atomic<bool> workerCancellationToken;
+	IndexResult currentIndexResult;
+	void launchWorker(ConcurrentQueue<QString> &queue, int batchsize);
+
+  public:
+	bool isRunning();
+
+	void beginIndexing();
+	void setIgnorePattern(QStringList ignorePattern);
+	void setTargetPaths(QVector<QString> pathsToScan);
+
+	void requestCancellation();
+
+	Indexer(SqliteDbService &db);
+	IndexResult getResult();
+
+  public slots:
+	void dirScanFinished();
+	void dirScanProgress(int current, int total);
+	void processFileScanResult(FileScanResult result);
+	void processFinishedWorker();
+
+  signals:
+	void pathsCountChanged(int total);
+	void fileScanResult(FileScanResult *result);
+	void indexProgress(unsigned int processedFiles, unsigned int added, unsigned int skipped, unsigned int failed,
+					   unsigned int totalPaths);
+	void finished();
+};
+
+#endif // INDEXER_H

shared/looqsquery.cpp

@@ -23,6 +23,16 @@ QueryType LooqsQuery::getQueryType()
 	return static_cast<QueryType>(tokensMask & COMBINED);
 }
 
+bool LooqsQuery::hasContentSearch()
+{
+	return (this->getTokensMask() & FILTER_CONTENT) == FILTER_CONTENT;
+}
+
+bool LooqsQuery::hasPathSearch()
+{
+	return (this->getTokensMask() & FILTER_PATH) == FILTER_PATH;
+}
+
 void LooqsQuery::addSortCondition(SortCondition sc)
 {
 	this->sortConditions.append(sc);
@@ -128,7 +138,7 @@ QVector<SortCondition> createSortConditions(QString sortExpression)
 			}
 			else
 			{
-				throw LooqsGeneralException("Unknown order specifier: " + order);
+				throw LooqsGeneralException("Unknown order specifier: " + orderstr);
 			}
 		}
 		else
@@ -157,15 +167,15 @@ void LooqsQuery::addToken(Token t)
  * thus, "Downloads zip" becomes essentailly "path.contains:(Downloads) AND path.contains:(zip)"
  *
  * TODO: It's a bit ugly still*/
-LooqsQuery LooqsQuery::build(QString expression)
+LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, bool mergeLoneWords)
 {
 	if(!checkParanthesis(expression))
 	{
 		throw LooqsGeneralException("Invalid paranthesis");
 	}
 
+	QStringList loneWords;
 	LooqsQuery result;
-	// TODO: merge lonewords
 	QRegularExpression rx("((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([\\w,])+)|(?<boolean>AND|OR)"
 						  "|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>\\w+))");
 	QRegularExpressionMatchIterator i = rx.globalMatch(expression);
@@ -233,7 +243,14 @@ LooqsQuery LooqsQuery::build(QString expression)
 
 		if(loneword != "")
 		{
-			result.addToken(Token(FILTER_PATH_CONTAINS, loneword));
+			if(mergeLoneWords)
+			{
+				loneWords.append(loneword);
+			}
+			else
+			{
+				result.addToken(Token(loneWordsTokenType, loneword));
+			}
 		}
 
 		if(filtername != "")
@@ -244,6 +261,10 @@ LooqsQuery LooqsQuery::build(QString expression)
 			{
 				value = m.captured("args");
 			}
+			if(value == "")
+			{
+				throw LooqsGeneralException("value cannot be empty for filters");
+			}
 
 			if(filtername == "path.contains")
 			{
@@ -292,7 +313,16 @@ LooqsQuery LooqsQuery::build(QString expression)
 		}
 	}
 
-	bool contentsearch = (result.getTokensMask() & FILTER_CONTENT) == FILTER_CONTENT;
+	if(mergeLoneWords)
+	{
+		QString mergedLoneWords = loneWords.join(' ');
+		if(!mergedLoneWords.isEmpty())
+		{
+			result.addToken(Token(loneWordsTokenType, mergedLoneWords));
+		}
+	}
+
+	bool contentsearch = result.hasContentSearch();
 	bool sortsForContent = std::any_of(result.sortConditions.begin(), result.sortConditions.end(),
 									   [](SortCondition c) { return c.field == CONTENT_TEXT; });
 

shared/looqsquery.h

@@ -52,9 +52,12 @@ class LooqsQuery
 	{
 		return tokensMask;
 	}
+	bool hasContentSearch();
+	bool hasPathSearch();
+
 	void addSortCondition(SortCondition sc);
 	static bool checkParanthesis(QString query);
-	static LooqsQuery build(QString query);
+	static LooqsQuery build(QString query, TokenType loneWordsTokenType, bool mergeLoneWords);
 };
 
 #endif // LOOQSQUERY_H

shared/migrations/migrations.qrc

@@ -0,0 +1,5 @@
+<RCC>
+    <qresource prefix="/looqs-migrations">
+        <file>1.sql</file>
+    </qresource>
+</RCC>

shared/pagedata.cpp

@@ -0,0 +1,13 @@
+#include "pagedata.h"
+
+QDataStream &operator<<(QDataStream &out, const PageData &pd)
+{
+	out << pd.pagenumber << pd.content;
+	return out;
+}
+
+QDataStream &operator>>(QDataStream &in, PageData &pd)
+{
+	in >> pd.pagenumber >> pd.content;
+	return in;
+}

shared/pagedata.h

@@ -1,6 +1,9 @@
 #ifndef PAGEDATA_H
 #define PAGEDATA_H
 #include <QString>
+#include <QMetaType>
+#include <QDataStream>
+
 class PageData
 {
   public:
@@ -10,10 +13,17 @@ class PageData
 	PageData()
 	{
 	}
+
 	PageData(unsigned int pagenumber, QString content)
 	{
 		this->pagenumber = pagenumber;
 		this->content = content;
 	}
 };
+
+Q_DECLARE_METATYPE(PageData);
+
+QDataStream &operator<<(QDataStream &out, const PageData &pd);
+QDataStream &operator>>(QDataStream &in, PageData &pd);
+
 #endif // PAGEDATA_H

shared/paralleldirscanner.cpp

@@ -0,0 +1,104 @@
+#include "paralleldirscanner.h"
+
+#include <QRunnable>
+#include <QMutex>
+#include <QDirIterator>
+#include <QThread>
+#include <QThreadPool>
+#include <functional>
+#include "dirscanworker.h"
+#include "logger.h"
+
+ParallelDirScanner::ParallelDirScanner()
+{
+	this->threadpool.setMaxThreadCount(QThread::idealThreadCount() / 2);
+}
+
+ConcurrentQueue<QString> &ParallelDirScanner::getResults()
+{
+	return this->resultPathsQueue;
+}
+
+void ParallelDirScanner::setIgnorePatterns(QStringList patterns)
+{
+	this->ignorePatterns = patterns;
+}
+
+void ParallelDirScanner::setPaths(QVector<QString> paths)
+{
+	this->paths = paths;
+}
+
+void ParallelDirScanner::cancel()
+{
+	this->stopToken.store(true, std::memory_order_seq_cst);
+}
+
+void ParallelDirScanner::handleWorkersProgress(unsigned int progress)
+{
+	this->processedPaths += progress;
+	if(!this->stopToken.load(std::memory_order_seq_cst))
+		emit this->progress(progress, this->processedPaths);
+}
+
+void ParallelDirScanner::handleWorkersFinish()
+{
+	Logger::info() << "Worker finished";
+	// no mutexes required due to queued connection
+	++finishedWorkers;
+	if(this->stopToken.load(std::memory_order_seq_cst) || finishedWorkers == getThreadsNum())
+	{
+		running = false;
+		emit scanComplete();
+	}
+}
+
+unsigned int ParallelDirScanner::getThreadsNum() const
+{
+	int threadsNum = this->threadpool.maxThreadCount();
+	if(threadsNum > this->paths.size())
+	{
+		threadsNum = this->paths.size();
+	}
+	return threadsNum;
+}
+
+void ParallelDirScanner::scan()
+{
+	Logger::info() << "I am scanning";
+	this->stopToken.store(false, std::memory_order_relaxed);
+	this->finishedWorkers = 0;
+	this->processedPaths = 0;
+	this->targetPathsQueue.clear();
+	this->resultPathsQueue.clear();
+
+	this->targetPathsQueue.enqueue(this->paths);
+	int threadsNum = getThreadsNum();
+	if(threadsNum == 0)
+	{
+		emit scanComplete();
+		return;
+	}
+	running = true;
+	for(int i = 0; i < threadsNum; i++)
+	{
+		DirScanWorker *runnable = new DirScanWorker(this->targetPathsQueue, this->resultPathsQueue,
+													this->ignorePatterns, 1000, this->stopToken);
+		runnable->setAutoDelete(false);
+		connect(runnable, &DirScanWorker::progress, this, &ParallelDirScanner::handleWorkersProgress,
+				Qt::QueuedConnection);
+		connect(runnable, &DirScanWorker::finished, this, &ParallelDirScanner::handleWorkersFinish,
+				Qt::QueuedConnection);
+		threadpool.start(runnable);
+	}
+}
+
+bool ParallelDirScanner::isRunning()
+{
+	return this->running;
+}
+
+unsigned int ParallelDirScanner::pathCount()
+{
+	return this->processedPaths;
+}

shared/paralleldirscanner.h

@@ -0,0 +1,48 @@
+#ifndef PARALLELDIRSCANNER_H
+#define PARALLELDIRSCANNER_H
+
+#include <QObject>
+#include <QMutex>
+#include <atomic>
+#include <QThreadPool>
+#include "concurrentqueue.h"
+class ParallelDirScanner : public QObject
+{
+	Q_OBJECT
+  protected:
+	QStringList ignorePatterns;
+	QThreadPool threadpool;
+
+	unsigned int finishedWorkers = 0;
+	unsigned int processedPaths = 0;
+
+	std::atomic<bool> stopToken;
+
+	bool running = false;
+
+	QVector<QString> paths;
+	ConcurrentQueue<QString> targetPathsQueue;
+	ConcurrentQueue<QString> resultPathsQueue;
+	unsigned int getThreadsNum() const;
+
+  public:
+	ParallelDirScanner();
+
+	ConcurrentQueue<QString> &getResults();
+	void setIgnorePatterns(QStringList patterns);
+	void setPaths(QVector<QString> paths);
+	void scan();
+	bool isRunning();
+
+	unsigned int pathCount();
+
+  signals:
+	void scanComplete();
+	void progress(int, int);
+  public slots:
+	void cancel();
+	void handleWorkersProgress(unsigned int progress);
+	void handleWorkersFinish();
+};
+
+#endif // PARALLELDIRSCANNER_H

shared/processor.h

@@ -10,6 +10,8 @@ enum DataSource
 	ARRAY
 };
 
+#define NOTHING_PROCESSED 4
+
 class Processor
 {
   public:

shared/sandboxedprocessor.cpp

@@ -0,0 +1,111 @@
+#include <QFile>
+#include <QFileInfo>
+#include <QDataStream>
+#include "sandboxedprocessor.h"
+#include "pdfprocessor.h"
+#include "defaulttextprocessor.h"
+#include "tagstripperprocessor.h"
+#include "nothingprocessor.h"
+#include "odtprocessor.h"
+#include "odsprocessor.h"
+#include "../submodules/exile.h/exile.h"
+#include "logger.h"
+
+static DefaultTextProcessor *defaultTextProcessor = new DefaultTextProcessor();
+static TagStripperProcessor *tagStripperProcessor = new TagStripperProcessor();
+static NothingProcessor *nothingProcessor = new NothingProcessor();
+static OdtProcessor *odtProcessor = new OdtProcessor();
+static OdsProcessor *odsProcessor = new OdsProcessor();
+
+static QMap<QString, Processor *> processors{
+	{"pdf", new PdfProcessor()},	{"txt", defaultTextProcessor}, {"md", defaultTextProcessor},
+	{"py", defaultTextProcessor},	{"xml", nothingProcessor},	   {"html", tagStripperProcessor},
+	{"java", defaultTextProcessor}, {"js", defaultTextProcessor},  {"cpp", defaultTextProcessor},
+	{"c", defaultTextProcessor},	{"sql", defaultTextProcessor}, {"odt", odtProcessor},
+	{"ods", odsProcessor}};
+
+void SandboxedProcessor::enableSandbox(QString readablePath)
+{
+	struct exile_policy *policy = exile_init_policy();
+	if(policy == NULL)
+	{
+		qCritical() << "Could not init exile";
+		exit(EXIT_FAILURE);
+	}
+	policy->namespace_options = EXILE_UNSHARE_NETWORK | EXILE_UNSHARE_USER;
+
+	if(!readablePath.isEmpty())
+	{
+		std::string readablePathLocation = readablePath.toStdString();
+		if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ, readablePathLocation.c_str()) != 0)
+		{
+			qCritical() << "Failed to add path policies";
+			exit(EXIT_FAILURE);
+		}
+	}
+	else
+	{
+		policy->no_fs = 1;
+	}
+	int ret = exile_enable_policy(policy);
+	if(ret != 0)
+	{
+		qDebug() << "Failed to establish sandbox: " << ret;
+		exit(EXIT_FAILURE);
+	}
+	exile_free_policy(policy);
+}
+
+void SandboxedProcessor::printResults(const QVector<PageData> &pageData)
+{
+	QFile fsstdout;
+	fsstdout.open(stdout, QIODevice::WriteOnly);
+	QDataStream stream(&fsstdout);
+
+	for(const PageData &data : pageData)
+	{
+		stream << data;
+		// fsstdout.flush();
+	}
+
+	fsstdout.close();
+}
+
+int SandboxedProcessor::process()
+{
+	QFileInfo fileInfo(this->filePath);
+	Processor *processor = processors.value(fileInfo.suffix(), nothingProcessor);
+
+	if(processor == nothingProcessor)
+	{
+		/* Nothing to do */
+		return NOTHING_PROCESSED;
+	}
+
+	QVector<PageData> pageData;
+	QString absPath = fileInfo.absoluteFilePath();
+
+	try
+	{
+		if(processor->PREFERED_DATA_SOURCE == FILEPATH)
+		{
+			/* Read access to FS needed... doh..*/
+			enableSandbox(absPath);
+			pageData = processor->process(absPath);
+		}
+		else
+		{
+			QByteArray data = Utils::readFile(absPath);
+			enableSandbox();
+			pageData = processor->process(data);
+		}
+	}
+	catch(LooqsGeneralException &e)
+	{
+		Logger::error() << "SandboxedProcessor: Error while processing" << absPath << ":" << e.message << Qt::endl;
+		return 3 /* PROCESSFAIL */;
+	}
+
+	printResults(pageData);
+	return 0;
+}

shared/sandboxedprocessor.h

@@ -0,0 +1,23 @@
+#ifndef SANDBOXEDPROCESSOR_H
+#define SANDBOXEDPROCESSOR_H
+#include <QString>
+#include "pagedata.h"
+
+class SandboxedProcessor
+{
+  private:
+	QString filePath;
+
+	void enableSandbox(QString readablePath = "");
+	void printResults(const QVector<PageData> &pageData);
+
+  public:
+	SandboxedProcessor(QString filepath)
+	{
+		this->filePath = filepath;
+	}
+
+	int process();
+};
+
+#endif // SANDBOXEDPROCESSOR_H

shared/shared.pro

@@ -4,15 +4,18 @@
 #
 #-------------------------------------------------
 
-QT       += sql
 
 QT       -= gui
+QT       += sql concurrent
 
 TARGET = shared
 TEMPLATE = lib
 CONFIG += staticlib
 CONFIG += c++17
 
+INCLUDEPATH += $$PWD/../sandbox/exile.h/
+INCLUDEPATH += /usr/include/poppler/qt5/ /usr/include/quazip5
+
 # The following define makes your compiler emit warnings if you use
 # any feature of Qt which has been marked as deprecated (the exact warnings
 # depend on your compiler). Please consult the documentation of the
@@ -25,19 +28,62 @@ DEFINES += QT_DEPRECATED_WARNINGS
 #DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0
 
 SOURCES += sqlitesearch.cpp \
+    concurrentqueue.cpp \
+    databasefactory.cpp \
+    dbmigrator.cpp \
+    defaulttextprocessor.cpp \
+    dirscanworker.cpp \
+    encodingdetector.cpp \
+    filesaver.cpp \
+    filescanworker.cpp \
+    indexer.cpp \
+    logger.cpp \
     looqsgeneralexception.cpp \
     common.cpp \
-    looqsquery.cpp
+    looqsquery.cpp \
+    nothingprocessor.cpp \
+    odsprocessor.cpp \
+    odtprocessor.cpp \
+    pagedata.cpp \
+    paralleldirscanner.cpp \
+    pdfprocessor.cpp \
+    processor.cpp \
+    sandboxedprocessor.cpp \
+    sqlitedbservice.cpp \
+    tagstripperprocessor.cpp \
+    utils.cpp \
+    ../submodules/exile.h/exile.c
 
 HEADERS += sqlitesearch.h \
+    concurrentqueue.h \
+    databasefactory.h \
+    dbmigrator.h \
+    defaulttextprocessor.h \
+    dirscanworker.h \
+    encodingdetector.h \
     filedata.h \
+    filesaver.h \
+    filescanworker.h \
+    indexer.h \
+    logger.h \
     looqsgeneralexception.h \
     looqsquery.h \
+    nothingprocessor.h \
+    odsprocessor.h \
+    odtprocessor.h \
+    pagedata.h \
+    paralleldirscanner.h \
+    pdfprocessor.h \
+    processor.h \
+    sandboxedprocessor.h \
     searchresult.h \
+    sqlitedbservice.h \
+    tagstripperprocessor.h \
     token.h \
-    common.h
+    common.h \
+    utils.h
 unix {
     target.path = /usr/lib
     INSTALLS += target
 }
-RESOURCES = create.qrc
+RESOURCES = migrations/migrations.qrc

shared/sqlitedbservice.cpp

@@ -7,7 +7,8 @@
 #include "logger.h"
 bool SqliteDbService::fileExistsInDatabase(QString path, qint64 mtime)
 {
-	auto query = QSqlQuery("SELECT 1 FROM file WHERE path = ? and mtime = ?", dbFactory->forCurrentThread());
+	auto query = QSqlQuery(dbFactory->forCurrentThread());
+	query.prepare("SELECT 1 FROM file WHERE path = ? and mtime = ?");
 	query.addBindValue(path);
 	query.addBindValue(mtime);
 	if(!query.exec())

shared/sqlitedbservice.h

@@ -12,7 +12,8 @@ enum SaveFileResult
 	OK,
 	SKIPPED,
 	DBFAIL,
-	PROCESSFAIL
+	PROCESSFAIL,
+	NOTFOUND
 };
 
 class SqliteDbService

shared/sqlitesearch.cpp

@@ -133,11 +133,17 @@ QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
 		throw LooqsGeneralException("Nothing to search for supplied");
 	}
 
-	for(const Token &token : query.getTokens())
+	bool ftsAlreadyJoined = false;
+	auto tokens = query.getTokens();
+	for(const Token &token : tokens)
 	{
 		if(token.type == FILTER_CONTENT_CONTAINS)
 		{
-			joinSql += " INNER JOIN content_fts ON content.id = content_fts.ROWID ";
+			if(!ftsAlreadyJoined)
+			{
+				joinSql += " INNER JOIN content_fts ON content.id = content_fts.ROWID ";
+				ftsAlreadyJoined = true;
+			}
 			whereSql += " content_fts.content MATCH ? ";
 			bindValues.append(token.value);
 		}
@@ -155,7 +161,11 @@ QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
 	{
 		if(sortSql.isEmpty())
 		{
-			sortSql = "ORDER BY rank";
+			if(std::find_if(tokens.begin(), tokens.end(),
+							[](const Token &t) -> bool { return t.type == FILTER_CONTENT_CONTAINS; }) != tokens.end())
+			{
+				sortSql = "ORDER BY rank";
+			}
 		}
 		prepSql =
 			"SELECT file.path AS path, group_concat(content.page) AS pages, file.mtime AS mtime, file.size AS size, "

submodules/exile.h

@@ -0,0 +1 @@
+Subproject commit ea66ef76ebb88a43ac25c9a86f8fcab8efa130b2