Compare commits

..

No commits in common. "9e3d3b1dd434fd9363c67b207ebcc4acd61e2e56" and "483ea04638c582e010067a9b26866031cee2646a" have entirely different histories.

46 mengubah file dengan 240 tambahan dan 1072 penghapusan

Melihat File

@ -1,29 +0,0 @@
# looqs: Release notes
## 2022-06-XX - v0.1
The first release comes with basic functionality. It's a start that can be considered useful to some degree.
looqs is still at an early stage and may exhibit some weirdness and contain bugs.
Tested architectures: amd64.
CHANGES:
- CLI command "looqs" to add/update/delete and search
- GUI: "looqs-gui" to search, render previews, and add files to index
- General: Add multi-threaded indexing of all files (paths, mtime)
- General: Generate sqlite based full-text search index for: .pdf,.odt,.ods, text files
- General: Sandboxed content processing
- GUI: Sandboxed IPC sub-process to render previews.
- GUI: Add previews for pdf: Render the page the search keywords were found. Highlight the keywords when rendering the page.
- GUI: Add previews for plaintext files: Extract snippets. Highlight the keywords when rendering the page.
- General: Add basic filters for query.
- GUI: Add icon. Special thanks to the following sources:
https://www.svgrepo.com/svg/151751/magnifier-with-small-handle
https://www.svgrepo.com/svg/52764/open-book
Thanks!
- General: Documentation.
- Add packages: Ubuntu 21.10, 22.04
Thanks to all those who provided feedback (and endured bugs) at various stages. You know who you are, thx!

Melihat File

@ -1,26 +0,0 @@
# looqs - Contributing
Contributions are welcome, please use the following guidelines.
## Repository
The github repository https://github.com/quitesimpleorg/looqs is the primary one. Pull-Requests and Issues go there, but you can also submit your feedback there.
Those who prefer a more classical approach can mail their patches etc. to looqs at quitesimple org. Ideally git-format patch and git send-email.
The repository at https://gitea.quitesimple.org/crtxcr/looqs was supposed to be the main one, but these
plans are on hold for the time being. Just ignore it.
## Pull-Requests & Rebasing
Your merge requests should be submitted against the dev branch. master branch won't be rebased. I'll try to avoid in the dev branch. I definitly rebase WIP/feature branches.
## Commit messages
Commit messages begin with the component your change affects, e. g. "gui:", "cli:", "shared:", followed by the class.
Then choose an appropriate verb in present tense. Wrong: Fixed, Fixes. Correct: Fix. Make sure lines are not too long,
I personally go by gut feeling in this matter.
If your change has an issue, link it at the end: Closes: https://github.com/quitesimpleorg/looqs/issues/1
Example: "shared: Indexer: Use WildcardMatcher to ignore paths"
## License
You license your changes under the GPLv3 by contributing.

Melihat File

@ -1,31 +0,0 @@
# looqs - Hacking
## Introduction
Without elaborating here, I hacked looqs because I was not satisfied with the state of desktop search on Linux.
Originally a set of CLI python scripts, it is now written in C++ and offers a GUI made using Qt. While a "web app" would have been an option, I prefer a desktop application for something like looqs. I chose Qt because I am more familiar with it than with any other GUI framework. To my knowledge, potential alternatives like GTK do not include as many "batteries" as Qt anyway, so the job presumably would have been harder there.
[CONTRIBUTING.md](CONTRIBUTING.md] contains the instructions on how to submit patches etc.
## Security
The architecture ensures that the parsing of documents and the preview generation is sandboxed by [exile.h](https://github.com/quitesimpleorg/exile.h). looqs uses a multi-process architecture to achieve this.
Qt code is considered trusted in this model. While one may criticize this, it was the only practical solution. looqs uses its serialization mechanism and other classes to communicate between the non-sandboxed GUI process and the sandboxed processes.
## Database
The heart is sqlite, with the FTS5 extensions behind the full-text search. I definitely did not
want to run some heavy Java based solutions. I explored other options like Postgresql, I've discard them due to some limitations back then.
Down the road, alternatives will be explored of course if sqlite should not suffice anymore.
## File format support
The pdf library is libpoppler. Files such as .odt or .docx documents are opened with libquazip. The XML files in there are not parsed,
looqs simply strips the tags and that seems to work fine so far. Naturally, this is not the "proper way", so there is room for improvement maybe here. However, those file formats are not a huge priority for me personally. libuchardet does encoding detection and conversion.
Naturally looqs won't be able to index and render previews for everything. Such approach would create a huge bloated binary. In the future, there will be some plugin system of some sorts, either we will load .so objects or use subprocesses.
## Name
looqs looks for files. You as the user can also look inside them. The 'k' was replaced by a 'q'. Originally wanted my projects to have "qs" (for quitesimple) in their name. While abandoned now, this got us to looqs.

Melihat File

@ -1,75 +1,46 @@
# looqs - Full-text search with previews for your files
looqs is a tool that creates a full-text search index for your files. It allows you to look at previews where your
# looqs - FTS for the Linux desktop with previews for search results
looqs creates a full text search index for your files. It allows you to look at previews where your
search terms have been found, as shown in the screenshots below.
## Screenshots
The screenshots in this section may occasionally be slightly outdated, but they are usually recent enough to get an overall impression of the current state of the GUI.
The screenshots in this section may occasionally be slightly outdated, but they are usually recent enough to get an overall impression of the current state.
### List
![Screenshot looqs results](https://garage.quitesimple.org/assets/looqs/opearting_systems_looqs.png)
### Preview
looqs allow you to look inside files. It marks what you have searched for.
![Screenshot looqs](https://garage.quitesimple.org/assets/looqs/orwell.png)
![Screenshot looqs search fstream](https://garage.quitesimple.org/assets/looqs/fstream_write.png)
### Results list
#### Classic results list
Just enter what you want to find, it will search paths and file content.
![Screenshot looqs results](https://garage.quitesimple.org/assets/looqs/looqs_diary.png)
#### Searching with filters
You can be more specific to get what you want with filters
**Filters (long form)**
![Screenshot looqs results](https://garage.quitesimple.org/assets/looqs/opearting_systems_looqs.png)
**Filters (short form)**
There is no need to write the long form of filters. There are also booleans available
![Screenshot looqs results](https://garage.quitesimple.org/assets/looqs/looqs_beatles_marley.png)
## Current status
Last version: 2022-06-0X, v0.1
Last version: 2022-0X-XX, v0.1
Please see [Changelog](CHANGELOG.md) for a human readable list of changes.
Please see [Changelog](CHANGELOG.md) for a human readable list of changes.
## Goals and principles
* **Find & Preview**. Instead of merely telling you where your search phrase has been found, it should also render the corresponding portion/pages of the documents and highlight the searched words.
* **No daemons**. As some other desktop search projects are prone to have annoying daemons running that eat system resources away, this solution should make do without daemons where possible.
* **Easy setup**. Similarly, there should be no need for heavy-weight databases. Instead, looqs tries to squeeze out the most from simple approaches. In particular, it relies on sqlite.
* **Easy setup**. Similiarly, there should be no need for heavy-weight databases. Instead, this solution tries to squeeze out the most from simple approaches. In particular, it relies on sqlite.
* **GUI & CLI**. Provide CLI interfaces and GUI interfaces
* **Sandboxing**. As reading and rendering lots of formats naturally opens the door for security bugs, those tasks are offloaded to small, sandboxed sub-processes to mitigate the effect of exploited vulnerabilities.
## Features
- GUI, CLI interface
- Indexing of file path and some metadata.
- Indexing of file file content for FTS search. Currently: .pdf, odt, docx, plaintext.
- Preview of file formats: Currently: .pdf, plaintext
- Highlight searched terms.
- Quickly open PDF viewer or text editor at location of preview
- Search filters
## Supported platforms
Linux (on amd64) is currently the main focus. Currently, I don't plan on supporting anything else and the sandboxing architecture does not make it likely. I suppose a version without sandboxing might be conceivable for other platforms, but I have no plans or resources to actively target anything but Linux at this point.
### Licence
GPLv3.
GPLv3.
### Contributing
Please see the [Contribution guidelines](CONTRIBUTING.md) file.
## Documentation
Please see [USAGE.md](USAGE.md) for the user manual. There is also [HACKING.md](HACKING.md) with more technical information.
Fow now, github issues and pull-requests are preferred, but you can also just email
your patches or issues to : looqs at quitesimple.org
## Build
### Ubuntu 21.10/22.04
To build on Ubuntu and Debian, clone the repo and then run:
```
git submodule init
git submodule update
@ -78,24 +49,10 @@ qmake
make
```
The GUI is located in `gui/looqs-gui`, the binary for the CLI is in `cli/looqs`
## Documentation
Please see [Usage.md](USAGE.md) for the user manual.
## Packages
At this point, looqs is not in any official distro package repo, but I maintain some packages.
### Ubuntu 21.10/22.04
Latest release can be installed using apt from the repo.
```
# First, obtain key, assume it's trusted.
wget -O- https://repo.quitesimple.org/repo.quitesimple.org.asc | gpg --dearmor > repo.quitesimple.org-keyring.gpg
cat repo.quitesimple.org-keyring.gpg | sudo tee -a /usr/share/keyrings/repo.quitesimple.org.gpg > /dev/null
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/repo.quitesimple.org.gpg] https://repo.quitesimple.org/debian/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/quitesimple.list
sudo apt-get update
sudo apt-get install looqs
```
### Other distros
I'll probably add a package for voidlinux at some point and maybe will provide a Gentoo ebuild. However, I would appreciate help for others distros. If you create a package, let me know!
As for distro-agnostic packages, I will also take a look into appimage / flatpak etc. and/or maybe just provide a self-contained archive.
Coming soon™

176
USAGE.md
Melihat File

@ -1,176 +0,0 @@
# looqs - User guide
This document is still work in progress.
## General points
Please consult the [README](README.md) for a description of what looqs is and on how to obtain it.
looqs is still at an early stage and may exhibit some weirdness and contain bugs.
## Current Limitations and things to know
You should be aware of the following:
- It may seem natural, but the GUI and CLI operate on the same database, so if you add files using the CLI, the GUI will search them too.
- If a file is listed in the "Search results" tab, it does not imply that a preview will be available in the "Previews" tab, as looqs can search more file formats than it can generate previews for currently.
- Database paths are stored inefficiently, not deduplicated to simplify queries. This may add up quickly. Also, each PDF text is stored twice. Each page separately + the whole document to simplify queries.
To give you some idea: At the time this section was written, 167874 files were in my index. A FTS index was built for 14280 of those, of which 4146 were PDF documents. The PDFs take around 10GB storage space on the filesystem. All files for which an FTS has been built are around 7GB in size on the filesystem. The looqs database had a size of 1.6 GB.
- Existing files are considered modified when the mtime has changed. looqs currently does not check whether the content
has changed.
## Config
The config file is in `$HOME/.config/quitesimple.org/looqs.conf`. It will be created on first execution of the CLI or GUI
interface. The GUI has a menu entry to quickly open this config file. This is to be considered temporary and will be removed once the GUI itself can edit all settings.
Database default path: `$HOME/.local/share/quitesimple.org/looqs/looqs.sqlite`. If this does not work for
you, move it and adjust adjust the path in the config file.
## GUI
The GUI is minimal at this point. Depending on what you want to do, you may need to open the config file and change the settings there. Chances are that you may not need to do that.
### First run
You will be presented with an empty list. Go to the **"Index"** tab, add some directories and click **"Start indexing"**.
For large directories the progress bar is essentially just decoration. As long as you see the counters
increase, everything is fine even if it seems the progress bar is stuck.
The indexing can be stopped. If you run it again you do not start from scratch, because looqs knows
which files have been modified or not since they have been added to the index. Thus, files will
only be reprocedded when necessary. Note that cancellation itself may take a moment as files finish processing.
### Search
The text field at the top is where you type your query. It can be selected quickly using **CTRL + L**. Filters are avalable,
see this document at the end. By default, both the full path and the content are searched. Path names take precedence.
### Configuring PDF viewer
It's most convenient if, when you click on a preview, the PDF reader opens the page you clicked. For that, looqs needs to know which viewer you want to launch.
It tries to auto detect some common viewers. You must set the value of the ```pdfviewer=``` config entry yourself if it doesn't do something you
like, such as not opening your favorite viewer. In the command line options, "%f" represents the filepath, "%p" the page number.
### Preview tab
The preview tab shows previews. It marks your search keywords too. Click on a preview to open the file.
A right click on a preview allows you to copy the file path, or to open the containing folder. Hovering tells you which file the preview originates from.
### Syncing index
Over time, files get deleted or their content changes. Go to **looqs** -> **Sync index**. looqs will reindex the content of files which have been changed. Files that cannot be found anymore will be removed from the index.
Reindexing a path using the "Index" tab will index new files and update existing ones. Currently however, this does not deal with deleted files.
I recommend doing a sync from time to time.
## CLI
The CLI command "looqs" comes with helptext. This documentation is incomplete at the moment.
### First run
There is no point in using the "search" command on the first run. Add some files if not done so already.
### Adding files
To add files to the index, run ```looqs add [path]```, where 'path' can be a directory or a single file.
If the path is a directory, the directory will be recursively descended, and all files in there added.
"Skipped" implies the file has not been changed since it has been added to the index. If it has changed, the index content will be updated.
### Searching files
Of course the CLI will not render any previews, but it can show you the paths where search results
have been found.
```
looqs search [terms...]
```
There is an implicit "AND" condition, meaning if you search for "photo" and "mountain", only paths
will be shown containing both terms, but not either alone.
### Deletion and Fixing Out of sync index
You sometimes delete files, to get rid of those from the index too, run:
```
looqs delete --deleted --dry-run
```
This commands lists all files which are indexed, but which cannot be found anymore.
Remove them using:
```
looqs delete --deleted --verbose
```
You can also delete by pattern:
```
looqs delete --pattern '*.java'
```
Delete never removes anything from the file system, it only operates on the database.
The equivalent of the GUI sync command is:
```
looks update -v --continue --delete
```
### Updating files
The content and metadata index for files can be updated:
```
looqs update -n
```
Those files still exist, but the content that has been indexed it out of date. This can be corrected with
```
looqs update
```
This will not add new files, you must run ```looqs add``` for this. For this reason, most users
will probably seldomly use the 'update' command alone.
## Tips
### Keeping index up to date
The most obvious way is to use the GUI and add your favorite paths in the "Index" tab. Then occasionally, just rescan. This works for me personally, looqs quickly picks up new files. This however may not be good enough for some users.
Some users may prefer setting up cronjobs or wire up the CLI interface with file system monitoring tools such as [adhocify](https://github.com/quitesimpleorg/adhocify).
### lh shell alias
If you are in a shell and you know your file is somewhere in your current directory or its subdirs, and those
are indexed by looqs, you may find the lh (look here) alias useful:
```
alias lh='looqs search $(pwd)'
```
So typing "lh recipes" searchs the current dir and its subdirs for a file containing 'recipes'.
## Query syntax / Search filters
A number of search filters are available.
| Filter (long) | Filter (short) | Explanation |
| ----------- | ----------- |----------- |
| path.contains:(term) | p:(term) | Pretty much a SQL LIKE '%term%' conditions, just searches the path string |
| path.ends:(term) | pe:(term) | Filters path ending with the specified term, e. g.: pe:(.ogg) |
| path.begins:(term) | pb:(term) | Filters path beginning with the specified term |
| contains:(terms) | c:(terms) | ull-text search, also understands quotes |
Filters can be combined. The booleans AND and OR are supported. Negations can be applied too, except for c:().
The AND boolean is implicit and thus entering it strictly optional.
Examples:
| Query | Explanation |
| ----------- | ----------- |
|pe:(.ogg) p:(marley)| Finds paths that end with .ogg and contain 'marley' (case-insensitive)
|p:(slides) support vector machine           |Performs a content search for 'support vector machine' in all paths containing 'slides'|
|p:(notes) (pe:(odt) OR pe:(docx))          |Finds files such as notes.docx, notes.odt but also any .docs and .odt when the path contains the string 'notes'|
|memcpy !(pe:(.c) OR pe:(.cpp))| Performs a FTS search for 'memcpy' but excludes .cpp and .c files.
|c:("I think, therefore")|Performs a FTS search for the phrase "I think, therefore".

Melihat File

@ -29,12 +29,8 @@ void CommandAdd::indexerFinished()
}
}
int ret = 0;
if(!keepGoing && failedPathsCount > 0)
{
ret = 1;
}
emit finishedCmd(ret);
/* TODO maybe not 0 if keepGoing not given */
emit finishedCmd(0);
}
int CommandAdd::handle(QStringList arguments)
@ -44,13 +40,20 @@ int CommandAdd::handle(QStringList arguments)
"Continue adding files, don't exit on first error. If this option is not given, looqs will "
"exit asap, but it's possible that a few files will still be processed. "
"Set -t 1 to avoid this behavior, but processing will be slower. "},
{{"a", "all"}, "On error, no files should be added, even already processed ones"},
{{"v", "verbose"}, "Print skipped and added files"},
{{"t", "threads"}, "Number of threads to use.", "threads"}});
parser.addHelpOption();
parser.addPositionalArgument("add", "Add paths to the index", "add [paths...]");
parser.process(arguments);
this->keepGoing = parser.isSet("continue");
bool keepGoing = parser.isSet("continue");
bool verbose = parser.isSet("verbose");
if(parser.isSet("all"))
{
throw LooqsGeneralException("To be implemented");
}
if(parser.isSet("threads"))
{
QString threadsCount = parser.value("threads");
@ -72,15 +75,17 @@ int CommandAdd::handle(QStringList arguments)
indexer = new Indexer(*this->dbService);
indexer->setTargetPaths(files.toVector());
indexer->setKeepGoing(keepGoing);
connect(indexer, &Indexer::pathsCountChanged, this,
[](int pathsCount) { Logger::info() << "Found paths: " << pathsCount << Qt::endl; });
connect(indexer, &Indexer::indexProgress, this,
[](int pathsCount, unsigned int added, unsigned int skipped, unsigned int failed, unsigned int totalCount)
{ Logger::info() << "Processed files: " << pathsCount << Qt::endl; });
connect(indexer, &Indexer::finished, this, &CommandAdd::indexerFinished);
/* TODO: keepGoing, verbose */
this->autoFinish = false;
indexer->beginIndexing();

Melihat File

@ -10,7 +10,6 @@ class CommandAdd : public Command
private:
SaveFileResult addFile(QString path);
Indexer *indexer;
bool keepGoing = true;
protected:
public:

Melihat File

@ -7,31 +7,29 @@ int CommandList::handle(QStringList arguments)
{
QCommandLineParser parser;
parser.addOptions({
{"pattern", "Only list files from index matching the pattern, e. g. '*.txt'", "pattern"},
{{"r", "reverse"}, "Print most-recent changed files first"},
{{"c", "count"}, "Counts the number of paths listed"},
{"pattern", "Only list files from index matching the pattern, e. g. */.git/*", "pattern"},
});
parser.addHelpOption();
parser.addPositionalArgument("list", "Lists paths in the index", "list [options]");
parser.process(arguments);
QString pattern = parser.value("pattern");
QVector<FileData> results;
int offset = 0;
int limit = 1000;
auto resultscount = dbService->getFiles(results, pattern, offset, limit);
while(resultscount > 0)
bool reverse = parser.isSet("reverse");
if(reverse)
{
for(FileData &fileData : results)
{
Logger::info() << fileData.absPath << Qt::endl;
}
offset += limit;
results.clear();
resultscount = dbService->getFiles(results, pattern, offset, limit);
throw LooqsGeneralException("Reverse option to be implemented");
}
QStringList files = parser.positionalArguments();
QString queryStrings = files.join(' ');
auto results = dbService->search(LooqsQuery::build(queryStrings, TokenType::FILTER_PATH_CONTAINS, false));
for(SearchResult &result : results)
{
Logger::info() << result.fileData.absPath << Qt::endl;
}
return 0;
}

Melihat File

@ -1,5 +1,5 @@
#ifndef COMMANDLIST_H
#define COMMANDLIST_H
#ifndef COMMANDSEARCH_H
#define COMMANDSEARCH_H
#include "command.h"
#include "../shared/sqlitesearch.h"
@ -11,4 +11,4 @@ class CommandList : public Command
int handle(QStringList arguments) override;
};
#endif // COMMANDLIST_H
#endif // COMMANDSEARCH_H

Melihat File

@ -8,19 +8,14 @@ int CommandSearch::handle(QStringList arguments)
QCommandLineParser parser;
parser.addOptions({
{{"r", "reverse"},
"Print most-recently changed files last. This is short for adding \"sort:(mtime asc)\" to the query."},
"Print most-recent changed files first. This is short for adding \"sort:(mtime asc)\" to the query."},
});
parser.addHelpOption();
parser.process(arguments);
QStringList terms = parser.positionalArguments();
if(terms.length() == 0)
{
Logger::error() << "Please enter search terms" << Qt::endl;
return 1;
}
QString queryStrings = terms.join(' ');
QStringList files = parser.positionalArguments();
QString queryStrings = files.join(' ');
LooqsQuery query = LooqsQuery::build(queryStrings, TokenType::FILTER_PATH_CONTAINS, false);
bool reverse = parser.isSet("reverse");
if(reverse)
@ -31,19 +26,11 @@ int CommandSearch::handle(QStringList arguments)
query.addSortCondition(sc);
}
try
{
auto results = dbService->search(query);
auto results = dbService->search(query);
for(SearchResult &result : results)
{
Logger::info() << result.fileData.absPath << Qt::endl;
}
}
catch(LooqsGeneralException &e)
for(SearchResult &result : results)
{
Logger::error() << "Exception:" << e.message << Qt::endl;
return 1;
Logger::info() << result.fileData.absPath << Qt::endl;
}
return 0;

Melihat File

@ -4,7 +4,6 @@
#include <QThreadPool>
#include "commandupdate.h"
#include "logger.h"
#include "../shared/indexsyncer.h"
int CommandUpdate::handle(QStringList arguments)
{
@ -13,17 +12,18 @@ int CommandUpdate::handle(QStringList arguments)
{{{"v", "verbose"}, "Print path of the files while updating them"},
{{"n", "dry-run"}, "Only print which files would be updated, don't actually update them"},
{"pattern", "Only consider to update files in the index matching the pattern, e. g. */.git/*.", "pattern"},
{{"d", "delete"}, "If a file does not exist anymore, delete it from the index"},
{{"d", "delete"}, "If a file does not exist anymore, delete it"},
{{"c", "continue"},
"Continue adding files, don't exit on first error. If this option is not given, looqs will exit asap, but "
"it's possible that a few files will still be processed. "
"Set -t 1 to avoid this behavior, but processing will be slower."},
{{"a", "all"}, "On error, no files should be updated, even already processed ones"},
{{"t", "threads"}, "Number of threads to use.", "threads"}
});
parser.addHelpOption();
parser.addPositionalArgument("update", "Checks files for changes and updates the index", "update");
parser.addPositionalArgument("update", "Checks files for changes and updates them", "update");
parser.process(arguments);
bool keepGoing = parser.isSet("continue");
@ -32,61 +32,85 @@ int CommandUpdate::handle(QStringList arguments)
bool dryRun = parser.isSet("dry-run");
QString pattern = parser.value("pattern");
if(parser.isSet("all"))
{
throw LooqsGeneralException("To be implemented");
}
if(parser.isSet("threads"))
{
QString threadsCount = parser.value("threads");
QThreadPool::globalInstance()->setMaxThreadCount(threadsCount.toInt());
}
FileSaver saver(*this->dbService);
QVector<FileData> files;
int offset = 0;
int limit = 1000;
int processedRows = dbService->getFiles(files, pattern, offset, limit);
bool hasErrors = false;
IndexSyncer *syncer = new IndexSyncer(*this->dbService);
syncer->setKeepGoing(keepGoing);
syncer->setVerbose(verbose);
syncer->setPattern(pattern);
syncer->setDryRun(dryRun);
syncer->setRemoveDeletedFromIndex(deleteMissing);
if(dryRun)
while(processedRows > 0)
{
connect(syncer, &IndexSyncer::removedDryRun, this,
[](QString path) { Logger::info() << "Would delete" << path << Qt::endl; });
connect(syncer, &IndexSyncer::updatedDryRun, this,
[](QString path) { Logger::info() << "Would update" << path << Qt::endl; });
}
else
{
connect(syncer, &IndexSyncer::removed, this,
[](QString path) { Logger::info() << "Removed " << path << Qt::endl; });
/* TODO: updated not printed, handled be verbose in FileSaver, but this can be improved */
}
connect(syncer, &IndexSyncer::finished, this,
[&](unsigned int totalUpdated, unsigned int totalRemoved, unsigned int totalErrors)
QVector<QString> filePathsToUpdate;
for(FileData &fileData : files)
{
QFileInfo fileInfo(fileData.absPath);
if(fileInfo.exists() && fileInfo.isFile())
{
Logger::info() << "Syncing finished" << Qt::endl;
if(!dryRun)
if(fileInfo.lastModified().toSecsSinceEpoch() != fileData.mtime)
{
Logger::info() << "Total updated:" << totalUpdated << Qt::endl;
Logger::info() << "Total removed from index: " << totalRemoved << Qt::endl;
Logger::info() << "Total deleted:" << totalErrors << Qt::endl;
if(!dryRun)
{
filePathsToUpdate.append(fileData.absPath);
}
else
{
Logger::info() << "Would update" << fileData.absPath << Qt::endl;
}
}
int retval = 0;
if(hasErrors && !keepGoing)
{
retval = 1;
}
emit finishedCmd(retval);
});
connect(syncer, &IndexSyncer::error, this,
[&](QString error)
}
else
{
Logger::error() << error << Qt::endl;
hasErrors = true;
});
if(deleteMissing)
{
if(!dryRun)
{
if(!this->dbService->deleteFile(fileData.absPath))
{
Logger::error()
<< "Error: Failed to delete" << fileData.absPath << "from databas" << Qt::endl;
if(!keepGoing)
{
return 1;
}
}
if(verbose)
{
Logger::info() << "Deleted" << fileData.absPath << Qt::endl;
}
}
else
{
Logger::info() << "Would delete" << fileData.absPath << Qt::endl;
}
}
}
}
int updatedFilesCount = saver.updateFiles(filePathsToUpdate, keepGoing, verbose);
int shouldHaveUpdatedCount = filePathsToUpdate.size();
if(updatedFilesCount != shouldHaveUpdatedCount)
{
if(!keepGoing)
{
Logger::error() << "Failed to update all files selected for updating. Updated" << updatedFilesCount
<< "out of" << shouldHaveUpdatedCount << "selected for upating" << Qt::endl;
return 1;
}
}
offset += limit;
files.clear();
processedRows = this->dbService->getFiles(files, pattern, offset, limit);
}
this->autoFinish = false;
syncer->sync();
/* Actual return code is handled by finishedCmd signal */
return 0;
}

Melihat File

@ -10,6 +10,7 @@
#include <QSettings>
#include <functional>
#include <QTimer>
#include <exception>
#include "encodingdetector.h"
#include "pdfprocessor.h"
@ -20,7 +21,6 @@
#include "commanddelete.h"
#include "commandupdate.h"
#include "commandsearch.h"
#include "commandlist.h"
#include "databasefactory.h"
#include "logger.h"
#include "sandboxedprocessor.h"
@ -50,10 +50,6 @@ Command *commandFromName(QString name, SqliteDbService &dbService)
{
return new CommandSearch(dbService);
}
if(name == "list")
{
return new CommandList(dbService);
}
return nullptr;
}

Melihat File

@ -78,4 +78,4 @@ else:win32:!win32-g++:CONFIG(release, debug|release): PRE_TARGETDEPS += $$OUT_PW
else:win32:!win32-g++:CONFIG(debug, debug|release): PRE_TARGETDEPS += $$OUT_PWD/../shared/debug/shared.lib
else:unix: PRE_TARGETDEPS += $$OUT_PWD/../shared/libshared.a
RESOURCES = ../looqs.svg
RESOURCES = ../icon.svg

Melihat File

@ -84,8 +84,7 @@ void IPCPreviewClient::start(RenderConfig config, const QVector<RenderTarget> &t
} while(!stream.commitTransaction() && socket->state() == QLocalSocket::ConnectedState);
if(numTarget != targets.count())
{
emit error("IPC Error: Server reports less targets than it should");
return;
throw std::runtime_error("Server reports less targets than it should");
}
}
else

Melihat File

@ -15,6 +15,25 @@
#include "../submodules/exile.h/exile.h"
#include "ipcserver.h"
void enableSandbox()
{
struct exile_policy *policy = exile_create_policy();
if(policy == NULL)
{
qCritical() << "Failed to init policy for sandbox";
exit(EXIT_FAILURE);
}
policy->namespace_options = 0;
policy->no_new_privs = 1;
int ret = exile_enable_policy(policy);
if(ret != 0)
{
qDebug() << "Failed to establish sandbox";
exit(EXIT_FAILURE);
}
exile_free_policy(policy);
}
void enableIpcSandbox()
{
struct exile_policy *policy = exile_create_policy();
@ -26,8 +45,7 @@ void enableIpcSandbox()
policy->namespace_options = EXILE_UNSHARE_NETWORK | EXILE_UNSHARE_USER;
policy->no_new_privs = 1;
policy->drop_caps = 1;
policy->vow_promises =
exile_vows_from_str("thread cpath wpath rpath unix stdio prot_exec proc shm fsnotify ioctl error");
policy->vow_promises = exile_vows_from_str("thread cpath wpath rpath unix stdio prot_exec proc shm fsnotify ioctl");
QString ipcSocketPath = Common::ipcSocketPath();
QFileInfo info{ipcSocketPath};
@ -70,16 +88,16 @@ int main(int argc, char *argv[])
if(arg == "process")
{
Common::setupAppInfo();
QCoreApplication a(argc, argv);
QApplication a(argc, argv);
QStringList args = a.arguments();
if(args.length() < 3)
if(args.length() < 1)
{
qDebug() << "Filename is required";
return 1;
}
QString file = args.at(2);
QString file = args.at(1);
SandboxedProcessor processor(file);
return processor.process();
}
@ -120,6 +138,15 @@ int main(int argc, char *argv[])
try
{
Common::ensureConfigured();
if(!parser.isSet("no-sandbox"))
{
enableSandbox();
qInfo() << "Sandbox: on";
}
else
{
qInfo() << "Sandbox: off";
}
}
catch(LooqsGeneralException &e)
{
@ -128,7 +155,7 @@ int main(int argc, char *argv[])
return 1;
}
QApplication a(argc, argv);
a.setWindowIcon(QIcon(":/looqs.svg"));
a.setWindowIcon(QIcon(":/icon.svg"));
QObject::connect(&a, &QApplication::aboutToQuit, &process, &QProcess::kill);
qRegisterMetaType<QVector<SearchResult>>("QVector<SearchResult>");
@ -138,9 +165,7 @@ int main(int argc, char *argv[])
qRegisterMetaType<RenderConfig>("RenderConfig");
qRegisterMetaType<QVector<RenderTarget>>("QVector<RenderTarget>");
qRegisterMetaType<QSharedPointer<PreviewResult>>("QSharedPointer<PreviewResult>");
MainWindow *w = new MainWindow{0, socketPath};
w->showMaximized();
int ret = a.exec();
process.waitForFinished(1000);
return ret;
MainWindow w{0, socketPath};
w.showMaximized();
return a.exec();
}

Melihat File

@ -14,7 +14,6 @@
#include <QMessageBox>
#include <QFileDialog>
#include <QScreen>
#include <QProgressDialog>
#include "mainwindow.h"
#include "ui_mainwindow.h"
#include "clicklabel.h"
@ -24,10 +23,8 @@
#include "ipcpreviewclient.h"
#include "previewgenerator.h"
MainWindow::MainWindow(QWidget *parent, QString socketPath)
: QMainWindow(parent), ui(new Ui::MainWindow), progressDialog(this)
MainWindow::MainWindow(QWidget *parent, QString socketPath) : QMainWindow(parent), ui(new Ui::MainWindow)
{
this->progressDialog.cancel(); // because constructing it shows it, quite weird
ui->setupUi(this);
setWindowTitle(QCoreApplication::applicationName());
this->ipcPreviewClient.moveToThread(&this->ipcClientThread);
@ -59,8 +56,6 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
db = this->dbFactory->forCurrentThread();
this->dbService = new SqliteDbService(*this->dbFactory);
this->indexSyncer = new IndexSyncer(*this->dbService);
this->indexSyncer->moveToThread(&this->syncerThread);
indexer = new Indexer(*(this->dbService));
indexer->setParent(this);
@ -77,9 +72,6 @@ MainWindow::MainWindow(QWidget *parent, QString socketPath)
QStringList indexPaths = settings.value("indexPaths").toStringList();
ui->lstPaths->addItems(indexPaths);
QString ignorePatterns = settings.value("ignorePatterns").toString();
ui->txtIgnorePatterns->setText(ignorePatterns);
ui->spinPreviewPage->setValue(1);
ui->spinPreviewPage->setMinimum(1);
}
@ -101,7 +93,6 @@ void MainWindow::addPathToIndex()
this->ui->lstPaths->addItem(path);
this->ui->txtPathScanAdd->clear();
}
void MainWindow::connectSignals()
{
connect(ui->txtSearch, &QLineEdit::returnPressed, this, &MainWindow::lineEditReturnPressed);
@ -172,70 +163,8 @@ void MainWindow::connectSignals()
}
}
});
connect(
ui->menuAboutAction, &QAction::triggered, this,
[this](bool checked)
{
QString html = "<h2>looqs</h2>";
html += "Full-text search with previews for your files<br><br>";
html += "Copyright (c) 2018-2022: Albert Schwarzkopf<br><br>";
html += QString("Version: %1<br><br>").arg(Common::versionText());
html += "Contact: looqs at quitesimple dot org<br><br>";
html += "Website: <a href=\"https://quitesimple.org\">https://quitesimple.org</a><br><br>";
html +=
"Source code: <a "
"href=\"https://github.com/quitesimpleorg/looqs\">https://github.com/quitesimpleorg/looqs</a><br><br>";
html += "License: GPLv3<br><br><br>";
html += "looqs is open source and free of charge in the hope it will be useful. The author(s) do not "
"give any warranty. In the unlikely event of any damage, the author(s) cannot be held responsible. "
"You are using looqs at your own risk";
QMessageBox::about(this, "About looqs", html);
});
connect(ui->menuOpenConfigInTextEditorAction, &QAction::triggered, this,
[this](bool checked)
{
QSettings setting;
QDesktopServices::openUrl(setting.fileName());
});
connect(ui->menuAboutQtAction, &QAction::triggered, this,
[this](bool checked) { QMessageBox::aboutQt(this, "About Qt"); });
connect(ui->menuSyncIndexAction, &QAction::triggered, this, &MainWindow::startIndexSync);
connect(indexSyncer, &IndexSyncer::finished, this,
[&](unsigned int totalUpdated, unsigned int totalDeleted, unsigned int totalErrored)
{
this->progressDialog.cancel();
QMessageBox::information(
this, "Syncing finished",
QString("Syncing finished\n\nTotal updated: %1\nTotal deleted: %2\nTotal errors: %3\n")
.arg(QString::number(totalUpdated))
.arg(QString::number(totalDeleted))
.arg(QString::number(totalErrored)));
});
connect(this, &MainWindow::beginIndexSync, indexSyncer, &IndexSyncer::sync);
connect(&this->progressDialog, &QProgressDialog::canceled, indexSyncer, &IndexSyncer::cancel);
}
void MainWindow::startIndexSync()
{
progressDialog.setWindowTitle("Syncing");
progressDialog.setLabelText("Syncing - this might take a moment, please wait");
progressDialog.setWindowModality(Qt::ApplicationModal);
progressDialog.setMinimum(0);
progressDialog.setMaximum(0);
progressDialog.setValue(0);
progressDialog.open();
indexSyncer->setKeepGoing(true);
indexSyncer->setVerbose(false);
indexSyncer->setDryRun(false);
indexSyncer->setRemoveDeletedFromIndex(true);
this->syncerThread.start();
emit beginIndexSync();
}
void MainWindow::spinPreviewPageValueChanged(int val)
{
makePreviews(val);
@ -267,12 +196,9 @@ void MainWindow::startIndexing()
pathSettingsValue.append(path);
}
this->indexer->setTargetPaths(paths);
QString ignorePatterns = ui->txtIgnorePatterns->text();
this->indexer->setIgnorePattern(ignorePatterns.split(";"));
this->indexer->beginIndexing();
QSettings settings;
settings.setValue("indexPaths", pathSettingsValue);
settings.setValue("ignorePatterns", ignorePatterns);
ui->btnStartIndexing->setText("Stop indexing");
}
@ -389,10 +315,6 @@ void MainWindow::lineEditReturnPressed()
ui->lblSearchResults->setText("Invalid paranthesis");
return;
}
if(indexerTabActive())
{
ui->tabWidget->setCurrentIndex(0);
}
// TODO: validate q;
ui->treeResultsList->clear();
ui->lblSearchResults->setText("Searching...");
@ -411,20 +333,10 @@ void MainWindow::lineEditReturnPressed()
if(addPathSearch)
{
LooqsQuery filesQuery = LooqsQuery::build(q, TokenType::FILTER_PATH_CONTAINS, false);
if(filesQuery.getLimit() == -1)
{
filesQuery.setLimit(1000);
}
results.append(searcher.search(filesQuery));
}
if(addContentSearch)
{
if(this->contentSearchQuery.getLimit() == -1)
{
this->contentSearchQuery.setLimit(1000);
}
results.append(searcher.search(this->contentSearchQuery));
}
return results;
@ -454,12 +366,9 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
bool exists = pathInfo.exists();
if(exists)
{
if(!pathInfo.suffix().contains("htm")) // hack until we can preview them properly...
if(PreviewGenerator::get(pathInfo) != nullptr)
{
if(PreviewGenerator::get(pathInfo) != nullptr)
{
this->previewableSearchResults.append(result);
}
this->previewableSearchResults.append(result);
}
}
else
@ -469,7 +378,6 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
}
ui->treeResultsList->resizeColumnToContents(0);
ui->treeResultsList->resizeColumnToContents(1);
ui->treeResultsList->resizeColumnToContents(2);
previewDirty = !this->previewableSearchResults.empty();
ui->spinPreviewPage->setValue(1);
@ -482,7 +390,7 @@ void MainWindow::handleSearchResults(const QVector<SearchResult> &results)
QString statusText = "Results: " + QString::number(results.size()) + " files";
if(hasDeleted)
{
statusText += " WARNING: Some files are inaccessible. No preview available for those. Index may be out of sync";
statusText += " WARNING: Some files don't exist anymore. No preview available for those. Index out of sync";
}
ui->lblSearchResults->setText(statusText);
}
@ -502,7 +410,7 @@ void MainWindow::makePreviews(int page)
scaleText.chop(1);
QVector<QString> wordsToHighlight;
QRegularExpression extractor(R"#("([^"]*)"|(\p{L}+))#");
QRegularExpression extractor(R"#("([^"]*)"|(\w+))#");
for(const Token &token : this->contentSearchQuery.getTokens())
{
if(token.type == FILTER_CONTENT_CONTAINS)
@ -620,11 +528,5 @@ void MainWindow::showSearchResultsContextMenu(const QPoint &point)
MainWindow::~MainWindow()
{
syncerThread.terminate();
ipcClientThread.terminate();
delete this->indexSyncer;
delete this->dbService;
delete this->dbFactory;
delete this->indexer;
delete ui;
}

Melihat File

@ -9,9 +9,7 @@
#include <QFutureWatcher>
#include <QSqlDatabase>
#include <QLocalSocket>
#include <QProgressDialog>
#include "../shared/looqsquery.h"
#include "../shared/indexsyncer.h"
#include "ipcpreviewclient.h"
#include "indexer.h"
namespace Ui
@ -36,9 +34,6 @@ class MainWindow : public QMainWindow
Ui::MainWindow *ui;
IPCPreviewClient ipcPreviewClient;
QThread ipcClientThread;
QThread syncerThread;
IndexSyncer *indexSyncer;
QProgressDialog progressDialog;
Indexer *indexer;
QFileIconProvider iconProvider;
@ -72,12 +67,10 @@ class MainWindow : public QMainWindow
void startIndexing();
void finishIndexing();
void addPathToIndex();
void startIndexSync();
signals:
void startIpcPreviews(RenderConfig config, const QVector<RenderTarget> &targets);
void stopIpcPreviews();
void beginIndexSync();
};
#endif // MAINWINDOW_H

Melihat File

@ -7,7 +7,7 @@
<x>0</x>
<y>0</y>
<width>1221</width>
<height>709</height>
<height>674</height>
</rect>
</property>
<property name="windowTitle">
@ -27,7 +27,7 @@
<enum>QTabWidget::South</enum>
</property>
<property name="currentIndex">
<number>0</number>
<number>1</number>
</property>
<widget class="QWidget" name="resultsTab">
<attribute name="title">
@ -333,7 +333,7 @@
</widget>
</item>
<item row="6" column="0">
<widget class="QLineEdit" name="txtIgnorePatterns"/>
<widget class="QLineEdit" name="lineEdit"/>
</item>
<item row="10" column="0">
<widget class="QPushButton" name="btnStartIndexing">
@ -363,47 +363,6 @@
</layout>
</widget>
<widget class="QStatusBar" name="statusBar"/>
<widget class="QMenuBar" name="menuBar">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>1221</width>
<height>35</height>
</rect>
</property>
<widget class="QMenu" name="menulooqs">
<property name="title">
<string>looqs</string>
</property>
<addaction name="menuOpenConfigInTextEditorAction"/>
<addaction name="menuSyncIndexAction"/>
<addaction name="menuAboutAction"/>
<addaction name="menuAboutQtAction"/>
<addaction name="separator"/>
</widget>
<addaction name="menulooqs"/>
</widget>
<action name="menuAboutAction">
<property name="text">
<string>About</string>
</property>
</action>
<action name="menuOpenConfigInTextEditorAction">
<property name="text">
<string>Open config in text editor</string>
</property>
</action>
<action name="menuAboutQtAction">
<property name="text">
<string>About Qt</string>
</property>
</action>
<action name="menuSyncIndexAction">
<property name="text">
<string>Sync index (remove deleted, update existing files)</string>
</property>
</action>
</widget>
<layoutdefault spacing="6" margin="11"/>
<resources/>

Melihat File

@ -1,4 +1,3 @@
#include "../shared/common.h"
#include "previewgenerator.h"
#include "previewgeneratorpdf.h"
#include "previewgeneratorplaintext.h"
@ -12,13 +11,5 @@ static QMap<QString, PreviewGenerator *> generators{
PreviewGenerator *PreviewGenerator::get(QFileInfo &info)
{
PreviewGenerator *result = generators.value(info.suffix(), nullptr);
if(result == nullptr)
{
if(Common::isTextFile(info))
{
return plainTextGenerator;
}
}
return result;
return generators.value(info.suffix(), nullptr);
}

Melihat File

@ -23,14 +23,12 @@ QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig c
int lastWordPos = 0;
QHash<QString, int> countmap;
const unsigned int maxSnippets = 7;
unsigned int currentSnippets = 0;
for(QString &word : config.wordsToHighlight)
{
int lastPos = 0;
int index = content.indexOf(word, lastPos, Qt::CaseInsensitive);
while(index != -1 && currentSnippets < maxSnippets)
while(index != -1)
{
countmap[word] = countmap.value(word, 0) + 1;
@ -54,7 +52,6 @@ QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig c
lastPos = index;
index = content.indexOf(word, lastPos + 1, Qt::CaseInsensitive);
++currentSnippets;
}
lastWordPos = lastPos;
}
@ -78,13 +75,8 @@ QSharedPointer<PreviewResult> PreviewGeneratorPlainText::generate(RenderConfig c
{
header += word + ": " + QString::number(countmap[word]) + " ";
}
if(currentSnippets == maxSnippets)
{
header += "(truncated)";
}
header += "<hr>";
result->setText(header + resulText.replace("\n", "<br>").mid(0, 1000));
result->setText(header + resulText.replace("\n", "<br>"));
return QSharedPointer<PreviewResultPlainText>(result);
}

Melihat File

Before

Width:  |  Height:  |  Ukuran: 13 KiB

After

Width:  |  Height:  |  Ukuran: 13 KiB

Melihat File

@ -1,6 +1,6 @@
[Desktop Entry]
Name=looqs
Exec=looqs-gui
Exec=/usr/bin/looqs-gui
Terminal=false
Type=Application
Icon=looqs

Melihat File

@ -7,7 +7,6 @@
#include <QSqlError>
#include <QTextStream>
#include <QDebug>
#include <QMimeDatabase>
#include "looqsgeneralexception.h"
#include "common.h"
#include "dbmigrator.h"
@ -18,8 +17,6 @@
#define SETTINGS_KEY_FIRSTRUN "firstrun"
#define SETTINGS_KEY_IPCSOCKETPATH "ipcsocketpath"
#define SETTINGS_KEY_PDFVIEWER "pdfviewer"
#define SETTINGS_KEY_EXCLUDEDPATHS "excludedpaths"
#define SETTINGS_KEY_MOUNTPATHS "mountpaths"
inline void initResources()
{
@ -165,73 +162,3 @@ QString Common::ipcSocketPath()
// QSettings settings;
// return settings.value(SETTINGS_KEY_IPCSOCKETPATH, "/tmp/.looqs/looqs-ipc-socket").toString();
}
static QStringList excludedPaths = {"/proc", "/sys", "/dev", "/tmp", "/var/run", "/run"};
QStringList Common::excludedPaths()
{
static int ran = false;
if(!ran)
{
QSettings settings;
QStringList userExcludedPaths = settings.value(SETTINGS_KEY_EXCLUDEDPATHS).toStringList();
ran = true;
::excludedPaths.append(userExcludedPaths);
}
return ::excludedPaths;
}
QStringList Common::mountPaths()
{
static int ran = false;
static QStringList mountPaths;
if(!ran)
{
QSettings settings;
mountPaths = settings.value(SETTINGS_KEY_MOUNTPATHS, QStringList{"/media", "/mnt"}).toStringList();
ran = true;
}
return mountPaths;
}
bool Common::isMountPath(QString path)
{
QStringList mountPaths = Common::mountPaths();
for(QString &mountPath : mountPaths)
{
if(path.startsWith(mountPath))
{
return true;
}
}
return false;
}
bool Common::isTextFile(QFileInfo fileInfo)
{
/* TODO: This is not sandboxed yet ... */
QMimeDatabase mimeDatabase;
QMimeType mimeType = mimeDatabase.mimeTypeForFile(fileInfo);
if(mimeType.name().startsWith("text/"))
{
return true;
}
else
{
for(QString &str : mimeType.allAncestors())
{
if(str.startsWith("text/"))
{
return true;
}
}
}
return false;
}
QString Common::versionText()
{
QString commitid = GIT_COMMIT_ID;
QString tag = GIT_TAG;
return tag + " (" + commitid + ") built " + __DATE__ + " " + __TIME__;
}

Melihat File

@ -1,7 +1,7 @@
#ifndef COMMON_H
#define COMMON_H
#include <QCoreApplication>
#include <QFileInfo>
namespace Common
{
void setupAppInfo();
@ -11,10 +11,5 @@ void setPdfViewer();
QString findInPath(QString needle);
bool initSqliteDatabase(QString path);
void ensureConfigured();
QStringList excludedPaths();
QStringList mountPaths();
bool isTextFile(QFileInfo fileInfo);
bool isMountPath(QString path);
QString versionText();
} // namespace Common
#endif

Melihat File

@ -7,7 +7,7 @@ DirScanWorker::DirScanWorker(ConcurrentQueue<QString> &queue, ConcurrentQueue<QS
{
this->queue = &queue;
this->resultQueue = &resultQueue;
this->wildcardMatcher.setPatterns(ignorePattern);
this->ignorePattern = ignorePattern;
this->progressReportThreshold = progressReportThreshold;
this->stopToken = &stopToken;
setAutoDelete(false);
@ -24,19 +24,10 @@ void DirScanWorker::run()
start new DirScanWorkers ourselves here... */
while(queue->dequeue(path))
{
if(wildcardMatcher.match(path))
{
continue;
}
QDirIterator iterator(path, QStringList{}, QDir::Files, QDirIterator::Subdirectories);
QDirIterator iterator(path, ignorePattern, QDir::Files, QDirIterator::Subdirectories);
while(iterator.hasNext())
{
QString entry = iterator.next();
if(wildcardMatcher.match(entry))
{
continue;
}
this->results.append(entry);
this->results.append(iterator.next());
++currentProgress;
if(currentProgress == progressReportThreshold)
{

Melihat File

@ -4,7 +4,6 @@
#include <QRunnable>
#include <QDirIterator>
#include "concurrentqueue.h"
#include "wildcardmatcher.h"
class DirScanWorker : public QObject, public QRunnable
{
Q_OBJECT
@ -13,7 +12,7 @@ class DirScanWorker : public QObject, public QRunnable
ConcurrentQueue<QString> *queue = nullptr;
ConcurrentQueue<QString> *resultQueue = nullptr;
WildcardMatcher wildcardMatcher;
QStringList ignorePattern;
QVector<QString> results;
std::atomic<bool> *stopToken;

Melihat File

@ -23,7 +23,6 @@ SaveFileResult FileSaver::addFile(QString path)
{
QFileInfo info(path);
QString absPath = info.absoluteFilePath();
auto mtime = info.lastModified().toSecsSinceEpoch();
if(this->dbService->fileExistsInDatabase(absPath, mtime))
{
@ -85,7 +84,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
}
else if(result == OK)
{
Logger::info() << "Saved" << path << Qt::endl;
Logger::info() << "Added" << path << Qt::endl;
}
}
}
@ -96,7 +95,7 @@ int FileSaver::processFiles(const QVector<QString> paths, std::function<SaveFile
SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
{
QVector<PageData> pageData;
QString canonicalPath = fileInfo.canonicalFilePath();
QString absPath = fileInfo.absoluteFilePath();
int status = -1;
@ -107,17 +106,9 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
if(fileInfo.isFile())
{
for(QString &excludedPath : this->excludedPaths)
{
if(canonicalPath.startsWith(excludedPath))
{
return SKIPPED;
}
}
QProcess process;
QStringList args;
args << "process" << canonicalPath;
args << "process" << absPath;
process.setProcessChannelMode(QProcess::ForwardedErrorChannel);
process.start("/proc/self/exe", args);
process.waitForStarted();
@ -141,7 +132,7 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
status = process.exitCode();
if(status != 0 && status != NOTHING_PROCESSED)
{
Logger::error() << "FileSaver::saveFile(): Error while processing" << canonicalPath << ":"
Logger::error() << "FileSaver::saveFile(): Error while processing" << absPath << ":"
<< "Exit code " << status << Qt::endl;
return PROCESSFAIL;
@ -151,7 +142,7 @@ SaveFileResult FileSaver::saveFile(const QFileInfo &fileInfo)
// Could happen if a file corrupted for example
if(pageData.isEmpty() && status != NOTHING_PROCESSED)
{
Logger::error() << "Could not get any content for " << canonicalPath << Qt::endl;
Logger::error() << "Could not get any content for " << absPath << Qt::endl;
}
return this->dbService->saveFile(fileInfo, pageData);

Melihat File

@ -5,12 +5,11 @@
#include "pagedata.h"
#include "filedata.h"
#include "sqlitedbservice.h"
#include "common.h"
class FileSaver
{
private:
SqliteDbService *dbService;
QStringList excludedPaths = Common::excludedPaths();
public:
FileSaver(SqliteDbService &dbService);
@ -21,6 +20,8 @@ class FileSaver
bool keepGoing, bool verbose);
int addFiles(const QVector<QString> paths, bool keepGoing, bool verbose);
int updateFiles(const QVector<QString> paths, bool keepGoing, bool verbose);
;
};
#endif // FILESAVER_H

Melihat File

@ -1,6 +1,5 @@
#include "indexer.h"
#include "logger.h"
#include "wildcardmatcher.h"
Indexer::Indexer(SqliteDbService &db)
{
@ -18,14 +17,8 @@ void Indexer::beginIndexing()
this->currentIndexResult.begin = QDateTime::currentDateTime();
QVector<QString> dirs;
WildcardMatcher wildcardMatcher;
wildcardMatcher.setPatterns(this->ignorePattern);
for(QString &path : this->pathsToScan)
{
if(wildcardMatcher.match(path))
{
continue;
}
QFileInfo info{path};
if(info.isDir())
{
@ -36,14 +29,10 @@ void Indexer::beginIndexing()
this->filePathTargetsQueue.enqueue(path);
}
}
this->dirScanner->setPaths(dirs);
this->dirScanner->setIgnorePatterns(this->ignorePattern);
if(!dirs.empty())
{
this->dirScanner->setPaths(dirs);
this->dirScanner->setIgnorePatterns(this->ignorePattern);
this->dirScanner->scan();
}
this->dirScanner->scan();
this->workerCancellationToken.store(false, std::memory_order_seq_cst);
launchWorker(this->filePathTargetsQueue, this->filePathTargetsQueue.remaining());
@ -59,16 +48,6 @@ void Indexer::setTargetPaths(QVector<QString> pathsToScan)
this->pathsToScan = pathsToScan;
}
void Indexer::setVerbose(bool verbose)
{
this->verbose = verbose;
}
void Indexer::setKeepGoing(bool keepGoing)
{
this->keepGoing = keepGoing;
}
void Indexer::requestCancellation()
{
this->dirScanner->cancel();
@ -82,7 +61,7 @@ IndexResult Indexer::getResult()
void Indexer::dirScanFinished()
{
Logger::info() << "Dir scan finished" << Qt::endl;
Logger::info() << "Dir scan finished";
if(!isRunning())
{
emit finished();
@ -106,24 +85,17 @@ void Indexer::dirScanProgress(int current, int total)
void Indexer::processFileScanResult(FileScanResult result)
{
if(result.second == DBFAIL || result.second == PROCESSFAIL || result.second == NOTFOUND)
if(verbose)
{
this->currentIndexResult.results.append(result);
if(!keepGoing)
{
this->requestCancellation();
emit finished();
return;
}
}
else
{
if(verbose)
if(result.second == DBFAIL || result.second == PROCESSFAIL || result.second == NOTFOUND)
{
this->currentIndexResult.results.append(result);
}
}
if(result.second == OK)
{
++this->currentIndexResult.addedPaths;

Melihat File

@ -67,8 +67,6 @@ class Indexer : public QObject
void beginIndexing();
void setIgnorePattern(QStringList ignorePattern);
void setTargetPaths(QVector<QString> pathsToScan);
void setVerbose(bool verbose);
void setKeepGoing(bool keepGoing);
void requestCancellation();

Melihat File

@ -1,134 +0,0 @@
#include <QDateTime>
#include "filesaver.h"
#include "indexsyncer.h"
IndexSyncer::IndexSyncer(SqliteDbService &dbService)
{
this->dbService = &dbService;
}
void IndexSyncer::setDryRun(bool dryRun)
{
this->dryRun = dryRun;
}
void IndexSyncer::setVerbose(bool verbose)
{
this->verbose = verbose;
}
void IndexSyncer::setKeepGoing(bool keepGoing)
{
this->keepGoing = keepGoing;
}
void IndexSyncer::setRemoveDeletedFromIndex(bool removeDeletedFromIndex)
{
this->removeDeletedFromIndex = removeDeletedFromIndex;
}
void IndexSyncer::setPattern(QString pattern)
{
this->pattern = pattern;
}
void IndexSyncer::sync()
{
this->stopToken.store(false, std::memory_order_relaxed);
FileSaver saver(*this->dbService);
QVector<FileData> files;
int offset = 0;
int limit = 10000;
unsigned int processedRows = dbService->getFiles(files, pattern, offset, limit);
unsigned int totalUpdatesFilesCount = 0;
unsigned int totalDeletedFilesCount = 0;
unsigned int totalErroredFilesCount = 0;
while(processedRows > 0)
{
QVector<QString> filePathsToUpdate;
for(FileData &fileData : files)
{
if(processedRows % 100 == 0 && this->stopToken.load(std::memory_order_relaxed))
{
emit finished(totalUpdatesFilesCount, totalDeletedFilesCount, totalErroredFilesCount);
return;
}
if(Common::isMountPath(fileData.absPath))
{
continue;
}
QFileInfo fileInfo(fileData.absPath);
if(fileInfo.exists())
{
if(fileInfo.isFile())
{
if(fileInfo.lastModified().toSecsSinceEpoch() != fileData.mtime)
{
if(!dryRun)
{
filePathsToUpdate.append(fileData.absPath);
}
else
{
emit updatedDryRun(fileData.absPath);
}
}
}
}
else
{
if(this->removeDeletedFromIndex)
{
if(!dryRun)
{
if(!this->dbService->deleteFile(fileData.absPath))
{
emit error("Error: Failed to delete " + fileData.absPath + " from the index");
if(!this->keepGoing)
{
emit finished(totalUpdatesFilesCount, totalDeletedFilesCount, totalErroredFilesCount);
return;
}
}
emit removed(fileData.absPath);
++totalDeletedFilesCount;
}
else
{
emit removedDryRun(fileData.absPath);
}
}
}
}
unsigned int updatedFilesCount = saver.updateFiles(filePathsToUpdate, keepGoing, verbose);
unsigned int shouldHaveUpdatedCount = static_cast<unsigned int>(filePathsToUpdate.size());
if(updatedFilesCount != shouldHaveUpdatedCount)
{
totalErroredFilesCount += (shouldHaveUpdatedCount - updatedFilesCount);
if(!keepGoing)
{
QString errorMsg = QString("Failed to update all files selected for updating in this batch. Updated") +
updatedFilesCount + "out of" + shouldHaveUpdatedCount + "selected for updating";
emit error(errorMsg);
emit finished(totalUpdatesFilesCount, totalDeletedFilesCount, totalErroredFilesCount);
}
}
offset += limit;
files.clear();
totalUpdatesFilesCount += updatedFilesCount;
processedRows = this->dbService->getFiles(files, pattern, offset, limit);
}
emit finished(totalUpdatesFilesCount, totalDeletedFilesCount, totalErroredFilesCount);
}
void IndexSyncer::cancel()
{
this->stopToken.store(true, std::memory_order_seq_cst);
}

Melihat File

@ -1,39 +0,0 @@
#ifndef INDEXSYNCER_H
#define INDEXSYNCER_H
#include "sqlitedbservice.h"
class IndexSyncer : public QObject
{
Q_OBJECT
private:
SqliteDbService *dbService = nullptr;
bool keepGoing = true;
bool removeDeletedFromIndex = true;
bool dryRun = false;
bool verbose = false;
QString pattern;
std::atomic<bool> stopToken{false};
public:
IndexSyncer(SqliteDbService &dbService);
public slots:
void sync();
void cancel();
void setDryRun(bool dryRun);
void setVerbose(bool verbose);
void setKeepGoing(bool keepGoing);
void setRemoveDeletedFromIndex(bool removeDeletedFromIndex);
void setPattern(QString pattern);
signals:
void error(QString error);
void removed(QString path);
void removedDryRun(QString path);
void updatedDryRun(QString path);
void updated(QString path);
void finished(unsigned int totalUpdated, unsigned int totalDeleted, unsigned int totalErrored);
};
#endif // INDEXSYNCER_H

Melihat File

@ -169,10 +169,6 @@ void LooqsQuery::addToken(Token t)
* TODO: It's a bit ugly still*/
LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, bool mergeLoneWords)
{
if(expression.isEmpty())
{
return LooqsQuery{};
}
if(!checkParanthesis(expression))
{
throw LooqsGeneralException("Invalid paranthesis");
@ -180,9 +176,8 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
QStringList loneWords;
LooqsQuery result;
QRegularExpression rx(
"((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([\\p{L},])+)|(?<boolean>AND|OR)"
"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[\"\\p{L}]+))");
QRegularExpression rx("((?<filtername>(\\.|\\w)+):(?<args>\\((?<innerargs>[^\\)]+)\\)|([\\w,])+)|(?<boolean>AND|OR)"
"|(?<negation>!)|(?<bracket>\\(|\\))|(?<loneword>[\"\\w]+))");
QRegularExpressionMatchIterator i = rx.globalMatch(expression);
auto previousWasBool = [&result] { return !result.tokens.empty() && ((result.tokens.last().type & BOOL) == BOOL); };
auto previousWas = [&result](TokenType t) { return !result.tokens.empty() && (result.tokens.last().type == t); };
@ -221,8 +216,19 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
{
throw LooqsGeneralException("Can't have two negations following each other");
}
if(!previousWasBool())
{
result.addToken(Token(BOOL_AND)); // Implicit and, our default operation
}
result.addToken(Token(NEGATION));
}
if(!result.tokens.isEmpty() && !previousWasBool() && !previousWas(NEGATION) && !previousWas(BRACKET_OPEN) &&
bracket != ")")
{
// the current token isn't a negation, isn't a boolean. Thus, implicit AND is required
result.addToken(Token(BOOL_AND));
}
if(bracket != "")
{
if(bracket == "(")
@ -249,7 +255,7 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
if(filtername != "")
{
TokenType tokenType = WORD;
TokenType tokenType;
QString value = m.captured("innerargs");
if(value == "")
{
@ -260,15 +266,15 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
throw LooqsGeneralException("value cannot be empty for filters");
}
if(filtername == "p" || filtername == "path.contains")
if(filtername == "path.contains")
{
tokenType = FILTER_PATH_CONTAINS;
}
else if(filtername == "pb" || filtername == "path.starts")
else if(filtername == "path.starts")
{
tokenType = FILTER_PATH_STARTS;
}
else if(filtername == "pe" || filtername == "path.ends")
else if(filtername == "path.ends")
{
tokenType = FILTER_PATH_ENDS;
}
@ -284,21 +290,21 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
{
tokenType = FILTER_CONTENT_PAGE;
}
// TODO: given this is not really a "filter", this feels slightly misplaced here
else if(filtername == "sort")
else if(filtername ==
"sort") // TODO: given this is not really a "filter", this feels slightly misplaced here
{
if(!result.sortConditions.empty())
{
throw LooqsGeneralException("Two sort statements are illegal");
}
// TODO: hack, since we are not a "filter", we must remove a preceeding (implicit) boolean
if((result.tokens.last().type & BOOL) == BOOL)
{
result.tokens.pop_back();
}
result.sortConditions = createSortConditions(value);
continue;
}
else if(filtername == "limit")
{
result.limit = value.toInt();
continue;
}
else
{
throw LooqsGeneralException("Unknown filter provided!");
@ -316,26 +322,6 @@ LooqsQuery LooqsQuery::build(QString expression, TokenType loneWordsTokenType, b
}
}
/* Add our default implicit AND boolean condition where appropriate */
QVector<Token> newTokens;
TokenType prevType = BOOL_AND;
int needsBoolean = FILTER_CONTENT | FILTER_PATH | NEGATION;
for(Token &t : result.tokens)
{
if(t.type == BRACKET_OPEN || t.type & needsBoolean)
{
if(!((prevType & BOOL) == BOOL) && !((prevType & NEGATION) == NEGATION) &&
!((prevType & BRACKET_OPEN) == BRACKET_OPEN))
{
newTokens.append(Token(BOOL_AND));
}
}
prevType = t.type;
newTokens.append(t);
}
result.tokens = newTokens;
bool contentsearch = result.hasContentSearch();
bool sortsForContent = std::any_of(result.sortConditions.begin(), result.sortConditions.end(),
[](SortCondition c) { return c.field == CONTENT_TEXT; });

Melihat File

@ -40,7 +40,6 @@ class LooqsQuery
/* Helper field to determine quertype as well as to quickly check what kind of filters etc.
* are being used in this query*/
int tokensMask = 0;
int limit = -1;
QVector<Token> tokens;
QVector<SortCondition> sortConditions;
void addToken(Token t);
@ -53,14 +52,6 @@ class LooqsQuery
{
return tokensMask;
}
int getLimit() const
{
return limit;
}
void setLimit(int limit)
{
this->limit = limit;
}
bool hasContentSearch();
bool hasPathSearch();

Melihat File

@ -50,7 +50,6 @@ void ParallelDirScanner::handleWorkersFinish()
running = false;
emit scanComplete();
}
delete QObject::sender();
}
unsigned int ParallelDirScanner::getThreadsNum() const

Melihat File

@ -41,7 +41,6 @@ class ParallelDirScanner : public QObject
void progress(int, int);
public slots:
void cancel();
private slots:
void handleWorkersProgress(unsigned int progress);
void handleWorkersFinish();
};

Melihat File

@ -9,7 +9,6 @@
#include "odtprocessor.h"
#include "odsprocessor.h"
#include "../submodules/exile.h/exile.h"
#include "common.h"
#include "logger.h"
static DefaultTextProcessor *defaultTextProcessor = new DefaultTextProcessor();
@ -23,7 +22,7 @@ static QMap<QString, Processor *> processors{
{"py", defaultTextProcessor}, {"xml", nothingProcessor}, {"html", tagStripperProcessor},
{"java", defaultTextProcessor}, {"js", defaultTextProcessor}, {"cpp", defaultTextProcessor},
{"c", defaultTextProcessor}, {"sql", defaultTextProcessor}, {"odt", odtProcessor},
{"ods", odsProcessor}, {"svg", nothingProcessor}};
{"ods", odsProcessor}};
void SandboxedProcessor::enableSandbox(QString readablePath)
{
@ -35,10 +34,9 @@ void SandboxedProcessor::enableSandbox(QString readablePath)
}
policy->namespace_options = EXILE_UNSHARE_NETWORK | EXILE_UNSHARE_USER;
std::string readablePathLocation;
if(!readablePath.isEmpty())
{
readablePathLocation = readablePath.toStdString();
std::string readablePathLocation = readablePath.toStdString();
if(exile_append_path_policies(policy, EXILE_FS_ALLOW_ALL_READ, readablePathLocation.c_str()) != 0)
{
qCritical() << "Failed to add path policies";
@ -52,7 +50,7 @@ void SandboxedProcessor::enableSandbox(QString readablePath)
int ret = exile_enable_policy(policy);
if(ret != 0)
{
qCritical() << "Failed to establish sandbox: " << ret;
qDebug() << "Failed to establish sandbox: " << ret;
exit(EXIT_FAILURE);
}
exile_free_policy(policy);
@ -76,16 +74,9 @@ void SandboxedProcessor::printResults(const QVector<PageData> &pageData)
int SandboxedProcessor::process()
{
QFileInfo fileInfo(this->filePath);
Processor *processor = processors.value(fileInfo.suffix(), nullptr);
if(processor == nullptr)
{
/* TODO: Not sandboxed */
if(Common::isTextFile(fileInfo))
{
processor = defaultTextProcessor;
}
}
if(processor == nullptr || processor == nothingProcessor)
Processor *processor = processors.value(fileInfo.suffix(), nothingProcessor);
if(processor == nothingProcessor)
{
/* Nothing to do */
return NOTHING_PROCESSED;

Melihat File

@ -1,14 +1,12 @@
#ifndef SANDBOXEDPROCESSOR_H
#define SANDBOXEDPROCESSOR_H
#include <QString>
#include <QMimeDatabase>
#include "pagedata.h"
class SandboxedProcessor
{
private:
QString filePath;
QMimeDatabase mimeDatabase;
void enableSandbox(QString readablePath = "");
void printResults(const QVector<PageData> &pageData);

Melihat File

@ -16,17 +16,12 @@ CONFIG += c++17
INCLUDEPATH += $$PWD/../sandbox/exile.h/
INCLUDEPATH += /usr/include/poppler/qt5/ /usr/include/quazip5
# The following define makes your compiler emit warnings if you use
# any feature of Qt which has been marked as deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS
DEFINES += GIT_COMMIT_ID="\\\"$(shell git rev-parse --short HEAD)\\\""
DEFINES += GIT_TAG="\\\"$(shell git describe --tags HEAD)\\\""
# You can also make your code fail to compile if you use deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
@ -42,7 +37,6 @@ SOURCES += sqlitesearch.cpp \
filesaver.cpp \
filescanworker.cpp \
indexer.cpp \
indexsyncer.cpp \
logger.cpp \
looqsgeneralexception.cpp \
common.cpp \
@ -58,8 +52,7 @@ SOURCES += sqlitesearch.cpp \
sqlitedbservice.cpp \
tagstripperprocessor.cpp \
utils.cpp \
../submodules/exile.h/exile.c \
wildcardmatcher.cpp
../submodules/exile.h/exile.c
HEADERS += sqlitesearch.h \
concurrentqueue.h \
@ -72,7 +65,6 @@ HEADERS += sqlitesearch.h \
filesaver.h \
filescanworker.h \
indexer.h \
indexsyncer.h \
logger.h \
looqsgeneralexception.h \
looqsquery.h \
@ -89,8 +81,7 @@ HEADERS += sqlitesearch.h \
tagstripperprocessor.h \
token.h \
common.h \
utils.h \
wildcardmatcher.h
utils.h
unix {
target.path = /usr/lib
INSTALLS += target

Melihat File

@ -62,11 +62,10 @@ bool SqliteDbService::deleteFile(QString path)
}
return result;
}
unsigned int SqliteDbService::getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit)
int SqliteDbService::getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit)
{
unsigned int processedRows = 0;
int processedRows = 0;
// TODO: translate/convert wildCardPattern to SQL where instead of regex
QString sql = "SELECT path, mtime, size, filetype FROM file";

Melihat File

@ -24,7 +24,7 @@ class SqliteDbService
public:
SqliteDbService(DatabaseFactory &dbFactory);
SaveFileResult saveFile(QFileInfo fileInfo, QVector<PageData> &pageData);
unsigned int getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit);
int getFiles(QVector<FileData> &results, QString wildCardPattern, int offset, int limit);
bool deleteFile(QString path);
bool fileExistsInDatabase(QString path);
bool fileExistsInDatabase(QString path, qint64 mtime);

Melihat File

@ -183,11 +183,6 @@ QSqlQuery SqliteSearch::makeSqlQuery(const LooqsQuery &query)
whereSql + " " + sortSql;
}
if(query.getLimit() > 0)
{
prepSql += " LIMIT " + QString::number(query.getLimit());
}
QSqlQuery dbquery(*db);
dbquery.prepare(prepSql);
for(const QString &value : bindValues)

Melihat File

@ -4,25 +4,24 @@
enum TokenType
{
WORD = 8,
NEGATION = 16,
BOOL = 32,
WORD,
NEGATION = 2,
BOOL = 4,
BOOL_AND,
BOOL_OR,
GROUP = 64,
GROUP = 8,
BRACKET_OPEN,
BRACKET_CLOSE,
SORT = 128,
FILTER_PATH = 256,
SORT = 16,
FILTER_PATH = 32,
FILTER_PATH_MTIME,
FILTER_PATH_CONTAINS,
FILTER_PATH_SIZE,
FILTER_PATH_ENDS,
FILTER_PATH_STARTS,
FILTER_CONTENT = 512,
FILTER_CONTENT = 64,
FILTER_CONTENT_CONTAINS,
FILTER_CONTENT_PAGE,
LIMIT = 1024
};
class Token

Melihat File

@ -1,29 +0,0 @@
#include "wildcardmatcher.h"
void WildcardMatcher::setPatterns(QStringList patterns)
{
this->regexes.clear();
for(QString &str : patterns)
{
QRegExp regexp;
regexp.setPattern(str);
regexp.setPatternSyntax(QRegExp::WildcardUnix);
this->regexes.append(regexp);
}
}
WildcardMatcher::WildcardMatcher()
{
}
bool WildcardMatcher::match(QString haystack) const
{
for(const QRegExp &regexp : this->regexes)
{
if(regexp.exactMatch(haystack))
{
return true;
}
}
return false;
}

Melihat File

@ -1,17 +0,0 @@
#ifndef WILDCARDMATCHER_H
#define WILDCARDMATCHER_H
#include <QStringList>
#include <QRegExp>
class WildcardMatcher
{
private:
QVector<QRegExp> regexes;
QStringList patterns;
public:
WildcardMatcher();
bool match(QString haystack) const;
void setPatterns(QStringList patterns);
};
#endif // WILDCARDMATCHER_H

@ -1 +1 @@
Subproject commit 42d44b0cc1e4ef35d0429e43a1dd005556450b44
Subproject commit ea66ef76ebb88a43ac25c9a86f8fcab8efa130b2