Commit Graph

284 Commits

Author SHA1 Message Date
Moritz Poldrack c882cf9960 lint: removed unused code (deadcode, structcheck, unused)
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:58:00 +02:00
Moritz Poldrack 5ca6022d00 lint: ensure errors are at least logged (errcheck)
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:57:57 +02:00
Moritz Poldrack ef599aa8fc lint: simplify code (gosimple)
Replaces infinite for loops containing a select on a channel with a
single case with a range over the channel.

Removes redundant assignments to blank identifiers.

Remove unnecessary guard clause around delete().

Remove `if condition { return true } return false` with return condition

Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:57:52 +02:00
Moritz Poldrack 03f9f4c3ab lint: remove ineffectual assignments (ineffassign)
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:57:50 +02:00
Moritz Poldrack 2534612c59 lint: remove unused structs and functions (unused)
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:57:48 +02:00
Moritz Poldrack 77a00de741 lint: remove redundant returns (S1023)
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:57:45 +02:00
Moritz Poldrack f70eecc7cb lint: remove empty branches (SA9003)
Empty branches are effectively dead code and should therefore be
removed.

Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-04 21:57:28 +02:00
Tim Culverhouse fdfec2c07a seqmap: refactor seqmap to use slice instead of map
The imap worker's seqmap is represented as a map of sequence number to
UID. This presents a problem when expunging group of messages from the
mailbox: each individual expunge decrements the sequence numbers by 1
(for every sequence number greater than the expunged). This requires a
looping around the map to update the keys. The use of a map also
requires that both the sequence number and the UID of a message be known
in order to insert it into the map. This is only discovered by fetching
individual message body parts (flags, headers, etc), leaving the seqmap
to be empty until we have fetched information about each message. In
certain instances (if a mailbox has recently been loaded), all
information is loaded in memory and no new information is fetched -
leaving the seqmap empty and the UI out of sync with the worker.

Refactor the seqmap as a slice, so that any expunge automatically
decrements the rest of the sequences.

Use the results of FetchDirectoryContents or FetchDirectoryThreaded to
initialize the seqmap with all discovered UIDs. Sort the UIDs in
ascending order: IMAP specification requires that sequence numbers start
at 1 increase in order of ascending UID.

Add individual messages to the map if they come via a MessageUpdate and
have a sequence number larger than our slice.

Update seqmap tests with new logic.

Reference: https://datatracker.ietf.org/doc/html/rfc3501#section-2.3.1.2
Fixes: https://todo.sr.ht/~rjarry/aerc/69
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-03 22:37:13 +02:00
Tim Culverhouse 9630d9d281 seqmap: compare ints instead of uints
When a test fails with a uint comparison, assert displays the hex code
instead of an int, making it harder to debug. Use ints in sequmap test
asserts instead of uints for better readability when tests fail

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-03 22:37:09 +02:00
Tim Culverhouse 23ee64b057 seqmap: re-order test asserts
Reorder seqmap asserts to properly show display expected and actual when
performing go test -v ./...

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-03 22:37:03 +02:00
Moritz Poldrack aaf0a0c656 lint: apply new formatting rules
Run `make fmt`.

Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-01 10:44:52 +02:00
Tim Culverhouse 44651b43b3 checkmail: error if check-mail-cmd is not set
Send error message to UI if check-mail-cmd is required but not set.

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-08-01 10:31:59 +02:00
Koni Marti 318f7d252c notmuch: fix cursor movement in threaded view
Set the SkipSort flag when sending directory infos for counting
purposes. Without this, the directory infos would trigger a directory
fetch which could bring the notmuch threads out of sync with the message
list. The notmuch backend sends these directory infos automatically
every minute.

To reproduce the weird cursor movement in notmuch's threaded view:
1. enter threaded view in notmuch
2. wait 1 min (until the auto directory infos are sent out)
3. move cursor around and notice how it jumps over threads

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
2022-07-31 20:02:46 +02:00
Tim Culverhouse ea718fe146 imap: remove unused expunge code
Remove unused code in the handleDeleteMessages routine. During
debugging, it was found that the channel for expunge updates was not
working and that all expunge details were coming through as
ExpungeUpdates. The reporting channel is unneeded.

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-26 22:24:28 +02:00
Koni Marti 26b9c3d966 sort: show warning when sort is not supported
Use the capabilities returned by the backend to check whether sort is
implemented when the user tries to use the sort command. Print a warning
to the log when a sort request is silently dropped by the backend.

Suggested-by: |cos|
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-26 22:22:58 +02:00
Tim Culverhouse fb5558da81 seqmap: sync seqNum to uid after expunge
This patch updates the seqNums after an Expunge operation. When an
expunge operation occurs, the seqNum of the deleted message is reported.
The Imap spec [0] states that an immediate decrement of all seqnums greater
than the deleted occurs, even before the next reporting of an expunge
update.

[0]: https://datatracker.ietf.org/doc/html/rfc3501#section-7.4.1

Fixes: https://todo.sr.ht/~rjarry/aerc/61
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-07-24 23:06:10 +02:00
Robin Jarry cd19995557 logging: use level-based logger functions
Do not pass logger objects around anymore. Shuffle some messages to make
them consistent with the new logging API. Avoid using %v when a more
specific verb exists for the argument types.

The loggers are completely disabled (i.e. Sprintf is not even called)
by default. They are only enabled when redirecting stdout to a file.

Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
2022-07-23 22:52:15 +02:00
Robin Jarry b188f19131 imap: disable debug logger
This causes all raw email bodies to be dumped along with actual
debugging messages. I don't believe we neither need nor want such
a thing.

Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
2022-07-23 22:37:08 +02:00
Robin Jarry 5dec1f09b1 imap: fix error when server returns a message without body section
When opening unread emails from certain people (I won't name any names,
sorry), an annoying error message is displayed on the status line:

 could not get section &imap.BodySectionName{BodyPartName:
 imap.BodyPartName{Specifier:"", Path:[]int(nil), Fields:[]string(nil),
 NotFields:false}, Peek:false, Partial:[]int(nil), value}

This does not occur for already read messages. This issue is similar to
the one that was fixed in commit 8ed95b0d2a ("imap: avoid crash when
replying to unread message").

This happens because the flags are updated in the callback that receives
the message itself. It causes the flag update to arrive in the same
channel/request. Ignore the messages that have an empty body (i.e. only
containing flag updates). This is inherently racy but there seems no way
to get rid of these extra messages.

Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Koni Marti <koni.marti@gmail.com>
2022-07-23 21:51:47 +02:00
Koni Marti e572087e58 account: import mbox file to a folder
Append all messages from an mbox file to the selected folder with the
import-mbox command.

User confirmation is required when the folder already contains messages.

A failed append will be retried a few times. If a backend timeout
occurs, the entire import is stopped to prevent a hang.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-14 23:14:56 +02:00
Koni Marti 845763cb1f account: export folder to mbox file
Export all message in the current folder to an mbox file. If an error
occurs during the export, aerc retries a few times before giving up to
prevent a hang.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-14 23:14:54 +02:00
Koni Marti a1a276e002 mbox: implement an mbox backend worker
Implement an mbox backend worker. Worker can be used for testing and
development by mocking a backend for the message store. Worker does not
modify the actual mbox file on disk; all operations are performed in
memory.

To use the mbox backend, create an mbox account in the accounts.conf
where the source uses the "mbox://" scheme, such as

source = mbox://~/mbox/

or

source = mbox://~/mbox/file.mbox

If the mbox source points to a directory, all files in this directory
with the .mbox suffix will be opened as folders.

If an outgoing smtp server is defined for the mbox account, replies can
be sent to emails that are stored in the mbox file.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-14 23:14:45 +02:00
Tim Culverhouse c2f4404fca threading: enable filtering of server-side threads
This patch enables the filtering of a threaded view which uses
server-built threads. Filtering is done server-side, in order to
preserve the use of server-built threads.

In adding this feature, the filtering of notmuch folders was brought up
to feature parity with the other workers. The filters function the same
(ie: they can be stacked). The notmuch filters, however, still use
notmuch syntax for the filtering.

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-10 21:15:12 +02:00
Tim Culverhouse f0c76fad72 threading: add backend capabilities to workers
This patch provides a method to report backend capabilities to the UI.
The intial capabilities included in the report are Sort and Thread.
Having these available to the UI enables the client to better handle
server side threading.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-07-10 21:15:12 +02:00
Adnan Maolood c5daf43460 worker/maildir: implement Maildir++ support
See https://www.courier-mta.org/maildir.html#maildircontents

Signed-off-by: Adnan Maolood <me@adnano.co>
Acked-by: Koni Marti <koni.marti@gmail.com>
2022-07-10 20:34:47 +02:00
Tim Culverhouse e2be2dd4c0 notmuch: fix server-side threads
Notmuch server-side threading added messages within a thread that didn't
match the query into the uidstore. By doing so, several UI issues
presented:

* All "hidden" messages displayed at the bottom of the msglist
* Selected messages wouldn't open properly

This patch stops these messages from being put into the message store,
thereby resolving the UI issues

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Koni Marti <koni.marti@gmail.com>
2022-07-02 18:24:41 +02:00
Moritz Poldrack 5205951149 lint: fix composite literal using unkeyed fields
Fix the following go vet error:

 # git.sr.ht/~rjarry/aerc/worker/notmuch
 worker/notmuch/worker.go:86:19:
 git.sr.ht/~rjarry/aerc/worker/types.Done composite literal uses unkeyed
 fields

Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
2022-06-26 11:23:21 +02:00
Robin Jarry 420f236d31 imap: fix data race on seqMap array
There are concurrent threads that are accessing and modifying
IMAPWorker.seqMap (the mapping of sequence numbers to message UIDs).
This can lead to crashes when trying to add and remove a message ID.

panic: runtime error: index out of range [391] with length 390

goroutine 1834 [running]:
git.sr.ht/~rjarry/aerc/logging.PanicHandler()
	logging/panic-logger.go:47 +0x6de
panic({0xa41760, 0xc0019b3290})
	/usr/lib/golang/src/runtime/panic.go:838 +0x207
git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessages.func1()
	worker/imap/fetch.go:214 +0x185
created by git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessages
	worker/imap/fetch.go:209 +0x12b

Use a map which makes more sense than a simple array for random access
operations. Also, it allows better typing for the key values. Protect
the map with a mutex. Add internal API to access the map. Add basic unit
tests to ensure that concurrent access works.

Fixes: https://todo.sr.ht/~rjarry/aerc/49
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
2022-06-24 21:08:12 +02:00
Tim Culverhouse 7aa71d334b imap: add option to cache headers
Add option to cache headers for imap accounts. Cache db is located at
$XDG_CACHE_DIR/aerc/{account name}. The cache is cleaned of stale
entries when aerc is first opened.

Two new account level configuration options are introduced:
* cache-headers (Default: false)
* cache-max-age (Default: 30 days (720 hours))

The change in worker/imap/open.go is to set the selected directory. This
is required to access the UIDVALIDITY field, which is used in
combination with the message ID to form the key for use in the cache db.
The key structure is: "header.{UIDVALIDITY}.{UID}"

Where reasonable, cache does not stop aerc from running. In general, if
there is an error in the cache, aerc should continue working as usual.
Errors are either displayed to the user or logged.

All messages are stored without flags, and when retrieved have the flags
set to SEEN. This is to prevent UI flashes. A new method to
FetchMessageFlags is introduced to update flags of cached headers. This
is done asynchronously, and the user will see their messages appear and
then any flags updated. The message will initially show as SEEN, but
will update to unread. I considered updating the cache with the
last-known flag state, however it seems prudent to spare the R/W cycle
and assume that - eventually - all messages will end up read, and if it
isn't the update will occur rather quickly.

Note that leveldb puts a lock on the database, preventing multiple
instances of aerc from accessing the cache at the same time.

Much of this work is based on previous efforts by Vladimír Magyar.

Implements: https://todo.sr.ht/~rjarry/aerc/2
Thanks: Vladimír Magyar <vladimir@mgyar.me>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: inwit <inwit@sindominio.net>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-06-22 11:26:13 +02:00
Tim Culverhouse af0e587976 maildir: fix dirinfo.Unseen and Exists counting
The maildir worker was adding Recent messages to the counts of Unseen
and Exists, however these messages were already included in those counts.

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-06-14 22:12:37 +02:00
Koni Marti 955f7683af parse: fix content-type parsing error
If an error occurs when parsing the content-type, check if the
content-type is quoted; if so, remove quotes. If this is not the case,
then return a text/plain content-type as a sane fallback option, so that
the message can be at least viewed in plaintext.

Fixes: https://todo.sr.ht/~rjarry/aerc/44
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-06-07 16:19:27 +02:00
Tim Culverhouse 8b6f9719a8 dirlist: update RUE counts for imap/maildir on move|copy|delete|archive
When moving/copying/deleting/archiving a message in imap, the RUE counts
displayed in the dirlist would not update properly. Maildir has (had) an
implementation that recounts the entire directory and updates the
DirectoryInfo after one of these actions.

This patch implements a more efficient method of updating, and also
enables it to apply to IMAP without any additional requests. Upon
completion of the action, the counts are manually updated with the count
of messages in the action and recent and/or unseen states of those
messages. This is more efficient for maildir, because we aren't counting
everything in the store. For IMAP, we get the updates for free because
we are only performing the update after confirmation from the server
that the action has happened.

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-06-07 16:19:23 +02:00
Tim Culverhouse 2551dd1bfa feat: add background mail polling option for all workers
Check for new mail (recent, unseen, exists counts) with an external
command, or for imap with the STATUS command, at start or on
reconnection and every X time duration

IMAP:
The selected folder is skipped, per specification. Additional config
options are included for including/excluding folders explicitly.

Maildir/Notmuch:
An external command will be run in the background to check for new mail.
An optional timeout can be used with maildir/notmuch. Default is 10s

New account options:
check-mail
check-mail-cmd (maildir/notmuch only)
check-mail-timeout (maildir/notmuch only), default 10s
check-mail-include (IMAP only)
check-mail-exclude (IMAP only)

If unset, or set less than or equal to 0, check-mail will be ignored

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-31 14:32:51 +02:00
Koni Marti b29293d7b5 imap: add timeout to tcp connect functions
Extract the tcp connection details and timeout the tcp connect functions
(net.ResolveTCPAddr and net.DialTCP). If timed out, ensure that the
connection is properly closed.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti b92efe4cd9 imap: add debouncer to the idler
Add a debouncer to the idle mode. Avoid unnecessary idling when another
job arrives within a certain time frame.

For example, the ui sends three messages to the worker at the same time
when we open a message (FlagMessage, FetchMessageBodyPart, and the
FetchMessageHeaders). The debouncer prevents the unnecessary entering
and leaving of the idle mode between those messages.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti e5b339702a imap: monitor the logout channel with an observer
Untangle the observer functionality from the message handling routine.
Observe the imap client's logout channel and trigger a connection error
when necessary to start the reconnect cycle.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti 397a6f267f imap: manage idle mode with an idler
Untangle the idle functionality from the message handling routine. Wait
for the idle mode to properly exit every time to ensure a consistent
imap state. Timeout when hanging in idle mode and inform the ui.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti 4d75137b20 imap: extract imap config and configure handling
Extract the imap config and move the configure part out of the message
handler.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti 8f976af17b imap: fix out-of-range panic for imap updates
Check slice bounds before using it for the message and expunge updates.
Log the error but ignore the affected updates.

Link: https://lists.sr.ht/~rjarry/aerc-devel/%3CCJEHBFFUI11T.1AYGOMVGZ87ZS%40rek2system%3E
Reported-by: ReK2 <rek2@hispagatos.org>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-04-29 13:41:35 +02:00
Robin Jarry 8ed95b0d2a imap: avoid crash when replying to unread message
When running `:reply -q` on an unread message, aerc crashes after
opening the editor:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x5d1019]

goroutine 63 [running]:
bufio.(*Reader).fill(0xc000086ef8)
	/usr/lib/golang/src/bufio/bufio.go:106 +0xd9
bufio.(*Reader).Peek(0xc00020bef8, 0x1)
	/usr/lib/golang/src/bufio/bufio.go:144 +0x5d
github.com/emersion/go-message/textproto.ReadHeader(0xc00004a700?)
	emersion/go-message@v0.15.0/textproto/header.go:525 +0x5f
git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessageBodyPart.func1(0xc00056e280)
	worker/imap/fetch.go:99 +0x1ab
git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessages.func1()
	worker/imap/fetch.go:178 +0xd7
created by git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessages
	worker/imap/fetch.go:172 +0x12b

This happens because the flags are updated in the callback that receives
the message itself. It causes the flag update to arrive in the same
channel/request. Ignore the messages that have an empty body (i.e. only
containing flag updates). This is inherently racy but there seems no way
to get rid of these extra messages.

Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Connor Kuehl <cipkuehl@gmail.com>
2022-04-28 17:55:57 +02:00
Koni Marti 698c0957d7 pgp: ensure CRLF line endings in pgpmail reader
Ensure CRLF line endings in the pgpmail reader. Fix the pgp signature
verification for maildir and notmuch.

These backends do not return the full message body with CRLF
line endings. But the accepted OpenPGP convention is for signed data to
end with a <CR><LF> sequence (see RFC3156).

If this is not the case the signed and transmitted data are considered
not the same and thus signature verification fails.

Link: https://datatracker.ietf.org/doc/html/rfc3156

Reported-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
2022-04-25 12:31:33 +02:00
Moritz Poldrack ae83373fa6 logging: added a log on panic
Since panics still regularly "destroy" the terminal, it is hard to get a
stack trace for panics you do not anticipate. This commit adds a panic
handler that automatically creates a logfile inside the current working
directory.

It has to be added to every goroutine that is started and will repair
the terminal on a panic.

Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-03-23 20:56:09 +01:00
Moritz Poldrack e30bd324a3 go vet: composite literal uses unkeyed fields
This commit fixes all occurrences of the abovementioned lint-error in
the codebase.

Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-03-18 13:33:16 +01:00
Koni Marti 65ae87a524 threading: honor user-defined sort criteria
Apply the user-defined sort criteria to the message with the highest
uid in a threaded discussion. Restore the default sort order when
leaving threading mode.

Commit 7811620eb8 ("threading: implement on-the-fly message
threading") introduced message threading with the threaded messages
being only sorted by their message uids irrespective of the defined sorting
criteria. It did not restore the default sort order either.

Reported-by: Sebastien Binet <s@sbinet.org>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-03-09 00:08:26 +01:00
wagner riffel 55ae3d2cab maildir: fix data race in maildir worker
Fix a data race due dirInfo pointer being read in the main goroutine
by NewMessageStore and written in the anonymous goroutine launched in
Worker.getDirectoryInfo.

To address the issue raised in https://todo.sr.ht/~rjarry/aerc/16, we
use readdir(3) once, parse and cache its results, this replaces
go-maildir library Dir.Flags based  stat(3) and filepath.Glob
causing the issue when N (emails) is large.

Signed-off-by: wagner riffel <w@104d.net>
2022-03-07 10:01:53 +01:00
Koni Marti 7811620eb8 threading: implement on-the-fly message threading
implement message threading on the message store level using the
jwz algorithm. Build threads on-the-fly when new message headers arrive.

Use the references header to create the threads and the in-reply-to
header as a fall-back option in case no references header is present.

Does not run when the worker provides its own threading (e.g. imap
server threads).

Include only those message headers that have been fetched and are
stored in the message store.

References: https://www.jwz.org/doc/threading.html
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Inwit <inwit@sindominio.net>
Tested-by: akspecs <akspecs@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-02-24 13:00:12 +01:00
Koni Marti 5eac8d603e thread: add method to append new node
implement a method function for a *types.Thread receiver to append
a new node to its linked list.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-23 21:09:01 +01:00
Koni Marti 48e01bd51f imap: start reconnect when initial connect fails
Start the reconnect cycle when the initial connect fails. Make the
connection observer send a connection error when the imap client is nil.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-20 22:04:16 +01:00
Koni Marti 11a4d5b71c imap: reconnect with exponential backoff
waits an increasing amount of time before attempting a reconnect.
Wait is capped at 16s. Prevents many reconnect attemps in a short time period.

Fixes commit 05ad96a30c ("imap: improve reconnect stability") that
improved the reliability of the reconnect mechanism but did not
implement controls to prevent the triggering of too many reconnects
within a short period of time.

Fixes: 05ad96a30c ("imap: improve reconnect stability")
Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-19 15:24:55 +01:00
Koni Marti 05ad96a30c imap: improve reconnect stability
improves the robustness of the imap reconnect feature which was
introduced in commit beae17a6da ("imap: auto-reconnects on connection
error").

If a connection error is emitted, the message list is cleared and a
corresponding error message is shown in the ui. Status bar is updated as
well. Upon reconnect, the directories and the message list will be
re-fetched (same behavior as the connect command).

Reconnect can be enabled and disabled with the connect and the
disconnect commands.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-06 17:06:20 +01:00
Robin Jarry a46468521b imap: do not abort on malformed messages
Some messages contain invalid headers, textproto.ReadHeader fails with
an error:

  malformed MIME header key: From nobody Fri Jan 14 19

And all other messages of the selected folder are ignored.

Return an explicit error message to the UI and continue processing other
messages. This is consistent with what the maildir worker does.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-02-06 16:04:15 +01:00
Robin Jarry 622802d3a5 maildir: defer the count of recent messages
Since commit 01c96e78df ("Update DirectoryInfo handling for maildir"),
flags are checked for every message of a folder when entering it.
Iterating over all messages of a folder takes a long time for large
collections of emails.

Only count the number of messages and state that the directory info
counts are not accurate. Defer the parsing of message flags in
a goroutine to have a more responsive UI.

Fixes: https://todo.sr.ht/~rjarry/aerc/16
Suggested-by: Koni Marti <koni.marti@gmail.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-01-30 20:49:30 +01:00
Nguyễn Gia Phong bf4abd309e maildir,notmuch: pass in-memory message to callback
This fixes piped full message (:pipe -m) being empty.

Fixes: 904ffacb0e ("maildir,notmuch: avoid leaking open files")
Signed-off-by: Nguyễn Gia Phong <mcsinyx@disroot.org>
2022-01-20 15:44:46 +01:00
Nguyễn Gia Phong 904ffacb0e maildir,notmuch: avoid leaking open files
Previously, Message.NewReader returned the wrapped buffered reader
without a reference to the opened file, so the files descriptors
were left unclosed after reading.  Now, the file reader is returned
directly and closed on the call site.  Buffering is not needed here
because it is an implementation detail of go-message.

Fixes: https://todo.sr.ht/~rjarry/aerc/9
2022-01-19 20:18:00 +01:00
Koni Marti beae17a6da imap: auto-reconnects on connection error
if the worker emits a connection error, the ui will automatically send back a
reconnect command. The worker then establishes a new connection. Auto-reconnect
is disabled when the user sends the disconnect command.

Fixes: https://todo.sr.ht/~rjarry/aerc/1
Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-01-19 17:40:52 +01:00
Koni Marti 1ace50a6b9 imap: emits connection error on logout
implements a new connection error message. This allows the worker to emit a
connection-related error message to the ui when the imap client closes the
loggedOut channel.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-01-19 17:34:42 +01:00
Koni Marti 022bf1a11f imap: fix panic when sending multiple connect cmds
fixes the panic when the user sends multiple connect commands and is
already connected. The panic is caused by closing an already closed
channel. This happens when the idle re-init code is not executed, e.g.
when there's a return statement in the switch block.  A defer func()
before the switch block will prevent this. The existing behavior of only
creating a new idleStop channel when properly connected is preseverd.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-01-19 17:34:41 +01:00
Robin Jarry 899d1cca3c Revert "imap: attempt automatic reconnection on error"
This reverts commit c605ada3dd.

This breaks reading message bodies. I am not sure why, I'll take some
time to fix it later.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-01-15 12:13:47 +01:00
Robin Jarry c605ada3dd imap: attempt automatic reconnection on error
Attempt to reconnect to the server when there is an unexpected
disconnection or network error.

Use the Client.LoggedOut() channel which is closed when the connection
is closed.

This patch is rather flaky and is certainly bugged. However, it is
a start.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-01-14 21:17:40 +01:00
Robin Jarry b331371a65 format: reformat code with go 1.17
There was a change in how build tags are formatted. Use this as new
reference.

Link: https://go.dev/doc/go1.17#gofmt
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-11 23:24:08 +01:00
Robin Jarry 15a4cc7d0a imap: fix build on macos
Fix the following build error on mac os:

 worker/imap/worker.go:368:29: undefined: syscall.TCP_KEEPCNT
 worker/imap/worker.go:376:29: undefined: syscall.TCP_KEEPINTVL

These symbols are not defined on darwin.

Fixes: 5dfeff75f3 ("imap: add tcp connection options")
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-11 21:45:51 +01:00
Robin Jarry 5dfeff75f3 imap: add tcp connection options
Allow fine tuning tcp connection options.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-07 21:28:13 +01:00
Robin Jarry 33aaf94663 imap: use low level tcp connection
In preparation for tcp keepalive options, we need access to the
net.TCPConn object associated with an IMAP connection. The only way to
do this is to create the connection ourselves.

No functional change.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-07 21:27:41 +01:00
Robin Jarry cc432eefd3 imap: move connect procedure into a dedicated function
This will prepare for extra tcp connection options support and for
automatic reconnect. No functional change.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-07 21:26:56 +01:00
Robin Jarry abcd327359 maildir: watch for message renames
Messages flags can also be changed without changing directories.
Changing flags in maildirs means renaming the message files. Also take
renames into account.

Link: https://cr.yp.to/proto/maildir.html
Fixes: f4d3c8fc77 ("maildir: watch for external changes")
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-06 23:50:04 +01:00
Robin Jarry f4d3c8fc77 maildir: watch for external changes
When a maildir is synchronized by an external process while aerc is
running (e.g. mbsync), some emails may be moved out of "new" to "cur" or
completely deleted.

These deletions are ignored and aerc may assume these messages are still
here, leading to errors.

Take file deletions into account. Also, add "cur" to the watched
folders since these can be modified as well.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-06 20:54:29 +01:00
Robin Jarry 6ddfc23e61 imap: fix segfault when disconnecting
Do not set client = nil, it breaks almost all message handlers:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8c7e51]
goroutine 25 [running]:
  git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleListDirectories
    worker/imap/list.go:32
  git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleMessage
    worker/imap/worker.go:174
  git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).Run
    worker/imap/worker.go:264
  created by git.sr.ht/~rjarry/aerc/widgets.NewAccountView
    widgets/account.go:85 +0x518

Simply leave the disconnected client object, it already returns explicit
error messages.

Fixes: e41ed82cf3 ("imap: add manual {dis,}connect support")
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-22 21:32:56 +01:00
Kalyan Sriram 402612fd97 notmuch: allow sort by file order
When using the notmuch backend, it often makes more sense to sort
folders (actual virtual folders, or queries) by the order specified in
the query-map file, rather than alphabetically. This patch introduces a
configuration option (disabled by default) that allows this.

Additionally, due to the notmuch backend previously using maps (which
are order-undefined) to store the list of queries, default query
selection on aerc startup fluctuated. This patch fixes that by using
slices to store query order.
2021-11-13 15:25:04 +01:00
Kalyan Sriram d7bda10f3e notmuch: allow empty searches
Don't error on empty search terms, just don't filter
by anything and return all items matching the current querymap
2021-11-13 15:20:54 +01:00
y0ast dc2a2c2dfd messages: allow displaying email threads
Display threads in the message list. For now, only supported by the
notmuch backend and on IMAP when the server supports the THREAD
extension.

Setting threading-enable=true is global and will cause the message list
to be empty with maildir:// accounts.

Co-authored-by: Kevin Kuehler <keur@xcf.berkeley.edu>
Co-authored-by: Reto Brunner <reto@labrat.space>
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-13 15:05:59 +01:00
Robin Jarry e41ed82cf3 imap: add manual {dis,}connect support
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-05 10:45:31 +01:00
Robin Jarry 0d645bcebd go.mod: change base git url
I'm not sure what are the implications but it seems required.

Link: https://github.com/golang/go/issues/20883
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-05 10:21:45 +01:00
Robin Jarry 4cf0ea2a05 imap: use builtin idle support
go-imap supports IDLE since 1.2.0. Remove dependency to go-imap-idle.

Link: https://github.com/emersion/go-imap/commit/ac3f8e195ef1b6d
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-02 11:24:33 +01:00
Reto Brunner bc48628839 maildir: track the recent flag correctly
In the maildir worker we manually need to track the Recent flag in order for the
notification command etc to work.

Push that responsibility to the container, we must make sure to manually add the
flag though if one grabs the message info.
2021-11-01 13:30:45 +01:00
Reto Brunner 1687e558d3 notmuch/maildir: remove double emit of the dirinfo
There was some bug which could be worked around by double emitting an event.
However that proofed to be brittle:

We send the first message here from the worker goroutine:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/maildir/worker.g=
o#L306

Then Tick() is waked in the main goroutine and calls ProcessMessage:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/widgets/account.go#L100

ProcessMessage in the main goroutine reads types.Message state with
msg.getId() and msg.InResponseTo():
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/types/worker.go#=
L74-76

Meanwhile in the worker goroutine we call PostMessage for a second
time with a pointer that points to the *same* previous message that
ProcessMessage is reading:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/maildir/worker.g=
o#L306

The second PostMessage call makes writes to message while
ProcessMessage in the main goroutine is possibly reading:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/types/worker.go#=
L59

This led to a data race in the event loop

Reported-By: Wagner Riffel <w@104d.net>
2021-04-28 07:54:16 +02:00
Reto Brunner fda424bebc lib/parse: simplify parseAddressList 2021-02-22 11:47:31 +01:00
Reto Brunner 8ecf0b73f4 maildir: don't read the full file into memory 2021-02-08 09:06:42 +01:00
Reto Brunner db6848fe1b notmuch: don't read the full file into memory 2021-02-08 08:40:07 +01:00
Kalyan Sriram b56a688589 notmuch: trim excluded tags
Trims whitespace in list of excluded notmuch tags. This allows a comma
separated list with spaces to be correctly processed.
2020-12-02 09:08:19 +01:00
Reto Brunner a9330f4c63 notmuch: remove gc close hooks
We frequently had issues with notmuch segfaulting and my guess is that this
was due to the garbage collection magic used in the module.

This changes to a fork that ripped the functionality out.
2020-12-02 09:06:34 +01:00
Reto Brunner daf214a1f8 imap: strip <> from message-ids 2020-11-14 15:40:13 +01:00
Reto Brunner 18b0ea571a lib/parse: use go-message msgid parsing if it succeeds 2020-11-14 15:40:13 +01:00
Reto Brunner fc9ccc3000 remove models.Address in favor of go-message Address
We made a new type out of go-message/mail.Address without any real reason.
This suddenly made it necessary to convert from one to the other without actually
having any benefit whatsoever.
This commit gets rid of the additional type
2020-11-14 15:40:13 +01:00
Reto Brunner b6bcf89784 imap: add sort support 2020-10-11 09:18:45 +02:00
Reto Brunner c48f228fa5 notmuch: rename method to SetFlag 2020-09-27 19:00:58 +02:00
Reto Brunner 6654c970e6 notmuch: close tag object 2020-09-24 21:24:44 +02:00
Jeff Martin 0acb28645f handle message unknown charset error
This change handles message parse errors by printing the error when the
user tries to view the message. Specifically only handling unknown
charset errors in this patch, but there are many types of invalid
messages that can be handled in this way.

aerc currently leaves certain messages in the msglist in the pending
(spinner) state, and I'm unable to view or modify the message. aerc also
only prints parse errors with message when they are initially loaded.
This UX is a little better, because you can still see the header info
about the message, and if you try to view it, you will see the specific
error.
2020-08-31 22:00:28 +02:00
Reto Brunner c846307144 base models.Address on the mail.Address type
This allows us to hook into the std libs implementation of parsing related stuff.
For this, we need to get rid of the distinction between a mailbox and a host
to just a single "address" field.

However this is already the common case. All but one users immediately
concatenated the mbox/domain to a single address.

So this in effects makes it simpler for most cases and we simply do the
transformation in the special case.
2020-08-20 19:18:57 +02:00
ARaspiK fe1cabb077 Add support for :rmdir
The `:rmdir` command removes the current directory (`-f` is required if
the directory is not empty).

This is not supported on the notmuch backend.

An issue with the maildir backend is that some sync programs (e.g.
offlineimap) may recover the directory after it is deleted.  They need
to specifically be configured to accept deletions, or special commands
need to be executed (e.g. `offlineimap --delete-folder`) to properly
delete folders.

A danger of using this on the IMAP backend is that it is possible for a
new message to be added to the directory and for aerc to not show it
immediately (due to a slow connection) - using `:rmdir` at this moment
(with `-f` if the directory already contains messages) would delete the
directory and the new message that just arrived (and all other
contents).  This is documented in aerc(1) so that users are aware of
possible risks.
2020-08-19 11:38:57 +02:00
Reto Brunner f1a0fd20d6 improve date parsing for notmuch/maildir
If a message date would fail to parse, the worker would never receive
the MessageInfo it asked for, and so it wouldn't display the message.

The problem is the spec for date formats is too lax, so trying to ensure
we can parse every weird date format out there is not a strategy we want
to pursue. On the other hand, preventing the user from reading and
working with a message due to the error format is also not a solution.

The maildir and notmuch workers will now fallback to the internal date, which
is based on the received header if we can't parse the format of the Date header.

The UI will also fallback to the received header whenever the date header can't
be parsed.

This patch is based on the work done by Lyudmil Angelov <lyudmilangelov@gmail.com>
But tries to handle a parsing error a bit more gracefully instead of just returning
the zero date.
2020-08-10 08:00:52 +02:00
Reto Brunner 7a9fe3824f notmuch: manually close notmuch objects
There seems to be some race with the automatic closing that should happen
in theory... close it manually where we can to avoid the issue
2020-08-07 09:49:57 +02:00
Tero Koskinen f81cc2803c maildir: Provide nicer error message on invalid url
If accounts.conf contains an invalid maildir url, return a nice
error instead of panicking.

Log a couple of different error cases to provide extra
information about the error to the user.
2020-07-27 18:26:57 +02:00
Reto Brunner c574a838fa Remove hard coded bodystruct path everywhere
Aerc usually used the path []int{1} if it didn't know what the proper path is.
However this only works for multipart messages and breaks if it isn't one.

This patch removes all the hard coding and extracts the necessary helpers to lib.
2020-07-27 09:19:27 +02:00
Tobias Wölfel 494bd674a9 Add flag based search options
Provide search and filter with the option to specify more flag based
conditions.
Use '-x <flag>' to search for messages with a flag (seen, answered,
flagged) and '-X <flag>' to search for messages without a flag.
2020-07-25 08:08:08 +02:00
Andrew Jeffery 126c9437e8 Only send directory info once from maildir
The directory info only needs to be sent once for all the messages, this
reduces unnecessary messages being sent which could lock up the ui.
2020-07-19 11:16:05 +02:00
Reto Brunner 8446d48664 run go fmt 2020-07-17 17:50:24 +02:00
Lyudmil Angelov cc44027545 Make it easier to debug date parsing errors
When message dates failed to parse, the error displayed would try to
include the time object it failed to obtain, which would display as
something like 0001-01-01 00:00:00 UTC, which isn't of much help.

Instead, display the text we were trying to parse into a date, which
makes the problem easier to debug.
2020-07-11 13:51:32 +02:00
ARaspiK 0535f6333f Add additional flagging functionality
More mail flags can now be set, unset, and toggled, not just the
read/seen flag.

This functionality is implemented with a new `:flag` and `:unflag`
command, which are extensions to the matching `:read` and `:unread`
commands, adding support for different flags.  In fact, the
`read`/`unread` commands are now recognized aliases to `flag`/`unflag`.
The new commands are also well documented in aerc(1).

The change mostly extends the previous read/unread setting functionality
by adding a selection for the flag to change.
2020-07-08 09:13:03 +02:00
Reto Brunner 3e6189f243 notmuch: fix docstring 2020-07-08 01:02:46 +02:00
Reto Brunner 5f9c005872 notmuch: handle the answered flag 2020-07-05 15:40:12 +02:00