The worker uses a buffered channel to queue tasks. Buffered channels
are effective at FIFO, but are prone to blocking. The design of aerc is
such that the UI must always accept a response from the backends, and
the backends must always accept a request from the UI. By using buffered
channels for both of these communication channels, a deadlock will
occur.
Break the chain by using a doubly linked list (container/list from the
standard library) to queue tasks for the worker. Essentially, this is an
infinitely buffered channel - but more memory efficient as it can change
size dynamically.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Subsitute the format specifier %w for %v in the logging facility. The
logging functions use a fmt.Sprintf call behind the scene which does not
recognize %w. %w should be used in fmt.Errorf when you want to wrap
errors. Hence, the log entries that use %w are improperly formatted like
this:
ERROR 2022/10/02 09:13:57.724529 worker.go:439: could not get message
info %!w(*fmt.wrapError=&{could not get structure: [snip] })
^
Links: https://go.dev/blog/go1.13-errors
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Commit 716ade8968 ("worker: lock access to callback maps") introduced
locks to the worker callback maps. The locks also locked the processing
of the callback, which had the unintended side effect of deadlocking the
worker if any callbacks attempted to post a new action or message.
Refactor the locks to only lock the worker while accessing the maps.
Fixes: 716ade8968 ("worker: lock access to callback maps")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Pass message containing remaining directories to check. Account widget
will recursively call CheckMail with the remaining directories until
a Done message is returned.
Only needed for IMAP worker as other workers run check-mail-cmd in
a separate goroutine.
Suggested-By: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: kt programs <ktprograms@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Worker callbacks are inherently set and called from different
goroutines. Protect access to all callback maps with a mutex.
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
When an error is encountered fetching a header, the backends respond
with a type.Error worker message. On receipt of this message, the UI
deletes all pending headers. The headers are all requested again as they
remain on the screen, resulting in an infinite request loop - and an
infinite logging loop. The user only ever sees the spinner unless they
check the logs.
A previous commit intended to fix this, however it introduced a
regression where any message that was part of the fetch request would
also be marked as erroneous. This commit is reverted with commit
2aad2fea7d36 ("msgstore: revert 9fdc7acf5b48").
Send an erroneous message info message from the backend when an error is
encountered for a specific UID.
Fixes: 01f80721e2 ("msgstore: post MessageInfo on erroneous fetch")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
When cached headers are fetched, an action is posted back to the Worker
to immediately fetch the flags for the message from the server (we can't
know the flags state, therefore it's not cached). When scrolling, a lag
occurs when loading cached headers because the n+1 message has to wait
for the flag request to return before the cached headers are retrieved.
Collect the message UIDs in the UI that need flags, and fetch them based
off a debounce timer in a single request. Post the action from the UI to
eliminate an (ugly) go routine in the worker.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
This reverts commit 7473571159.
The commit has introduced a regression that lead to the pager not being
filled with content, thereby making reading mails impossible.
Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
Send to worker.Messages in goroutine to prevent deadlocks: the UI can
fill the worker.Actions channel. The worker can generate more than one
Message per action, and if it generates enough to fill the
worker.Messages channel from a single message while the worker.Actions
channel is full, a deadlock occurs.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The IMAP worker has a few methods that post a new Action to itself. This
can create a deadlock when the worker.Actions channel is full: The
worker can't accept a new Action because it's trying to post an action.
This is most noticeable when cached headers are enabled and the message
list is scrolled fast.
Use a goroutine to post actions to the worker when posting from within
the worker.
Fixes: https://todo.sr.ht/~rjarry/aerc/45
Fixes: 7aa71d334b ("imap: add option to cache headers")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Introduce a MoveMessages worker message to use for improved message
moving in subsequent patches.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Mark or unmark the shown message threads. Threads must be available in the
message store. Use the -T option for the mark or unmark commands. Can be
used in combination with the toggle flag (-t).
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Error wrapping as introduced in Go 1.13 adds some additional logic to
use for comparing errors and adding information to it.
Signed-off-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
Do not pass logger objects around anymore. Shuffle some messages to make
them consistent with the new logging API. Avoid using %v when a more
specific verb exists for the argument types.
The loggers are completely disabled (i.e. Sprintf is not even called)
by default. They are only enabled when redirecting stdout to a file.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
This patch enables the filtering of a threaded view which uses
server-built threads. Filtering is done server-side, in order to
preserve the use of server-built threads.
In adding this feature, the filtering of notmuch folders was brought up
to feature parity with the other workers. The filters function the same
(ie: they can be stacked). The notmuch filters, however, still use
notmuch syntax for the filtering.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add option to cache headers for imap accounts. Cache db is located at
$XDG_CACHE_DIR/aerc/{account name}. The cache is cleaned of stale
entries when aerc is first opened.
Two new account level configuration options are introduced:
* cache-headers (Default: false)
* cache-max-age (Default: 30 days (720 hours))
The change in worker/imap/open.go is to set the selected directory. This
is required to access the UIDVALIDITY field, which is used in
combination with the message ID to form the key for use in the cache db.
The key structure is: "header.{UIDVALIDITY}.{UID}"
Where reasonable, cache does not stop aerc from running. In general, if
there is an error in the cache, aerc should continue working as usual.
Errors are either displayed to the user or logged.
All messages are stored without flags, and when retrieved have the flags
set to SEEN. This is to prevent UI flashes. A new method to
FetchMessageFlags is introduced to update flags of cached headers. This
is done asynchronously, and the user will see their messages appear and
then any flags updated. The message will initially show as SEEN, but
will update to unread. I considered updating the cache with the
last-known flag state, however it seems prudent to spare the R/W cycle
and assume that - eventually - all messages will end up read, and if it
isn't the update will occur rather quickly.
Note that leveldb puts a lock on the database, preventing multiple
instances of aerc from accessing the cache at the same time.
Much of this work is based on previous efforts by Vladimír Magyar.
Implements: https://todo.sr.ht/~rjarry/aerc/2
Thanks: Vladimír Magyar <vladimir@mgyar.me>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: inwit <inwit@sindominio.net>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
When moving/copying/deleting/archiving a message in imap, the RUE counts
displayed in the dirlist would not update properly. Maildir has (had) an
implementation that recounts the entire directory and updates the
DirectoryInfo after one of these actions.
This patch implements a more efficient method of updating, and also
enables it to apply to IMAP without any additional requests. Upon
completion of the action, the counts are manually updated with the count
of messages in the action and recent and/or unseen states of those
messages. This is more efficient for maildir, because we aren't counting
everything in the store. For IMAP, we get the updates for free because
we are only performing the update after confirmation from the server
that the action has happened.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Check for new mail (recent, unseen, exists counts) with an external
command, or for imap with the STATUS command, at start or on
reconnection and every X time duration
IMAP:
The selected folder is skipped, per specification. Additional config
options are included for including/excluding folders explicitly.
Maildir/Notmuch:
An external command will be run in the background to check for new mail.
An optional timeout can be used with maildir/notmuch. Default is 10s
New account options:
check-mail
check-mail-cmd (maildir/notmuch only)
check-mail-timeout (maildir/notmuch only), default 10s
check-mail-include (IMAP only)
check-mail-exclude (IMAP only)
If unset, or set less than or equal to 0, check-mail will be ignored
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
Apply the user-defined sort criteria to the message with the highest
uid in a threaded discussion. Restore the default sort order when
leaving threading mode.
Commit 7811620eb8 ("threading: implement on-the-fly message
threading") introduced message threading with the threaded messages
being only sorted by their message uids irrespective of the defined sorting
criteria. It did not restore the default sort order either.
Reported-by: Sebastien Binet <s@sbinet.org>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
implement message threading on the message store level using the
jwz algorithm. Build threads on-the-fly when new message headers arrive.
Use the references header to create the threads and the in-reply-to
header as a fall-back option in case no references header is present.
Does not run when the worker provides its own threading (e.g. imap
server threads).
Include only those message headers that have been fetched and are
stored in the message store.
References: https://www.jwz.org/doc/threading.html
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Inwit <inwit@sindominio.net>
Tested-by: akspecs <akspecs@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
if the worker emits a connection error, the ui will automatically send back a
reconnect command. The worker then establishes a new connection. Auto-reconnect
is disabled when the user sends the disconnect command.
Fixes: https://todo.sr.ht/~rjarry/aerc/1
Signed-off-by: Koni Marti <koni.marti@gmail.com>
implements a new connection error message. This allows the worker to emit a
connection-related error message to the ui when the imap client closes the
loggedOut channel.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Display threads in the message list. For now, only supported by the
notmuch backend and on IMAP when the server supports the THREAD
extension.
Setting threading-enable=true is global and will cause the message list
to be empty with maildir:// accounts.
Co-authored-by: Kevin Kuehler <keur@xcf.berkeley.edu>
Co-authored-by: Reto Brunner <reto@labrat.space>
Signed-off-by: Robin Jarry <robin@jarry.cc>
The `:rmdir` command removes the current directory (`-f` is required if
the directory is not empty).
This is not supported on the notmuch backend.
An issue with the maildir backend is that some sync programs (e.g.
offlineimap) may recover the directory after it is deleted. They need
to specifically be configured to accept deletions, or special commands
need to be executed (e.g. `offlineimap --delete-folder`) to properly
delete folders.
A danger of using this on the IMAP backend is that it is possible for a
new message to be added to the directory and for aerc to not show it
immediately (due to a slow connection) - using `:rmdir` at this moment
(with `-f` if the directory already contains messages) would delete the
directory and the new message that just arrived (and all other
contents). This is documented in aerc(1) so that users are aware of
possible risks.
More mail flags can now be set, unset, and toggled, not just the
read/seen flag.
This functionality is implemented with a new `:flag` and `:unflag`
command, which are extensions to the matching `:read` and `:unread`
commands, adding support for different flags. In fact, the
`read`/`unread` commands are now recognized aliases to `flag`/`unflag`.
The new commands are also well documented in aerc(1).
The change mostly extends the previous read/unread setting functionality
by adding a selection for the flag to change.
- Add maildir flags to complement a messages imap flags
- Set the "seen" flag on sent messages when using the maildir backend
- Cleanup AppendMessage interface to use models.Flag for both IMAP and
maildir
Actions such as read / unread or the addition of new messages do change
the read/unread/recent count. Hence we request an update from the workers.
Workers going over the network should probably cache the information and invalidate
it only if necessary
Previously the workers returned a mixture of decoded / encoded parts.
This lead to a whole bunch of issues.
This commit changes the msgviewer and the commands to assume parts to already
be decoded
There is a command and config option. The criteria are a list of the
sort criterion and each can be individually reversed.
This only includes support for sorting in the maildir backend currently.
The other backends are not supported in this patch.
A sequence-set is an IMAP-specific implementation detail. Throughout the
UI, aerc simply operates using lists of opaque identifiers. In order to
loosen the coupling between the UI and IMAP in particular, replace most
usages of imap.SeqSet with []uint32, leaving the translation to a SeqSet
to the IMAP backend as needed.
Before, the information needed to display different parts of the UI was
tightly coupled to the specific messages being sent back and forth to
the backend worker. Separating out a models package allows us to be more
specific about exactly what a backend is able to and required to
provide for the UI.
Adds an archive command that moves the current message into the folder
specified in the account config entry.
Supports three layouts at this point:
- flat: puts all messages next to each other
- year: creates a folder per year
- month: same as above, plus folders per month
This also adds a "-p" argument to "cp" and "mv" that works like
"--parents" on mkdir(1). We use this to auto-create the directories
for the archive layout.
