Refactor the filtering and paging logic to use several fewer goroutines.
Fixes data race condition with the timing of filter -> pager ->
terminal, can be found when switching message views fast.
Check if filter -> pager process is currently running before calling it
to start again. Fixes data race between fetching message body and
terminal starting. Both can initiate the copying process, and for long
running filters and fast message fetching, it is possible to have
fetched a message but still be running the filter when the terminal
starts.
Move StripAnsi to it's own file in lib/parse, similar to the hyperlinks
parser.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Prevent the embarrassing forgotten attachment scenario by warning the
user before sending a message that may need an attachment but does not
have one. Whether a message needs an attachment is determined by testing
a configurable regex against the message body.
Signed-off-by: Jason Cox <dev@jasoncarloscox.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Fix out-of-bounds panic by updating the focused int variable when
headers change in the switch-account commands.
Fixes: d371c1ac8 ("commands: add switch-account command for composer")
Reported-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
In order to make things easier for newcomers from notmuch, add a tag command
which is just an alias for modify-labels.
Signed-off-by: inwit <inwit@sindominio.net>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
These two commands have virtually zero in common. Move open-link in its
own file.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Export AERC_MIME_TYPE and AERC_FILENAME in the filters command
environment. This allows dynamic coloring with tools that require
a filename and/or a mime type to determine the syntax.
Update docs and add example use in the default config file.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Now that the share/filters folders are in $PATH when running the
commands, let's reference the scripts by their name.
Add more filter examples, some of them using the built-in filters, some
of them not...
Suggested-by: Teo Luppi <me@luppi.uk>
Suggested-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
To allow referencing built-in filters without hard coding a path during
installation, append the following folders to the exec PATH when running
the filter commands:
~/.config/aerc/filters
~/.local/share/aerc/filters
$PREFIX/share/aerc/filters
/usr/share/aerc/filters
If a filter script has the same name than a command in the default exec
PATH, it will not shadow it. In that edge case, the absolute path to the
filter script must be specified.
Suggested-by: Teo Luppi <me@luppi.uk>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
The MouseEvent method of the composer passes on the mouse event to it's
underlying grid while the composer is locked. The underlying grid then
passes on the mouse event to child objects of the grid, which are
referenced via fields of the composer (c.editor is a field in composer
but a child of c.grid, for example). When the grid attempts to pass on
the mouse event, it is referencing a pointer which is locked, and a
deadlock occurs due to the original lock in composer.MouseEvent.
Unlock before calling the grid.MouseEvent, and lock the composer again
after it is called.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Some programs like Skanpage allow sharing files via email and attaching
them automatically from the mailto: link.
This patch introduces parsing of the attach query argument in mailto
links and attaches the listed files.
A potential file:// URL has it's prefix removed.
Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
Remove invalidatable type and all associated calls. All items can
directly invalidate the UI.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
AddEditor acquires the lock and calls FocusEditor which also attempts to
acquire it. Since the lock is not re-entrant, it ends in deadlock.
Add an internal focusEditor fonction that does not acquire the lock.
Fixes: bf2bf8c242 ("compose: prevent out of bounds access")
Reported-by: Moritz Poldrack <moritz@poldrack.dev>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Thomas Vigouroux <thomas.vigouroux@protonmail.com>
The algorithm is broken, there may be more than one header editor with
focused=true. Reset the focused flag before forwarding the mouse event
to the composer grid.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
The vulnerability database is evolving with time. It can cause the lint
step to fail suddenly without any source code changes on our side.
Moreover, sometimes, there is nothing we can do to fix the issue nor to
silence that specific error.
Found 1 known vulnerability.
Vulnerability #1: GO-2022-1039
Programs which compile regular expressions from untrusted
sources may be vulnerable to memory exhaustion or denial of
service. The parsed regexp representation is linear in the size
of the input, but in some cases the constant factor can be as
high as 40,000, making relatively small regexps consume much
larger amounts of memory. After fix, each regexp being parsed is
limited to a 256 MB memory footprint. Regular expressions whose
representation would use more space than that are rejected.
Normal use of regular expressions is unaffected.
Call stacks in your code:
config/config.go:1000:46:
git.sr.ht/~rjarry/aerc/config.AercConfig.LoadBinds calls
regexp.Compile, which eventually calls regexp/syntax.Parse
Found in: regexp/syntax@go1.18.6
Fixed in: regexp/syntax@go1.19.2
More info: https://pkg.go.dev/vuln/GO-2022-1039
Move govulncheck into its own make target to be executed manually.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
The Invalidatable struct is designed so that a widget can have a
callback function ran when it is Invalidated. This is used to cascade up
the widget tree, marking things as Invalid along the way so that only
Invalid widgets are drawn. However, this is only implemented at the grid
cell level for checks if the cell is invalidated -- and the grid cells
are never set back to a "valid" state. The effect of this is that no
matter what is invalidated, the entire UI gets drawn again.
The calling through the Invalidate callbacks creates *several* race
conditions, as Invalidate is called from several different goroutines,
and many widgets call invalidate on their parent or children.
Tcell has optimizations to only rerender screen cells that have changed
their rune and style. The only performance penalty by redrawing the
entire screen for aerc is the operations *within the aerc draw methods*.
Most of these are not expensive and have relatively no impact on
performance.
Skip all of the OnInvalidates, and directly invalidate the UI when
DoInvalidate is called by a widget. This reduces data races, and
simplifies the widget redraw logic signficantly.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The terminal widget uses it's own redraw logic to improve performance.
With the addition of a main event loop, the redraw logic can happen in
the main loop via the standard Invalidate logic.
Use the Invalidate method to mark aerc invalid, and immediately trigger
a redraw with ui.QueueRedraw. The follow up call to QueueRedraw is
needed because the terminal update happens in a separate goroutine. This
can result in the main event loop finishing it's process of the current
event, redrawing the screen, and the terminal having additional updates
to be drawn.
This fixes race conditions by drawing and calling screen.Show in a
separate goroutine.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The render method sets everything as invalid if there was a popover.
This is no longer necessary, as everything is redrawn anyways.
Remove the check and extra atomic set of dirty and invalidate.
Remove unused return value
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Combine tcell events with WorkerMessages to better synchronize state
with IO and UI. Remove Tick loop for rendering. Use events to trigger
renders.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add AercMsg as a main interface for internal communication in aerc in
preparation for a main event loop. Add a QueueRedraw function to to
trigger a redraw. This will be needed for widgets which should be drawn
after some delay (completions, terminal, for example)
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Fix charset to UTF-8 in part attachments. The forward and recall
commands fetch message parts with the go-message package which decodes
to UTF-8. Hence, we should set the charset of the part attachment to
utf-8 and not just copying over the one from the original message.
Reported-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The MouseEvent locks the composer, and also calls FocusEditor which
attempts to lock the composer. This results in a deadlock.
No need to call FocusEditor which takes a name as parameter and needs to
iterate over all editors to find the correct one. We already have the
headerEditor object, use it directly.
Fixes: bf2bf8c242 ("compose: prevent out of bounds access")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Fixes updating the flags in the imap backend. Before, the silent flag
was set incorrectly by 75fc42e ("imap: send message info updates for
bulk flag ops") which caused some imap servers to not send the updated
flags. By disabling the silent flag, the flag update will return a
corrsponding value that we can send back to the message store to update
the flags correctly.
Fixes: 75fc42e ("imap: send message info updates for bulk flag ops")
Reported-by: Jens Grassel <jens@wegtam.com>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Jens Grassel <jens@wegtam.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add a peek flag -p to the view commands to open the message viewer
without setting the "seen" flag. If the flag is set, it would ignore the
"auto-mark-read" config.
The SetSeen flag will be propagated in case the message viewer moves on
to other messages, i.e. with the delete or archive commands.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add option to open a message in the message viewer without setting the
seen flag. Enables the message viewer to be used as a preview pane
without changing the message flags unintentionally. Before, the message
viewer would set the seen flag by default. The IMAP backend will now
always fetch the message body with the peek option enabled (same as we
fetch the headers).
An "auto-mark-read" option is added to the ui config which is set to
true by default. If set the false, the seen flag is not set by the
message viewer.
Co-authored-by: "James Cook" <falsifian@falsifian.org>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The worker uses a buffered channel to queue tasks. Buffered channels
are effective at FIFO, but are prone to blocking. The design of aerc is
such that the UI must always accept a response from the backends, and
the backends must always accept a request from the UI. By using buffered
channels for both of these communication channels, a deadlock will
occur.
Break the chain by using a doubly linked list (container/list from the
standard library) to queue tasks for the worker. Essentially, this is an
infinitely buffered channel - but more memory efficient as it can change
size dynamically.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
When checking for an exKey, aerc inspects the key and the rune of the
event vs the exkey binding. Runes should only be inspected if the key is
a tcell.KeyRune. Some Ctrl-[:alpha:] keys report a rune in tcell, but
aerc does not have these bound to the keystroke definition. Only <C-x>
has a rune bound, and is one of the very few <C-> keys that can actually
be bound to exKey
Only compare the Rune field if the key is of type KeyRune. Otherwise,
compare the Key. Also compare any modifiers with the keystroke/key
event. These changes allow for any control or alt key combination to be
bound to the exkey.
Update documentaiton to reflect that the default keybind is ':', and not
<semicolon>
Fixes: https://todo.sr.ht/~rjarry/aerc/67
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The maildir worker currently populates the list of mail folders by
listing all the filesystem subdirectories in the maildir directory.
Although there's no official specification for maildir subfolders,
they should all have cur/ new/ and tmp/ subdirectories to be valid.
This patch prevents directories that don't have those subdirectories
present on the filesystem from appearing in the account folder list.
This is useful for example to prevent ".notmuch" and ".notmuch/xapian"
from showing up in the folder list if using notmuch to index emails
while using aerc's maildir backend.
Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
Flag fetching is debounced in the UI, creating a race condition where
fields are accessed in the AfterFunc. Protect the needsFlags field with
a mutex.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Subsitute the format specifier %w for %v in the logging facility. The
logging functions use a fmt.Sprintf call behind the scene which does not
recognize %w. %w should be used in fmt.Errorf when you want to wrap
errors. Hence, the log entries that use %w are improperly formatted like
this:
ERROR 2022/10/02 09:13:57.724529 worker.go:439: could not get message
info %!w(*fmt.wrapError=&{could not get structure: [snip] })
^
Links: https://go.dev/blog/go1.13-errors
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Upgrade tcell-term to v0.2.0
Use Start method from tcell-term. This prevents aerc from needing to
wait until the command has started to continue. The tcell-term start
method blocks until the command is started, similar to cmd.Start. By
doing so, we prevent a race condition between aerc and tcell-term on
access to cmd.Process.
Remove cleanup of cmd, this is all already handled by tcell-term when
Close is called.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Add XOAUTH2 authentication support for IMAP and SMTP. Although XOAUTH2
is now deprecated in favor of OAuthBearer, it is the only way to connect
to Office365 since Basic Auth is now completely removed.
Since XOAUTH2 is very similar to OAuthBearer and uses the same
configuration parameters, this is basically a copy-paste of the existing
OAuthBearer code.
However, XOAUTH2 support was removed from go-sasl library, so this
change reimports the code that was removed from go-sasl and offers it
a new home in lib/xoauth2.go. Hopefully it shouldn't be too hard to
maintain, being less than 50 SLOC.
Link: https://github.com/emersion/go-sasl/commit/7bfe0ed36a21
Implements: https://todo.sr.ht/~rjarry/aerc/78
Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
Instead of xdg-open (or open on MacOS), allow forcing a program to open
a message part. The program is determined in that order of priority:
1) If :open has arguments, they will be used as command to open the
attachment. If the arguments contain the {} placeholder, the
temporary file will be substituted, otherwise the file path is added
at the end of the arguments.
2) If a command is specified in the [openers] section of aerc.conf for
the part MIME type, then it is used with the same rules of {}
substitution.
3) Finally, fallback to xdg-open/open with the file path as argument.
Update the docs and default config accordingly with examples.
Fixes: https://todo.sr.ht/~rjarry/aerc/64
Co-authored-by: Jason Stewart <support@eggplantsd.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
There is no need for convoluted channels and other async fanciness.
Expose a single XDGOpen static function that runs a command and returns
an error if any.
Caller is responsible of running this in an async goroutine if needed.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
The terminal widget has two invalidation methods, one exported and one
private. The private one does nothing special.
Remove the private method and only use the exported method.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
The read command calls store.Flag in a separate goroutine unnecessarily.
Calling this method on store should be very fast, as it only sends a
message to the backend worker and does not wait on IO.
Call the store.Flag method from the main thread. Remove wrapper function
and call store.Flag directly for cleaner code.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Commit 716ade8968 ("worker: lock access to callback maps") introduced
locks to the worker callback maps. The locks also locked the processing
of the callback, which had the unintended side effect of deadlocking the
worker if any callbacks attempted to post a new action or message.
Refactor the locks to only lock the worker while accessing the maps.
Fixes: 716ade8968 ("worker: lock access to callback maps")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Call SelectedMessage() in the mark command only when the uid of the
currently selected message is actually needed. If no message is
selected, i.e. after some filter operations where the previously
selected message is not in the results, 'mark -a' would fail since no
message is selected and an error is returned from SelectedMessage() even
though this is not necessary to mark or unmark all messages.
Reported-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
Pass message containing remaining directories to check. Account widget
will recursively call CheckMail with the remaining directories until
a Done message is returned.
Only needed for IMAP worker as other workers run check-mail-cmd in
a separate goroutine.
Suggested-By: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: kt programs <ktprograms@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
Fix the following panic, seen while switching accounts:
runtime error: index out of range [4] with length 4
goroutine 6 [running]:
git.sr.ht/~rjarry/aerc/widgets.(*Composer).Focus(0xc005cfbe30?, 0x40?)
git.sr.ht/~rjarry/aerc/widgets/compose.go:618 +0x51
git.sr.ht/~rjarry/aerc/widgets.(*Aerc).focus(0xc00034c000, {0x0?, 0x0})
git.sr.ht/~rjarry/aerc/widgets/aerc.go:568 +0xec
git.sr.ht/~rjarry/aerc/widgets.(*Aerc).BeginExCommand.func2()
git.sr.ht/~rjarry/aerc/widgets/aerc.go:590 +0x4c
git.sr.ht/~rjarry/aerc/widgets.(*ExLine).Event(0xc009453860, {0xbb6820?, 0xc009baa320?})
git.sr.ht/~rjarry/aerc/widgets/exline.go:81 +0xbc
git.sr.ht/~rjarry/aerc/widgets.(*Aerc).Event(0xc009ab1950?, {0xbb6820?, 0xc009baa320?})
git.sr.ht/~rjarry/aerc/widgets/aerc.go:285 +0x470
git.sr.ht/~rjarry/aerc/lib/ui.(*UI).ProcessEvents(0xc000327540)
git.sr.ht/~rjarry/aerc/lib/ui/ui.go:117 +0x202
created by main.main
git.sr.ht/~rjarry/aerc/aerc.go:244 +0x94c
Protect Composer.editable and Composer.focus with a mutex.
Reported-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Moves logic for creating dynamic folders from the dirlist widget to the
backend. Since dynamic folders are notmuch-specific, the notmuch backend
should be responsible for correctly setting up those folders. It does
that by sending two DirectoryInfos: the first to create the message
store, the second to fetch the directory content.
This approach also fixes a deadlock introduced by 716ade8968
("worker: lock access to callback maps").
Reported-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
When parsing address header fields, the field charset is automatically
decoded to UTF-8. If the charset is unknown, use the raw field value.
Fixes: https://todo.sr.ht/~rjarry/aerc/91
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
The exline widget works by matching actual keystrokes to a map of
keybinds, and if a match is found sending simulated keystrokes through
aerc. This has the effect of aerc thinking we are actually typing in the
expanded command, and aerc attempts to draw the completions. This
results in even basic navigation having two screen draws:
For example, pressing 'j' to select the next message (:next), draws once
for the initial key event and state change, and again after the
completion debounce timer.
Disable tab completion while aerc is simulating keystrokes. If the
exline still has focus after simulating keystrokes, restore tab
completion.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Koni Marti <koni.marti@gmail.com>
Send message info updates back to to ui instead of posting a fetch
header action to the worker when performing a bulk flag operation. This
prevents the worker channels from filling up which can result in a
deadlock.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Tim Culverhouse <tim@timculverhouse.com>
When using a tiling window manager, aerc terminal dimensions may be
greatly reduced after a new window has been created by :open. When the
ui attempts to render to formerly-valid coordinates, SetCell & Printf
may panic. Replace panic() with no-op in both functions to prevent
aerc from crashing after a window shrink.
Signed-off-by: Jason Stewart <support@eggplantsd.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Protect access to fields in textinput. Concurrent access can happen in
the main event loop and the completion debounce function.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Protect access to fields idleing and waiting via a mutex.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
