Commit graph

108 commits

Author SHA1 Message Date
Tim Culverhouse
2551dd1bfa feat: add background mail polling option for all workers
Check for new mail (recent, unseen, exists counts) with an external
command, or for imap with the STATUS command, at start or on
reconnection and every X time duration

IMAP:
The selected folder is skipped, per specification. Additional config
options are included for including/excluding folders explicitly.

Maildir/Notmuch:
An external command will be run in the background to check for new mail.
An optional timeout can be used with maildir/notmuch. Default is 10s

New account options:
check-mail
check-mail-cmd (maildir/notmuch only)
check-mail-timeout (maildir/notmuch only), default 10s
check-mail-include (IMAP only)
check-mail-exclude (IMAP only)

If unset, or set less than or equal to 0, check-mail will be ignored

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-31 14:32:51 +02:00
Koni Marti
b29293d7b5 imap: add timeout to tcp connect functions
Extract the tcp connection details and timeout the tcp connect functions
(net.ResolveTCPAddr and net.DialTCP). If timed out, ensure that the
connection is properly closed.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti
b92efe4cd9 imap: add debouncer to the idler
Add a debouncer to the idle mode. Avoid unnecessary idling when another
job arrives within a certain time frame.

For example, the ui sends three messages to the worker at the same time
when we open a message (FlagMessage, FetchMessageBodyPart, and the
FetchMessageHeaders). The debouncer prevents the unnecessary entering
and leaving of the idle mode between those messages.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti
e5b339702a imap: monitor the logout channel with an observer
Untangle the observer functionality from the message handling routine.
Observe the imap client's logout channel and trigger a connection error
when necessary to start the reconnect cycle.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti
397a6f267f imap: manage idle mode with an idler
Untangle the idle functionality from the message handling routine. Wait
for the idle mode to properly exit every time to ensure a consistent
imap state. Timeout when hanging in idle mode and inform the ui.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti
4d75137b20 imap: extract imap config and configure handling
Extract the imap config and move the configure part out of the message
handler.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-05-04 14:07:15 +02:00
Koni Marti
8f976af17b imap: fix out-of-range panic for imap updates
Check slice bounds before using it for the message and expunge updates.
Log the error but ignore the affected updates.

Link: https://lists.sr.ht/~rjarry/aerc-devel/%3CCJEHBFFUI11T.1AYGOMVGZ87ZS%40rek2system%3E
Reported-by: ReK2 <rek2@hispagatos.org>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-04-29 13:41:35 +02:00
Robin Jarry
8ed95b0d2a imap: avoid crash when replying to unread message
When running `:reply -q` on an unread message, aerc crashes after
opening the editor:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x5d1019]

goroutine 63 [running]:
bufio.(*Reader).fill(0xc000086ef8)
	/usr/lib/golang/src/bufio/bufio.go:106 +0xd9
bufio.(*Reader).Peek(0xc00020bef8, 0x1)
	/usr/lib/golang/src/bufio/bufio.go:144 +0x5d
github.com/emersion/go-message/textproto.ReadHeader(0xc00004a700?)
	emersion/go-message@v0.15.0/textproto/header.go:525 +0x5f
git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessageBodyPart.func1(0xc00056e280)
	worker/imap/fetch.go:99 +0x1ab
git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessages.func1()
	worker/imap/fetch.go:178 +0xd7
created by git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleFetchMessages
	worker/imap/fetch.go:172 +0x12b

This happens because the flags are updated in the callback that receives
the message itself. It causes the flag update to arrive in the same
channel/request. Ignore the messages that have an empty body (i.e. only
containing flag updates). This is inherently racy but there seems no way
to get rid of these extra messages.

Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Connor Kuehl <cipkuehl@gmail.com>
2022-04-28 17:55:57 +02:00
Moritz Poldrack
ae83373fa6 logging: added a log on panic
Since panics still regularly "destroy" the terminal, it is hard to get a
stack trace for panics you do not anticipate. This commit adds a panic
handler that automatically creates a logfile inside the current working
directory.

It has to be added to every goroutine that is started and will repair
the terminal on a panic.

Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-03-23 20:56:09 +01:00
Moritz Poldrack
e30bd324a3 go vet: composite literal uses unkeyed fields
This commit fixes all occurrences of the abovementioned lint-error in
the codebase.

Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
2022-03-18 13:33:16 +01:00
Koni Marti
48e01bd51f imap: start reconnect when initial connect fails
Start the reconnect cycle when the initial connect fails. Make the
connection observer send a connection error when the imap client is nil.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-20 22:04:16 +01:00
Koni Marti
11a4d5b71c imap: reconnect with exponential backoff
waits an increasing amount of time before attempting a reconnect.
Wait is capped at 16s. Prevents many reconnect attemps in a short time period.

Fixes commit 05ad96a30c ("imap: improve reconnect stability") that
improved the reliability of the reconnect mechanism but did not
implement controls to prevent the triggering of too many reconnects
within a short period of time.

Fixes: 05ad96a30c ("imap: improve reconnect stability")
Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-19 15:24:55 +01:00
Koni Marti
05ad96a30c imap: improve reconnect stability
improves the robustness of the imap reconnect feature which was
introduced in commit beae17a6da ("imap: auto-reconnects on connection
error").

If a connection error is emitted, the message list is cleared and a
corresponding error message is shown in the ui. Status bar is updated as
well. Upon reconnect, the directories and the message list will be
re-fetched (same behavior as the connect command).

Reconnect can be enabled and disabled with the connect and the
disconnect commands.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-02-06 17:06:20 +01:00
Robin Jarry
a46468521b imap: do not abort on malformed messages
Some messages contain invalid headers, textproto.ReadHeader fails with
an error:

  malformed MIME header key: From nobody Fri Jan 14 19

And all other messages of the selected folder are ignored.

Return an explicit error message to the UI and continue processing other
messages. This is consistent with what the maildir worker does.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-02-06 16:04:15 +01:00
Koni Marti
beae17a6da imap: auto-reconnects on connection error
if the worker emits a connection error, the ui will automatically send back a
reconnect command. The worker then establishes a new connection. Auto-reconnect
is disabled when the user sends the disconnect command.

Fixes: https://todo.sr.ht/~rjarry/aerc/1
Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-01-19 17:40:52 +01:00
Koni Marti
1ace50a6b9 imap: emits connection error on logout
implements a new connection error message. This allows the worker to emit a
connection-related error message to the ui when the imap client closes the
loggedOut channel.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-01-19 17:34:42 +01:00
Koni Marti
022bf1a11f imap: fix panic when sending multiple connect cmds
fixes the panic when the user sends multiple connect commands and is
already connected. The panic is caused by closing an already closed
channel. This happens when the idle re-init code is not executed, e.g.
when there's a return statement in the switch block.  A defer func()
before the switch block will prevent this. The existing behavior of only
creating a new idleStop channel when properly connected is preseverd.

Signed-off-by: Koni Marti <koni.marti@gmail.com>
2022-01-19 17:34:41 +01:00
Robin Jarry
899d1cca3c Revert "imap: attempt automatic reconnection on error"
This reverts commit c605ada3dd.

This breaks reading message bodies. I am not sure why, I'll take some
time to fix it later.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-01-15 12:13:47 +01:00
Robin Jarry
c605ada3dd imap: attempt automatic reconnection on error
Attempt to reconnect to the server when there is an unexpected
disconnection or network error.

Use the Client.LoggedOut() channel which is closed when the connection
is closed.

This patch is rather flaky and is certainly bugged. However, it is
a start.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2022-01-14 21:17:40 +01:00
Robin Jarry
15a4cc7d0a imap: fix build on macos
Fix the following build error on mac os:

 worker/imap/worker.go:368:29: undefined: syscall.TCP_KEEPCNT
 worker/imap/worker.go:376:29: undefined: syscall.TCP_KEEPINTVL

These symbols are not defined on darwin.

Fixes: 5dfeff75f3 ("imap: add tcp connection options")
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-11 21:45:51 +01:00
Robin Jarry
5dfeff75f3 imap: add tcp connection options
Allow fine tuning tcp connection options.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-07 21:28:13 +01:00
Robin Jarry
33aaf94663 imap: use low level tcp connection
In preparation for tcp keepalive options, we need access to the
net.TCPConn object associated with an IMAP connection. The only way to
do this is to create the connection ourselves.

No functional change.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-07 21:27:41 +01:00
Robin Jarry
cc432eefd3 imap: move connect procedure into a dedicated function
This will prepare for extra tcp connection options support and for
automatic reconnect. No functional change.

Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-12-07 21:26:56 +01:00
Robin Jarry
6ddfc23e61 imap: fix segfault when disconnecting
Do not set client = nil, it breaks almost all message handlers:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8c7e51]
goroutine 25 [running]:
  git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleListDirectories
    worker/imap/list.go:32
  git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).handleMessage
    worker/imap/worker.go:174
  git.sr.ht/~rjarry/aerc/worker/imap.(*IMAPWorker).Run
    worker/imap/worker.go:264
  created by git.sr.ht/~rjarry/aerc/widgets.NewAccountView
    widgets/account.go:85 +0x518

Simply leave the disconnected client object, it already returns explicit
error messages.

Fixes: e41ed82cf3 ("imap: add manual {dis,}connect support")
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-22 21:32:56 +01:00
y0ast
dc2a2c2dfd messages: allow displaying email threads
Display threads in the message list. For now, only supported by the
notmuch backend and on IMAP when the server supports the THREAD
extension.

Setting threading-enable=true is global and will cause the message list
to be empty with maildir:// accounts.

Co-authored-by: Kevin Kuehler <keur@xcf.berkeley.edu>
Co-authored-by: Reto Brunner <reto@labrat.space>
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-13 15:05:59 +01:00
Robin Jarry
e41ed82cf3 imap: add manual {dis,}connect support
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-05 10:45:31 +01:00
Robin Jarry
0d645bcebd go.mod: change base git url
I'm not sure what are the implications but it seems required.

Link: https://github.com/golang/go/issues/20883
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-05 10:21:45 +01:00
Robin Jarry
4cf0ea2a05 imap: use builtin idle support
go-imap supports IDLE since 1.2.0. Remove dependency to go-imap-idle.

Link: https://github.com/emersion/go-imap/commit/ac3f8e195ef1b6d
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-02 11:24:33 +01:00
Reto Brunner
daf214a1f8 imap: strip <> from message-ids 2020-11-14 15:40:13 +01:00
Reto Brunner
fc9ccc3000 remove models.Address in favor of go-message Address
We made a new type out of go-message/mail.Address without any real reason.
This suddenly made it necessary to convert from one to the other without actually
having any benefit whatsoever.
This commit gets rid of the additional type
2020-11-14 15:40:13 +01:00
Reto Brunner
b6bcf89784 imap: add sort support 2020-10-11 09:18:45 +02:00
Reto Brunner
c846307144 base models.Address on the mail.Address type
This allows us to hook into the std libs implementation of parsing related stuff.
For this, we need to get rid of the distinction between a mailbox and a host
to just a single "address" field.

However this is already the common case. All but one users immediately
concatenated the mbox/domain to a single address.

So this in effects makes it simpler for most cases and we simply do the
transformation in the special case.
2020-08-20 19:18:57 +02:00
ARaspiK
fe1cabb077 Add support for :rmdir
The `:rmdir` command removes the current directory (`-f` is required if
the directory is not empty).

This is not supported on the notmuch backend.

An issue with the maildir backend is that some sync programs (e.g.
offlineimap) may recover the directory after it is deleted.  They need
to specifically be configured to accept deletions, or special commands
need to be executed (e.g. `offlineimap --delete-folder`) to properly
delete folders.

A danger of using this on the IMAP backend is that it is possible for a
new message to be added to the directory and for aerc to not show it
immediately (due to a slow connection) - using `:rmdir` at this moment
(with `-f` if the directory already contains messages) would delete the
directory and the new message that just arrived (and all other
contents).  This is documented in aerc(1) so that users are aware of
possible risks.
2020-08-19 11:38:57 +02:00
Tobias Wölfel
494bd674a9 Add flag based search options
Provide search and filter with the option to specify more flag based
conditions.
Use '-x <flag>' to search for messages with a flag (seen, answered,
flagged) and '-X <flag>' to search for messages without a flag.
2020-07-25 08:08:08 +02:00
ARaspiK
0535f6333f Add additional flagging functionality
More mail flags can now be set, unset, and toggled, not just the
read/seen flag.

This functionality is implemented with a new `:flag` and `:unflag`
command, which are extensions to the matching `:read` and `:unread`
commands, adding support for different flags.  In fact, the
`read`/`unread` commands are now recognized aliases to `flag`/`unflag`.
The new commands are also well documented in aerc(1).

The change mostly extends the previous read/unread setting functionality
by adding a selection for the flag to change.
2020-07-08 09:13:03 +02:00
Srivathsan Murali
b1eb7ad18d Set AnsweredFlag on successful reply 2020-05-25 11:29:53 -04:00
Reto Brunner
bae678e8f2 imap: Remove FetchMessageBodyPart.{Encoding,Charset}
Fixes https://todo.sr.ht/~sircmpwn/aerc2/352 exactly as suggested by emersion.
2020-05-16 20:03:42 +02:00
Ben Fiedler
92f59ef94a imap/fetch.go: report error when fetching headers fails
Correct me if I'm wrong, but shouldn't this raise an error when it
fails?
2020-04-24 12:59:21 -04:00
Amir Yalon
ae7595e945 Ignore missing message part in getDecodedPart
The code that calls this function handles nil without issues.
2020-03-09 09:12:33 -04:00
Simon Ser
f3535703b0 worker/imap: rely on go-imap for charset handling
Set imap.CharsetReader so that go-imap can automatically decode all
encoded fields.
2020-03-03 16:48:00 -05:00
Galen Abell
6ff3c7a1ba Mark sent messages as "seen" in maildir
- Add maildir flags to complement a messages imap flags
- Set the "seen" flag on sent messages when using the maildir backend
- Cleanup AppendMessage interface to use models.Flag for both IMAP and
  maildir
2020-03-03 20:08:37 +01:00
Reto Brunner
f3dbecaaea remove the dirInfoUpdateRequest functionality
The notmuch worker followed suit in handling the dirInfo submission manually.
That removes the last user so we might as well remove the functionality.
2020-02-29 13:16:33 -05:00
Reto Brunner
5dd0f454c1 lib/msgstore: debounce directoryUpdateRequests
Apparently sending an event for every incoming messageInfo slows down
the application significantly.

Therefore this slows down the emmision rate, on the cost of being out of date
in some cases.
2020-02-25 11:05:26 -05:00
Reto Brunner
30c0a9fba4 imap: fix double closing idleStop
The idle restart code is at the end of handleMessage in the worker.
However if an unsupported msg comes in, we returned early, skipping the re-init.
That lead to a crash due to double closing idleStop in the next iteration.
2020-02-16 10:41:11 -05:00
Reto Brunner
df5d9a3ec7 imap: emit messageinfo when changing read state.
We need to emit the changed msgInfo whenever we modify the state
2020-01-24 10:51:09 -05:00
Reto Brunner
472c421e85 worker/imap: don't decode in FetchFullMessage.
Doing that breaks `git am` as it expected the encoded variant.
Same is probably true for any sort of signature validation (gpg / dkim)
2020-01-18 19:00:02 -05:00
Reto Brunner
3d85f75d9c imap: decode reader prior to returning them 2020-01-05 16:02:45 -05:00
Leszek Cimała
e84e402e48 decode(RFC 2047) imap headers to fix encoding in subject & filenames
Me again,
this time fixing encoding of subjects and attachments. It was problem in
IMAP backend. While other backends user MessageInfo() function which
generates MessageInfo decoded via go-message methodes, IMAP worker is
creating MessageInfo directly, so all non-utf8 subjects and filenames
were in raw form.

This patch fixes it. Not sure if we should care about errors (if
DecodeHeader fails it returns raw string back).

>From what I see, this should solve all encoding problem (tested only
IMAP). So, now I can focus on features. ;-)

Have a great weekend!
Leszek
2019-12-07 14:30:33 -05:00
Kevin Kuehler
8a848303fe worker/imap: Fix seqMap race condition
When deleting a message, sometimes FetchDirectoryContents will fire.
FetchDirectoryContents will return a smaller set of UIDs since messages
have been deleted. This operation races with fetching from the seqMap in
client.ExpungeUpdate. Only recreate the seqMap if it can grow so that
messages will continue to be expunged.

Signed-off-by: Kevin Kuehler <keur@xcf.berkeley.edu>
2019-11-10 17:19:23 -05:00
Frode Aannevik
550ef0bc1f Fix: oauthbearer runtime error
Configure an oauthbearer source without a token_endpoint
parameter would panic due to nil pointer dereference

  Example
    source=imaps+oauthbearer://frode.aa%40gmail.com@imap.gmail.com:993
    source-cred-cmd=pass oatuh2 frode.aa@gmail.com

token_endpoint is not required as it will use the provided
password as access_token when it is not set
2019-10-16 11:04:41 -04:00